Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Herald: Python surpasses Perl in popularity!

2 views
Skip to first unread message

Xah Lee

unread,
Nov 24, 2008, 8:06:32 PM11/24/08
to
herald: Python surpasses Perl in popularity!

According to
“TIOBE Programming Community Index for November 2008” at
http://www.tiobe.com/content/paperinfo/tpci/index.html

it seems that Python has surpassed Perl in popularity this month!

Good for Python!

From my own personal experience in the programing industry, i find it
hard to believe that Python actually surpassed Perl in terms of use in
companies. Python is used in, Google, as everybody knows, but where
else? Perl is used in, umm, about every company except Google (and
probably Google too! in fact).

A quick search in monster.com, i find that perl returns 2673 results,
and Python returns 879 results. Perlers, you still safe!

Looking at other lang popularity site,
http://www.langpop.com/
it indicates that it's pretty much a tie.

So, i think it's not all peaches and cream for Python yet.

However, am pretty sure it'll be so in the next couple of years.

(btw, for those perlers who wishes to learn Python, see a comparative
tutorial:

• Xah's Perl and Python Tutorial
http://xahlee.org/perl-python/index.html
)

Xah
http://xahlee.org/


david...@preisshare.net

unread,
Nov 24, 2008, 8:25:51 PM11/24/08
to pytho...@python.org

Interesting topic !

I would venture so far to say that perl is todays 'C'..

Perl is todays language of technical complexity. It is obscure,
complex, and is oriented towards the supremely intelligent (or equally
- those seeking to get 'lost' in programming).

Python, whilst very powerful, doesn't have the sheer scale of
contributors that Perl has. ie cpan. Many libraries, aren't as
sophisticated - ie spreadsheet reading and writing.

Python is very clever.. but it goes in a 'purest' direction.

It is being used in stockbrocking and many other fields. It has a good
following. It is being taught in schools here.. meaning it is
extremely healthy.

More popular doesn't always mean better...

Quoting Xah Lee <xah...@gmail.com>:

> --
> http://mail.python.org/mailman/listinfo/python-list
>

John Machin

unread,
Nov 24, 2008, 10:08:08 PM11/24/08
to
On Nov 25, 12:25 pm, david.l...@preisshare.net wrote:

> Python, whilst very powerful, doesn't have the sheer scale of  
> contributors that Perl has. ie cpan. Many libraries, aren't as  
> sophisticated - ie spreadsheet reading and writing.

Care to elaborate on the relative unsophistication of Python
spreadsheet reading and writing libraries?

Cheers,
John

david...@preisshare.net

unread,
Nov 24, 2008, 10:59:41 PM11/24/08
to pytho...@python.org
Quoting John Machin <sjma...@lexicon.net>:

> Care to elaborate on the relative unsophistication of Python
> spreadsheet reading and writing libraries?
>
> Cheers,
> John

Not really.

But one only has to use both languages on a regular basis to realise
that perl is well ahead on the libaray/module front.

Some things in python are still pretty primitive...

but I know a lot of work has gone into them... just not anywhere near
like what you see in perl.

David

John Machin

unread,
Nov 24, 2008, 11:14:07 PM11/24/08
to
On Nov 25, 2:59 pm, david.l...@preisshare.net wrote:

I'll try again: On what grounds do you assert that "Many libraries,
aren't as sophisticated - ie spreadsheet reading and writing."? What
Python spreadsheet reading and writing libraries have you used? In
what way are they less sophisticated than their perl counterparts? Do
you have any particular enhancement requests?


alex23

unread,
Nov 24, 2008, 11:20:08 PM11/24/08
to
On Nov 25, 11:25 am, david.l...@preisshare.net wrote:
> Perl is todays language of technical complexity. It is obscure,  
> complex, and is oriented towards the supremely intelligent [...]

I think you misspelled "insular".

Richard Riley

unread,
Nov 25, 2008, 12:03:40 AM11/25/08
to
alex23 <wuw...@gmail.com> writes:

Sounds like eLisp :-;

david...@preisshare.net

unread,
Nov 25, 2008, 12:57:20 AM11/25/08
to pytho...@python.org
Quoting John Machin <sjma...@lexicon.net>:

> I'll try again: On what grounds do you assert that "Many libraries,
> aren't as sophisticated - ie spreadsheet reading and writing."? What
> Python spreadsheet reading and writing libraries have you used? In
> what way are they less sophisticated than their perl counterparts? Do
> you have any particular enhancement requests?

Hi John,

Since you asked a second time and appear genuinely interested - I will
answer properly. :-)

pyExcelorator - Setting column widths. Doesn't seem to work.

Nor setting attrributes in sheets. Things seem to be only half done
when compared to what I have seen in Perl.

Perl was much easier to get the same functionality working.

Here is my particular complaint:

..
from pyExcelerator import *
..
..
# -- Adjust the column width
if dbf1.fieldDefinitions()[f].fieldInfo()[1] == 'C':
ws.col(f).width = 0x0d00 + (f * 600)

in perl....

$sheet1->set_column(0, 0, 20);


perl works nicely... I couldn't find a way to so easily set a column width...

plus.. colours are all broken...

If you can shed any light on it for me, I would be thankful.

Regards

David


John Machin

unread,
Nov 25, 2008, 2:13:26 AM11/25/08
to
On Nov 25, 4:57 pm, david.l...@preisshare.net wrote:
> Quoting John Machin <sjmac...@lexicon.net>:
>
> > I'll try again: On what grounds do you assert that "Many libraries,
> > aren't as sophisticated - ie spreadsheet reading and writing."? What
> > Python spreadsheet reading and writing libraries have you used? In
> > what way are they less sophisticated than their perl counterparts? Do
> > you have any particular enhancement requests?
>
> Hi John,
>
> Since you asked a second time and appear genuinely interested - I will  
> answer properly. :-)
>
> pyExcelorator  - Setting column widths. Doesn't seem to work.

pyExcelerator is abandonware. Check out xlwt (a fork of pyExcelerator)
at http://pypi.python.org/pypi/xlwt

What are the symptoms of setting column widths not seeming to be
working?

>
> Nor setting attrributes in sheets. Things seem to be only half done  
> when compared to what I have seen in Perl.
>
> Perl was much easier to get the same functionality working.

If you mean cell formatting, check out the new easyxf functionality in
xlwt.

> Here is my particular complaint:

> from pyExcelerator import *


> # -- Adjust the column width

> if dbf1.fieldDefinitions()[f].fieldInfo()[1] == 'C':

The above code looks like it's accessing the metadata for the f'th
column/field in a dBase file; nothing to do with pyExcelerator.

>      ws.col(f).width = 0x0d00 + (f * 600)

With slightly more meaningful names:
worksheet.col(col_index).width = 0x0d00 + col_index * 600

Not sure why you're multiplying by the dbf field number; wouldn't the
dbf field size be a better bet?

In any case, this is merely an elaboration of the basic functionality
column_object.width = expression
How would you prefer to express it?

> in perl....

>      $sheet1->set_column(0, 0, 20);

> perl works nicely... I couldn't find a way to so easily set a column width...

I'm guessing that this is equivalent to the pyExcelerator/xlwt code
sheet1.col(0).width = 20 * 256
(column width in Excel is in 256ths of a standardised character)

> plus.. colours are all broken...

"All broken" meaning what?

> If you can shed any light on it for me, I would be thankful.

I hope that helped a little on the write side; what were your problems
on the read side?

You might like to join the discussion group at http://groups.google.com/group/python-excel

Cheers,
John

Lawrence D'Oliveiro

unread,
Nov 25, 2008, 2:34:52 AM11/25/08
to
david...@preisshare.net wrote:

> I would venture so far to say that perl is todays 'C'..

Perl is the FORTRAN of scripting languages. Python is in some ways like Pascal.

Java is like COBOL.

C? Who knows...

Jorgen Grahn

unread,
Nov 25, 2008, 5:47:12 PM11/25/08
to
On Mon, 24 Nov 2008 20:25:51 -0500, david...@preisshare.net <david...@preisshare.net> wrote:

> Quoting Xah Lee <xah...@gmail.com>:
>
>> herald: Python surpasses Perl in popularity!
>>
>> According to

>> ?TIOBE Programming Community Index for November 2008? at


>> http://www.tiobe.com/content/paperinfo/tpci/index.html
>>
>> it seems that Python has surpassed Perl in popularity this month!

> Interesting topic !

Hard to take a popularity index seriously when Logo is at #19 and
Bourne shell at #32 ... and then they suggest that their readers can
use it "to make a strategic decision about what programming language
should be adopted when starting to build a new software system".

/Jorgen

--
// Jorgen Grahn <grahn@ Ph'nglui mglw'nafh Cthulhu
\X/ snipabacken.se> R'lyeh wgah'nagl fhtagn!

Roy Smith

unread,
Nov 25, 2008, 6:07:22 PM11/25/08
to
In article <slrngip03p.c...@frailea.sa.invalid>,

Jorgen Grahn <grahn...@snipabacken.se> wrote:

> Hard to take a popularity index seriously when Logo is at #19 and
> Bourne shell at #32 ... and then they suggest that their readers can
> use it "to make a strategic decision about what programming language
> should be adopted when starting to build a new software system".

When you've seen some of the shell scripts I've seen, it's not hard to
imagine that logo might be a better choice of programming language.

david...@preisshare.net

unread,
Nov 25, 2008, 10:28:12 PM11/25/08
to pytho...@python.org
Quoting John Machin <sjma...@lexicon.net>:

> pyExcelerator is abandonware. Check out xlwt (a fork of pyExcelerator)
> at http://pypi.python.org/pypi/xlwt

Thanks John.

That is very helpful.

I will move to that product..

david...@preisshare.net

unread,
Nov 25, 2008, 10:29:59 PM11/25/08
to pytho...@python.org
Quoting Lawrence D'Oliveiro <l...@geek-central.gen.new_zealand>:

> Perl is the FORTRAN of scripting languages. Python is in some ways
> like Pascal.
>
> Java is like COBOL.
>
> C? Who knows...

Your memory goes way back...

haha


Steve Holden

unread,
Nov 26, 2008, 8:19:24 AM11/26/08
to pytho...@python.org

:)

In fact all that's really happened is that Perl has slid down the ranks,
at least temporarily. Python has been around the 6/7 mark for a while now.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/

Marco Mariani

unread,
Nov 26, 2008, 9:34:45 AM11/26/08
to
Steve Holden wrote:

> In fact all that's really happened is that Perl has slid down the ranks,
> at least temporarily. Python has been around the 6/7 mark for a while now.

Also.. can someone attempt to explain the funny correlation in
popularity over time between, for instance, Python and Delphi? :-)

Roy Smith

unread,
Nov 26, 2008, 10:37:51 AM11/26/08
to
In article <a8dXk.39476$8%.37958@tornado.fastwebnet.it>,
Marco Mariani <ma...@sferacarta.com> wrote:

It's explained in http://en.wikipedia.org/wiki/Delphi.

Xah Lee

unread,
Nov 26, 2008, 11:00:32 AM11/26/08
to
On Nov 25, 2:47 pm, Jorgen Grahn <grahn+n...@snipabacken.se> wrote:

> On Mon, 24 Nov 2008 20:25:51 -0500, david.l...@preisshare.net <david.l...@preisshare.net> wrote:
> > QuotingXahLee<xah...@gmail.com>:
>
> >> herald: Python surpasses Perl in popularity!
>
> >> According to
> >> ?TIOBE Programming Community Index for November 2008? at
> >>http://www.tiobe.com/content/paperinfo/tpci/index.html
>
> >> it seems that Python has surpassed Perl in popularity this month!
> > Interesting topic !
>
> Hard to take a popularity index seriously when Logo is at #19 and
> Bourne shell at #32 ... and then they suggest that their readers can
> use it "to make a strategic decision about what programming language
> should be adopted when starting to build a new software system".

your remark is a bit overzealous. After all, we all know that site is
websearh based. Although it not some kinda scientific report, but it
does give some good indication of language popularity, however you
define that.

it is conceivable that logo is somewhat used more than bourne shell.

first of all, Logo is a lisp dialect. (it's one of the rare lisp sans
the parens.) The most famous logo book is the triology titled
something like Computer Science Logo Style, by Brian Harvey, who
teaches at UC Berkeley and now and then still post to
“comp.lang.scheme”. (who, like some many veteran Scheme Lisp
dignitaries, cries out against the utter fuckup Scheme 6 (aka R6RS))

Bourne Shell, is pretty much replaced by Bash since several years ago.
For example, as far as i know, linuxes today don't have Bourne Shell
anymore. “sh” is just a alias to bash with some compatibility
parameter. That immediately wipe out a huge sector of unixes that
lives on Bourne Shell. This is a good thing. In about 2000 i called
for this. The fucking asshole Sun Microsystems insists on installing
at least 3 versions of shell utilities in several directories... (and
the BSD unixes insist on their inferior stupid versions of shell
tools)

To be sure, Logo is very much a academic lang, mostly for teaching and
for younsters. Much of its code is about drawing graphics. Some other
major use of Logo is in robotics, much associated with the Lego
robotics toys.

While Bourne Shell, as far as i can venture a guess, is still the
primary startup scripts in various unixes.

It is hard to put down exactly which is “popular”. We have to first
define what we mean by popular, of course. Is it number of programers/
users? Popularity in the sense of awareness? Number of software using
them out there? etc.

However, as mentioned before, all things considered, it is conceivable
that Logo is more popular than sh. For one thing, for any use of shell
script other than the machine startup scripts, people don't use bourne
shell anymore. They use bash, maybe tcsh, and probably vast majority
of unix/server shell oriented installation scripts are done in Perl or
python today.

For those interested in languages, see:

• Proliferation of Computing Languages
http://xahlee.org/UnixResource_dir/writ/new_langs.html

It would be fruitful to actually set aside some 3 hours in some
weekend, to read thru these and the Wikipedia articles linked. You'll
get a survey of today's languages, what they are, what they do, their
nature, their field, and where the landscape of languages might be
tomorrow.

plain text version follows.
-------------------------------

Back to Computing and Its People.
Proliferation of Computing Languages

Xah Lee, 2008-07, 2008-11

There is a proliferation of computer languages today like never
before. In this page, i list some of them.

In the following, i try to list some of the langs that are created
after 2000, or become very active after 2000.

Functional langs:

Erlang↗. Functional, concurrent. Mostly used in a telecomunication
industry for corcurrency and continuous up-time features.
Haskell↗ Oldish, classic functional lang.
Mercury↗. Logic, functional.
Q↗. Functional lang, based on term rewriting. To be replaced by Pure↗.
Oz↗. Concurrent. Multiparadigm.
ML Family:

OCaml↗
Alice↗. Concurrent, ML derivative. Saarland University, Germany.
F#↗. Microsoft's functional lang.
Lisp family or similar:

Mathematica↗. Computer algebra system background. Used mostly for math
and research.
NewLisp↗. Lisp scripting style.
Arc↗. Paul Graham squeezing juice out of his celebrity status.
Qi↗. Common Lisp added with modern functional lang features.
Clojure↗. A new lisp dialect on Java platform.
Scheme↗, notably PLT Scheme↗. Used mostly for teaching.
(Dead. Dylan↗. Apple's re-invention of lisp for industrial programers,
active in the 1990s.)
Computer Algebra and Proof systems:

Coq↗. For formal proofs.
For much more, see Category:Computer algebra systems↗ and Automated
theorem proving↗.
Perl Family or derivative:

PHP↗. Perl derivative for server side web apps. One of the top 10 used
langs post 2000.
Ruby↗. Perl with rectified syntax and semantics.
Perl6↗. Next gen of perl.
Sleep↗. A scripting lang, perl syntax. On Java platform.
On Java Virtual Machine:

Scala↗. A FP+OOP lang on Java platform as a Java alternative.
Groovy↗. Scritping lang on Java platform.
C derivatives:

ObjectiveC↗. Strict superset of C. Used as the primary language by
Apple for Mac OS X.
C#↗. Microsoft's answer to Java. Quickly becoming top 10 lang with
Microsoft's “.NET” architecture.
D↗. Clean up of C++.
2D graphics related.

Scratch↗. Derived from SmallTalk + Logo.
Adobe Flash↗'s ActionScript↗. 2D graphics. Quickly becomes top 10 lang
post 2000 due to popularity of Flash.
Processing↗. 2D graphics on Java platform. Primarily used for art and
teaching.
Misc:

Linden_Scripting_Language↗. Used in virtual world Second Life.
Lua↗. Scripting.
Tcl↗. Scripting, esp GUI.
JavaScript↗. Web browser scripting.
Some Random Thoughts

Following are some random comments on comp langs.

Listing Criterion and Popularity

In the above, i tried to not list implementations. (e.g. huge number
of Scheme implemented in JVM with fluffs here and there; also e.g.
JPython, JRuby, and quite a lot more.) Also, i tried to avoid minor
derivatives or variations. Also, i tried to avoid langs that's one-
man's fancy with little followings.

In the above, i tried to list only “new” langs that are born or seen
with high activity or awareness after 2000. But without this
criterion, there are quite a few staples that still have some user
base. e.g. APL↗, Fortran↗, Cobol↗, Forth↗, Logo↗ (many variants),
Pascal↗ (Ada, Modula, Delphi). And others that are today top 10 most
popular langs: C++, ObjectiveC, Visual Basic.

The user base of the langs differ by some magnitude. Some, such as for
example PHP, C#, are within the top 10 most popular lang with active
users (which is perhaps in order of hundreds of millions). Some
others, are niche but still with huge users (order of tens or hundreds
of thousands), such as LSL, Erlang, Mathematica. Others are niche but
robust and industrial (counting academic use), such as Coq (a proof
system), Processing, PLT Scheme, AutoLisp↗. Few are mostly academic
followed with handful of experimenters, Qi, Arc, Mercury, Q,
Concurrent Clean are probably examples.

For those of you developers of Java, Perl, Python for example, it
would be fruitful to spend a hour or 2 to look at the Wikipedia
articles about these, or their home pages. Wikipedia has several pages
that is a listing of comp langs, of which you can read about perhaps
over 2 hundreds of langs.

Why The List

I was prompted to have a scan at these new lang because recently i
wrote a article titled Fundamental Problems of Lisp, which mentioned
my impression of a proliferation of languages (and all sorts of
computing tools and applications). Quote:

10 years ago, in the dot com days (~1998), where Java, Javascript,
Perl are screaming the rounds. It was my opinion, that lisp will
inevitably become popular in the future, simply due to its inherent
superior design, simplicity, flexibility, power, whatever its existing
problems may be. Now i don't think that'll ever happen as is. Because,
due to the tremendous technological advances, in particular in
communication (i.e. the internet and its consequences, e.g. Wikipedia,
youtube, youporn, social networks sites, blogs, Instant chat, etc)
computer languages are proliferating like never before. (e.g. erlang,
OCaml, Haskell, PHP, Ruby, c#, f#, perl6, arc, NewLisp, Scala, Groovy,
Goo, Nice, E, Q, Qz, Mercury, Scratch, Flash, Processing, ..., helped
by the abundance of tools, libraries, parsers, existence of
infrastructures) New langs, basically will have all the advantages of
lisps or lisp's fundamental concepts or principles. I see that,
perhaps in the next decade, as communication technologies further hurl
us forward, the proliferation of langs will reduce to a trend of
consolidation (e.g. fueled by virtual machines such as
Microsoft's .NET.).

Creating A Lang Is Easy

In general, creating a lang is relatively easy to do in comparison to
equivalent-sized programing tasks in the industry (such as, for
example, writing robust signal processing lib, a web server (e.g.
video web server), a web app framework, a game engine ...etc.).
Computing tasks typically have a goal, where all sorts of complexities
and nit-gritty detail arise in the solving process. Creating a lang
often is simply based on a individual's creativity that doesn't have
much fixed constraints, much as in painting or sculpting. Many langs
that have become popular, in fact arose this way. Popularly known
examples includes Perl, Python Ruby, Perl6, Arc. Creating a lang
requires the skill of writing a compiler though, which isn't trivial,
but today with mega proliferation of tools, even the need for compiler
writing skill is reduced. (e.g. Arc, various langs on JVM. (10 years
ago, writing a parser is mostly not required due to existing tools
such as lex/yacc))

Some lang are created to solve a immediate problem or need.
Mathematica, Adobe Flash's ActionScript, Emacs Lisp, LSL would be good
examples. Some are created as computer science research byproducts,
usually using or resulting a new computing model. Lisp, Prolog,
SmallTalk, Haskell, Qi, Concurrent Clean, are of this type.

Some are created by corporations from scratch for one reasons or
another. e.g. Java, Javascript, AppleScript, Dylan, C#. The reason is
mostly to make money by creating a lang that solves perceived problems
or need, as innovation. The problem may or may not actually exist. (C#
is a lang created primarily to overrun Java. Java was created first as
a lang for embedded devices, then Sun Microsystems pushed it to ride
the internet wave to envision “write once run everywhere” and
interactivity in web browser. In hindsight, Java's contribution to the
science of computer languages is probably just a social one, mainly in
popularizing the concept of a virtual machine.)

Infinite Number Of Syntaxes And Semantics

Looking at some tens of langs, one might think that there might be
some unifying factor, some unifying theory or model, that limits the
potential creation to a small set of types, classes, models. With
influence from Stephen Wolfram book “A New Kind of Science” (see:
Notes on A New Kind of Science) , i'd think this is not so. That is to
say, different languages are potentially endless, and each can become
quite useful or important or with sizable user base. In other words, i
think there's no theoretical basis that would govern what languages
will be popular due to its technical/mathematical properties. Perhaps
another way to phrase this imprecise thought is that, languages will
keep proliferating, and even if we don't count langs that created by
one-man's fancy, there will still probably be forever birth of
languages, and they will all be useful or solve some niche problem,
because there is no theoretical or technical reason that sometimes in
the future there would be one lang that can be fittingly used to solve
all computing problems.

Also, the possibilities of lang's syntax are basically unlimited, even
considering that they be practical and human readable. So, any joe,
can potentially create a new syntax. The syntaxes of existing langs,
when compared to the number of all potentially possible (human
readable) syntaxes, are probably a very small fraction. That is to
say, even with so many existing langs today with their wildly
differing syntax, we probably are just seeing a few pixels in a
computer screen.

Also note here all langs mentioned here are all plain-text linear
ones. Spread sheet and visual programing langs↗ would be example of 2D
syntax... but i haven't thought about how they can be classified as
syntax. (nor do i fully understand the ontology of syntax↗ )

----------------------------
This post is posted to:
comp.lang.python,comp.lang.perl.misc,comp.lang.lisp,comp.lang.functional

Xah
http://xahlee.org/


Steve Holden

unread,
Nov 26, 2008, 11:17:21 AM11/26/08
to pytho...@python.org
Xah Lee wrote:
> On Nov 25, 2:47 pm, Jorgen Grahn <grahn+n...@snipabacken.se> wrote:
>> On Mon, 24 Nov 2008 20:25:51 -0500, david.l...@preisshare.net <david.l...@preisshare.net> wrote:
>>> QuotingXahLee<xah...@gmail.com>:
>>>> herald: Python surpasses Perl in popularity!
>>>> According to
>>>> ?TIOBE Programming Community Index for November 2008? at
>>>> http://www.tiobe.com/content/paperinfo/tpci/index.html
>>>> it seems that Python has surpassed Perl in popularity this month!
>>> Interesting topic !
>> Hard to take a popularity index seriously when Logo is at #19 and
>> Bourne shell at #32 ... and then they suggest that their readers can
>> use it "to make a strategic decision about what programming language
>> should be adopted when starting to build a new software system".
>
> your remark is a bit overzealous. After all, we all know that site is
> websearh based. Although it not some kinda scientific report, but it
> does give some good indication of language popularity, however you
> define that.
>
[... rant omitted ...]
I wondered how long you would be able to resist making your vituperative
remarks. Please stop this rubbish.

Hrvoje Niksic

unread,
Nov 26, 2008, 4:47:40 PM11/26/08
to
Xah Lee <xah...@gmail.com> writes:

> Bourne Shell, is pretty much replaced by Bash since several years ago.
> For example, as far as i know, linuxes today don't have Bourne Shell
> anymore. “sh” is just a alias to bash with some compatibility
> parameter.

That used to be the case, but these days 'sh' is as often an alias to
a lighter shell program, a free reimplementation of sh that includes a
much smaller superset of the original sh functionality than that of
bash. For example, on default Ubuntu installations, sh is a symlink
to dash, "a lightweight POSIX-compliant shell derived from ash."

Steven D'Aprano

unread,
Nov 26, 2008, 7:59:31 PM11/26/08
to
On Wed, 26 Nov 2008 11:17:21 -0500, Steve Holden wrote:

> Xah Lee wrote:
...


>> your remark is a bit overzealous. After all, we all know that site is
>> websearh based. Although it not some kinda scientific report, but it
>> does give some good indication of language popularity, however you
>> define that.
>>
> [... rant omitted ...]
> I wondered how long you would be able to resist making your vituperative
> remarks. Please stop this rubbish.

Oh please Steve. Did you read Xah's post or stop after the second
paragraph? It was amazingly *non* vituperative, and I don't just mean
"for Xah".

I only counted two uses of the f-word and one of the a-word, none of
which I personally would object to (we're grown-ups, naughty words
shouldn't shock us). The only things which were even *close* to a
vituperative rant were a short comment about Brian Harvey being unhappy
with Scheme 6, and that Sun Microsystems use inferior versions of shell
tools. Maybe half a dozen lines expressing strong opinions, out of a 300
line post.

--
Steven

Jorgen Grahn

unread,
Nov 28, 2008, 5:05:05 AM11/28/08
to

I have seen them, too. But I have also written and maintained
medium-sized shell scripts which are very successful.

(I could get away with using Bash in these cases. It has functions,
local variables and so on. Writing portable Bourne shell is not as
much fun.)

Not to mention how useful interactive shell scripting is. I use it
more and more: pipelines which combine xargs, while, for, sort and
perl oneliners. This is surely a kind of programming, too.

Paul Boddie

unread,
Nov 28, 2008, 6:40:43 AM11/28/08
to
On 27 Nov, 01:59, Steven D'Aprano

<ste...@REMOVE.THIS.cybersource.com.au> wrote:
>
> Oh please Steve. Did you read Xah's post or stop after the second
> paragraph? It was amazingly *non* vituperative, and I don't just mean
> "for Xah".

Agreed, although I had to look "vituperative" up first. Is the mere
presence of Xah Lee's name now shocking in itself?

> I only counted two uses of the f-word and one of the a-word, none of
> which I personally would object to (we're grown-ups, naughty words
> shouldn't shock us). The only things which were even *close* to a
> vituperative rant were a short comment about Brian Harvey being unhappy
> with Scheme 6, and that Sun Microsystems use inferior versions of shell
> tools. Maybe half a dozen lines expressing strong opinions, out of a 300
> line post.

And I'd definitely want to express frustration with the latter
situation if I had to use a Sun operating system without the GNU suite
of programs installed. I haven't checked, but perhaps even the latest
release of OpenSolaris still has the /usr/ucb, /usr/ccs and /usr/xpg5
directories (or whatever they're called), although these probably no
longer provide the default tools.

Paul

Tam Ha

unread,
Nov 29, 2008, 11:23:49 AM11/29/08
to
Jorgen Grahn <grahn...@snipabacken.se> wrote:
>(I could get away with using Bash in these cases. It has functions,
>local variables and so on. Writing portable Bourne shell is not as
>much fun.)

Can you explain this? Bourne is always more portable than Bash.
That's why you'll find experienced shell programmers writing everything
that doesn't absolutely require a bash feature in /bin/sh. Boot scripts,
install scripts, etc. should never be written in bash and if where you
find one using bash you can be sure a Linux-only newbie has written it.
For one there are too many versions of bash, for two it is not installed
by default on every Unix/Linux OS, for three it has poor backwards
(and forwards) compatibility. It is also found at different places on
the path.

As to the "study" that started this thread, clearly 100% pure garbage out.

Tam

Stephane CHAZELAS

unread,
Nov 29, 2008, 12:03:41 PM11/29/08
to
2008-11-29, 16:23(+00), Tam Ha:
[...]

There's a common confusion in this in the nature of /bin/sh.
There's no standard (neither POSIX nor Unix) that specifies that
/bin/sh should be any variant of the Bourne shell. Over the
years and on the different flavours of Unix, /bin/sh has been a
Thomson shell, Mashey shell, Bourne shell and all its variants,
Almquist shell, bash, ksh, zsh...

For a very long time, on a wide range of Unices, it has been the
Bourne shell (modified slightly in different ways by the
different Unix vendors), so that's probably why it is still
nowadays confused with the Bourne shell even though it is still
so only on very few Unices (mainly only Solaris, Tru64 and some
SCO OSes).

Nowadays, what the POSIX and Unix standards say is that a conformant Unix
should have somewhere (and the location is not specified but on
most Unices except those quoted above is generally /bin) a
command called "sh" that is an interpreter of the POSIX shell
language it defines.

Shells that do implement an interpreter for that language
(though most shells of course have extensions which the
specification doesn't prohibit, such as many C compilers (like
gcc) have extensions to the standard C language as published by
the ISO) include (with lesser or greater accuracy) bash (2 or
above), the AT&T Korn shell (88<x> or 93<x> versions) and its
variants, the public domain Korn shell and its derivatives such
as posh, mksh or OpenBSD sh, modern variants of the Almquist
shell such as the NetBSD sh or dash.

That shell language is based on a subset of the ksh88 language
and the Bourne shell is not a conformant implementation. Amongst
the feature of the POSIX/Unix sh that are not in the Bourne
shell are the $(...) form of command substitution, arithmetic
expansions as in $((...)), the processing of IFS that differs,
redirections of compound commands that no longer invoke a
subshell... That language is mostly backward compatible with the
Bourne syntax though there are a few small differences.

To sum up, /bin/sh is no longer the Bourne shell. It is only so
on very old systems and on systems that have chosen to keep the
Bourne shell as /bin/sh for backward compatibility (and for
those, the standard sh is in another place such as
/usr/xpg4/bin on Solaris). Free software Unices have never had
the Bourne shell as their /bin/sh as the source code of the
Bourne shell has not been released until recently (except for
some early versions of BSD and they were then breaching the
law).

--
Stéphane

Xah Lee

unread,
Nov 29, 2008, 1:34:13 PM11/29/08
to
Great to see quality post from real expert once in a while. Thanks!

Xah
http://xahlee.org/

On Nov 29, 9:03 am, Stephane CHAZELAS <stephane_chaze...@yahoo.fr>
wrote:

Tam Ha

unread,
Nov 30, 2008, 1:11:46 AM11/30/08
to
Stephane CHAZELAS wrote:
> There's a common confusion in this in the nature of /bin/sh.
> There's no standard (neither POSIX nor Unix) that specifies that
> /bin/sh should be any variant of the Bourne shell.

Sure there is, POSIX. Or rather their Austin Group. And while they done
an extremely poor job of it the POSIX shell is still the Bourne shell,
/bin/sh as it's always been known, with a few enhancements such as tilde
expansion and such.

> Over the years and on the different flavours of Unix, /bin/sh
> has been a Thomson shell, Mashey shell, Bourne shell and all its
> variants, Almquist shell, bash, ksh, zsh...

Zsh? Seriously? Where? The fact that various flavors of Linux have
used various poorly designed "compatiblity mode" hacks to circumvent
Bourne/POSIX regardless of compatibility consequences does not change
the fact that /bin/sh is still there in all distributions, is still the
cross-platform compatibility winner by a county mile, and can still be
relied on for shell scripts.

>should have somewhere (and the location is not specified but on
>most Unices except those quoted above is generally /bin) a
>command called "sh" that is an interpreter of the POSIX shell
>language it defines.

Right, that's /bin/sh. It's common name is the Bourne shell even though
it is not the code that Mr Bourne wrote years ago, and even though
GNU/POSIX and ATT/David Korn want to remake it into a "POSIX shell"
regardless of the compatibility consequences.

>That shell language is based on a subset of the ksh88 language
>and the Bourne shell is not a conformant implementation.

Only if you limit your use of the term "Bourne shell" to something
that the rest of us don't recognize as a Bourne shell. The fact that
POSIX has been known to jerry-rig ksh features (most of which werre
derived from tcsh by the way) into /bin/sh (and sort, and a few other
APIs they really should not be changing) only reflects the fact that
A) David Korn (still with ATT) has been pushing his vision, and B)
systems administrators (the ones who suffer most from cross-platform
incompatibilities) are not well represented within POSIX in general,
and not well represented within its Austin Group in particular.

Tam

Stephane CHAZELAS

unread,
Nov 30, 2008, 4:46:27 AM11/30/08
to
2008-11-30, 06:11(+00), Tam Ha:

> Stephane CHAZELAS wrote:
>> There's a common confusion in this in the nature of /bin/sh.
>> There's no standard (neither POSIX nor Unix) that specifies that
>> /bin/sh should be any variant of the Bourne shell.
>
> Sure there is, POSIX. Or rather their Austin Group. And while they done
> an extremely poor job of it the POSIX shell is still the Bourne shell,
> /bin/sh as it's always been known, with a few enhancements such as tilde
> expansion and such.

No, saying that /bin/sh is the Bourne shell is as wrong as
saying /bin/csh the Thomson shell. It's at best a very
inaccurate approximation but it's true it's a very commonly made
mistake. All you can say is that "sh" is meant to be a POSIX
compliant shell (which the Bourne shell is not) or that /bin/sh
(nowadays) is a Bourne-like shell. If you want to write a shell
script for the /bin/sh of any modern Unix, you have to write it
in the language that is the intersection of that of the Bourne
shell and the POSIX shell syntax.

As I said, the POSIX shell specification is based on a subset of
ksh88, has incompatibilities with the Bourne shell. The Bourne
shell, as can still be found on some systems either in some
non-standard place (/bin on Solaris, /usr/old/bin on HPUX) or
named differently (see
http://www.in-ulm.de/~mascheck/various/shells/ for more details)
is not conforming to that standard.

On most commercial Unices, the "sh" utility is implemented by
one version of the other of ksh88 (possibly slightly modified).
That's the case at least of Solaris (where it is in
/usr/xpg4/bin/sh, and you can see that /usr/xpg4/bin is before
/bin in the output of "getconf PATH"), HPUX since at least
10.10, AIX since at least 4.3.2, IRIX...

See on Tru64 http://nixdoc.net/man-pages/Tru64/sh.1.html:

sh(1)> [Tru64 UNIX] Subsequent shells spawned from the initial
sh(1)> shell depend on the value in the environment variable
sh(1)> BIN_SH. If this variable is set to xpg4, the POSIX shell
sh(1)> is started. If this variable is set to svr4, an SVR4
sh(1)> compliant version of the shell is started. If this
sh(1)> variable is unset, the Bourne shell is started. If this
sh(1)> variable is set to any other value, an error is
sh(1)> reported and the results are unpredictable. See the
sh(1)> EXAMPLES section for information on setting this
sh(1)> variable.

The POSIX one being ksh. On Linux and all the GNU OSes, it's
generally bash. On BSDs, it initially was the Almquist shell, a
public domain rewriting of the System V shell (the Bourne shell)
with a few modifications, and nowadays FreeBSD and NetBSD have
adapted that ash so as to become POSIX compliant, and OpenBSD is
using pdksh. MacOS/X has been using zsh and now bash


>> Over the years and on the different flavours of Unix, /bin/sh
>> has been a Thomson shell, Mashey shell, Bourne shell and all its
>> variants, Almquist shell, bash, ksh, zsh...
>
> Zsh? Seriously? Where?

Some versions of MacOSX/Darwin.

> used various poorly designed "compatiblity mode" hacks to circumvent
> Bourne/POSIX regardless of compatibility consequences does not change
> the fact that /bin/sh is still there in all distributions, is still the
> cross-platform compatibility winner by a county mile, and can still be
> relied on for shell scripts.

Could you please clarify what you mean? Yes, most systems have a
/bin/sh (for those that have a /bin directory), but as I said
/bin/sh can be any of a Thomson, Mashey (only in museums
nowadays), Almquist, Bourne, Korn, bash...

Nowadays, they are most likely to be Bourne-like.

If you want to write a portable script, today, the best advice
to give is to write it in a syntax conforming to POSIX (note
that of course that applies to the shell but also to the
standard utilities). Then you'll be guaranteed to find a "sh" on
any compliant Unix (most at least try to be nowadays) that will
interpret it. It is a much better situations than a few decades
ago where every vendor had its own variant of the Bourne shell
and the utilities.

>>should have somewhere (and the location is not specified but on
>>most Unices except those quoted above is generally /bin) a
>>command called "sh" that is an interpreter of the POSIX shell
>>language it defines.
>
> Right, that's /bin/sh.

Not always. As I said, on a few systems, the POSIX sh is not in
/bin. That's the case of at least Solaris, where /bin/sh is a
Bourne shell and therefore not a POSIX compliant sh.

> It's common name is the Bourne shell even though
> it is not the code that Mr Bourne wrote years ago, and even though
> GNU/POSIX and ATT/David Korn want to remake it into a "POSIX shell"
> regardless of the compatibility consequences.

Saying that would be denying the reality.

>>That shell language is based on a subset of the ksh88 language
>>and the Bourne shell is not a conformant implementation.
>
> Only if you limit your use of the term "Bourne shell" to something
> that the rest of us don't recognize as a Bourne shell. The fact that
> POSIX has been known to jerry-rig ksh features (most of which werre
> derived from tcsh by the way) into /bin/sh (and sort, and a few other
> APIs they really should not be changing) only reflects the fact that
> A) David Korn (still with ATT) has been pushing his vision, and B)
> systems administrators (the ones who suffer most from cross-platform
> incompatibilities) are not well represented within POSIX in general,
> and not well represented within its Austin Group in particular.

[...]

A few tcsh features have made it to bash and even more to zsh
but mostly only on the interactive usage side not the syntax
but hardly any to ksh, ksh has added a few features from csh
such as tilde expansion, brace expansion (though differently),
FIGNORE.

The reality is that nowadays the Bourne shell has almost
disappeared, but it's syntax is still used in scripts, which is
most cases, modern shs can still interpret.

I would strongly recommend the reading of
http://www.in-ulm.de/~mascheck/bourne/ and
http://www.in-ulm.de/~mascheck/various/shells/ which is a very
good reference on sh and the Bourne shell.

--
Stéphane

Stephane Chazelas

unread,
Nov 30, 2008, 7:16:53 AM11/30/08
to
2008-11-30, 06:11(+00), Tam Ha:
> Stephane CHAZELAS wrote:
>> There's a common confusion in this in the nature of /bin/sh.
>> There's no standard (neither POSIX nor Unix) that specifies that
>> /bin/sh should be any variant of the Bourne shell.
>
> Sure there is, POSIX.
[...]

And on this. First, POSIX has no juridiction on defining the
Bourne shell, let alone its location as the Bourne shell is
completely outside POSIX, it's a legacy shell and as been for
years. Then, POSIX makes it clear that the location of its sh
utility (again, which the Bourne shell is not) is unspecified,
only that the lookup of "sh" via $PATH in a conformant
environment should resolve to a conformant "sh".

And if you need some examples to be convinced, see the different
choices of those 4 different POSIX conformant Unices:

Solaris (7, 8, 9 at least):
/bin/sh: legacy Bourne/SVr4 shell
/usr/xpg4/bin/sh: a POSIX compliant shell (actually ksh88)
Solaris choice was to keep /bin/sh as the Bourne shell for
backward compatibility (as changing it for a POSIX shell
introduces a slight chance of breaking some existing scripts
as there are a few corner-case areas where the POSIX shells
are not backward compatible with the Bourne shell), and make
/bin a non standard place. How to get a conformant environment
(typically one where /usr/xpg4/bin/sh is before /bin) is
described in standards(5). The default one is not, which makes
Solaris a pain when porting scripts.

HPUX (10.10 and newer at least):
/usr/bin/sh (and /bin/sh if there's a symlink /bin ->
usr/bin): a POSIX compliant shell (based on ksh88)

/usr/old/bin/sh: the Bourne shell (though I think nowadays,
it's part of an optional package).

Tru64:
/bin/sh can behave either as a Bourne shell or a POSIX shell
(ksh88) depending on the environment

most Linux based systems and most other Unices:
/bin/sh is a POSIX compliant shell (bash generally for Linux)
and there's no Bourne shell (either there has never been or it
has been removed/replaced by a POSIX shell).

--
Stéphane

Sven Mascheck

unread,
Nov 30, 2008, 7:10:27 PM11/30/08
to
In comp.unix.shell Stephane CHAZELAS wrote:

> The Bourne shell, as can still be found on some systems either in some
> non-standard place (/bin on Solaris, /usr/old/bin on HPUX) or named

> differently [...]

What do you mean with "non-standard place" here?

Stephane CHAZELAS

unread,
Dec 1, 2008, 1:36:16 AM12/1/08
to
2008-12-1, 01:10(+01), Sven Mascheck:

It's true it was vague and misleading,

/bin is not the standard place to look for "sh" as far as the
"POSIX" standard is concerned. That doesn't mean that standard
commands (POSIX or not) cannot be found in /bin. But /bin/sh has
been made a non-standard place for "sh". It's the "legacy"
place. Those commands that have not undergone non-backward
compatible changes with POSIX can still be found in /bin, the
others are to be found in /usr/xpg<x>/bin (for the standard
versions).

--
Stéphane

Casper H.S. Dik

unread,
Dec 1, 2008, 3:51:58 AM12/1/08
to
Stephane CHAZELAS <stephane...@yahoo.fr> writes:

>It's true it was vague and misleading,

>/bin is not the standard place to look for "sh" as far as the
>"POSIX" standard is concerned. That doesn't mean that standard
>commands (POSIX or not) cannot be found in /bin. But /bin/sh has
>been made a non-standard place for "sh". It's the "legacy"
>place. Those commands that have not undergone non-backward
>compatible changes with POSIX can still be found in /bin, the
>others are to be found in /usr/xpg<x>/bin (for the standard
>versions).

The proper place is "`getconf PATH`". POSIX doesn't list any
pathnames.

Of course, the problem is then "where did getconf come from?".

Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

Andre Majorel

unread,
Dec 1, 2008, 5:16:36 AM12/1/08
to

How does it decide ? argv[0] ? isatty (STDIN_FILENO) ?

> most Linux based systems and most other Unices:
> /bin/sh is a POSIX compliant shell (bash generally for Linux)
> and there's no Bourne shell (either there has never been or it
> has been removed/replaced by a POSIX shell).

Two other Unixen I know of that have a non-POSIX sh are SCO Open
Server and UnixWare. I'm not sure they even provide a
POSIX-compliant alternative like Solaris does.

On the other hand, their utilities (awk, grep, sed et al.) seem
to be POSIX-compliant, at least to the extent that I've always
managed. Unlike Solaris where I've often had to use the ones in
/usr/xpg4/bin.

Many people assume "standard" equates "portable". As far as
shell programming is concerned, the safest way to get portability
is to ignore the standard and code for the Bourne shell.

--
André Majorel <URL:http://www.teaser.fr/~amajorel/>
"After 15 minutes, I wanted to marry her. After a half hour, I completely
gave up the idea of snatching her purse." -- _Take the Money and Run_

Stephane Chazelas

unread,
Dec 1, 2008, 6:53:18 AM12/1/08
to
2008-12-01, 08:51(+00), Casper H.S Dik:

> Stephane CHAZELAS <stephane...@yahoo.fr> writes:
>
>>It's true it was vague and misleading,
>
>>/bin is not the standard place to look for "sh" as far as the
>>"POSIX" standard is concerned. That doesn't mean that standard
>>commands (POSIX or not) cannot be found in /bin. But /bin/sh has
>>been made a non-standard place for "sh". It's the "legacy"
>>place. Those commands that have not undergone non-backward
>>compatible changes with POSIX can still be found in /bin, the
>>others are to be found in /usr/xpg<x>/bin (for the standard
>>versions).
>
> The proper place is "`getconf PATH`". POSIX doesn't list any
> pathnames.
>
> Of course, the problem is then "where did getconf come from?".
[...]

Sorry, I was specifically speaking of Solaris 7, 8, 9. "getconf
PATH" is not enough, you have to pick the correct getconf, and
that getconf be called in the relevant environment. For
instance, in later versions of Solaris, I beleive you get
/usr/xpg6/bin or /usr/xpg4/bin first depending on the
environment.

Also, PATH is not necessarily the only thing needed to get you
the conformant utilities. See BIN_SH on Tru64, POSIXLY_CORRECT
on GNU...

I beleive the only thing POSIX requires is that how to get into
the proper environment be documented (but it is unspecified).
On Solaris, I beleive it's in the standards(5) man page.

--
Stéphane

Stephane Chazelas

unread,
Dec 1, 2008, 11:08:16 AM12/1/08
to
2008-12-1, 10:16(+00), Andre Majorel:
[...]

>> Tru64:
>> /bin/sh can behave either as a Bourne shell or a POSIX shell
>> (ksh88) depending on the environment
>
> How does it decide ? argv[0] ? isatty (STDIN_FILENO) ?

That was answered in another article with a quote of the sh man
page on Tru64: via an environment variable: BIN_SH.

>> most Linux based systems and most other Unices:
>> /bin/sh is a POSIX compliant shell (bash generally for Linux)
>> and there's no Bourne shell (either there has never been or it
>> has been removed/replaced by a POSIX shell).
>
> Two other Unixen I know of that have a non-POSIX sh are SCO Open
> Server and UnixWare. I'm not sure they even provide a
> POSIX-compliant alternative like Solaris does.

Again, see http://www.in-ulm.de/~mascheck/various/shells/

[...]


> Many people assume "standard" equates "portable". As far as
> shell programming is concerned, the safest way to get portability
> is to ignore the standard and code for the Bourne shell.

Yes, it's enough and relatively safe in most cases. It's the
only option if you want to be portable to systems over 15 years
old.

But nowadays, especially since the specifications have been made
public, I find that it is sufficiently widespread to make POSIX a
safer bet. At least having a specification gives you some sort
of guarantee: if you write your script by the rules that are
clearly (most of the time) specified, then it should work on any
compliant system. If it doesn't work, then it's not your fault
(though it's true in many case, you'll still be the one who'll
have to do something about it).

--
Stéphane

Jorgen Grahn

unread,
Dec 5, 2008, 7:10:52 AM12/5/08
to
["Followup-To:" header set to comp.unix.shell.]

On 29 Nov 2008 16:23:49 GMT, Tam Ha <not-fo...@sonic.net> wrote:
> Jorgen Grahn <grahn...@snipabacken.se> wrote:
>>(I could get away with using Bash in these cases. It has functions,
>>local variables and so on. Writing portable Bourne shell is not as
>>much fun.)
>
> Can you explain this?

Sorry for the late answer. No, it's actually what I said:

I could get away with using bash in these cases, so I did it. It was
an in-house application, it had already unknown dependencies to bash,
and to Linux versions of other utilities. If I had asked for time to
modify it to make sure it was portable Bourne shell, people would have
laughed at me.

> Bourne is always more portable than Bash.
> That's why you'll find experienced shell programmers writing everything
> that doesn't absolutely require a bash feature in /bin/sh. Boot scripts,
> install scripts, etc. should never be written in bash and if where you
> find one using bash you can be sure a Linux-only newbie has written it.

Sure, nothing controversial about that. I don't argue that people
should use bash features in any random script they write. I just noted
that if you decide to use it, it's a pretty useful language. Probably
more useful than Python in my case, where most of the work was about
starting and managing external commands and pipelines.

> For one there are too many versions of bash, for two it is not installed
> by default on every Unix/Linux OS, for three it has poor backwards
> (and forwards) compatibility.

Worse compatibility than Perl or Python? The Bourne shell timescale
is probably impressive, but often you aren't interested in decades.

> It is also found at different places on
> the path.

Surely that applies to almost any interpreter, like perl and python.
It's a problem (on the non-free Unixes at least) but if you let it be
the deciding factor, you could use no scripting language except
/bin/sh and awk.

0 new messages