Python's biggest compromises
Anthony_Barker
programming language. The author states that Basic - particularly
Microsoft's version is full of compromises which crept in along the
language's 30+ year evolution.
What to you think python largest compromises are?
The three that come to my mind are significant whitespace, dynamic
typing, and that it is interpreted - not compiled. These three put
python under fire and cause some large projects to move off python or
relegate it to prototyping.
Whitespace is an esthetic preference that make Andrew Hunt and David
Thomas (of Pragmatic Programmer fame) prefer Ruby. Personally, I love
it - but I can see why some people might not like it (30 years of
braces).
Dynamic typing causes the most fuss. I like Guido's answer to the
question -
> "Doesn't dynamic typing cause more errors to creep into the code
because you catch them later than compile time?".
> "No, we use Unit Testing in Zope".
That said, obvious Basic compromised by using things such as "Option
Explicit", thereby allowing both dynamic and more static style
variables. Yahoo groups moved from python to C due to dynamic typing.
Non-compiled - obviously there are times when performance matters more
than other things. Google I believe uses python to prototype (or used)
and then turns to c++ for heavy lifting.
What about immutable strings? I'm not sure I understand Guido's
preference for them.
Anthony
http://xminc.com/anthony
John Roth
"Anthony_Barker" <anthony...@hotmail.com> wrote in message
news:899f842.03073...@posting.google.com...
> I have been reading a book about the evolution of the Basic
> programming language. The author states that Basic - particularly
> Microsoft's version is full of compromises which crept in along the
> language's 30+ year evolution.
>
> What to you think python largest compromises are?
>
> The three that come to my mind are significant whitespace, dynamic
> typing, and that it is interpreted - not compiled. These three put
> python under fire and cause some large projects to move off python or
> relegate it to prototyping.
>
> Whitespace is an esthetic preference that make Andrew Hunt and David
> Thomas (of Pragmatic Programmer fame) prefer Ruby. Personally, I love
> it - but I can see why some people might not like it (30 years of
> braces).
Since I'm not particularly masochistic, I will respectfully disagree.
Unnecessary braces are one of the reasons I haven't looked at
Ruby.
That said, I've come to the conclusion that the editor should take
care of these things for you. If you prefer a brace free notation,
you should be able to tell your editor to present the program to you
that way. If you prefer braces, then it should be able do that for
you as well. That kind of stylistic thing doesn't belong in the
language.
In fact, if I didn't have to deal with the braces, I think I'd come
around to the view that the text should have them. Explicit is
better than implicit, and there are a few things that the
automatic indentation makes rather difficult in the design area.
> Dynamic typing causes the most fuss. I like Guido's answer to the
> question -
> > "Doesn't dynamic typing cause more errors to creep into the code
> because you catch them later than compile time?".
> > "No, we use Unit Testing in Zope".
>
> That said, obvious Basic compromised by using things such as "Option
> Explicit", thereby allowing both dynamic and more static style
> variables. Yahoo groups moved from python to C due to dynamic typing.
Basic was compromised from day 1. Its designers tried to water
down Fortran to produce something that they could use to teach
programming in an interactive system. The late Ejdsger Dykstra called
Fortran an "infantile disorder". While he was rather outspoken, he had
his reasons. The only reason that Basic exists today is that Bill Gates
has a certain fondness for that interpreter he wrote that got him his
start in the business.
> Non-compiled - obviously there are times when performance matters more
> than other things. Google I believe uses python to prototype (or used)
> and then turns to c++ for heavy lifting.
High performance isn't Python's target. If PyPy ever gets their act
off the ground, then we will have a shot at a good quality JIT
interpreter. Then watch it fly.
> What about immutable strings? I'm not sure I understand Guido's
> preference for them.
Immutable objects make it much easier to do dictionaries. That's
why Python has both lists and tuples. The actual problem with
immutable strings is performance. While you can concatinate
strings with '+', if you've got more than two strings to put together,
it's faster (and clearer) to use the '%' formatting operator. This
isn't real obvious to experianced programmers that are new to the
language, though.
I don't see any of these as compromises. Python has a specific
location on the spectrum that makes it better for certain things than
for others.
John Roth
>
> Anthony
> http://xminc.com/anthony
Christopher Koppler
(Anthony_Barker) wrote:
>I have been reading a book about the evolution of the Basic
>programming language. The author states that Basic - particularly
>Microsoft's version is full of compromises which crept in along the
>language's 30+ year evolution.
>
>What to you think python largest compromises are?
>
[snip whitespace, dynamic typing, interpreted]
I don't see those as compromises, but mostly as assets.
Significant whitespace (you probably mean significant indentation -
whitespace isn't more or less significant in Python than in other
modern languages) I have only experienced as a boost in readability,
clarity and, most of all, consistence; and there's no possibility of
'brace style wars'.
Dynamic typing vs. static typing has already long ago reached the
status of holy war, so I'll decline to comment.
That python is not (yet) compiled, is mostly a non-issue (and if PyPy
is a success, it won't even be that). If it was just about
performance, then coding the really performance-intensive parts in C
should suffice, apart from kernel hacking and similar . In my
experience, the decision to convert a (successfully functioning)
project from 'a scripting language' to C/C++/Java has always been a
political one, and not really based on technical considerations.
That said, the only large compromise in Python language design I can
detect, is the decision to be quite strictly backwards-compatible
between versions, which is definitely not a bad thing, as long as the
language doesn't go baroque because of it. And Python 3.0 will
hopefully throw out any accumulated cruft.
--Christopher
Edward K. Ream
There aren't any.
You want to base significant projects on the highest level, most dynamic
tools available (Python), then use Python as a wrapper to hide static
inflexibilities and inferiorities when descending to lower levels for
whatever (usually spurious) reasons. For example, there is a huge
difference between using wxWindows and wxPython, but the performance
difference between wxWindows and wxPython is insignificant.
Edward
--------------------------------------------------------------------
Edward K. Ream email: edre...@charter.net
Leo: Literate Editor with Outlines
Leo: http://webpages.charter.net/edreamleo/front.html
--------------------------------------------------------------------
Bruno Desthuilliers
(snip)
>
> What to you think python largest compromises are?
>
> The three that come to my mind are significant whitespace, dynamic
> typing, and that it is interpreted - not compiled.
IMHO these are not compromises, but features.
Bruno
Robin Becker
<newsg...@jhrothjr.com> writes
>
.....
>High performance isn't Python's target. If PyPy ever gets their act
>off the ground, then we will have a shot at a good quality JIT
>interpreter. Then watch it fly.
>
speed things up that much. If all the variables are known to be of a
specific type then you could expect C like speeds from a good JIT, but
without global analysis it's hard to see how we infer/guarantee python
types.
--
Robin Becker
John J. Lee
[...]
> That said, I've come to the conclusion that the editor should take
> care of these things for you. If you prefer a brace free notation,
> you should be able to tell your editor to present the program to you
> that way. If you prefer braces, then it should be able do that for
> you as well. That kind of stylistic thing doesn't belong in the
> language.
100% agreed: once-and-only-once dictates this. Manually maintaining
both braces and indentation (as in C, with some editors) is Just Plain
Bad for this reason.
> In fact, if I didn't have to deal with the braces, I think I'd come
> around to the view that the text should have them. Explicit is
> better than implicit,
At least in the absence of proper research results, I think this part
of it *is* religious. To some people, it's obvious that whitespace is
better on the eyes. To others, it's obvious that whitespace + braces
is better on the eyes. One group may well be wrong <0.5 wink>.
This argument (that braces are another visual cue which make code
easier to read -- provided that they're automatically maintained in
sync with the indentation), is the only argument against
pure-whitespace that has ever made any sense to me. Irrespective of
whether you pick whitespace or braces as the thing the compiler takes
notice of, though, the optimal solution probably still involves an
editor that supports showing/hiding braces, so the syntax is
(theoretically!) a non-issue. In practice, editors don't seem to
currently support showing and hiding braces automatically. At least
editors do get you out of most mistakes caused by brace-y languages.
Of course, whitespace *is* still superior because the storage format
doesn't duplicate state -- so people with poor editors can't produce
ambiguous code :-) (remember code can be ambiguous to *people* even if
it isn't to a compiler). OTOH, *Python's* scheme is inferior to a
pure-space-character indentation scheme because off the tab-vs.-space
issue :-(
> and there are a few things that the
> automatic indentation makes rather difficult in the design area.
[...]
What are those things??
John
Mark VandeWettering
Anthony_Barker wrote:
> I have been reading a book about the evolution of the Basic
> programming language. The author states that Basic - particularly
> Microsoft's version is full of compromises which crept in along the
> language's 30+ year evolution.
>
> What to you think python largest compromises are?
>
> The three that come to my mind are significant whitespace, dynamic
> typing, and that it is interpreted - not compiled. These three put
> python under fire and cause some large projects to move off python or
> relegate it to prototyping.
I don't view any of these as "compromises". That word suggests that
something was conceded, or that an intermediate position between two
extremes was chosen to appease. I don't think that either sense really
applies to these features.
The three items that you listed are merely design choices. While arguments
over them are continuous, two of the design choices (interpreter, dynamic
typing) are consistent with Python's intended use as a language which
excels at rapid prototyping. The third (white space) is merely a stylistic
choice which is designed to encourage readable programs.
"Compromises" in language design occur usually when a committee tries to
standardize a language, and each has differing views about how the language
should be used. While this occurs somewhat in Python, other languages
have suffered more mightily from this particular disorder.
Mark
> Anthony
> http://xminc.com/anthony
Aaron Leung
namespaces are usually represented by dictionaries, and that Python
exposes this fact to the programmer. This would seem to limit the
possible optimizations that can easily be performed by a compiler.
BTW, I have only read about Python out of interest, and haven't
actually used it for anything, so I hope my remark isn't ignorant.
Best regards,
Aaron
Terry Reedy
"Anthony_Barker" <anthony...@hotmail.com> wrote in message
news:899f842.03073...@posting.google.com...
A compromise is an in-between position or decision. Example: wife
wants to go to a horse show, husband to an auto race, so they
compromise and go to a horse race.
> The three that come to my mind are significant whitespace, dynamic
> typing, and that it is interpreted - not compiled.
The first two are end-point positions, not in-between compromises.
The third is a matter of definition and implementation. CPython
compiles to version-dependent but otherwise portable PyCode. PyRex,
Weave, and Psyco all compile to C or machine code.
> These three put python under fire
Anything can bo put under fire by anyone who wants to shoot.
> and cause some large projects to move off python or
This sort of statement remains an opinion or impression until backed
by evidence.
> relegate it to prototyping.
This is one of its intended uses.
> Whitespace is an esthetic preference that make Andrew Hunt and David
> Thomas (of Pragmatic Programmer fame) prefer Ruby.
Evidence? And I mean evidence that whitespace is *the* reason and not
just a convenient summary of an overall esthetic preference.
In any case, so what? Different strokes for different folks. Do they
also use indentation for human readers? If so, they have assigned
themselves the task of keeping brackets and indents in sync so that
human and machine 'see' the same structure.
I see two good uses for brackets:
1. machine-generated code never intended for human eyes
2. redundancy for transmission error detection by a processor that
compares brackets and indents and raises a flag on mismatches.
A compromise in the area of structure indication would be accepting
either brackets or indents or both.
> Yahoo groups moved from python to C due to dynamic typing.
Evidence? Evidence as to what exactly happened (it is not common
knowledge that I know of) and that any such change was a reasoned
technical decision and not politics.
If memory serves me right, Guido has tried a couple of compromises to
slightly limit dynamicity that he could not see much use for and has
backed off at least partly when current users presented use cases that
the change would break.
> Non-compiled - obviously there are times when performance matters
more
> than other things. Google I believe uses python to prototype (or
used)
> and then turns to c++ for heavy lifting.
This is a simple matter of economic tradeoff. A week of programmer
time costs roughly the same as, say, a year of pc time. A roaring
success like Google has hundreds (thousands?) of servers around the
world running the same relatively stable code. Faster code means
machines not bought, installed, and maintained. But I imagine that
their main production code is pretty far out on the frequency-of-use
curve.
Terry J. Reedy
Skip Montanaro
Anthony> What to you think python largest compromises are?
Anthony> The three that come to my mind are significant whitespace,
Anthony> dynamic typing, and that it is interpreted - not compiled.
Anthony> These three put python under fire and cause some large projects
Anthony> to move off python or relegate it to prototyping.
Your message is sure to get the pot boiling. I don't think of any of the
above as compromises. They were all design decisions. Considering the
whitespace issue, calling it a compromise suggests that Guido had to cave in
to some outside forces. He couldn't decide between BEGIN/END or {/} as
block delimiters, so he chose significant whitespace. It doesn't make
sense.
Anthony> What about immutable strings? I'm not sure I understand Guido's
Anthony> preference for them.
Performance is one reason. The Python interpreter creates a huge number of
strings at runtime. Knowing exactly how long the string is going to be and
that it will not grow means that a single malloc can be used to allocate the
object header and the storage for the data. If strings were mutable, the
structure of the string object storage would probably be much different and
you'd need at minimum two mallocs per string, one for the object header and
one for the data itself.
Skip
Ian Bicking
> The three that come to my mind are significant whitespace, dynamic
> typing, and that it is interpreted - not compiled. These three put
> python under fire and cause some large projects to move off python or
> relegate it to prototyping.
I think these are valid as "compromises", not to be confused with
flaws. These are all explicit choices, and ones for which the original
justifications remain valid. A lot of stuff in Basic is simply flaws,
or based on justifications that no longer apply to today's programming
world.
Anyway, I might add mine: the nature of modules as executed code is a
compromise. That is, a module isn't a declaration, it's a program to be
executed in its own namespace. When you import a module, you are
executing the module then looking at the namespace.
There are some advantages to this, particularly in the transparency of
the implementation -- things don't always work the way you might want
(e.g., circular imports), but it's usually not that hard to understand
why (and often the way you want things to work has nasty gotchas that
you wouldn't have thought of). It also opens up a lot of possibilities
for dynamicism in class and function declaration, like doing this in the
top level:
if something:
def func(x): ...
else:
def func(x): ...
But it's a compromise, because it makes things more difficult as well.
It's a *big* problem in any long-running process, where you may want to
modify code underlying the system without rebuilding the entire state.
Classes aren't declared, they are simply constructed, so by reloading a
module all the persistent instances still exist and refer to the defunct
class. You can modify classes at runtime, but this is different from
simply rerunning the class definition. (A clever metaclass *could* make
those equivalent, though... hmmm...)
A related problem is that Python objects generally can accept any
instance variable names, and names are not declared anywhere. Again,
this makes it difficult to deal with long-lived objects. If you change
the class so that all instances get a new attribute, old objects won't
be updated. I'm thinking about both of these things in terms of
Smalltalk, where they make tools possible that really add to the ease of
developing in its environment.
Not that I don't like the fun tricks Python lets you do. Prototype-like
programming (as in Self) is very accessible in Python, and classes are
only a suggestion not a dominant concept. So, it's a compromise.
There are lots and lots of compromises in Python -- every aspect has
pluses and minuses to it. Personally I like whitespace sensitivity well
enough, but in the larger sense I think it probably was the wrong choice
-- but that's based on how I weigh various benefits and problems, and
other people will validly weigh them differently.
Ian
John Roth
"John J. Lee" <j...@pobox.com> wrote in message
news:87el06d...@pobox.com...
> "John Roth" <newsg...@jhrothjr.com> writes:
> [...]
> > That said, I've come to the conclusion that the editor should take
> > care of these things for you. If you prefer a brace free notation,
> > you should be able to tell your editor to present the program to you
> > that way. If you prefer braces, then it should be able do that for
> > you as well. That kind of stylistic thing doesn't belong in the
> > language.
>
> 100% agreed: once-and-only-once dictates this. Manually maintaining
> both braces and indentation (as in C, with some editors) is Just Plain
> Bad for this reason.
>
>
> > In fact, if I didn't have to deal with the braces, I think I'd come
> > around to the view that the text should have them. Explicit is
> > better than implicit,
>
> At least in the absence of proper research results, I think this part
> of it *is* religious. To some people, it's obvious that whitespace is
> better on the eyes. To others, it's obvious that whitespace + braces
> is better on the eyes. One group may well be wrong <0.5 wink>.
>
> This argument (that braces are another visual cue which make code
> easier to read -- provided that they're automatically maintained in
> sync with the indentation), is the only argument against
> pure-whitespace that has ever made any sense to me.
I believe there was quite a bit of research on this issue in ABC,
which was the research system that Guido learned on when he
was in university. One of the fallouts from that is the colon at the
end of block headers: it's syntactically unnecessary, but it does
seem to help the eyes.
I wouldn't characterize it as religious. Brains don't work the
way most people think they do, and if you've programmed yours
one way, it seems to depend on a proper choice of grandparents,
or the proper diet, or maybe the phase of the moon whether you'll
be comfortable changing it.
> Irrespective of
> whether you pick whitespace or braces as the thing the compiler takes
> notice of, though, the optimal solution probably still involves an
> editor that supports showing/hiding braces, so the syntax is
> (theoretically!) a non-issue. In practice, editors don't seem to
> currently support showing and hiding braces automatically. At least
> editors do get you out of most mistakes caused by brace-y languages.
>
> Of course, whitespace *is* still superior because the storage format
> doesn't duplicate state -- so people with poor editors can't produce
> ambiguous code :-) (remember code can be ambiguous to *people* even if
> it isn't to a compiler). OTOH, *Python's* scheme is inferior to a
> pure-space-character indentation scheme because off the tab-vs.-space
> issue :-(
Well, that's been recognized as a problem for a long time. I believe
that the intention is to mandate spaces only in Python 3.0. On the
other hand, I'm not holding my breath waiting for Guido to decide
he's had enough 2.x releases and start work on 3.0 To be frank
about it, I don't see a crying need for 2.4 before 3.0,
and if he's after mind share, he's got both Ruby and the redesign
of Perl breathing down his neck. The new Perl regex package looks
nice! Three years ago, he didn't have any credible competition in
the high quality but easy to use scripting language arena. Now he
does.
> > and there are a few things that the
> > automatic indentation makes rather difficult in the design area.
> [...]
>
> What are those things??
Embedding Python in other languages, such as HTML, for
starters. It can be done, but it isn't pretty.
I've run across a couple of other places where I wished I
had braces, but they weren't important enough to stick in
my memory.
John Roth
>
>
> John
John Roth
"Robin Becker" <ro...@jessikat.fsnet.co.uk> wrote in message
news:TBO13LAmBVK$Ew...@jessikat.fsnet.co.uk...
> In article <viibo7t...@news.supernews.com>, John Roth
> <newsg...@jhrothjr.com> writes
> >
> .....
> >High performance isn't Python's target. If PyPy ever gets their act
> >off the ground, then we will have a shot at a good quality JIT
> >interpreter. Then watch it fly.
> >
> .... doesn't psyco already attempt to do JIT? It certainly doesn't
> speed things up that much. If all the variables are known to be of a
> specific type then you could expect C like speeds from a good JIT, but
> without global analysis it's hard to see how we infer/guarantee python
> types.
Well, that's certainly a problem, but Bicycle Repair Man seems to
do a pretty good job of type inference, at least for refactoring.
One of the things to consider here is that a decent JIT interpreter
would automatically change the playing field for what is good
practice and what isn't, at least if you expect performance.
John Roth
> --
> Robin Becker
John Roth
"Ian Bicking" <ia...@colorstudy.com> wrote in message
news:mailman.1059679007...@python.org...
> On Thu, 2003-07-31 at 08:55, Anthony_Barker wrote:
> > The three that come to my mind are significant whitespace, dynamic
> > typing, and that it is interpreted - not compiled. These three put
> > python under fire and cause some large projects to move off python or
> > relegate it to prototyping.
>
> I think these are valid as "compromises", not to be confused with
> flaws. These are all explicit choices, and ones for which the original
> justifications remain valid. A lot of stuff in Basic is simply flaws,
> or based on justifications that no longer apply to today's programming
> world.
I think I'd go with your first assessment. These are design choices.
Compromises are things that you can't figure out how to get right
within the structure of the major design choices.
> Anyway, I might add mine: the nature of modules as executed code is a
> compromise. That is, a module isn't a declaration, it's a program to be
> executed in its own namespace. When you import a module, you are
> executing the module then looking at the namespace.
I wouldn't call it a compromise, but it does have its bad points
when you're trying to construct a large, reliable system.
Well, while that is true in detail, if you put the prototype in the
class object, all instances will find it the next time they try to
access it. References will get the class version, and assignments
will update the instance. Of course, you had to change some of
the methods to even get into this situation...
John Roth
>
> Ian
>
>
>
Andrew Dalke
> There aren't any [compromises in Python]
There are definitely compromises in Python. As Christopher Koppler
pointed out, Python does compromise for backwards compatibility.
If you want, you can write "1 <> 2" and it will work, even though that
feature has been deprecated for about 8 years. Similarly, string exceptions
were deprecated about 5 years ago, but you can still use them.
And now with new-style types you can use
class OldSpam:
eggs = attribute(_get_eggs, _set_eggs)
class NewSpam(object):
eggs = attribute(_get_eggs, _set_eggs)
and get slightly different behaviours.
These exist as a compromise between the old way (plenty of existing
source code) and the new way (which provides better ways to do
older behaviours).
Andrew
da...@dalkescientific.com
sism...@hebmex.com
> Sent: Jueves, 31 de Julio de 2003 01:41 p.m.
Hello Aaron.
Actually, this which you call a compromise, is one of Python's
biggest strengths: it allows you, as a programmer, to perform
many things which in other languages would be simply impossible,
or at least would force you to jump through plenty loops and
hoops.
Take a look at Alex Martelli's Borg pattern, it shows what can
be done by having open access to class *and* object namespaces,
and being able to manipulate them. In Python it's easy and
straightforward (eek), but in other languages, well... you have
whole books dedicated to that.
We're all adults, and all that. :-)
-gustavo
Advertencia:La informacion contenida en este mensaje es confidencial y
restringida, por lo tanto esta destinada unicamente para el uso de la
persona arriba indicada, se le notifica que esta prohibida la difusion de
este mensaje. Si ha recibido este mensaje por error, o si hay problemas en
la transmision, favor de comunicarse con el remitente. Gracias.
Gary Herron
What's more, these are three of Python's greatest *strengths*. We
resist all attempts to change these, and we (at least I) avoid other
languages because they do not supply these features.
Gary Herron
Ian Bicking
> "Robin Becker" <ro...@jessikat.fsnet.co.uk> wrote in message
> news:TBO13LAmBVK$Ew...@jessikat.fsnet.co.uk...
> > In article <viibo7t...@news.supernews.com>, John Roth
> > <newsg...@jhrothjr.com> writes
> > >
> > .....
> > >High performance isn't Python's target. If PyPy ever gets their act
> > >off the ground, then we will have a shot at a good quality JIT
> > >interpreter. Then watch it fly.
> > >
> > .... doesn't psyco already attempt to do JIT? It certainly doesn't
> > speed things up that much. If all the variables are known to be of a
> > specific type then you could expect C like speeds from a good JIT, but
> > without global analysis it's hard to see how we infer/guarantee python
> > types.
>
> Well, that's certainly a problem, but Bicycle Repair Man seems to
> do a pretty good job of type inference, at least for refactoring.
And Java's JIT is based on (at least originally) work done on Self,
which had to do type inference. And actually in many circumstances Java
requires type inference, because you can substitute in an instance of a
subclass.
Anyway, JIT is all about runtime analysis -- if you could infer types
completely before running the program, you would just put in the
optimizations statically (i.e., compiling optimizations). JIT does
those optimizations at runtime by definition.
And Bicycle Repair Man is inspired by the Refactoring Browser, an IDE
tool based on another dynamic language (Smalltalk), not on a tool from a
static language (like Java).
Ian
Aaron Leung
I've seen the Borg pattern; it's nice. I suppose I should have said
that using dictionaries everywhere is a very flexible and convenient
feature, but the compromise is in efficiency. Of course, I realize
that efficiency isn't always important.
Best regards,
Aaron
Hannu Kankaanpää
> What to you think python largest compromises are?
I think reference counting is. Added to this the fact that
garbage collection is also possible (as in Jython). So we get
the worst of both worlds.
1. Idioms like this won't work portably:
def foo():
f = file('x')
return f.read()
Even though f.__del__ closes the file, in garbage
collected environment the f object might not get deleted
in a good while, and would cause problems.
2. And then, we still need to use weakref to ensure
that our crossreferenced stuff works both with and without GC.
Worst of both indeed. Maybe the decision to choose reference
counting was driven by speed considerations. That might've
been reasonable back in early 90's, but GC techniques have
evolved from those days and so GC would be a superior
technique now.
Well, since no one else pointed this out yet, maybe there's
some flaw in my reasoning.
Andrew Dalke
> Whereas BASIC started life as an interpreted language wherein
every
> statement had a line number, conditionals (IF) did jumps to line
> numbers, and all variables were global (even across subroutine calls).
Not interpreted.
See http://wombat.doc.ic.ac.uk/foldoc/foldoc.cgi?Dartmouth+BASIC
] Dartmouth BASIC
] <language> The original BASIC language [...] Unlike most later
] BASIC dialects, Dartmouth BASIC was compiled
or a more detailed description at http://www.kbasic.org/1/history.php3
which says the first interpreted BASIC was the Micro-Soft one for
the Altair.
In more seriousness, compiled v. interpreted is an implementation
detail which has very little impact on the language. The other examples
you gave are more relevant.
One thing to consider is - what's the *compromise* in the different
versions of BASIC? That is, how was the support for backwards-
compatible operations in BASIC (which you list as a compromise)
any different than Python's backwards compatibility support?
Andrew
da...@dalkescientific.com
Gerald Klix
Hash: SHA1
Anthony_Barker wrote:
| I have been reading a book about the evolution of the Basic
| programming language. The author states that Basic - particularly
| Microsoft's version is full of compromises which crept in along the
| language's 30+ year evolution.
|
| What to you think python largest compromises are?
|
IHMO it is the lambda expression.
These "functions" are not real functions. You can not use
statements in them.
What Python realy needs here is some means to make an expression
from a list of statements (called suite in the syntax defintion).
That given PEP 308 (If-then-else expression) or PEP 318 (Function/Method
Decorator Syntax) are mostly pointless.
Let me give an example:
Suppose there some special braces like (: :) and a exit operator like a
unary ^ one can write a conditional expession like that:
(:
~ if f():
~ ^trueValue
~ else:
~ ^falseValue :)
classmethods can be declared as follows:
cm = (:
~ def im( arg0, arg1 ):
~ return answer
~ ^classmethod( im ) :)
or
cm = classmethod( (:
~ def im( arg0, arg1 ):
~ return answer
~ ^im
~ :) )
obvously this demands for some means to write anonymous functions like
def ( arg0, arg1 ):
~ return arg0 - arg1
semanticly this should transform to
(:
~ def newName( arg0, arg1 ):
~ return arg0 - arg1
~ ^newName :)
giving
cm = def ( arg0, arg1 ):
~ return answer
Ok, I admit that this is difficult to be integrated in
the existing syntax. Perhaps we can not drop the braces
around such expression.
Is this worth writing a PEP?
|
|>"No, we use Unit Testing in Zope".
I am still missing a simple testing framework. doctest
is a good idea, but conflicts with syntax hilighting in most
editors.
|
|
| That said, obvious Basic compromised by using things such as "Option
| Explicit", thereby allowing both dynamic and more static style
| variables. Yahoo groups moved from python to C due to dynamic typing.
This is not a problem. They key to speed is using extension written
in C for performance. Normaly you will find one that sloves your problem.
|
| Non-compiled - obviously there are times when performance matters more
| than other things. Google I believe uses python to prototype (or used)
| and then turns to c++ for heavy lifting.
|
| What about immutable strings? I'm not sure I understand Guido's
| preference for them.
In fact the array module provides mutable strings.
|
| Anthony
| http://xminc.com/anthony
HTH,
Gerald
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)
Comment: Using GnuPG with Debian - http://enigmail.mozdev.org
iD8DBQE/KjWgEDg9cqFA1jQRAovzAJkBygPzdfHsoVXu9H2QHnxTD9sMmACdHvY5
dB1j4kXTzejml7fG0oSwhUg=
=WGR+
-----END PGP SIGNATURE-----
Robin Becker
Bicking <ia...@colorstudy.com> writes
>And Java's JIT is based on (at least originally) work done on Self,
>which had to do type inference. And actually in many circumstances Java
>requires type inference, because you can substitute in an instance of a
>subclass.
>
>Anyway, JIT is all about runtime analysis -- if you could infer types
>completely before running the program, you would just put in the
>optimizations statically (i.e., compiling optimizations). JIT does
>those optimizations at runtime by definition.
>
but Java does at least require specifying every type and that must at
least cut down on the amount of work required.
>And Bicycle Repair Man is inspired by the Refactoring Browser, an IDE
>tool based on another dynamic language (Smalltalk), not on a tool from a
>static language (like Java).
>
> Ian
I don't have any data here, but I believe Python is just a little too
weakly typed for compiling to float*float type assembler efficiently.
--
Robin Becker
John Roth
The trick with JITs is that they don't depend on absolute type
consistency. They depend on the observation that 99.44% of your
code is type consistent, and that consistency will turn up at run time. So
the code they generate depends on that discovered consistency, and
checks in front of each section to discover if the types are what the
code expects.
If it is, they execute it, if it isn't, they abandon it and go back to
the intepreter to discover what happened.
Michael Hudson
> Worst of both indeed. Maybe the decision to choose reference
> counting was driven by speed considerations.
Ease of implementation, portability and playing nicely with C
extensions are more likely candidates, IMO.
> That might've been reasonable back in early 90's, but GC techniques
> have evolved from those days and so GC would be a superior technique
> now.
<button nature="hot">
Reference counting *is* a form of garbage collection.
</button>
Saying "Ref. counting sucks, let's use GC instead" is a statement near
as dammit to meaningless.
Given the desires above, I really cannot think of a clearly better GC
strategy for Python that the one currently employed. AFAICS, the
current scheme's biggest drawback is its memory overhead, followed by
the cache-trashing tendencies of decrefs.
What would you use instead?
Cheers,
mwh
--
After a heavy night I travelled on, my face toward home - the comma
being by no means guaranteed. -- paraphrased from cam.misc
John Roth
"Hannu Kankaanpää" <hanz...@yahoo.com.au> wrote in message
news:840592e1.03073...@posting.google.com...
There's a flaw in your reasoning. The various techniques that
descend from mark and sweep (which is what you're
calling garbage collection) depend on being able to
identify all of the objects pointed to. For objects that are
owned by Python, that's a lengthy (that is, inefficient)
process, and it's not possible in general for objects that
are created by extensions.
Reference counting only depends on having the
object itself, and control of the creation and removal
of references. The latter is a frequent source of bugs
and memory leaks in extensions.
It's easy to say that various languages would be improved
by adding "real" garbage collection, but those techniques
impose significant design constraints on the implementation
model.
John Roth
Daniel Dittmar
> <button nature="hot">
> Reference counting *is* a form of garbage collection.
> </button>
>
> Saying "Ref. counting sucks, let's use GC instead" is a statement near
> as dammit to meaningless.
This statement is not meaningless because most programmers will correctly
identify GC in this context as something like mark-and-sweep, generation
scavenging etc.
see also: DOS sucks, let's use an operating system instead.
> current scheme's biggest drawback is its memory overhead, followed by
> the cache-trashing tendencies of decrefs.
plus it doesn't support independent threads as all reference counting would
have to be protected, leading to poor performance.
But a lot of Python code depends on reference counting or more exactly it
depends on the timely call of the destructor. So even if a much better GC is
added to Python, reference counting would perhaps be kept for backwards
compatibility (see Python's biggest compromises)
Daniel
Daniel Dittmar
> There's a flaw in your reasoning. The various techniques that
> descend from mark and sweep (which is what you're
> calling garbage collection) depend on being able to
> identify all of the objects pointed to. For objects that are
> owned by Python, that's a lengthy (that is, inefficient)
That's what generation scavenging was developed for. One shouldn't argue by
tradition alone, but the fact that the major implementations of dynamic
languages like LISP and Smalltalk don't use reference counting should carry
some weight.
> process, and it's not possible in general for objects that
> are created by extensions.
This is generally handled by registering and unregistering objects in the
extension code. Error prone as well, but probably less so than reference
counting.
> It's easy to say that various languages would be improved
> by adding "real" garbage collection, but those techniques
> impose significant design constraints on the implementation
> model.
True. But one could review these constraints from time to time.
Daniel
Michael Hudson
> John Roth wrote:
> > There's a flaw in your reasoning. The various techniques that
> > descend from mark and sweep (which is what you're
> > calling garbage collection) depend on being able to
> > identify all of the objects pointed to. For objects that are
> > owned by Python, that's a lengthy (that is, inefficient)
>
> That's what generation scavenging was developed for.
But most generational collectors are copying aren't they? That makes
it challenging to meet the "play nice with C" goal I talked about (not
impossible, but significantly difficult).
> One shouldn't argue by tradition alone, but the fact that the major
> implementations of dynamic languages like LISP and Smalltalk don't
> use reference counting should carry some weight.
True. But the major implementations of these languages are also
usually less portable, and something more of a fiddle to write C
extensions for (at least, for the implementations I know about, which
are mostly CL impls).
If you're writing a native code compiler, it doesn't seem that much of
a drag to have your code about e.g. the processor's registers.
> > It's easy to say that various languages would be improved
> > by adding "real" garbage collection, but those techniques
> > impose significant design constraints on the implementation
> > model.
>
> True. But one could review these constraints from time to time.
Indeed. People have tried using the Boehm GC with Python, but
generally found that performance got worse, or at least not strikingly
better. I don't think anyone has tried implementing a really good
generational-type collector.
(And unfortunately, I think one constraint that's not going away is
"not breaking every C extension that's ever been written).
Cheers,
mwh
--
US elections
For those of you fearing that the rest of the world might be
making fun of the US because of this: Rest assured, we are.
-- http://www.advogato.org/person/jameson/diary.html?start=12
Paul Rubin
> > One shouldn't argue by tradition alone, but the fact that the major
> > implementations of dynamic languages like LISP and Smalltalk don't
> > use reference counting should carry some weight.
>
> True. But the major implementations of these languages are also
> usually less portable, and something more of a fiddle to write C
> extensions for (at least, for the implementations I know about, which
> are mostly CL impls).
I'd say the opposite, the Lisp implementations I've worked on are
considerably easier to write C extensions for, partly BECAUSE you
don't have to worry about constantly tweaking ref counts. In GNU
Emacs Lisp, for example, if you cons a new heap object and put it in a
C variable and (iirc) then call eval, you have to call a macro that
tells the GC not to sweep the object. But many C functions don't make
new objects, and most don't call eval, and you don't have to remember
what objects you've called the macro for. There's another macro that
you call before your function returns, and that cleans up all the GC
records in your stack frame made by any invocations of the first macro.
Robin Becker
<newsg...@jhrothjr.com> writes
>>
>> I don't have any data here, but I believe Python is just a little too
>> weakly typed for compiling to float*float type assembler efficiently.
>
>The trick with JITs is that they don't depend on absolute type
>consistency. They depend on the observation that 99.44% of your
>code is type consistent, and that consistency will turn up at run time. So
>the code they generate depends on that discovered consistency, and
>checks in front of each section to discover if the types are what the
>code expects.
>
>If it is, they execute it, if it isn't, they abandon it and go back to
>the intepreter to discover what happened.
>
>John Roth
'float' object must carry along a whole lot of baggage (I guess I mean
be a more generic object) to allow for the testing. Loops without method
or function calls would be good candidates for JIT as methods and
functions could alter attribute types.
Is the JIT object literally just a union of
type,values
or would it be an actual Python object? For example would an
innerproduct be over a pair of lists or would the magic convert these
into actual double arrays.
--
Robin Becker
Mel Wilson
"Andrew Dalke" <ada...@mindspring.com> wrote:
>Dennis Lee Bieber:
>> Whereas BASIC started life as an interpreted language wherein
>every
>> statement had a line number, conditionals (IF) did jumps to line
>> numbers, and all variables were global (even across subroutine calls).
>
>Not interpreted.
>See http://wombat.doc.ic.ac.uk/foldoc/foldoc.cgi?Dartmouth+BASIC
>] Dartmouth BASIC
>] <language> The original BASIC language [...] Unlike most later
>] BASIC dialects, Dartmouth BASIC was compiled
>
>or a more detailed description at http://www.kbasic.org/1/history.php3
>which says the first interpreted BASIC was the Micro-Soft one for
>the Altair.
Hmm, OK. Let's say then that Darmouth BASIC was designed
as a very lighweight compiler, for quick response.
'Designed for interaction' might cover it. Lightweightness
would encoourage things like the 'LET' statment,
single-character (+ optional type-suffix) variable names.
My catch-phrase lately is "Python is to C++ as BASIC was
to FORTRAN". I should be careful who I use it around,
maybe.
Regards. Mel.
Anthony_Barker
> >
> > python under fire and cause some large projects to move off python or
> > relegate it to prototyping.
>
> something was conceded, or that an intermediate position between two
> extremes was chosen to appease. I don't think that either sense really
> applies to these features.
>
> The three items that you listed are merely design choices. While arguments
> over them are continuous, two of the design choices (interpreter, dynamic
> typing) are consistent with Python's intended use as a language which
> excels at rapid prototyping. The third (white space) is merely a stylistic
> choice which is designed to encourage readable programs.
>
> "Compromises" in language design occur usually when a committee tries to
> standardize a language, and each has differing views about how the language
> should be used. While this occurs somewhat in Python, other languages
> have suffered more mightily from this particular disorder.
>
> Mark
Excellent points - you are correct the ones I listed are design
choices.
Some people could be interpreted them as design "compromises". The
kind of people who would like to use the same tool for all problems.
Ian Bicking
> Worst of both indeed. Maybe the decision to choose reference
> counting was driven by speed considerations.
Reference counting spreads the speed hit over the entire program, while
other techniques tend to hit performance hard every so often. But all
together I think reference counting is usually slower than a good GC
algorithm, and incremental garbage collection algorithms can avoid
stalling. And I'm sure that the current state -- references counting
plus another kind of garbage collection for circular references -- must
be worse than either alone. The advantage is predictable collection
(unless you are using Jython), without memory leaks (due to circular
references).
Oh well...
Ian
OKB (not okblacke)
> What Python realy needs here is some means to make an expression
> from a list of statements (called suite in the syntax defintion).
I believe this is called a "function".
--
--OKB (not okblacke)
"Do not follow where the path may lead. Go, instead, where there is
no path, and leave a trail."
--author unknown
Brian Quinlan
while
> other techniques tend to hit performance hard every so often. But all
> together I think reference counting is usually slower than a good GC
> algorithm, and incremental garbage collection algorithms can avoid
> stalling. And I'm sure that the current state -- references counting
> plus another kind of garbage collection for circular references --
must
> be worse than either alone.
I'm not sure that I believe this. In Python, everything is an object
that participates in GC including integers and other light-weight types.
Imagine that you have a mathematic expression that creates a dozen
floats objects as part of its evaluation. Do you believe that it is
faster to add those dozen floats to a list for later collection than to
decrement an integer, notice that the decremented value is 0 and
immediately reclaim the memory? That would not be my intuition (but my
intuition is often wrong :-)).
Cheers,
Brian
John Roth
As far as I'm aware, the JIT code doesn't fiddle with the data;
it just does the equivalent of assert tests at the beginning of the
blocks to verify that it's got the type of object it expects.
In other words, it does a very large amount of run-time type
checking. This only pays off if it can save even more expense
by compiling the code.
Now, this is going to be difficult for short segments of code,
but it can be quite a time saver if the JIT generated code can
make intermediate objects vanish so they don't have to be
created just to be discarded a short time later.
John Roth
"Anthony_Barker" <anthony...@hotmail.com> wrote in message
news:899f842.03073...@posting.google.com...
> I have been reading a book about the evolution of the Basic
> programming language. The author states that Basic - particularly
> Microsoft's version is full of compromises which crept in along the
> language's 30+ year evolution.
>
I'm not sure if we've beaten this one to death or not, but a real
example of a compromise just floated through my head.
Consider <list>.sort() and <list>.reverse(). These two otherwise
admirable methods don't return the object, so they can't be chained.
Why not? Because they update the list in place, and Guido decided
that not returning the object was a cheap way to make it clear that
they were doing something unusual.
Now, *that's* a compromise. The worst of both worlds.
John Roth
Hannu Kankaanpää
> <button nature="hot">
> Reference counting *is* a form of garbage collection.
> </button>
You apparently have such a loose definition for garbage
collection, that even C programs have "a form of garbage
collection" on modern OSes: All garbage is reclaimed by
the OS when the program exits. It's just a very lazy collector.
I don't consider something a garbage collector unless it
collects all garbage (ref.counting doesn't) and is a bit more
agile than the one provided by OS.
> Saying "Ref. counting sucks, let's use GC instead" is a statement near
> as dammit to meaningless.
You, I and everyone knows what I was talking about, so it could
hardly be regarded as "meaningless".
> Given the desires above, I really cannot think of a clearly better GC
> strategy for Python that the one currently employed. AFAICS, the
> current scheme's biggest drawback is its memory overhead, followed by
> the cache-trashing tendencies of decrefs.
It's not "the one currently employed". It's the *two* currently
employed and that causes grief as I described in my previous post.
And AFAIK, Ruby does use GC (mark-and-sweep, if you wish) and
seems to be working. However, this is rather iffy knowledge. I'm
actually longing for real GC because I've seen it work well in
Java and C#, and I know that it's being used successfully in many
other languages.
> What would you use instead?
A trick question?
Marc
Python.
To me there is a compromise in Python's dependence on C. It seems that
at some point I will hit a performance or feature issue that will
require me to write a C extension. It seems to me VB6 has a similarly
awkward relationship with C++. Clearly the creators of Python were
expert C programmers; that should not be a requirement to become an
expert Python programmer.
- Marc
John Roth
"Marc" <marcs...@yahoo.com> wrote in message
news:ac7dc9b9.03080...@posting.google.com...
As a number of people have said: if PyPy ever gets working...
John Roth
>
> - Marc
Paul Rubin
> I'm not sure that I believe this. In Python, everything is an object
> that participates in GC including integers and other light-weight types.
> Imagine that you have a mathematic expression that creates a dozen
> floats objects as part of its evaluation. Do you believe that it is
> faster to add those dozen floats to a list for later collection than to
> decrement an integer, notice that the decremented value is 0 and
> immediately reclaim the memory? That would not be my intuition (but my
> intuition is often wrong :-)).
In a fast GC system you would just allocate the floats from a
contiguous block. Later you would copy the reachable floats from that
block to another block. If no floats are reachable, you don't copy
anything. There are all kinds of optimizations you can use, including
making clever use of memory protection hardware to notice when your
allocator has run over the end of a block so you don't have to even
make an explicit pointer comparison, that can make all this run very fast.
Andrew Appel has a book about garbage collection where he discusses
these methods in detail.
Andrew Dalke
> I'd say the opposite, the Lisp implementations I've worked on are
> considerably easier to write C extensions for, partly BECAUSE you
> don't have to worry about constantly tweaking ref counts.
I've mentioned in c.l.py before a library I used which can be called
from both C and FORTRAN. The latter doesn't support pointers,
so instead the library has a global instance table, indexed by integers.
The C/FORTRAN code just passes integers around.
In addition, there are dependencies between the objects, which
means that user code object deallocation can only occur in a
certain order.
With CPython it's possible to put a high-level OO interface to
that library, and provide hooks for the ref-counted gc to call
the proper deallocators in the correct order. This is done by
telling the finalizer how to do it and paying careful attention to
order.
The library also has Java bindings. As far as I can tell, it's
impossible to hook into Java's automatic gc. A C-level
gc like Boehm can't ever tell that data is no longer needed,
because the global table keeps a reference to every created
object, and Java's native gc doesn't make the proper
guarantees on finalization order.
Andrew
da...@dalkescientific.com
Andrew Dalke
> IHMO it is the lambda expression.
> These "functions" are not real functions. You can not use
> statements in them.
True. That's why they are "lambda expressions" and not
"lambda functions"
(Okay, that was a cheap shot ... but still true ;)
Andrew
da...@dalkescientific.com
Andrew Dalke
> To me there is a compromise in Python's dependence on C.
Then explain Jython, which is an implementation of Python-the-language
on top of Java.
> It seems that
> at some point I will hit a performance or feature issue that will
> require me to write a C extension.
change "will" to "may"
And if you write in C, at some point you may hit a performance
or feature issue that will require you to write assembly code.
> Clearly the creators of Python were
> expert C programmers; that should not be a requirement to become an
> expert Python programmer.
Lack of knowledge of C does not strongly preclude becoming an
expert Python programmer.
To be an expert Python programmer, you should know how other
programming langauges work too, but that can be done by
learning other languages: Eiffel, Java, APL, Haskel, Caml, Ada,
Prolog, Java, ...
Andrew
da...@dalkescientific.com
Terry Reedy
"Paul Rubin" <http://phr...@NOSPAM.invalid> wrote in message
news:7xr8456...@ruckus.brouhaha.com...
The proof of such assertions, either way, is in real code. Before
circular ref GC was added, at least one person tried replacing ref
count GC with 'real' GC. The resulting benchmarks were about the
same. So Guido stuck with what was familiar, simpler to understand,
and seemed less likely to interfere with other improvements. The type
of GC didn't seem to matter too much. But anyone is welcome to prove
this wrong. However, acceptance into the core requires some degree of
system/hardware independence.
As I understand it, Psyco unboxes some types and does away with some
of the need for any GC. *That* does seem to make a difference.
Terry J. Reedy
Paul Rubin
> The proof of such assertions, either way, is in real code. Before
> circular ref GC was added, at least one person tried replacing ref
> count GC with 'real' GC. The resulting benchmarks were about the
> same. So Guido stuck with what was familiar, simpler to understand,
> and seemed less likely to interfere with other improvements. The type
> of GC didn't seem to matter too much. But anyone is welcome to prove
> this wrong. However, acceptance into the core requires some degree of
> system/hardware independence.
I think ref counting pervades CPython so thoroughly that it would be
ridiculous to try to change it. However, in a complete re-implementation,
different GC schemes should be considered.
I also don't think traditional GC is harder to understand than ref
counting. I think Guido once mentioned that he chose ref counting
because it was what he was used to. Having implemented GC by both
mark-sweep (for a Lisp interpreter) and ref counting (in GNU Awk), I'd
say that mark-sweep is easier to keep straight.
> As I understand it, Psyco unboxes some types and does away with some
> of the need for any GC. *That* does seem to make a difference.
I think very large performance improvements can come from changing the
language semantics in ways that would cause some backwards
incompatibility. But I think that should be considered for Python
3000 which should be implemented with a native-code compiler from the
very outset. I don't see why a good Python system can't have the
performance of a good Lisp system.
Courageous
>> The three that come to my mind are significant whitespace, ...
"Significant whitespace" isn't a "compromise," it's a design choice.
The Python interpreter actually inserts explicit scope tokens into
the symbol stream at the lexer; the parser deals with the symbols as
does any parser. It's really not all that hard, actually. One just
has to understand the bit that making the *parser* deal with the white
space is not the right thing.
C//
Brian Quinlan
> contiguous block.
This is how Python works now.
> Later you would copy the reachable floats from that block to another
> block. If no floats are reachable, you don't copy anything.
Is determining if a float is reachable faster or slower than
dereferencing a pointer and comparing an integer to zero? 'cause that is
all reference counting requires.
I'm not saying that the reference counting (+ something else) is the
optimal GC strategy for Python but it seems to have advantages when
working with a lot of objects with short lifetimes.
> Does checking to see if a There are all kinds of optimizations you
> can use, including making clever use of memory protection hardware
> to notice when your allocator has run over the end of a block so
> you don't have to even make an explicit pointer comparison, that
> can make all this run very fast.
Of course the PythonLabs implementation can never require such a trick.
Cheers,
Brian
Paul Rubin
> > In a fast GC system you would just allocate the floats from a
> > contiguous block.
>
> This is how Python works now.
Yes, but in a fast GC system you normally never free the floats
individually back into that block. You copy the live ones to another
block and then free the original block all at once.
> > Later you would copy the reachable floats from that block to another
> > block. If no floats are reachable, you don't copy anything.
>
> Is determining if a float is reachable faster or slower than
> dereferencing a pointer and comparing an integer to zero? 'cause that is
> all reference counting requires.
If the float is reachable, finding it is comparably fast (maybe
faster, maybe slower) than checking a ref count. The more frequent
case is that the float is unreachable, in which case it's not checked
at all. The idea is you traverse the heap to find all the live data
and copy it to a new block, but you never have to touch the dead data.
> I'm not saying that the reference counting (+ something else) is the
> optimal GC strategy for Python but it seems to have advantages when
> working with a lot of objects with short lifetimes.
Yes, "ephemeral" copying GC schemes are designed for that too.
> > Does checking to see if a There are all kinds of optimizations you
> > can use, including making clever use of memory protection hardware
> > to notice when your allocator has run over the end of a block so
> > you don't have to even make an explicit pointer comparison, that
> > can make all this run very fast.
>
> Of course the PythonLabs implementation can never require such a trick.
In a serious optimizing implementation with native code compilers, I
don't see any reason not to use such GC tricks on OS's that support them.
Gerald Klix
Hash: SHA1
This is not a function. This just the same as blocks in Algol 68,
because both do not defer execution until the fuction is called.
Of course you can define the semantics of such a block in terms
of a function defintion and an immedirate call to the fuction.
This is done in the Scheme languges let syntax.
HTH,
Gerald
OKB (not okblacke) wrote:
| Gerald Klix wrote:
|
|
|>What Python realy needs here is some means to make an expression
|>from a list of statements (called suite in the syntax defintion).
|
|
| I believe this is called a "function".
|
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)
Comment: Using GnuPG with Debian - http://enigmail.mozdev.org
iD8DBQE/K4adEDg9cqFA1jQRAnLkAJ0WXxqAwdzyqrKfOy2O1ycod1aCmQCfeKRH
doeQdEuwBJ8LS+gy6IOYzZQ=
=KB0f
-----END PGP SIGNATURE-----
Andy C
> > pure-space-character indentation scheme because off the tab-vs.-space
> > issue :-(
>
> Well, that's been recognized as a problem for a long time. I believe
> that the intention is to mandate spaces only in Python 3.0. On the
> other hand, I'm not holding my breath waiting for Guido to decide
What's bad about tabs? I'm new to Python. Tabs are good because then you
can view the source however you want. I can't write in 4 space tabs and
someone else can change it to 2 if they prefer. But I can see the problem
with continuation lines. Also it must be ALL tabs or ALL spaces. This
shouldn't be too hard -- most editors have the "tabify" option.
Andy
John Roth
"Andy C" <ayc8N...@cornell.edu> wrote in message
news:y3VWa.261$_p6.23...@newssvr21.news.prodigy.com...
The basic problem with tabs is that many tools don't handle them "right."
Especially when there's no agreement about what "right" means.
Originally, they were metal shims that were put onto the back of
typewriter carriages to stop the carriage at a particular point, and that's
the "official" meaning of tab to this day: it advances to the next multiple
of 8. According to this interpretation (which is *the* official
interpretation)
it's impossible to have only tabs and have any other indentation than 8
characters.
Many tools don't allow you to configure tabs, and of those that do,
each uses its own incompatible syntax and has its own quirks. In other
words, tabs may seem like a good thing if you use just one or two tools
that do what you want, but as soon as your program moves out into
the wild world, things change.
The only reason that Python works as well as it does with tabs is
that there is an undocumented hack buried in the guts somewhere
that identifies the comment line that emacs sticks into the program to
remember your settings. It does this for a couple of other popular
editors as well.
John Roth
>
> Andy
>
>
Andy C
> each uses its own incompatible syntax and has its own quirks. In other
> words, tabs may seem like a good thing if you use just one or two tools
> that do what you want, but as soon as your program moves out into
> the wild world, things change.
What are these quirks? By far the most common I've seen is mixing tabs and
spaces, but this should be relatively easily solved by requiring one or the
other (minus continuation lines, which are still a problem). Using spaces
has some disadvantages too, since not everyone will use the same number of
spaces, and editors don't behave as nicely. I like when you hit the arrow
key at a tab, and it jumps the full tab, rather than having to press an
arrow key 4 times.
Terry Reedy
"Andy C" <ayc8N...@cornell.edu> wrote in message
news:y3VWa.261$_p6.23...@newssvr21.news.prodigy.com...
Because there is no 'official' meanings, despite claims to the
contrary, some software like Outlook Express ignores them on receipt.
<flameshield on> tjr
John Roth
I'm used to a certain minimum standard from my editors, and smart
tabs are one of those things that's part of the price of admission these
days, just like syntax highlighting. A programming editor's job is to help
where it's useful, and get out of the way when it's not. When you want
to indent, you hit the tab button. It's the editor's job to know I want,
for example, four spaces, and deliver them. In Python, it's the editor's
job to know that when I hit return at the end of a line, there are only one
or two legitimate places to put the cursor on the next line, and to put
it in the most likely of them.
As to the different number of spaces between developers, that's
another thing I'd expect from my editors. It's easy enough in Python
to figure out what's an indent and infer the number of spaces. I'd
expect a decent editor to be able to load a program and tell me
what the indentation policy was! I'd also expect to be able to tell
it to change it, and have it automatically reindent the program for
me.
John Roth
John Roth
"Terry Reedy" <tjr...@udel.edu> wrote in message
news:8Z-cnS25-8y...@comcast.com...
You can always blame that on Micro$haft. However, it does
make e-mailing a program indented with tabs something of an
adventure.
John Roth
>
>
>
Andy C
you. The editor should be able to handle tabs in a satisfactory manner as
well.
Ben Finney
> OK, then unless I'm missing something, tabs vs. spaces shouldn't
> matter for you. The editor should be able to handle tabs in a
> satisfactory manner as well.
Not if spaces *and* tabs are used for indentation within the one file.
Without editor-specific hacks (the "vim:" comment line, the Emacs mode
line, etc.) there's no way to know what size the previous person's
editor had tab stops set to. The standard size of a tab stop is 8, but
many programmers change this in their editors, resulting in indentation
that gets completely messed up when the tabs are shown at 8 columns
against other lines in the same file that use spaces for indentation.
--
\ "The best is the enemy of the good." -- Voltaire |
`\ |
_o__) |
Ben Finney <http://bignose.squidly.org/>
Terry Hancock
Yeah, it does. For some reason, my installation of Kmail insists on using
a proportional font. So I really have to *count the spaces* to get my
python examples to render right for those with correctly functioning email
clients. ;-)
Someday I'm going to get around to fixing that.
It can also be a pain when editing python code in a web-form, as you do
with Zope if you are using the through-the-web interface. Some browsers,
like Galeon, behave oddly regarding tabs: they accept them when pasted in
from another program, but it is impossible to type them! (Because the tab
key is configured to jump to the next entry blank in the form!). This is
sort of annoying, and I always wind up converting my code to spaces-only
when writing Python scripts in Zope.
I actually use spaces and tabs together to create 4-space indents with
8-character tabs. I try very hard to make sure that "as many tabs as
possible" are used (i.e. no 8-space indents). I really need to set things
up to filter the tabs out and replace them for archival code, though.
Cheers,
Terry
--
Terry Hancock
Anansi Spaceworks http://www.AnansiSpaceworks.com/
Andrew Dalke
> OK, then unless I'm missing something, tabs vs. spaces shouldn't matter
for
> you. The editor should be able to handle tabs in a satisfactory manner as
> well.
python-mode for Emacs is an example of how to support different
styles of tab/space uses.
But code goes through other tools. Suppose you only use tabs
and you have your editor set for 2 character tabs. Now print
your code. It'll look different because your printer is likely set
for 8 character tabs.
Some people like writing code like this
if ((this_is_a_value > that_long_variable) or
(this_is_a_value == something_else):
....
that is, align terms of an expression vertically to emphasize
similarities between the lines. Were the second line done in
tabs then differing tab styles would change the alignment,
which ruins the intent. And different tools do have different
tab styles.
Just Use Spaces.
Andrew
da...@dalkescientific.com
Andrew Dalke
> Because there is no 'official' meanings, despite claims to the
> contrary, some software like Outlook Express ignores them on receipt.
I sent email to a friend of mine. It contained a copy&paste from
the interactive prompt. Something like
>>> class Spam:
... pass
>>>
Something on his side (either Exchange or OE, dropped
the "... pass" part of what I sent.
Andrew
da...@dalkescientific.com
Andrew Dalke
> I'd swear that the BASIC I learned on -- via a timeshare
> Honeywell-Bull using a dial-up from an ASR-33 -- back in 1972 was
> interpreted...
Well, my background is all microcomputer basics in the 80s, including
some (very little) HP BASIC for lab equipment, so I'm not a good
judge. Were I to implement a compiled BASIC I would have done
the "compile on <ret>" you suggested. One of the links described the
Dartmouth BASIC commands and they would be easy to implement
that way.
> Python's compatibility doesn't /feel/ like a different language;
> through all the changes in Python, a Python 1.3 program /looks/ like a
> Python 2.2+ program.
In playing around with some of the new Python 2.3 features, I noticed
myself using a more functional style, like
d = {"A": 1, "B": 9", ...} # all uppercase keys
d.update(dict([(k.lower(), v) for k, v in d.items()]))
instead of the more readable and faster
for k, v in d.items():
d[k.lower] = v
It's easier in Python 2.2+ to make Python code which doesn't
look like Python 1.3. Which I think is the version I started with. ;)
When I got QuickBasic in 1988 or so, it would run just about
all of my old GW-BASIC programs, which was distributed some
5 years previous. But it's true, the GW-BASIC program wouldn't
look like a QB program.
Thinking some more ... the 1.3 code would have string
exceptions, so that's one age indicator. It would use the
"while 1 / readline / if not line / break" idiom instead of the
more modern 'for line in file'. It didn't have module support,
IIRC, and only the regex module for regular expressions.
Ahh, and no string methods. That's very noticible when I
look at pre-2.0 code.
Okay, so the two main things that would make a 1.3-style
code stick out is using string.* and raising string exceptions.
In other words, I mostly agree with you, but couldn't let
you get a complete bye :)
> But between early 70s BASIC and what passes for BASIC today looks
like
> totally different languages.
I stopped about 10 years ago. I do recall reading an article by
a VB book author saying VB_(N+1) was too different from VB(N)
and the differences were driven by marketing and not for good
programming reasons.
There's also the TrueBASIC folks. I believe they started in the
mid-80s and argue their BASIC is essentially the same.
> quick glance... Let a non-programmer look at source files from K&K
> type BASIC, Visual BASIC, F-IV, F77, and F90... and he likely will be
> able to identify the three Fortrans as being related -- but would not
> consider K&K to be a relation of VB.
Interesting test. Nasty idea: get that same person to judge if
Lisp and Scheme are closely related then post the results on c.l.lisp.
Andrew
da...@dalkescientific.com
Lulu of the Lotus-Eaters
|I don't have any data here, but I believe Python is just a little too
|weakly typed for compiling to float*float type assembler efficiently.
Repeat this every morning:
PYTHON IS A STRONGLY TYPED LANGUAGE WITH LATE BINDING!
(objects are typed in Python, names are not)
--
mertz@ _/_/_/_/ THIS MESSAGE WAS BROUGHT TO YOU BY: \_\_\_\_ n o
gnosis _/_/ Postmodern Enterprises \_\_
.cx _/_/ \_\_ d o
_/_/_/ IN A WORLD W/O WALLS, THERE WOULD BE NO GATES \_\_\_ z e
Lulu of the Lotus-Eaters
|But a lot of Python code depends on reference counting or more exactly it
|depends on the timely call of the destructor. So even if a much better GC is
|added to Python, reference counting would perhaps be kept for backwards
|compatibility (see Python's biggest compromises)
Did this thread get caught in a time warp, and posts from two years ago
get posted again. Exactly this all happened years ago.
--
mertz@ | The specter of free information is haunting the `Net! All the
gnosis | powers of IP- and crypto-tyranny have entered into an unholy
.cx | alliance...ideas have nothing to lose but their chains. Unite
| against "intellectual property" and anti-privacy regimes!
-------------------------------------------------------------------------
Lulu of the Lotus-Eaters
Beni Cherniavsky
> What to you think python largest compromises are?
>
As everybody said, whitespace, dynamic typing and interpretedness are
not
compromises.
One real compromise is the fact that you can't rebind an outer
variable from a
closure. That allows lexical scoping without any variable declrations
at the
price of giving up a useful ability of closures. When you do want it,
you must
either use the singleton list hack (but charactertistically, it isn't
used even
once in the standard library - I once performed an almost reliable
search), or
turn your nested function into a class just for that, which is
annoying (see
Java <wink>). It's noty impossible to fix, it will just require some
sort of "declration" (per assignment, per variable per function (like
``global``) or per
function) for the cases when you want this).
Robin Becker
Lotus-Eaters <me...@gnosis.cx> writes
>Robin Becker <ro...@jessikat.fsnet.co.uk> wrote previously:
>|I don't have any data here, but I believe Python is just a little too
>|weakly typed for compiling to float*float type assembler efficiently.
>
>Repeat this every morning:
>
> PYTHON IS A STRONGLY TYPED LANGUAGE WITH LATE BINDING!
>
>(objects are typed in Python, names are not)
>
strictly true, but I program with the names not the objects. I thought
that objects are always typed, if they weren't it would be hard to do
things with them :)
>--
> mertz@ _/_/_/_/ THIS MESSAGE WAS BROUGHT TO YOU BY: \_\_\_\_ n o
>gnosis _/_/ Postmodern Enterprises \_\_
>.cx _/_/ \_\_ d o
> _/_/_/ IN A WORLD W/O WALLS, THERE WOULD BE NO GATES \_\_\_ z e
>
>
--
Robin Becker
Andrew Dalke
> Unless I'm mistaken, the folks behind TrueBASIC /were/ Kemeny and
> Kurtz (spellings)... IE, the creators of the original BASIC. <G>
Yup. From truebasic.com
] John G. Kemeny and Thomas E. Kurtz invented BASIC in 1964
] for use at Dartmouth College. They made it freely available to
] everyone who wanted to learn how to program computers. It
] soon became a world standard.
]
] In 1983 they created True BASIC to incorporate and showcase
] all the advanced developments they had added to their language,
] and offered it as a commercial product.
But that doesn't implement the original BASIC language. OTOH,
it does say it can convert older BASIC to TrueBASIC, so there
is still backwards compatibility. That's gotta warm someone's heart
knowing code written back in the 1960s on a teletype machine
will still run today.. and even on a handheld.
Andrew
da...@dalkescientific.com
Alex Martelli
> OK, then unless I'm missing something, tabs vs. spaces shouldn't matter
> for
> you. The editor should be able to handle tabs in a satisfactory manner as
> well.
There seems to be an unspoken assumption here that an editor is the only
program that will need to deal with Python code I receive. It isn't --
mail and news readers, for example, are often involved -- and those often
can't deal satisfactorily with tabs. Thus, it should be the job of the
editor of the person who PRODUCES code (if said code is to be sent outside
at all) to store spaces, not tabs.
Alex
Michael Hudson
> Michael Hudson <m...@python.net> wrote in message news:<7h3brv9...@pc150.maths.bris.ac.uk>...
> > <button nature="hot">
> > Reference counting *is* a form of garbage collection.
> > </button>
>
> You apparently have such a loose definition for garbage
> collection, that even C programs have "a form of garbage
> collection" on modern OSes: All garbage is reclaimed by
> the OS when the program exits. It's just a very lazy collector.
>
> I don't consider something a garbage collector unless it
> collects all garbage (ref.counting doesn't) and is a bit more
> agile than the one provided by OS.
Well, OK, but people do tend to allow a bit of 'conservativeness' in
their collectors, don't they? Boehm's GC-for-C is usually considered
a 'real GC' and that doesn't necessarily collect everything (AIUI, I'm
not an expert in this field).
CPython's 'ref counting + gimmicks' certainly *would* seem to qualify
as a GC, by your definitions.
> > Saying "Ref. counting sucks, let's use GC instead" is a statement near
> > as dammit to meaningless.
>
> You, I and everyone knows what I was talking about, so it could
> hardly be regarded as "meaningless".
Well, OK, but: even if we don't allow refcounting as 'GC' I would
(really!) like to know which form of garbage collection you would use
instead.
> > Given the desires above, I really cannot think of a clearly better GC
> > strategy for Python that the one currently employed. AFAICS, the
> > current scheme's biggest drawback is its memory overhead, followed by
> > the cache-trashing tendencies of decrefs.
>
> It's not "the one currently employed". It's the *two* currently
> employed and that causes grief as I described in my previous post.
You mean two, as in the ones used by Jython and CPython? If have to
admit, I
> And AFAIK, Ruby does use GC (mark-and-sweep, if you wish) and
> seems to be working.
With finalizers? Just curious.
> However, this is rather iffy knowledge. I'm actually longing for
> real GC because I've seen it work well in Java and C#, and I know
> that it's being used successfully in many other languages.
>
> > What would you use instead?
>
> A trick question?
Not at all!
I think it would be practically very difficult to move to a radically
different memory management methodology without breaking the vast
majority of C extensions out there, but I would like to see some
more concrete speculation about alternatives.
Cheers,
mwh
--
Considering that this thread is completely on-topic in the way only
c.l.py threads can be, I think I can say that you should replace
"Oblivion" with "Gravity", and increase your Radiohead quotient.
-- Ben Wolfson, comp.lang.python
Michael Hudson
> "Daniel Dittmar" <daniel....@sap.com> wrote previously:
> |But a lot of Python code depends on reference counting or more exactly it
> |depends on the timely call of the destructor. So even if a much better GC is
> |added to Python, reference counting would perhaps be kept for backwards
> |compatibility (see Python's biggest compromises)
>
> Did this thread get caught in a time warp, and posts from two years ago
> get posted again. Exactly this all happened years ago.
Welcome to USENET!
Cheers,
mwh
--
Not only does the English Language borrow words from other
languages, it sometimes chases them down dark alleys, hits
them over the head, and goes through their pockets. -- Eddy Peters
Michael Hudson
> > As I understand it, Psyco unboxes some types and does away with some
> > of the need for any GC. *That* does seem to make a difference.
>
> I think very large performance improvements can come from changing the
> language semantics in ways that would cause some backwards
> incompatibility. But I think that should be considered for Python
> 3000 which should be implemented with a native-code compiler from the
> very outset. I don't see why a good Python system can't have the
> performance of a good Lisp system.
I can think of a few reasons, actually, with Python as it is today,
mostly along the lines of things like "you can't extend at runtime the
functionality of #'cl:=".
Tangentially, I'm not aware of any work that does Pysco-style dynamic
specialization in a Common Lisp environment, which surprises me to the
extent that I suspect I just don't know about it. Can anyone provide
a pointer?
Cheers,
mwh
--
... with these conditions cam the realisation that ... nothing
turned a perfectly normal healthy individual into a great political
or military leader better than irreversible brain damage.
-- The Hitch-Hikers Guide to the Galaxy, Episode 11
Mel Wilson
"Andy C" <ayc8N...@cornell.edu> wrote:
>What's bad about tabs? I'm new to Python. Tabs are good because then you
>can view the source however you want. I can't write in 4 space tabs and
>someone else can change it to 2 if they prefer. But I can see the problem
>with continuation lines. Also it must be ALL tabs or ALL spaces. This
>shouldn't be too hard -- most editors have the "tabify" option.
Nothing essentially. But if a program combines the two
forms, say somebody pastes space-using code into a tab-using
program, then strange things can happen for no obvious
reason. There's no standard exchange rate between tabs and
spaces in the Python syntax.
(Idea for obfuscated Python: a program that mixes spaces
and tabs in the indenting so as to perform two or more
distinct useful functions depending on the space-to-tab
rate. Bonus points for a program that undoes itself at
different settings.)
My own preference is for tabs, because my favourite
editor works well with them (i.e. Tab serves as indent and
Backspace as undent), and convert to spaces for people on
the outside, per convention.
Regards. Mel.
Andrew Dalke
> (Idea for obfuscated Python: a program that mixes spaces
> and tabs in the indenting so as to perform two or more
> distinct useful functions depending on the space-to-tab
> rate. Bonus points for a program that undoes itself at
> different settings.)
That's just not possible because Python only has one setting
for a tag - 8 spaces. At most there would appear to be a
difference if your editor was configured to display tabs as,
say, 4 spaces. But Python's understanding of it would not
change.
Andrew
da...@dalkescientific.com
Dang Griffith
..
> What about immutable strings? I'm not sure I understand Guido's
> preference for them.
..
Strings are also immutable in Java. Maybe Guido likes Java? ;-)
--dang
Aahz
Dang Griffith <dmgriff...@tasc.com> wrote:
>Anthony_Barker wrote:
>>
>> What about immutable strings? I'm not sure I understand Guido's
>> preference for them.
>
Python predates Java.
--
Aahz (aa...@pythoncraft.com) <*> http://www.pythoncraft.com/
This is Python. We don't care much about theory, except where it intersects
with useful practice. --Aahz
enoch
> I have been reading a book about the evolution of the Basic
> programming language. The author states that Basic - particularly
> Microsoft's version is full of compromises which crept in along the
> language's 30+ year evolution.
>
> What to you think python largest compromises are?
Its non existant SMP scalability.
If you try to sell clients somewhat bigger server apps, they don't
want to hear that having these run on a SMP system might (and will)
actually _hurt_ performance without special administrative
interference (processor binding), which isn't even possible on some
older operating systems.
We seem to be more or less at the end concerning ramping up single
processor speeds, and SMP like hardware becomes more and more
ubiquitous, e.g. Intel's Hyperthreading, IBM's SMP-on-a-chip, Sun's
"Throughput Computing".
As far as I know, there seems to be no interest to get rid of the GIL.
To do this might be a big technical problem, but I fear if it stays
the way it is, the actual trend in the hardware industry might work
against python (in server apps) big time.
sism...@hebmex.com
> Sent: Martes, 05 de Agosto de 2003 08:05 p.m.
>
> In article <bgp1e7$i3g$1...@gateway.northgrum.com>,
> Dang Griffith <dmgriff...@tasc.com> wrote:
> >Anthony_Barker wrote:
> >>
> >> What about immutable strings? I'm not sure I understand Guido's
> >> preference for them.
> >
> > Strings are also immutable in Java. Maybe Guido likes Java? ;-)
>
> Python predates Java.
>
Yeah, snakes drink coffee too.
;-)
-gustavo
Advertencia:La informacion contenida en este mensaje es confidencial y
restringida, por lo tanto esta destinada unicamente para el uso de la
persona arriba indicada, se le notifica que esta prohibida la difusion de
este mensaje. Si ha recibido este mensaje por error, o si hay problemas en
la transmision, favor de comunicarse con el remitente. Gracias.
Aahz
enoch <en...@gmx.net> wrote:
>anthony...@hotmail.com (Anthony_Barker) wrote in message news:<899f842.03073...@posting.google.com>...
>>
>
>Its non existant SMP scalability.
Would you care to back up your claim with some actual evidence?
(Yes, there are issues with Python on SMP machines, but to call Python's
built-in threading "non-existent SMP scalability" is either a lie or
revelatory of near-complete ignorance. That doesn't even count the
various IPC mechanisms.)
Robin Becker
writes
....
>
>(Yes, there are issues with Python on SMP machines, but to call Python's
>built-in threading "non-existent SMP scalability" is either a lie or
>revelatory of near-complete ignorance. That doesn't even count the
>various IPC mechanisms.)
prefer either java or c/c++, I suspect that those schemes aren't really
using threads in main, after all they seem to be running between
machines in different parts of the world even. I suspect Python would be
in better shape if we could migrate threads or tasklets from one
processor to another.
I believe pyro can almost do that, but I haven't tried it.
--
Robin Becker
Gerrit Holl
> Yeah, snakes drink coffee too.
Both for Python and Java is true that it is unclear when in emerged.
Pythons have existed for a looooooooong time, so has Jawa. But only
since Mankind has emerged they are called as such. I think Pythons
were known earlier in the west. Do Pythons live on Jawa? I don't know...
Are Pearls and Ruby's organic? I don't think so - which must mean that
they are even older.
C is definetely newer because it requires the alphabet.
Pascal is even newer, he lived very recently in this context.
Gerrit.
--
190. If a man does not maintain a child that he has adopted as a son
and reared with his other children, then his adopted son may return to his
father's house.
-- 1780 BC, Hammurabi, Code of Law
--
Asperger Syndroom - een persoonlijke benadering:
http://people.nl.linux.org/~gerrit/
Het zijn tijden om je zelf met politiek te bemoeien:
http://www.sp.nl/
Syver Enstad
> (Yes, there are issues with Python on SMP machines, but to call
> Python's built-in threading "non-existent SMP scalability" is either
> a lie or revelatory of near-complete ignorance. That doesn't even
> count the various IPC mechanisms.)
It's an interesting subject though. How does python threading on SMP
machines compare with f.ex. Java and C++. I know that at least the
MSVC compiler has a GIL like problem with heap access (new, malloc,
delete, free), which is guarded with a global lock.
Would migrating the global data for a thread to some sort of thread
local storage help Python SMP performance? If Java has better
threading performance than Python how have they solved the interpreter
state problem. Java is interpreted isn't it?
--
Vennlig hilsen
Syver Enstad
Syver Enstad
> Are Pearls and Ruby's organic? I don't think so - which must mean
> that they are even older.
I'll agree with Ruby's but perls are made my shells. Huh, I didn't
get that joke before I wrote it.
enoch
> In article <ad02da8c.03080...@posting.google.com>,
> enoch <en...@gmx.net> wrote:
> >anthony...@hotmail.com (Anthony_Barker) wrote in message news:<899f842.03073...@posting.google.com>...
> >>
> >> What to you think python largest compromises are?
> >
> >Its non existant SMP scalability.
>
> Would you care to back up your claim with some actual evidence?
>
> (Yes, there are issues with Python on SMP machines, but to call Python's
> built-in threading "non-existent SMP scalability" is either a lie or
> revelatory of near-complete ignorance.
Ok, I confess, the term you cited might be little bit exaggerated. But
there's no need to get personal. I'm surely not a liar (w.r.t. to this
thread, everything else is not a matter of public concern ;) ). The
ignorance part, well, we can talk about that ...
> That doesn't even count the various IPC mechanisms.)
Correct me if I'm wrong, but I don't think any form of IPC is a
measurement of scalability of something like the python interpreter.
Here are some sources which show that I'm not alone with my assessment
that python has deficiencies w.r.t. SMP systems:
http://www.python.org/pycon/papers/deferex/
"""
It is optimal, however, to avoid requiring threads for any part of a
framework. Threading has a significant cost, especially in Python. The
global interpreter lock destroys any performance benefit that
threading may yield on SMP systems, [...]
"""
http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=utf-8&safe=off&selm=bekg96%24mi%241%40panix2.panix.com
(note the author of that post)
"""
>My project will be running on an SMP box and requires scalability.
>However, my test shows that Python threading has very poor
performance
>in terms of scaling. In fact it doesn't scale at all.
That's true for pure Python code.
"""
I'm aware that you know quite well about these facts, so I'll leave it
at that. But let me just add one more link which maybe you don't know:
http://www.zope.org/Members/glpb/solaris/multiproc
"""
Well, in worst case, it can actually give you performance UNDER 1X.
The latency switching the GIL between CPUs comes right off your
ability to do work in a quanta. If you have a 1 gigahertz machine
capable of doing 12,000 pystones of work, and it takes 50 milliseconds
to switch the GIL(I dont know how long it takes, this is an example)
you would lose 5% of your peak performance for *EACH* GIL switch.
Setting sys.setchechinterval(240) will still yield the GIL 50 times a
second. If the GIL actually migrates only 10% of the time its
released, that would 50 * .1 * 5% = 25% performance loss. The cost
to switch the GIL is going to vary, but will probably range between .1
and .9 time quantas (scheduler time intervals) and a typical time
quanta is 5 to 10ms.
[...]
However, I have directly observed a 30% penalty under MP constraints
when the sys.setcheckinterval value was too low (and there was too
much GIL thrashing).
"""
So, although python is capable of taking advantage of SMP systems
under certain circumstances (I/O bound systems etc. etc.), there are
real world situations where python's performance is _hurt_ by running
on a SMP system.
Btw. I think even IPC might not help you there, because the different
processes might bounce betweeen CPUs, so only processor binding might
help.
I did quite a bit of googling on this problem - several times -
because I'm selling zope solutions. Sometimes, the client wants to run
the solution on an existing SMP system, and worse, the system has to
fulfill some performance requirements. Then I have the problem of
explaining to him that his admins need to undertake some special tasks
in order for zope to be able to exploit the multiple procs in his
system.
Aazh, I'm lurking this newsgroup since approx. 3 years, so I know who
you are. You have participated in nearly any discussion about threads,
I know your slides, and there's no doubt that you have forgotten more
about this subject than I'll never know.
Alex Martelli
>> > Python predates Java.
>> Yeah, snakes drink coffee too.
>
> Both for Python and Java is true that it is unclear when in emerged.
> Pythons have existed for a looooooooong time, so has Jawa. But only
Not really. What existed (in Ethiopia, only) as a wild plant variant of
coffee, before man got to it, has by now a rather thin relation with the
widespread coffee plants -- and it didn't grow on the island of Java,
nor anywhere else in Indonesia, only in a small corner of Ethiopia.
Generally, man has had a huge influence on plants and animals it
raises, a much thinner one on those it doesn't; often it makes sense
to consider different species for the agriculturally or pastorally
"enhanced" (from man's viewpoint) variants, with respect to the
preexistent wild "cousins". Thus, I would suggest "Java" didn't
exist before (at most) about 1000 years ago -- while phythons most
assuredly did.
> Pascal is even newer, he lived very recently in this context.
"Ada", on the other hand, being the start of "Adam", might claim
older genesis.
"C sharp" presumably requires musical instruments on a well-
tempered scale -- if so, then it seems to me it should be even
newer than Pascal.
Alex
Carl Banks
> "Ada", on the other hand, being the start of "Adam", might claim
> older genesis.
I disagree. Ada refers to a specific person, Ada Lovelace, who lived
in the 19th century, and was credited as the world's first programmer.
Python, Pascal, Java, and C (which goes back to the Etruscans, at
least) all predate Ada.
--
CARL BANKS
"You don't run Microsoft Windows. Microsoft Windows runs you."
Aahz
enoch <en...@gmx.net> wrote:
>aa...@pythoncraft.com (Aahz) wrote in message news:<bgr96h$a9$1...@panix3.panix.com>...
>> In article <ad02da8c.03080...@posting.google.com>,
>> enoch <en...@gmx.net> wrote:
>>>anthony...@hotmail.com (Anthony_Barker) wrote in message news:<899f842.03073...@posting.google.com>...
>>>>
>>>> What to you think python largest compromises are?
>>>
>>>Its non existant SMP scalability.
>>
>> Would you care to back up your claim with some actual evidence?
>>
>> (Yes, there are issues with Python on SMP machines, but to call Python's
>> built-in threading "non-existent SMP scalability" is either a lie or
>> revelatory of near-complete ignorance.
>
>Ok, I confess, the term you cited might be little bit exaggerated. But
>there's no need to get personal. I'm surely not a liar (w.r.t. to this
>thread, everything else is not a matter of public concern ;) ). The
>ignorance part, well, we can talk about that ...
Since, as you say, you've done some research, that's why I flamed you.
There's just no call for making such an overstated claim -- it is *NOT*
"a little bit exaggerated".
>> That doesn't even count the various IPC mechanisms.)
>
>Correct me if I'm wrong, but I don't think any form of IPC is a
>measurement of scalability of something like the python interpreter.
Depends what your context is. If you're restricting yourself to
discussing the Python interpreter, that's correct; if you're discussing
ways to boost the performance of a Python application on an SMP box,
it's entirely appropriate.
>Here are some sources which show that I'm not alone with my assessment
>that python has deficiencies w.r.t. SMP systems:
That I won't argue. But Python's approach also has some benefits even
on SMP systems. And if you choose a multi-process approach, the same
advantages that accrue to Python's approach on a single-CPU box apply
just as much to an SMP system.
>http://www.python.org/pycon/papers/deferex/
>"""
>It is optimal, however, to avoid requiring threads for any part of a
>framework. Threading has a significant cost, especially in Python. The
>global interpreter lock destroys any performance benefit that
>threading may yield on SMP systems, [...]
>"""
Just because it's a published PyCon paper doesn't mean that it's correct.
The multi-threaded spider that I use as my example is a toy version of a
spider that was used on an SMP box. (That's why I became a threading
expert in the first place -- Tim Peters probably remembers me pestering
him with questions four years ago. ;-) I guarantee you that SMP made
that spider much faster.
>So, although python is capable of taking advantage of SMP systems
>under certain circumstances (I/O bound systems etc. etc.), there are
>real world situations where python's performance is _hurt_ by running
>on a SMP system.
Absolutely. But that's true of any system with threading that isn't
designed and tuned for the needs of a specific application. Python
trades performance in some situations for a clean and simple model of
threading.
>Btw. I think even IPC might not help you there, because the different
>processes might bounce betweeen CPUs, so only processor binding might
>help.
My understanding that most OSes are designed to avoid this; I'd be
interested in seeing some information if I'm wrong. In any event, I do
know that IPC speeds things up in real-world applications on SMP boxes.
>I did quite a bit of googling on this problem - several times -
>because I'm selling zope solutions. Sometimes, the client wants to run
>the solution on an existing SMP system, and worse, the system has to
>fulfill some performance requirements. Then I have the problem of
>explaining to him that his admins need to undertake some special tasks
>in order for zope to be able to exploit the multiple procs in his
>system.
Even if Zope is the 800-pound gorilla of the Python world, Python isn't
going to change just for Zope. If you want to talk about ways of
improving Zope's performance on SMP boxes, I'll be glad to contribute
what I can. But spreading false information isn't the way to get me
interested.
Keep in mind that one reason IPC has gained popularity is because it
scales more than threading does, in the end. Blade servers are cheaper
than big SMP boxes, and IPC works across multiple computers.
Aahz
Syver Enstad <syver-e...@online.no> wrote:
>aa...@pythoncraft.com (Aahz) writes:
>>
>> (Yes, there are issues with Python on SMP machines, but to call
>> Python's built-in threading "non-existent SMP scalability" is either
>> a lie or revelatory of near-complete ignorance. That doesn't even
>> count the various IPC mechanisms.)
>
>It's an interesting subject though. How does python threading on SMP
>machines compare with f.ex. Java and C++. I know that at least the
>MSVC compiler has a GIL like problem with heap access (new, malloc,
>delete, free), which is guarded with a global lock.
Sure, but that's not where a C++ application usually spends its time.
>Would migrating the global data for a thread to some sort of thread
>local storage help Python SMP performance? If Java has better
>threading performance than Python how have they solved the interpreter
>state problem. Java is interpreted isn't it?
Well, that's a good question. *Does* Java have better threading
performance than Python? If it does, to what extent is that performance
bought at the cost of complexity for the programmer?
Keep in mind that the GIL exists not because of issues with thread-local
storage but because every Python object is global and can have bindings
to it in any -- or every -- thread. Python uses objects *everywhere*;
the GC uses Python objects, stack frames are Python objects, modules are
Python objects. To create "thread-local" storage as you suggest would
require a wholesale revision of Python's object model that would make it
something other than what Python is today.
Based on recent discussions about restricted execution, I suspect that
security would be much more likely to drive such changes; if that
happens, perhaps revisiting the way GIL works might happen with it.
Andrew Dalke
> Would you care to back up your claim with some actual evidence?
Aaaz? What do you define as SMP scalability?
For me it's that when my program runs on an SMP machine then
I should expect non-trivial performance advantages. (Excepting
the effects of othe programs on the system.)
This can mean many things. For example, a FORTRAN compiler
might parallelize array operations even though to the programmer
the FORTAN code is not threaded. Still, for most people the
idea of "SMP scalability" is the ability to have separate threads
run at the same time on different CPUs.
Which doesn't work well for Python if all the threads are busy
running native code, due to the GIL.
If the threads are usually blocked on I/O then the GIL is
released. Your own essay on a multithreaded spider shows
you know this well. And for this case Python does have good
SMP support, so enoch's complaint about its "non existant" is
overstated, but still true in his context of writing a large server
app.
> (Yes, there are issues with Python on SMP machines, but to call Python's
> built-in threading "non-existent SMP scalability" is either a lie or
> revelatory of near-complete ignorance. That doesn't even count the
> various IPC mechanisms.)
Given that it is a problem (and one I've had to deal with), calling
it an "issue" seems too weak, while dissing enoch this way is way
too strong.
There are IPC mechanisms, but if that's your definitions of SMP
scalability then any language with socket support (or bindings
for PVM/MPI/CORBA/Linda/...) is SMP scalable, which kinda
ruins the usefulness of that phrase.
I would say Python's SMP scalability is poor for Python-bound
threads. Better would be a language where I can share objects
between free-running SMP threads, as with Greg Stein's
experimental patches years ago. Not-as-good are ones which
let me pass data through shared memory, as POSH, and worse
are those which assume pickles always work (I use some
extensions with unpicklable objects). Even worse (for my
development style) are those that require working through some
IDL.
I also am developing a heavy web app and want scalability
on SMP machines, and I also bemoan Python's poor threading
applicability for my case. I'll work around it with some IPC
mechanism, but it is a workaround.
(OTOH, that means bugs in the extensions won't crash the
server.)
Andrew
da...@dalkescientific.com
enoch
> <snip>
> Since, as you say, you've done some research, that's why I flamed you.
> There's just no call for making such an overstated claim -- it is *NOT*
> "a little bit exaggerated".
Well, I based this phrase on the fact that while under some
circumstances (e.g. your web spider) python does scale somewhat, under
others (e.g. zope) it may perform even worse on a SMP system. If you
sum these two facts up ...
> <snip IPC>
> >Here are some sources which show that I'm not alone with my assessment
> >that python has deficiencies w.r.t. SMP systems:
>
> That I won't argue. But Python's approach also has some benefits even
> on SMP systems. And if you choose a multi-process approach, the same
> advantages that accrue to Python's approach on a single-CPU box apply
> just as much to an SMP system.
Yes, and these advantages also include a simpler threading model, as
far as I understand it, on every system. It's a compromise, that's why
I posted in this thread.
>
> >http://www.python.org/pycon/papers/deferex/
> >"""
> >It is optimal, however, to avoid requiring threads for any part of a
> >framework. Threading has a significant cost, especially in Python. The
> >global interpreter lock destroys any performance benefit that
> >threading may yield on SMP systems, [...]
> >"""
>
> Just because it's a published PyCon paper doesn't mean that it's correct.
> The multi-threaded spider that I use as my example is a toy version of a
> spider that was used on an SMP box. (That's why I became a threading
> expert in the first place -- Tim Peters probably remembers me pestering
> him with questions four years ago. ;-) I guarantee you that SMP made
> that spider much faster.
But how big is the significance of software which has the same
characteristics as your web spider example versus application servers?
> >So, although python is capable of taking advantage of SMP systems
> >under certain circumstances (I/O bound systems etc. etc.), there are
> >real world situations where python's performance is _hurt_ by running
> >on a SMP system.
>
> Absolutely. But that's true of any system with threading that isn't
> designed and tuned for the needs of a specific application. Python
> trades performance in some situations for a clean and simple model of
> threading.
Again, the compromise we were talking about. I'm not in a position to
weigh the pros and cons of it against each other, but I think I can
point out some cons of the current approach. I'm not doing that to
spread FUD, but to give an outsiders perspective on what I think might
hurt python in the future, and I want python to thrive because I like
using it alot.
> >Btw. I think even IPC might not help you there, because the different
> >processes might bounce betweeen CPUs, so only processor binding might
> >help.
>
> My understanding that most OSes are designed to avoid this; I'd be
> interested in seeing some information if I'm wrong. In any event, I do
> know that IPC speeds things up in real-world applications on SMP boxes.
For example, there are always lots of discussions about CPU affinity
on linux-kernel, and it seems to be a hard problem. Hyperthreading and
other non-symmetric architectures make this problem even harder.
Add to that the problem of the GIL getting shuffled around and you
have a system where you'll have trouble to predict the performance
characteristics. Admins don't like that. Though, it's not like there
are no problems without the GIL, it just adds to the complication.
> >I did quite a bit of googling on this problem - several times -
> >because I'm selling zope solutions. Sometimes, the client wants to run
> >the solution on an existing SMP system, and worse, the system has to
> >fulfill some performance requirements. Then I have the problem of
> >explaining to him that his admins need to undertake some special tasks
> >in order for zope to be able to exploit the multiple procs in his
> >system.
>
> Even if Zope is the 800-pound gorilla of the Python world, Python isn't
> going to change just for Zope. If you want to talk about ways of
> improving Zope's performance on SMP boxes, I'll be glad to contribute
> what I can. But spreading false information isn't the way to get me
> interested.
I wasn't even aware that zope is the "800-pound gorilla" of the python
world. I used it just as an example for a typical larger server app,
because, well, I know it.
incidentally, the pycon paper above, which you seem to dismiss as
false, is also from a guy which is working on a larger server app.
Maybe there's a pattern?
> Keep in mind that one reason IPC has gained popularity is because it
> scales more than threading does, in the end. Blade servers are cheaper
> than big SMP boxes, and IPC works across multiple computers.
Allow me some comment of the nature of this discussion (python and SMP
in general, not just this thread). I've seen it before and the
ingredients are:
- a major open source project
- developers which love this project
- some "outsider" which points out some perceived deficiency of said
project
- said developers pointing out (rightly or wrongly) reasons why this
deficiency doesn't matter, or that there are other (better) ways for
the "outsider" to achieve what he wants
In most cases this discussion then develops in to a big fat flamewar
;).
Two examples are linux and its threading capabilities, and mysql and
ACID compliancy.
A nice quote from the linux discussion btw. was from Alan Cox:
"A Computer is a state machine. Threads are for people who can't
program state machines."
But today, linux' thread support is magnitudes better than it was.
You wrote in another message in this thread:
> Well, that's a good question. *Does* Java have better threading
> performance than Python? If it does, to what extent is that performance
> bought at the cost of complexity for the programmer?
While I can't comment on the second question, here's an article which
sheds some light on the SMP scalability of an older java JDK, the meat
is on the third page:
http://www.javaworld.com/javaworld/jw-08-2000/jw-0811-threadscale.html
Seems that java does indeed have better threading performance than
python.
Bengt Richter
>Alex Martelli wrote:
>> "Ada", on the other hand, being the start of "Adam", might claim
>> older genesis.
>
>I disagree. Ada refers to a specific person, Ada Lovelace, who lived
>in the 19th century, and was credited as the world's first programmer.
>Python, Pascal, Java, and C (which goes back to the Etruscans, at
>least) all predate Ada.
>
The world's first programmer was Eve, when she gave Adam a list of things to do ;-)
Real time programming emerged when she told him to what to do when, in order to
get all the food served hot at the same time ;-)
Regards,
Bengt Richter
Cliff Wells
> The world's first programmer was Eve, when she gave Adam a list of things to do ;-)
> Real time programming emerged when she told him to what to do when, in order to
> get all the food served hot at the same time ;-)
And choosing Apple was a mistake even back then <wink>
Regards,
Cliff
--
People climbing up the walls, breaking all of my wretched dolls
Fingernails they scratch outside, in the attic is where I'll hide
-Switchblade Symphony
Matej Cepl
> Pythons have existed for a looooooooong time, so has Jawa. But only
But Jawa is neither programming language nor coffee, but trademark of
motorcycles from my native Czech Republic :-).
Matej
--
Matej Cepl,
GPG Finger: 89EF 4BC6 288A BF43 1BAB 25C3 E09F EF25 D964 84AC
138 Highland Ave. #10, Somerville, Ma 02143, (617) 623-1488
Irmen de Jong
> I'm not an expert, but the various grid computation schemes seem to
> prefer either java or c/c++, I suspect that those schemes aren't really
> using threads in main, after all they seem to be running between
> machines in different parts of the world even. I suspect Python would be
> in better shape if we could migrate threads or tasklets from one
> processor to another.
>
> I believe pyro can almost do that, but I haven't tried it.
Could you please elaborate on this a bit?
What exactly did you have in mind when talking about
"migrating threads or tasklets" ?
Does this involve transporting code across nodes,
or only the 'execution' (and data)?
Pyro supports transporting code, but with a few important limitations,
such as "once loaded, not reloaded".
--Irmen de Jong
Robin Becker
<irmen@-NOSPAM-REMOVETHIS-xs4all.nl> writes
>Robin Becker wrote:
>
.....
>> I believe pyro can almost do that, but I haven't tried it.
>
>Could you please elaborate on this a bit?
>What exactly did you have in mind when talking about
>"migrating threads or tasklets" ?
>
Well I had in mind the grid concept, which I believe implies the
distribution of code to multiple nodes and then the ability to execute
on them (I suppose that includes re-sending data to already distributed
instances).
I imagine that a proper grid would allow reloading of modules as the
overall application requires, but that would be relatively trivial if we
could capture 'execution state'.
Moving a running thread to another process would be fairly hard I
imagine, but I guess that's what we want for load balancing etc.
>Does this involve transporting code across nodes,
>or only the 'execution' (and data)?
>
>Pyro supports transporting code, but with a few important limitations,
>such as "once loaded, not reloaded".
>
>--Irmen de Jong
>
--
Robin Becker