A better self
David LeBlanc
So, why not make a leading '.' mean self? AFAIK, it's otherwise currently
illegal. The only drawback that I can think of is that it's less visible as
a first argument for class defs.
class foo():
def __init__(., me, me2, me3)
.a = me
.b = me2
.c = me3
def fee(.)
print .a, .b, .c
def bar(.)
.fee()
What I like about it is that it's resonant with the use of '.' to denote
members/components of modules (i.e: sys.path) and suggests that the
identifier is a member of the local "capsule" (or class scope if you
prefer). I see it as answering the needs of both camps: it's easy to typy,
unobtrusive and yet supports the "belt and suspenders" crowd who are not
comfortable with an implied self.
Before you all react (programmers can be SO conservative), please think
about it!
David LeBlanc
Seattle, WA USA
Erik Max Francis
> I notice that in every discussion of self, there's always a '.' in
> there.
> So, why not make a leading '.' mean self? AFAIK, it's otherwise
> currently
> illegal. The only drawback that I can think of is that it's less
> visible as
> a first argument for class defs.
>
> class foo():
> def __init__(., me, me2, me3)
> .a = me
> .b = me2
> .c = me3
> def fee(.)
> print .a, .b, .c
> def bar(.)
> .fee()
This doesn't totally revolt me, but I would prefer that the "self"
argument still be an explicit self:
def __init__(self, me, me2, me3) ...
so that the notation `.attribute' is just a shortcut for
`self.attribute'. Having the "self" argument written as . suggests that
accessing an attribute in it should involve an extra space, e.g.
`..attribute' which is clearly not what you meant.
`self' as the name of that first argument is already so entrenched it's
really hard to come up with reasonable arguments about replacing it.
--
Erik Max Francis / m...@alcyone.com / http://www.alcyone.com/max/
__ San Jose, CA, US / 37 20 N 121 53 W / ICQ16063900 / &tSftDotIotE
/ \ See the son in your bad day / Smell the flowers in the valley
\__/ Chante Moore
Bosskey.net: Aliens vs. Predator 2 / http://www.bosskey.net/avp2/
A personal guide to Aliens vs. Predator 2.
Sean 'Shaleh' Perry
> Before you all react (programmers can be SO conservative), please think
> about it!
>
I have played with ideas like this in python and other languages, and witnessed
others do the same.
.foo is REALLY hard to see if the font is not just right.
.size = .r + .l
is even worse.
I am not 100% against the idea as an idea, but the actual usage is not all that
fun.
David LeBlanc
the identifier most often used for the first argument.
This is essentially the same as the posted example:
class foo:
def __init__(this, me, me2, me3):
this.a = me
this.b = me2
this.c = me3
def bar(this):
print this.a, this.b, this.c
b = foo("ba", "bb", "bc")
b.bar()
I have no problem with "self" (or whatever) being equivelent to a leading
'.', but '..' would be confused with "up directory" syntax of U*x - although
it's tempting to suggest it as an alternative for global <grin>.
I'm not sure what you meant by ".attribute should have an extra space...".
David LeBlanc
Seattle, WA USA
Erik Max Francis
> Except, of course, "self" is not the name of the first argument, it's
> merely
> the identifier most often used for the first argument.
Sure, my suggestion was not meant to give any preference to `self' as
the Magical Self Argument*; the `.attribute' notation would simply be a
shortcut for `(first variable in argument list).attribute'.
Again, I'm not particularly fond of the proposal, although it certainly
is better than introducing some new punctuation pattern (I've seen
`@attribute' suggested for the same thing), but it doesn't seem right to
me to pretend that . is a legal name and then pretend like it isn't.
> I'm not sure what you meant by ".attribute should have an extra
> space...".
Thinko; I meant to say an extra _dot_, although the example which
followed probably got my point across anyway: If `.' is the self
argument, then its attribute must be referenced by `..attribute', not
`.attribute'.
.
* Perversity of choosing to name it something other than `self' aside, I
seem to recall some official Python documentation somewhere suggesting
that using a name other than `self' is not supported behavior (i.e., a
warning that at some future time its name of `self' might be enforced),
but a few furtive searches from time to time have failed to turn up what
I might have been remembering. Does that ring any bells with anybody?
Greg Ewing
> so that the notation `.attribute' is just a shortcut for
> `self.attribute'.
Or it could be a shorthand for <whatever_the_first_arg_is>.attribute,
so you could still call the first arg something else.
But I tend to agree that a leading "." is a bit
*too* unobtrusive!
--
Greg Ewing, Computer Science Dept,
University of Canterbury,
Christchurch, New Zealand
http://www.cosc.canterbury.ac.nz/~greg
Matt Gerrans
> '.', but '..' would be confused with "up directory" syntax of U*x -
although
> it's tempting to suggest it as an alternative for global <grin>.
In keeping with that concept, '..' would be referring to the base class
(equivalent to java's "super"). Or would that be mixing metaphors? (in
Ruby, mixin metaphors).
Erik Max Francis
> In keeping with that concept, '..' would be referring to the base
> class
> (equivalent to java's "super").
I think that's definitely mixing metaphors. For one thing, which base
class? Python supports multiple inheritance.
Erik Max Francis
> Or it could be a shorthand for <whatever_the_first_arg_is>.attribute,
> so you could still call the first arg something else.
Right, that's actually what I intended to get across, though obviously I
didn't do a very good job. (I wouldn't do anything so perverse as name
the self argument anything other than `self'.)
> But I tend to agree that a leading "." is a bit
> *too* unobtrusive!
It doesn't bother me too bad. I wouldn't mind it being done, but I
wouldn't fight for it either.
Delaney, Timothy
>
> Matt Gerrans wrote:
>
> > In keeping with that concept, '..' would be referring to the base
> > class
> > (equivalent to java's "super").
>
> I think that's definitely mixing metaphors. For one thing, which base
> class? Python supports multiple inheritance.
Well, obviously it would only be available for new-style classes, and the
following would be equivalent ...
# current
class A (object):
def __init__(self)
super(A, self).__init__()
class B (object):
def __init__(self)
super(B, self).__init__()
class C (A, B):
def __init__(self)
super(C, self).__init__()
# new
class A (object):
def __init__(self)
..__init__()
class B (object):
def __init__(self)
..__init__()
class C (A, B):
def __init__(self)
..__init__()
thus getting rid of the need to do a self.__super = super(cls, self) or use
a metaclass for this ...
;)
Tim Delaney
Duncan Booth
news:3D2BD37D...@alcyone.com:
> Matt Gerrans wrote:
>
>> In keeping with that concept, '..' would be referring to the base
>> class
>> (equivalent to java's "super").
>
> I think that's definitely mixing metaphors. For one thing, which base
> class? Python supports multiple inheritance.
>
Python already has a 'super' which works properly with multiple
inheritance, so presumably it would be a shorthand for super with
appropriate arguments rather than referring to a specific base class.
--
Duncan Booth dun...@rcp.co.uk
int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3"
"\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?
Bo M. Maryniuck
> .size = .r + .l
> is even worse.
I still can't catch why people raise and reply to this really nonsense thread
("self" vs. "dot" or "this" or "pumpkin" or whatever)? Just go and design +
develope your own language... This is more worse example, if somebody like
use a dot, instead "self":
.__.foo = .o
.__.__doc__ = .__.foo.__doc__
This is more like Morse language...
bo@oberon:(~) morse -s '.__.foo'
.-.-.-,
.-.-.-,
..-.,
---,
---,
What we will do if some Perl-hooked folk starts bite us "Use "->" instead
".", and use @$% symbols with variables":
.$__->$foo = .$o
.$__->$__doc__ = .$__->$foo->$__doc__
Damn!..
--
Sincerely yours, Bogdan M. Maryniuck
"Whip me. Beat me. Make me maintain AIX."
(By Stephan Zielinski)
Huaiyu Zhu
A better (and currently available) convention is
_.size = _.r + _.l
It has been mentioned several times in this thread. I've used it several
times in the past and liked it, esp in iterative algorithms. Some usage
patterns include
def __init__(_, x, y, z):
_.x, _.y, _.z = x, y, z
def process(_, x, y, z):
actions involving _.x, _.y, _.z, x, y, z
_.x, _.y, _.z = x, y, z
def update(_):
x, y, z = _.x, _.y, _.z
actions involving x, y, z
_.x, _.y, _.z = x, y, z
One could even think of _. as a magic persistence prefix that makes the
variable keep its value between method calls. It is also easy to replace
_.x with self.x or vice versa in a good editor.
Huaiyu
Gabe Newcomb
'self' is useful, we may as well stick with the word 'self' as is. It's
the explicit version--it makes things clear, and if anyone's really
dying to use the shortest identifiers possible, they might consider
Perl.
Yes that last statement was partially tongue-in-cheek, but seriously,
readability is worth preserving IMO.
Gabe
-----Original Message-----
From: Huaiyu Zhu [mailto:hua...@gauss.almadan.ibm.com]
Sent: Wednesday, July 10, 2002 10:01 AM
To: pytho...@python.org
Subject: Re: A better self
Huaiyu
--
http://mail.python.org/mailman/listinfo/python-list
Peter Hansen
>
> I still can't catch why people raise and reply to this really nonsense thread
> ("self" vs. "dot" or "this" or "pumpkin" or whatever)? Just go and design +
> develope your own language... This is more worse example, if somebody like
> use a dot, instead "self":
>
> .__.foo = .o
> .__.__doc__ = .__.foo.__doc__
>
> This is more like Morse language...
>
> bo@oberon:(~) morse -s '.__.foo'
> .-.-.-,
> .-.-.-,
> ..-.,
> ---,
> ---,
Actually, the Morse is much easier to write, since it doesn't
require the Shift key (god how I hate underscores) and easier
to read, since it's harder to tell how many underscores there
are in a series of them since fonts generally don't have any
space between them, unlike hyphens.
If this change were implemented, it would make Python less
readable and writable than Morse code, and I therefore nominate
it for honor of being the most damaging change at the smallest cost!
<_.7 wink>
-Peter
Louis M. Pecora
<pe...@engcorp.com> wrote:
> Actually, the Morse is much easier to write, since it doesn't
> require the Shift key (god how I hate underscores)
Just go and design + develope your own keyboard.
--
Lou Pecora
- My views are my own.
Huaiyu Zhu
>It seems to me that, given that most folks here agree the concept of
>'self' is useful, we may as well stick with the word 'self' as is. It's
>the explicit version--it makes things clear, and if anyone's really
>dying to use the shortest identifiers possible, they might consider
>Perl.
>
>Yes that last statement was partially tongue-in-cheek, but seriously,
>readability is worth preserving IMO.
Using self is indeed the most readable most of the time.
However, there are situations, mostly in numerical computations, where it is
necessary to have, say, a dozen variables in one expression, and a dozen
statements containing such expressions and some other statements in one
method. It is far more readable to have them all appear in one screen in a
form that resembles written equations on paper, than to dogmatically follow
a convention in a particular programming language. For anyone spending a
few weaks sorting out these equations and several days making sure that they
are correctly represented in the program, it is no big deal to notice that _
is used in place of self in this situation.
Just like the shortest programs are not always the most readable programs,
programs that follow a particular style convention (like using self) are not
always the most readable programs.
What is required is a convention that allows shorter notions when necessary
in a disciplined way. I find that switching from self to _ is quite
satisfactory for this purpose under the right circumstances. And, as such,
there is no need to extend Python's syntax for this purpose, which was the
initial topic of this thread.
Huaiyu
Louis M. Pecora
<hua...@gauss.almadan.ibm.com> wrote:
> Using self is indeed the most readable most of the time.
>
> However, there are situations, mostly in numerical computations, where it is
> necessary to have, say, a dozen variables in one expression, and a dozen
> statements containing such expressions and some other statements in one
> method. It is far more readable to have them all appear in one screen in a
> form that resembles written equations on paper, than to dogmatically follow
> a convention in a particular programming language.
[cut - remainder]
Huaiyu,
Nicely said.
Terry Reedy
"Huaiyu Zhu" <hua...@gauss.almadan.ibm.com> wrote in message
news:slrnairg48...@gauss.almadan.ibm.com...
> What is required is a convention that allows shorter notions when
necessary
> in a disciplined way. I find that switching from self to _ is quite
> satisfactory for this purpose under the right circumstances. And,
as such,
> there is no need to extend Python's syntax for this purpose, which
was the
> initial topic of this thread.
Another alternative is to copy instance vars to local vars and then
use the latter without the 'self.' or 'self.__' prefix. If there
are multiple references to the instance vars within a method, this may
be just as fast or faster. IE
def quadsolve(self):
a,b,c = self.a, self.b, self.c
discrim = b*b - a*c
...
Terry J. Reedy
Peter Hansen
>
> In article <3D2CEDA1...@engcorp.com>, Peter Hansen
> <pe...@engcorp.com> wrote:
>
> > Actually, the Morse is much easier to write, since it doesn't
> > require the Shift key (god how I hate underscores)
>
> Just go and design + develope your own keyboard.
Since you post without smileys, I can only assume you are serious?
To which my response is: you can't be serious. :-)
(Hint, the original was a joke... note the partial wink. I do
hate underscores, but concluding from that that my best option
is to design and develop my own keyboard is just bizarre.)
-Peter
Erik Max Francis
> Since you post without smileys, I can only assume you are serious?
Yes, that's right. Any post that doesn't contain a smiley is always
absolutely dead serious.
> To which my response is: you can't be serious. :-)
:-)
Peter Hansen
>
> Peter Hansen wrote:
>
> > Since you post without smileys, I can only assume you are serious?
>
> Yes, that's right. Any post that doesn't contain a smiley is always
> absolutely dead serious.
It always helps when the smileyless posts contain actual humour,
no matter how subtle. Like yours, unlike the other.
> > To which my response is: you can't be serious. :-)
>
> :-)
:-|
(The best attempt I can manage with my current keyboard at a
straight-faced stare, daring you to smile.)
-Peter
Tim Peters
> Since you post without smileys, I can only assume you are serious?
[Erik Max Francis]
> Yes, that's right. Any post that doesn't contain a smiley is always
> absolutely dead serious.
Yes, that is the rule. There's a subtler one, too: whenever you read a
post without a smiley, you're entitled to interpret its meaning in any way
that strikes your fancy, and its author must concede that your reading is
the intended one in all respects, and was probably intended with malice
aforethought too. There are no exceptions.
without-strong-rules-usenet-would-become-chaotic-ly y'rs - tim
Erik Max Francis
> It always helps when the smileyless posts contain actual humour,
> no matter how subtle. Like yours, unlike the other.
Sheesh. Even with a smiley, you still get pissy.
Peter Hansen
>
> Peter Hansen wrote:
>
> > It always helps when the smileyless posts contain actual humour,
> > no matter how subtle. Like yours, unlike the other.
>
> Sheesh. Even with a smiley, you still get pissy.
I didn't think *any* of the posts were pissy yet, except
Louis' comment that I ought to just develope (sic) my own
keyboard.
Strange how lack of face-to-face contact tends to destroy
the friendliness of a thread, and the posters.
Louis M. Pecora
<pe...@engcorp.com> wrote:
Yes, I should have added a smilely. :-) Sorry.
But my proposal is no more bizzare than yours suggesting we design our
own language or such.
Both proposals are in smilely land. I hope you agree. ;-)
Louis M. Pecora
<pe...@engcorp.com> wrote:
> Louis' comment that I ought to just develope (sic) my own
^^^^^^^^
Verbatim from the original. :-)
Robin Siebler
With Text1:
.Height = 100
.Width = 200
End With
Duncan Booth <dun...@NOSPAMrcp.co.uk> wrote in message news:<Xns92477232B31...@127.0.0.1>...
Andy Salnikov
"Louis M. Pecora" <pec...@anvil.nrl.navy.mil> wrote in message
news:110720021224320303%pec...@anvil.nrl.navy.mil...
> In article <3D2CEDA1...@engcorp.com>, Peter Hansen
> <pe...@engcorp.com> wrote:
>
> > Actually, the Morse is much easier to write, since it doesn't
> > require the Shift key (god how I hate underscores)
>
>
> Just go and design + develope your own keyboard.
>
Why bother, look here is one: http://anekdotov.net/pic/comp2/2404morze.jpg
It should have smileys on it, I bet :)
Andy.
Peter Hansen
No problem about the smileys, although you're mistaken about
the origin of the suggestion that someone design his own language.
It was Bo, to whom I was replying (humorously, I hoped) who
suggested that, not I. As it appears he was also the origin of
the 'develope' which I thought you first used, it seems we both
made the wrong attributions! :-)
Cheers,
-Peter
Tom Loredo
>
> However, there are situations, mostly in numerical computations, where it is
> necessary to have, say, a dozen variables in one expression, and a dozen
> statements containing such expressions and some other statements in one
> method. It is far more readable to have them all appear in one screen in a
> form that resembles written equations on paper, than to dogmatically follow
> a convention in a particular programming language.
Extremely well put.
I've been using Python almost exclusively for scientific programming in
the past 3 years or so. I've followed threads about many controversial
aspects of Python (additions or long-held characteristics), and I've
been impressed time after time with the choices of the development team.
Even when my first instinct has been to disagree with a choice or policy,
I have always come to see a great deal of wisdom in the final choice.
But the one thing that continues to irk me about the language is "self,"
for just the reasons Huaiyu describes. I think non-numerical programmers
are likely not in a position to appreciate the awkwardness of "self..."
in numerical settings, where in a many-variable expression it serves
to hinder understanding (by making the expression very long, often
broken across multiple lines) rather than to enhance it. Yet I think the
general arguments for "self" are strong and valid. Though I often wish there
were something like a "with" statement in such contexts, Huaiyu's
suggestion seems like a reasonable middle ground.
But if there were a "with" statement, I'd probably welcome it!
Cheers,
Tom Loredo
Max M
> But the one thing that continues to irk me about the language is "self,"
> for just the reasons Huaiyu describes. I think non-numerical programmers
> are likely not in a position to appreciate the awkwardness of "self..."
> in numerical settings, where in a many-variable expression it serves
> to hinder understanding (by making the expression very long, often
> broken across multiple lines) rather than to enhance it. Yet I think the
> general arguments for "self" are strong and valid. Though I often wish there
> were something like a "with" statement in such contexts, Huaiyu's
> suggestion seems like a reasonable middle ground.
I don't especially like "self" myself. But wouldn't it be easy enough to
just:
class ProjectScheduler:
pi = 3.14
def setEstimatedTime(hours):
self.estimatedTime = hours
def calculateRealisticTime(estimatedTime):
# rebinding vars to make the calculation easier on the eye
pi = self.pi
estimatedTime = self.estimatedTime
# the formula
realisticTime = estimatedTime * pi
# return result
return realisticTime
regards Max M
Erik Max Francis
> I don't especially like "self" myself. But wouldn't it be easy enough
> to
> just:
An obvious issue is what happens with respect to bound and unbound
methods. The self argument is often implicit in normal work, but it
_is_ there, and needs to be set up properly. In your scheme, what
happens when you write:
def f(x, y): ... # not a method, no self, implicit or explicit
class C:
def m(x, y): ... # method, implicit self
g = f # what does this mean?
c = C()
um = C.m
um(c, ...) # how do I call this?
bm = c.m
bm(...) # or this?
Trying to hide the self argument is destined to only lead to massive
confusion.
Max M
> Max M wrote:
>>I don't especially like "self" myself. But wouldn't it be easy enough
>>to
>>just:
> An obvious issue is what happens with respect to bound and unbound
> methods. The self argument is often implicit in normal work, but it
> _is_ there, and needs to be set up properly. In your scheme, what
> happens when you write:
Arh bugger. I don't disagree, and I made the typical typo that I often
do when creating classes.
I forgot to add self as the prameter. So my example should have looked like:
class ProjectScheduler:
pi = 3.14
def setEstimatedTime(self, hours):
self.estimatedTime = hours
def calculateRealisticTime(self, estimatedTime):
# rebinding vars to make the calculation easier on the eye
pi = self.pi
estimatedTime = self.estimatedTime
# the formula
realisticTime = estimatedTime * pi
# return result
return realisticTime
/me slapping himself on forehead repeatedly
regards Max M
Louis M. Pecora
stuff so I feel the pain of self. -- sheeze, that was almost Zen). But
I agree with Tom Loredo that there are good general arguments for self.
Many people have suggested doing something like the following (which is
reasonable) to eliminate self. from mathematical expressions:
t, x, y, z=self.t, self.x, self.y, self.z
result=sin(t)*x**y+sqrt(z) # A pretend formula
My question is how much overhead is added by the assignments in the top
line? I suspect not much at all, but then I do not know how the
interpreter handles these things.
-- Lou Pecora
In article <3D3724C0...@astro.cornell.edu>, Tom Loredo
Louis M. Pecora
> I forgot to add self as the prameter. So my example should have looked like:
I do that a lot, too. Sometimes I get pissed at myself sometimes I get
pissed at self. (Zen, again) But (and I love this about Python) the
interpreter gracefully stops and says it doesn't recogize variables or
functions (since the "self" is missing). I correct and in 10 sec am on
my way again.
Terry Reedy
> Many people have suggested doing something like the following (which
is
> reasonable) to eliminate self. from mathematical expressions:
>
> t, x, y, z=self.t, self.x, self.y, self.z
> result=sin(t)*x**y+sqrt(z) # A pretend formula
>
> My question is how much overhead is added by the assignments in the
top
> line? I suspect not much at all, but then I do not know how the
> interpreter handles these things.
You are correct. Reference to local variables is optimized to be fast
via array lookup*. If you use a variable at least twice (as in
quadratic formula, for example), you might actually gain by copying to
a local. Certainly, copying a global or builtin method to a local for
repeated use is a standard Python speedup trick.
Note that when you do the copying, you can also rename, thus allowing
both self-explanatory attribute names and short formulas in standard
form. Consider
def mortgage_pay(self):
i = self.interest_rate
n = 12*self.years
p = self.principal
return <standard formula using i, n, and p>
Terry J. Reedy
* IE, when "meth(self, a): b = self.attr; return (a+math.sin(b))/b" is
compiled names 'self', 'a', and 'b' become slots 0, 1, and 2 in a
behind-the-scenes localvars array. There is no runtime dict lookup as
there is for globals and self attributes.
Tom Loredo
>
> Many people have suggested doing something like the following (which is
> reasonable) to eliminate self. from mathematical expressions:
>
> t, x, y, z=self.t, self.x, self.y, self.z
> result=sin(t)*x**y+sqrt(z) # A pretend formula
Yes, this was suggested earlier. I don't consider it a great solution
to the problem. For example, in a current calculation I have a class
with many member variables and many member functions that use them
to return various quantities. In a language that boasts of how
simple and straightforward it is, and how it shrinks code size (by
omitting braces, declarations, etc.), it is rather silly to have
lines like:
t, x, y, z=self.t, self.x, self.y, self.z
in every one of many member functions, often with the same variables.
You look at that code, with all those redundant lines of negligible
content, and it yells out to you: something is missing here that
should make this silly copying unnecessary in order to have expressions
that are easy to read/understand.
The language is so great in other respects, and on a fundamental level
the arguments for "self." are so sound, that I just live with this.
But it sure is a nuisance.
-Tom Loredo
Louis M. Pecora
Terry Reedy <tjr...@udel.edu> wrote:
> Certainly, copying a global or builtin method to a local for
> repeated use is a standard Python speedup trick.
I learn something every day. We should have a collection of speedup
tricks listed somewhere.
Thanks.
Grant Griffin
>
>"Louis M. Pecora" wrote:
>>
>> Many people have suggested doing something like the following (which is
>> reasonable) to eliminate self. from mathematical expressions:
>>
>> t, x, y, z=self.t, self.x, self.y, self.z
>> result=sin(t)*x**y+sqrt(z) # A pretend formula
>
>Yes, this was suggested earlier. I don't consider it a great solution
>to the problem. For example, in a current calculation I have a class
>with many member variables and many member functions that use them
>to return various quantities. In a language that boasts of how
I don't remember any "boasting". Python is for the humble (except for maybe Tim
and Fredrik <wink>).
>simple and straightforward it is, and how it shrinks code size (by
>omitting braces, declarations, etc.), it is rather silly to have
>lines like:
>
> t, x, y, z=self.t, self.x, self.y, self.z
>
>in every one of many member functions, often with the same variables.
>You look at that code, with all those redundant lines of negligible
>content, and it yells out to you: something is missing here that
>should make this silly copying unnecessary in order to have expressions
>that are easy to read/understand.
>
>The language is so great in other respects, and on a fundamental level
>the arguments for "self." are so sound, that I just live with this.
>But it sure is a nuisance.
I was thinking about this just a few days ago. I had found a book about Ada on
our "unloved software books" shelf at work. (In fact, there were six copies of
it, so Ada must be pretty unloved <wink>.) I read a little of it, and it turns
out that Ada provides "with", which is the one-and-only construct I missed when
I moved from Pascal (Ada's daddy) to C many years ago.
For example:
with self:
result=sin(t)*x**y+sqrt(z) # A pretend formula
I don't know if there's a technical reason (in terms of either Python grammar or
C implementation) why this wouldn't work, but perhaps it's not in Python
because:
1) explicit is better than implicit
2) there should be one--and preferably only one--way to do things
or maybe even
3) it hasn't happened yet
(At one time, I used to think "augmented assignment" was left out of Python
because of 1) and 2), but it turned out to be 3). Silly me <wink>.)
any-construct-that's-in-pascal-and-ada-but-not-in-C
-can't-be-all-bad-<wink>-ly y'rs,
=g2
or, as we would say in Pascal:
:=g2
_________________________________________________________________________
Grant R. Griffin g...@dspguru.com
Publisher of dspGuru http://www.dspguru.com
Iowegian International Corporation http://www.iowegian.com
bru...@tbye.com
> > t, x, y, z=self.t, self.x, self.y, self.z
> > result=sin(t)*x**y+sqrt(z) # A pretend formula
[snip]
> to return various quantities. In a language that boasts of how
> simple and straightforward it is, and how it shrinks code size (by
> omitting braces, declarations, etc.),
[snip]
The 'boasts' of course apply in the general sense, and in the general
sense they are very true. From your perspective this is quite a nuisance
because the combination of the types of programs you write and your design
style make you encounter this problem often.
From your posts it sounds like you're trying to remain pretty positive
about this nuisance (good job!), but I hope you also realize that for a
lot of people writing a lot of programs there is no 'problem' - from my
perspective I hope that this whole 'self' thing never changes at all.
> in every one of many member functions, often with the same variables.
> You look at that code, with all those redundant lines of negligible
> content, and it yells out to you: something is missing here that
> should make this silly copying unnecessary in order to have expressions
> that are easy to read/understand.
Hehe... well, changing the language is of course only one of your options,
and probably the least likely to ever pan out, especially for the
aforementioned reason that lots of people and lots of code already use
and/or like the other way.
Every time this topic comes up I'm surprised at how desperately some
people want the language changed (not necessarily you, since you've
already said that it's not enough of a nuisance to abandon Python), and
I'm equally surprised that no combination of the following alternatives
are acceptable:
1) Learn to like it - I've spent more time reading about getting rid of
self than time actually typing 'self.' (i.e. the costs are minimal and it
makes my code more readable and explicit - it's worth it to me!)
2) Learn to just accept it (perhaps you find it annoying, but at least
it's not something you deal with on every line of code)
3) Use the smarts of your editor - macros, special syntax highlighting,
etc.
4) Use a shorter name than self, e.g. 's'
5) Use the x,y,z = self.x, self.y, self.z idiom
6) Have Python generate the function bodies for you, e.g:
class SomeClass:
method = MyCoolUnboundMethodMaker('arg:x = y*z + t*arg')
7) Change your design - if something is awkward it *may* be a flaw in the
language, but there's also a really, really good chance that a different
approach to your program would be best.
> The language is so great in other respects, and on a fundamental level
> the arguments for "self." are so sound, that I just live with this.
Fundamental, *and* practical, IMO.
-Dave
bru...@tbye.com
> > Certainly, copying a global or builtin method to a local for
> > repeated use is a standard Python speedup trick.
>
> I learn something every day. We should have a collection of speedup
> tricks listed somewhere.
We do, but we keep it fairly hidden because consulting the list too early
or too often is worse than not having a list. <0.5 wink>
-Dave
Peter Hansen
>
> In article <c9YZ8.176223$iX5.9...@bin3.nnrp.aus1.giganews.com>,
> Terry Reedy <tjr...@udel.edu> wrote:
>
> > Certainly, copying a global or builtin method to a local for
> > repeated use is a standard Python speedup trick.
>
> I learn something every day. We should have a collection of speedup
> tricks listed somewhere.
This text by Guido sort of already does, in the conclusion section:
http://www.python.org/doc/essays/list2str.html
-Peter
Bengt Richter
>I notice that in every discussion of self, there's always a '.' in there.
>So, why not make a leading '.' mean self? AFAIK, it's otherwise currently
>illegal. The only drawback that I can think of is that it's less visible as
>a first argument for class defs.
>
>class foo():
> def __init__(., me, me2, me3)
> .a = me
> .b = me2
> .c = me3
> def fee(.)
> print .a, .b, .c
> def bar(.)
> .fee()
>
>What I like about it is that it's resonant with the use of '.' to denote
>members/components of modules (i.e: sys.path) and suggests that the
>identifier is a member of the local "capsule" (or class scope if you
>prefer). I see it as answering the needs of both camps: it's easy to typy,
>unobtrusive and yet supports the "belt and suspenders" crowd who are not
>comfortable with an implied self.
>
>Before you all react (programmers can be SO conservative), please think
>about it!
>
Ok, I thought about it some ;-)
I think I may even have proposed or seconded virtually the same self-less-dot in the past,
so I'm not going to criticize that too much, but I think there is more to think about ;-)
ISTM that self in self.x is really a named name-space selector identifying a
list of "directories" to search in orderly fashion for the appended 'x',
or a default place to bind 'x' if not already bound.
A bare 'x' implies a search through another namespace, also a list of
"directories", depending on context. (By quoting I mean that some may
not literally be directories).
If these lists of directories could be exposed simply for purposes of declaration
and access, we might be able to do some interesting things.
Suppose 'namepath' were magically defined in all execution contexts to be
the list of directories to search for a bare name in that context (i.e.,
pretty much what dir() sees now). Hence namepath[0] would always be the local
namespace first accessed for a bare name, unless other factors apply.
In a method, 'self' would normally be found in namepath[0], and self.x would
be found in self.namepath[0] if it was an instance attribute, and self.namepath[1]
if it was in self.__class__.__dict__, etc.
Now if namepath were dynamically modifiable, you could get bare name access
to instance attributes as in this example:
class FOO:
def __init__(self, v): self.v = v # old style init
def myMethod(self):
x = 123 # default local
namepath.insert(0, self.namepath[0])
y = 456 # bare means self.y here!!
print v, x, y # found in namepath[0], namepath[1], and namepath[0] resp.
namepath.pop(0) # back to default
print self.v, x, self.y # duplicate previous output
An alternative (additional) way to spell the above namepath operations might be:
def myMethod(self):
x = 123 # default local
with self.namepath[0]: # [1]
y = 456
print v, x, y
print self.v, x, self.y # [2] duplicate previous output
[1] meaning namepath.insert(0, self.namepath[0]) until dedent.
A list of directories might also be allowed for "with ... :"
[2] dedent accomplishes namepath.pop(0) if it's a with-block # back to default
Note that namepath always appears locally as if namepath == namepath[0]['namepath']
This still leaves "global x" to be handled generally in terms of namepath.
IOW, a kind of private namepath.insert(0, namepath[-1]) for 'x' but not
all names, and no need to search beyond the one directory. The general problem
is to control where a bare 'x' will be re/bound for assignment. Perhaps a placement
function, e.g.,
place('x', namepath[-1]) # effect of "global x"
Thus the equivalent of the default for any bare name would be
place('name', namepath[0])
Given the above, we are perhaps in a position to write
place(('x','y'), namepath[1])
and expect to use it with nested scopes, e.g.,
def foo():
def init_xy():
place(('x','y'), namepath[1]) # force binding in immediately enclosing scope
f = file('blah') # local f goes to namepath[0] # just a weird example
x, y = map(int, f.readline().split()) # x and y bindings are created in enclosing scope
def bar(dx,dy):
with namepath[1]: # could/should use place
x += dx
y += dy
If you wanted to specify name-specific lookup search paths it could be like place:
lookup(('x','y'), namepath[1:], {'x': 'no NameError for x'}) # y will raise NameError if not found.
I am implying that default assignment goes to namepath[0] unless place() has defined
other places for the name in question, and that lookup by default does not just go from
local to global, but through lexically enclosing scopes in between (as found in the
default namepath list). Additionally, namepath can be modified dynamically, which will
control search for names, unless overridden for specific names by lookup().
Also, for a given statment, rebinding should occur in the same namepath directory where
a lookup succeeds for the same name, so that x += 1 will not create a local based on an 'x'
in an enclosing scope. However x = 1 would bind in namepath[0] unless overridden by
a place('x', somewhere). This would allow bar() above to leave out the "with namepath[1]:"
statement. However, place() might be better optimizable, since it would not change default
lookup for dx and dy.
For convenience, namepath(n) could present the relevant dictionary as a virtual "self" of
a virtual object whose attributes are stored in that directory. I.e.,
namepath(1).x = 123 would be equivalent to
namepath[1]['x'] = 123
which would bind x in the immediately enclosing scope.
This is hot OTTOMH, offered in the hope of stimulating useful and/or enjoyable discussion.
ISTM these issues are bound (non-Python sense ;-) to play an evolving role in Python given
nested scopes etc. Obviously a naive implementation of the above would have a lot of overhead.
OTOH, place() and lookup() might provide optimization opportunities.
Hm, a further thought: If place() had an optional keyword argument named 'prop',
we could write
place('p', myModule.namepath[0], prop=0) # ensure that p is not already a property at myModule.p
p = property(get_p, set_p) # myModule.p is property object itself here
place('p', myModule.namepath[0], prop=1) # myModule.p triggers property methods after this
or if we know p is ordinary or nonexistent, just
myModule.p = property(get_p, set_p) # myModule.p is property object itself here
place('p', myModule.namepath[0], prop=1) # myModule.p triggers property methods after this
and expect writing myModule.p to trigger the get_p or set_p functions. If you wanted to
rebind p in that directory, you would have to do another place('p', ... prop=0). One way
to do this, and keep ordinary lookups fast, might be to associate a flag byte with names
in dictionaries, so that it's a one-instruction matter to decide whether special processing
such as for properties is required. A C switch on this byte could also handle special-casing
for slots, I would imagine. The place() function would change bits in the flag byte.
With this methodology for properties, the magic actions could be triggered on any access
to a name via any specified directory, not just class attribute directories accessed via instances.
Sharp tools are dangerous, of course ;-)
I'm not sure whether tying property magic to name lookup is a better idea than tying the magic
to the looked-up object and its state independent of access path, but that is another discussion.
Regards,
Bengt Richter
Alex Martelli
...
>>to return various quantities. In a language that boasts of how
>
> I don't remember any "boasting". Python is for the humble (except for
> maybe Tim and Fredrik <wink>).
Nah, ALL of us bots are unhumble -- even I, the one programmed for
occasional human-like errors in order to better fool you all.
> with self:
> result=sin(t)*x**y+sqrt(z) # A pretend formula
>
> I don't know if there's a technical reason (in terms of either Python
> grammar or C implementation) why this wouldn't work, but perhaps it's not
If we adopted the Visual Basic variant of this, with a leading dot
to distinguish fields of the with-ed objects (as others have often
suggested), technical feasibility would be guaranteed.
Without such a tweak, it's something of a mess. Is this statement
computing a local variable result, or a field self.result, for
example? Either interpretation could be quite desirable, and any
choice between them seems rather arbitrary. Access to local
variables inside a with would be slow, since each time it would
have to be preceded by a lookup in the with-ed object. Etc, etc.
WITH the leading-dot convention, and some drastic simplification
about what happens on nested with (e.g.: all leading-dot names
always only refer to the innermost with -- there is just NO way
to refer to outer-nested withs), I think it would be feasible.
Desirable, I dunno...:
> any-construct-that's-in-pascal-and-ada-but-not-in-C
> -can't-be-all-bad-<wink>-ly y'rs,
yes but, if it's also in Visual Basic ... ???
Alex
Louis M. Pecora
In article <mailman.1027108242...@python.org>,
<bru...@tbye.com> wrote:
> 1) Learn to like it - I've spent more time reading about getting rid of
> self than time actually typing 'self.' (i.e. the costs are minimal and it
> makes my code more readable and explicit - it's worth it to me!)
Well, readability _is_ the issue in numerics and math apps.
> 2) Learn to just accept it (perhaps you find it annoying, but at least
> it's not something you deal with on every line of code)
True, but more lines than most programmers when you do numerics.
> 3) Use the smarts of your editor - macros, special syntax highlighting,
> etc.
Not practical for me (not sure about others) since I tend to develop
using an edit/run style. Python is so quick that with the Macintosh
IDE I just code a little, then quickly run a test. Correct code until
satisfied, then code more new stuff and test. etc. etc. etc. But that
eliminates the macros or fancy editor stuff. It's a great way to code
though (only with Python :-) )
> 4) Use a shorter name than self, e.g. 's'
Yes, that helps.
> 5) Use the x,y,z = self.x, self.y, self.z idiom
Yes, also works.
> 6) Have Python generate the function bodies for you, e.g:
>
> class SomeClass:
> method = MyCoolUnboundMethodMaker('arg:x = y*z + t*arg')
Hmmm...what's that? I don't get it (I hope I am about to learn
something here.)
> 7) Change your design - if something is awkward it *may* be a flaw in the
> language, but there's also a really, really good chance that a different
> approach to your program would be best.
Not too relevant to the math required. You have to write a math
expression.
Louis M. Pecora
<bru...@tbye.com> wrote:
> > I learn something every day. We should have a collection of speedup
> > tricks listed somewhere.
>
> We do, but we keep it fairly hidden because consulting the list too early
> or too often is worse than not having a list. <0.5 wink>
So, only the Python secret brotherhood gets to see The List?
:-)
bru...@tbye.com
> > 3) Use the smarts of your editor - macros, special syntax highlighting,
> > etc.
>
> Not practical for me (not sure about others) since I tend to develop
> using an edit/run style. Python is so quick that with the Macintosh
> IDE I just code a little, then quickly run a test. Correct code until
> satisfied, then code more new stuff and test. etc. etc. etc. But that
> eliminates the macros or fancy editor stuff. It's a great way to code
> though (only with Python :-) )
I guess I don't understand what you mean - in my mind the two are
independent. For example, if you're bothered by typing 'self.' then bind
that text to a hotkey, e.g. Ctrl-s. The benefit doesn't depend on your
coding style that much. If readability is your gripe, change the syntax
highlighting to make 'self.' be lighter (so it doesn't jump out as much).
Again, using an edit/run approach benefits.
> > 6) Have Python generate the function bodies for you, e.g:
> >
> > class SomeClass:
> > method = MyCoolUnboundMethodMaker('arg:x = y*z + t*arg')
>
> Hmmm...what's that?
Just an idea - since Python is so dynamic, instead of writing the
function you write a string describing the function and have another
function read it and generate a method that does all the 'self.' stuff.
Obviously you'd have to think it through some, but this is definitely
possible.
In the above example, I assume MyCoolUnboundMethodMaker would read
the string, and use it to generate a string like this:
'''def NewMethod(self, arg):
self.x = self.y * self.z + self.t * arg
'''
after which it would exec the string and then do a 'return NewMethod', and
bingo, your class now has a method that performs that operation. It'd take
some work to get it right the first time, but from then on you could reuse
it for all your numeric work.
> > 7) Change your design - if something is awkward it *may* be a flaw in the
> > language, but there's also a really, really good chance that a different
> > approach to your program would be best.
>
> Not too relevant to the math required. You have to write a math
> expression.
Sure it's relevant. For example, you have to specify 'self.' on all your
variables because through your design you have *chosen* to make them all
members of an object. I think that was a reasonable decision on your part,
but it's certainly not the only possibility.
-Dave
Alex Martelli
...
>> > 6) Have Python generate the function bodies for you, e.g:
>> >
>> > class SomeClass:
>> > method = MyCoolUnboundMethodMaker('arg:x = y*z + t*arg')
>>
>> Hmmm...what's that?
>
> Just an idea - since Python is so dynamic, instead of writing the
> function you write a string describing the function and have another
> function read it and generate a method that does all the 'self.' stuff.
> Obviously you'd have to think it through some, but this is definitely
> possible.
>
> In the above example, I assume MyCoolUnboundMethodMaker would read
> the string, and use it to generate a string like this:
>
> '''def NewMethod(self, arg):
> self.x = self.y * self.z + self.t * arg
> '''
>
> after which it would exec the string and then do a 'return NewMethod', and
> bingo, your class now has a method that performs that operation. It'd take
> some work to get it right the first time, but from then on you could reuse
> it for all your numeric work.
Indeed, code generation is not a bad idea. However, I think I
would take a different approach to it, based on hacking bytecode
in a metaclass rather than generating and compiling source. It's
just py-in-the-sky musing, but -- imagine being able to code, e.g.:
class Myself(Selfer):
__slots__ = 'x y z'.split()
def __init__(ax, ay, az):
x, y, z = ax, ay, az
def compute():
return x**y/z
and have type(Selfer), e.g. SelferMeta, tweak the code so it's
just as if you had put self in all the needed places. It should
be feasible (with the obvious limitations -- all the slot names
become reserved as variable names and you can't use them for any
argument, global, local, etc).
The point is that SelferMeta.__new__ gets to look at all of
class Myself's dictionary *and tamper with it as needed* before
delegating the rest to class type. Specifically, it would get
to look at the various methods, already compiled down to
bytecode... and change the bytecode as needed, i.e., each
of the various LOAD_GLOBAL, STORE_FAST, LOAD_FAST, would be
checked to see if the name it refers to is in __slots__, and,
if so, transformed into LOAD_FAST+LOAD_ATTR, or LOAD_FAST+
STORE_ATTR. Of course, all the arguments of the various
JUMP_* operators would have to be adjusted as the code's
length changes. Hmmm, would you want to allow calling methods
by bare-name, too, or would self.name() be still required there?
No matter -- it's your language, design it as you wish, of
course. Implementation is pretty obvious anyway. Oh I was
almost forgetting -- insert a first argument named 'self' if
not already present, and then change any LOAD_GLOBAL for 'self'
to LOAD_FAST instead, of course.
Sounds like about a week's worth of serious hacking, and all
in Python -- no C needed.
Me, I like 'self' -- but, as you see, those who dislike it
have only a modest effort to spend to make it optional in
their own classes. Not right now (I should be writing the
Nutshell, darn, NOT reading and writing on c.l.py...!!!),
but when the Nutshell's done I'll be available for only
moderately outrageous hourly rates (Strakt only takes 40
hours a week of my time, as Agile development principles
dictate), and I'm quite willing to make a fixed price bid
for this if we can nail down the specs to something easy
to implement this way. I'm sure lots more people who are
comfortable hacking on bytecode will also be happy to bid
(assuming every self-hater is unable or unwilling to dirty
his or her hands with this approach).
The point is: custom metaclasses allow you to do a LOT of
things more productively than by whining on c.l.py. Not
all, by a long shot, since a class with a custom meta must
still be syntactically valid Python (what you can't
express that way you must hack into docstrings, or via
naming-conventions, and the like). But a heck of a *LOT*.
So why not 'import dis' and start studying bytecode on
small examples, and how you could hack on it? DIY is
cheaper and more fun -- and you're sure to learn a lot
even if you end up throwing the results away!-)
Alex
Fredrik Lundh
> 5) Use the x,y,z = self.x, self.y, self.z idiom
unfortunately, that actually creates a 3-item tuple,
only to tear it apart again. this is better:
x = self.x; y = self.y; z = self.z
</F>
Louis M. Pecora
<bru...@tbye.com> wrote:
> I guess I don't understand what you mean - in my mind the two are
> independent. For example, if you're bothered by typing 'self.' then bind
> that text to a hotkey, e.g. Ctrl-s.
Yep, did that. I use "_" in place of self.
>The benefit doesn't depend on your
> coding style that much. If readability is your gripe, change the syntax
> highlighting to make 'self.' be lighter (so it doesn't jump out as much).
Can't do that on the current IDE. No syntax highlighting.
Louis M. Pecora
> Just an idea - since Python is so dynamic, instead of writing the
> function you write a string describing the function and have another
> function read it and generate a method that does all the 'self.' stuff.
> Obviously you'd have to think it through some, but this is definitely
> possible.
>
> In the above example, I assume MyCoolUnboundMethodMaker would read
> the string, and use it to generate a string like this:
>
> '''def NewMethod(self, arg):
> self.x = self.y * self.z + self.t * arg
> '''
>
> after which it would exec the string and then do a 'return NewMethod', and
> bingo, your class now has a method that performs that operation. It'd take
> some work to get it right the first time, but from then on you could reuse
> it for all your numeric work.
I gotta think about that one. Thanks.
Louis M. Pecora
<fre...@pythonware.com> wrote:
If that's faster, that's what I'd use. Thanks.
Delaney, Timothy
>
> The point is that SelferMeta.__new__ gets to look at all of
> class Myself's dictionary *and tamper with it as needed* before
> delegating the rest to class type. Specifically, it would get
> to look at the various methods, already compiled down to
> bytecode... and change the bytecode as needed, i.e., each
> of the various LOAD_GLOBAL, STORE_FAST, LOAD_FAST, would be
> checked to see if the name it refers to is in __slots__, and,
> if so, transformed into LOAD_FAST+LOAD_ATTR, or LOAD_FAST+
> STORE_ATTR. Of course, all the arguments of the various
> JUMP_* operators would have to be adjusted as the code's
Wouldn't work in Jython ... (when Jython gets to 2.2 ...).
Tim Delaney
bru...@tbye.com
> Yep, did that. I use "_" in place of self.
>
> >The benefit doesn't depend on your
> > coding style that much. If readability is your gripe, change the syntax
> > highlighting to make 'self.' be lighter (so it doesn't jump out as much).
>
> Can't do that on the current IDE. No syntax highlighting.
Aha! The real problem comes to light! ;-) A good syntax highlighting
editor (IDE or not) will give a bigger boost to code readability than any
change the to Python language, but oh well...
-Dave
Alex Martelli
>> From: Alex Martelli [mailto:al...@aleax.it]
>>
>> The point is that SelferMeta.__new__ gets to look at all of
>> class Myself's dictionary *and tamper with it as needed* before
>> delegating the rest to class type. Specifically, it would get
...
> Wouldn't work in Jython ... (when Jython gets to 2.2 ...).
Or more precisely: the bytecode manipulations needed will be specific
to the underlying virtual machine. I.e., unless a VM is designed in
really crazy ways, it will probably be possible to use exactly the
same technique (metaclass computing what bytecode is needed to achieve
the desired results and substituting it in place of the bytecode as
given), but the details of the transformations will change.
Source-to-source transformations would be more general, but getting
at the relevant sources is not trivial. When Python is run with -O
you lose the line-numbers; when it's run with -OO you even lose the
docstrings, so hiding semantically crucial info there is not fully
kosher either. Unfortunately, the metaclass gets ahold of things
only AFTER the class body is done. I suspect the only techniques
that would let you use the same metaclass independently of underlying
VM and runtime options are those that boil down to putting the
source in string attributes of the class object, e.g.:
class Selfish(SourceSelfer):
__slots__ = "x y z".split()
amethod = """(t, u, v):
return x*t + u*y + z*v
"""
Alex
Delaney, Timothy
>
> Delaney, Timothy wrote:
>
> >> From: Alex Martelli [mailto:al...@aleax.it]
> >>
> >> The point is that SelferMeta.__new__ gets to look at all of
> >> class Myself's dictionary *and tamper with it as needed* before
> >> delegating the rest to class type. Specifically, it would get
> ...
> > Wouldn't work in Jython ... (when Jython gets to 2.2 ...).
>
> Or more precisely: the bytecode manipulations needed will be specific
> to the underlying virtual machine. I.e., unless a VM is designed in
> really crazy ways, it will probably be possible to use exactly the
> same technique (metaclass computing what bytecode is needed to achieve
> the desired results and substituting it in place of the bytecode as
> given), but the details of the transformations will change.
May I suggest that I was awed by the hack, and felt that my only possible
contribution was to point out the unfeasibility of such a method in case
anyone actually wanted to *do* something this perverse?
I mean - it was a truly evil suggestion, well and truly worthy of the
Martellibot :)
Tim Delaney
Duncan Booth
news:ah9p0...@drn.newsguy.com:
> For example:
>
> with self:
> result=sin(t)*x**y+sqrt(z) # A pretend formula
>
> I don't know if there's a technical reason (in terms of either Python
> grammar or C implementation) why this wouldn't work, but perhaps it's
> not in Python because:
>
> 1) explicit is better than implicit
> 2) there should be one--and preferably only one--way to do things
It isn't in Python for a variety or reasons (your two among them), but
perhaps more because there is no way in your scheme to tell which of
result, t, x, y, and z are members of the self instance, and which are
global (or in result's case local) variables. Languages which require you
to declare all the member variables in advance can get away with this sort
of usage, but they then introduce new problems. Even in C++ it can be
impossible to tell whether a variable in a function is implicitly
referencing a member variable or a global of the same name.
An alternative which is proposed from time to time would be implementable:
with self:
result=sin(.t)*.x**.y+sqrt(.z) # A pretend formula
This has the advantage of tagging the member variables, but is thought by
many to reduce readability and be error prone. Perhaps someone should put
it into a patch though so people could try it out. If Python supported this
format you might even allow functions to default to assuming a 'with' block
on their first argument. Then again, see your #1 above.
Of course, Python being what it is, there are ways to simulate some of what
people want without changing the language at all. Try the code below:
----- begin magic.py -----
# How to make methods magically access member variables.
#
import new
from math import sin, sqrt
def withself(method):
'''withself(method)
Converts an unbound method into a method where all global variables
are assumed to come from the instance.
Global variables are accessible only if the instance sets
self.__builtins__=globals()
Builtins are not accessible unless you do:
self.__builtins__=__builtins__
'''
def callwith(self, *args, **kw):
f = new.function(method.func_code,
self.__dict__,
method.func_name,
method.func_defaults or ())
return f(self, *args, **kw)
return callwith
class C:
def __init__(self, x, y, z):
self.x, self.y, self.z = x, y, z
# Makes global variables visible to formulae
self.__builtins__ = globals()
def formula(self):
return x + y - z
formula = withself(formula)
def updater(self):
global x
x += 1
updater = withself(updater)
def formula2(self):
return sqrt(y*z)
formula2 = withself(formula2)
test = C(3, 7, 2)
print test.formula()
test.updater()
print test.formula()
print test.formula2()
----- end magic.py -----
--
Duncan Booth dun...@rcp.co.uk
int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3"
"\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?
Michael Chermside
>> Many people have suggested doing something like the following (which is
>> reasonable) to eliminate self. from mathematical expressions:
>>
>
> lines like:
>
>
> You look at that code, with all those redundant lines of negligible
> content, and it yells out to you: something is missing here that
> should make this silly copying unnecessary in order to have expressions
> that are easy to read/understand.
I agree. Something is missing here. Your problem is that you really
aren't dealing with "objects", per se, you have these "formulas" in your
mind. So why don't we use the closest Python equivalent?
# WARNING: Untested code below
def pretendFormula(x,y,z,t):
return sin(t) * x**y + sqrt(z)
def MyClass:
def pretend(self):
return pretendFormula(self.x, self.y, self.z, self.t)
Notice how this has an additional advantage... it allows you to use
short, simple names for the variables within the formula (the
traditional to write formula), while still allowing longer, more
descriptive names within the object:
...
return pretendFormula( self.position.x,
self.position.y,
self.position.z,
self.time )
And while you're at it, you can give a descriptive name to the formula
itself.
Can't we get the best of both worlds?
-- Michael Chermside
Louis M. Pecora
Chermside <mch...@destiny.com> wrote:
> # WARNING: Untested code below
>
> def pretendFormula(x,y,z,t):
> return sin(t) * x**y + sqrt(z)
>
> def MyClass:
> def pretend(self):
> return pretendFormula(self.x, self.y, self.z, self.t)
>
> Notice how this has an additional advantage... it allows you to use
> short, simple names for the variables within the formula (the
> traditional to write formula), while still allowing longer, more
> descriptive names within the object
What's the overhead of the function calls versus either
(1) t, x, y, z=self.t, self.x, self.y, self.z
or
(2)t=self.t; x=self.x; y=self.y; z=self.z;
??
Seems like (1) and (2) have less overhead and someone in this thread
said (1) creates and then destroys a tuple (unnecessarily so) and (2)
is better (no intermediate tuple).
I still prefer (1) -- most readable, but I like minimum overhead.
Delaney, Timothy
>
> What's the overhead of the function calls versus either
>
> (1) t, x, y, z=self.t, self.x, self.y, self.z
>
> or
>
> (2)t=self.t; x=self.x; y=self.y; z=self.z;
>
> ??
>
> Seems like (1) and (2) have less overhead and someone in this thread
> said (1) creates and then destroys a tuple (unnecessarily so) and (2)
> is better (no intermediate tuple).
>
> I still prefer (1) -- most readable, but I like minimum overhead.
Do some profiling ... (untested code follows)
---
import time
class Vars:
def __init__(self):
self.t = 1
self.x = 2
self.y = 3
self.z = 4
def f (t, x, y, z):
pass
def bench (f=f, time=time): # to remove global lookups
r = range(1000000) # modify as required to get usable times
v = Vars()
start = time.clock()
for i in r:
t, x, y, z = v.t, v.x, v.y, v.z
end = time.clock()
print (end - start)
start = time.clock()
for i in r:
t = v.t
x = v.x
y = v.y
z = v.z
end = time.clock()
print (end - start)
start = time.clock()
for i in r:
f(v.t, v.x, v.y, v.z)
end = time.clock()
print (end - start)
if __name__ == '__main__':
bench()
---
On my machine, the results are:
2.30967285289
2.14318686404
2.982054833
This is *purely* the overhead of the various function call, tuple packing
and unpacking, etc - once you start doing things with these they start
having much less effect (because you're only using local variables in each
case).
Tim Delaney
Delaney, Timothy
>
> Do some profiling ... (untested code follows)
Followed by results of a test run ... ;)
Tim Delaney
Greg Ewing
>>>
> def pretendFormula(x,y,z,t):
> return sin(t) * x**y + sqrt(z)
>
> def MyClass:
> def pretend(self):
> return pretendFormula(self.x, self.y, self.z, self.t)
You could do slightly better than that, if you
don't want to be bothered passing in the args
one at a time (untested):
def pretendFormula(x, y, z, t, **_):
return sin(t) * x**y + sqrt(z)
def MyClass:
def pretend(self):
return pretendFormula(**self.__dict__)
--
Greg Ewing, Computer Science Dept,
University of Canterbury,
Christchurch, New Zealand
http://www.cosc.canterbury.ac.nz/~greg
Louis M. Pecora
Timothy <tdel...@avaya.com> wrote:
> On my machine, the results are:
>
> 2.30967285289 (1)
> 2.14318686404 (2)
> 2.982054833 (3)
[I added the (numbers) above for clarity]
First, thanks for that nice study. Very interesting.
(1) is using t, x, y, z = v.t, v.x, v.y, v.z
(2) is t = v.t; x = v.x; y = v.y; z = v.z
(3) a function call to a formula f(v.t, v.x, v.y, v.z)
It looks like some of the guesses people had on this NG are correct.
(2) is fastest
(1) is slower (by ~ 8%)
(3) is slowest (by ~ 40%)
I'd call (1) and (2) a near draw, but (3), if it accurately reflects a
function call has non-negligible overhead, IMO. I understand these
were quick and dirty tests. More tests would be nicer. I may try a
few. Thanks, again.
Michael Chermside
> (2)t=self.t; x=self.x; y=self.y; z=self.z;
[...]
> I still prefer (1) -- most readable, but I like minimum overhead.
Then I recomend that you avoid Python.
Seriously, Python is interpreted, and the overhead for performing a
simple mathematical calculation like
sin(t) * x**y + sqrt(z)
in Python vs doing it directly in C, Fortran, or some other
close-to-the-machine language is FAR, FAR greater than the difference
between (1) and (2) above.
Think about it like this. Suppose there were two ways to write your
code. Method (A) was clear and easy to read, and not particularly
difficult to modify. Method (B) was much harder to read, and a real pain
to modify. But method (B) runs X microseconds faster.
If X makes the difference between your whole program running in 5
minutes vs 5 hours, or the difference between your user interface
responding in 0.5 seconds vs locking for 5 seconds, then you probably
care deeply, and you probably want to use method (B). If X makes the
difference between your whole program running in 5 minutes vs 6 minutes,
or the user interface responding in 0.3 seconds or 0.5 seconds, then
many programmers would prefer method (A).
If you are one of those who would choose method (B) even when the
difference in runtime was between 5 and 6 minutes, then you probably
shouldn't be using Python. Yes, Python is a nice language, and yes, it
can be sped up by recoding key portions of an application in C/C++, but
it is never going to be as fast as an algorithm hand coded in assembly.
I guess what I'm trying to say here is, yes... having the functions
separate WILL slow things down, and by a *LOT* more than the difference
between (1) and (2) above. But you REALLY shouldn't be worrying about
the difference in execution time between (1) and (2)... if you ARE
worrying about such things, then there are MUCH bigger gains to be had
by trying a different kind of language.
-- Michael Chermside
Louis M. Pecora
I did some tests as suggested in the "Better self" thread on the timing
overhead of assigning to local variables and then using them in a
formula. The issue in that thread was readability which can be gained
several ways. The code and results below compares those ways. I
present the explanation for the four tests, first, then show the
results (a bit surprising...to me, at least), and then at the end I
give the code. I am not claiming this is the best way to do these
tests. It is typical of coding I have done. Feel free to criticize or
better, suggest other tests, or even better, do other tests and post
them.
-- Lou Pecora
EXPLANATION OF TESTS =====================================
# Tests overhead for "assigning" object variables
# (e.g. self.t, self.x, self.y, self.z) to
# local variables or function arguments so as to use simpler names
# in a formula without the "self." prefix. Such assignements
# will cause "overhead". This module's aim is to compare the
# overhead involved in each approach.
# The approaches are:
# (1) Assign in tuple, thus: t,x,y,z=self.t,self.x,self.y,self.z,
# then use t,x,y, and z in the formula
# (2) Assign in line, thus: t=self.t
# x=self.x
# y=self.y
# z=self.z, then use t,x,y, and z
# in the formula
# (3) Assign to class function arguments, thus:
# self.fcntst(self.t,self.x,self.y,self.z),
# where the class function contains the formula
# (4) Assign to class function arguments using local
# variable f assigned to the class function, thus:
# f(self.t,self.x,self.y,self.z), where f=self.fcntst
#
# Note, an attempt is made to _exclude_ the loop overhead by
# timing an empty xrange loop, first, and subtracting that
# time, tx, from the tests (1) to (4)
RESULTS AND COMMENTS =====================================
Note: the results (the ratios) appeared to be independent of the
number of loops used (n, below) from n=1000 to n=1000000
xRange time= 0.15076
tuple assign time= 0.958203999999
line time= 0.799539
fcn1 (class call) time= 1.508272
fcn2 (local call) time= 1.256834
Ratios to the minimum time:
(1): 1.24455939542
(2): 1.0
(3): 2.09241051267
(4): 1.70485481189
Method (2) is fastest as some people suggested. (1) does seem to
involve some time creating and destroying a tuple as suggested. (3) is
the worst, by a factor of 2! So if you have lots of class function
calls, assign the function to a local variable, but even better, avoid
using the function calls to just calculate a formula, if possible.
CODE =====================================
# Just imports stuff commonly used
import time
from Numeric import *
class lcltst:
def __init__(self,n):
self.t=1
self.x=2
self.y=3
self.z=4
self.n=n # Numb.loops
# Calculate the for loop time so as to exclude it later
t1=time.clock()
for i in xrange(self.n):
pass
self.tx=time.clock()-t1 # loop time
print "xRange time=",self.tx
# (1) Assign variables in a tuple
def linasg(self):
t1=time.clock()
for i in xrange(self.n):
t,x,y,z=self.t,self.x,self.y,self.z
# After assigning local variables normally the
# formula for doing
# calculations using the local variables would follow
t2=time.clock()
print "tuple assign time=",t2-t1
return t2-t1
# (2) Assign variables one at a time, in line
def indasg(self):
t1=time.clock()
for i in xrange(self.n):
t=self.t
x=self.x
y=self.y
z=self.z
# After assigning local variables normally the
# formula for doing
# calculations using the local variables would follow
t2=time.clock()
print "line time=",t2-t1
return t2-t1
# Dummy function
def fcntst(self,t,x,y,z):
# Would normally contain the formula and do calculations using
# the argument variables
pass
# (3) Call dummy formula function
def fcn1(self):
t1=time.clock()
for i in xrange(self.n):
# Use class function call
self.fcntst(self.t,self.x,self.y,self.z)
t2=time.clock()
print "fcn1 (class call) time=",t2-t1
return t2-t1
# (4) Call local assignment of dummy formula function
def fcn2(self):
t1=time.clock()
# Assign class function to local variable, first
f=self.fcntst
for i in xrange(self.n):
f(self.t,self.x,self.y,self.z)
t2=time.clock()
print "fcn2 (local call) time=",t2-t1
return t2-t1
#---- Run the test for n loops ----------------------------
print "--------------------------------------"
n=200000 # Number of loops (tested from n=1000, to n=1000000)
lt=lcltst(n)
tx=lt.tx
tlist=[lt.linasg()-tx]
tlist=tlist+[lt.indasg()-tx]
tlist=tlist+[lt.fcn1()-tx]
tlist=tlist+[lt.fcn2()-tx]
tmax=min(tlist)
tfract=divide(tlist,tmax)
print
print "Ratios to the minimum time:"
print "(1):",tfract[0]
print "(2):",tfract[1]
print "(3):",tfract[2]
print "(4):",tfract[3]
Kerim Borchaev
sorry if this message sounds offensive - it isn't supposed to ;-)
Tuesday, July 23, 2002, 8:11:00 PM, you wrote:
...
LMP> xRange time= 0.15076
LMP> tuple assign time= 0.958203999999
LMP> line time= 0.799539
LMP> fcn1 (class call) time= 1.508272
LMP> fcn2 (local call) time= 1.256834
...
LMP> Method (2) is fastest as some people suggested. (1) does seem to
LMP> involve some time creating and destroying a tuple as suggested. (3) is
LMP> the worst, by a factor of 2! So if you have lots of class function
LMP> calls, assign the function to a local variable, but even better, avoid
LMP> using the function calls to just calculate a formula, if possible.
It feels like a general advice.
"lots of class function" - it lets everyone decide the meaning of
"lots" for himself; or how should I treat "to just calculate a
formula" - is it for any code that involves "the math"?
So I'd give another general advice:
- Make the code as clean as possible(using functions, methods - all
the best of the language ) BEFORE you discover that it works too slow.
- And only whan you're absolutely sure(the tests show) that
it's slow for your needs - think about optimization.
- Then profile test cases of interest to find out which pieces of code are
worth optimizing to get a valuable speedup.
- Then optimize. ...or not yet?
Best regards,
Kerim mailto:war...@storm.ru
Louis M. Pecora
Borchaev <war...@storm.ru> wrote:
> It feels like a general advice.
> "lots of class function" - it lets everyone decide the meaning of
> "lots" for himself; or how should I treat "to just calculate a
> formula" - is it for any code that involves "the math"?
Yes, but it's for assigning local variables to avoid using self.
That's it. If you have other ways you might do it, please test those
and post the results.
> So I'd give another general advice:
>
> - Make the code as clean as possible(using functions, methods - all
> the best of the language ) BEFORE you discover that it works too slow.
>
> - And only whan you're absolutely sure(the tests show) that
> it's slow for your needs - think about optimization.
>
> - Then profile test cases of interest to find out which pieces of code are
> worth optimizing to get a valuable speedup.
>
> - Then optimize. ...or not yet?
>
> Best regards,
> Kerim mailto:war...@storm.ru
All good advice, but I don't have the time to do all that. It was
meant to be a quick test in response to some suggestions on the "Best
self." thread that some of the code people were suggesting should be
tried out. Some other quick tests by others appear on that thread. I
think what I got was informative since I've seen code and suggestions
just like what I offered. You may disagree. Maybe I should put
disclaimers in it. Well, live and learn.
Thanks.
Michele Simionato
tradition and to use the standard self. However many people don't like self.
It has not been mentioned in this thread, but there a simple way to cope
with this problem without changing the language: a Python preprocessor.
Suppose one define the pseudocode
--begin pseudo.py--
from math import sin,sqrt
class Example:
y,z,t=1.0,1.0,1.0
def f(,x):
return sin(.t)*x**.y+sqrt(.z)
example=Example()
print example.f(1.0)
--end pseudo.py--
where the comma in f(,x) remind us that the first argument self is omitted.
One can easily create a preprocessor that convert this code on standard
python code converting
(, ---> (self,
and
.something ---> self.something
For instance the following code does the job (if you work on Red Hat
Linux the first line has to be #!/usr/bin/python2):
--begin pp--
#!/usr/bin/python
from re import compile
import sys
def process(pseudocode):
comma=compile(r'\(\s*,')
dot=compile(r'[^\w_]\.[a-zA-Z_]+')
code=comma.sub('(self,',pseudocode)
end=0; newcode=''
for found in dot.finditer(code):
start=found.start()
newcode+=code[end:start+1]+'self'
end=found.end()
newcode+=code[start+1:end]
return newcode+code[end:]
try:
name=sys.argv[1]
except:
print "Usage: ./p2 programname.py"
sys.exit()
inp=file(name)
out=process(inp.read())
inp.close()
print out
--end pp--
This is not perfect because it converts "(," and ".something" everywhere in
the file, even in comments and in strings, however it can be refined.
The point is that pp (Python Preprocessor) convert the pseudo code in
normal code which can be executed. After a chmod u+x pp the user can type
$ ./pp pseudo.py > normal.py
to convert the pseudocode in standard code or even directly execute the
pseudocode with
$ ./pp pseudo.py | python
Notice that the preprocessor also works with function with no explicit
arguments, since f(,) is converted to f(self,) which is equivalent to f(self).
The preprocessor idea is interesting, not only for people who want to create
their own shortcuts to the language (I think it is better to use good habits
and to use the standard language), but even for more serious jobs.
--
Michele Simionato - Dept. of Physics and Astronomy
210 Allen Hall Pittsburgh PA 15260 U.S.A.
Phone: 001-412-624-9041 Fax: 001-412-624-9163
Home-page: http://www.phyast.pitt.edu/~micheles/
Irmen de Jong
> It has not been mentioned in this thread, but there a simple way to cope
> with this problem without changing the language: a Python preprocessor.
This will work if you know that the code you're dealing with has to be
processed first before it is real Python code.
It will not work if you are somehow unable to run the preprocessor first.
Think about importing modules. They have to be processed first,
otherwise the import will not work because the module is not correct
Python code.
Irmen.
Alex Martelli
> Personally, I have no problems in conforming myself to the established
> tradition and to use the standard self. However many people don't like
> self. It has not been mentioned in this thread, but there a simple way to
> cope with this problem without changing the language: a Python
> preprocessor.
>
> Suppose one define the pseudocode
>
> --begin pseudo.py--
>
> from math import sin,sqrt
>
> class Example:
> y,z,t=1.0,1.0,1.0
> def f(,x):
> return sin(.t)*x**.y+sqrt(.z)
...
> The preprocessor idea is interesting, not only for people who want to
> create their own shortcuts to the language (I think it is better to use
> good habits and to use the standard language), but even for more serious
> jobs.
Yes, but it doesn't fit smoothly with Python's normal work organization.
Using an extension different than .py for the preprocessor-sources makes
it OK for languages normally handled with makefiles, but that's not how
you usually work in Python.
If you want preprocessing, I suggest you integrate it differently:
class Example(Prepro):
'''
y,z,t=1.0,1.0,1.0
def f(,x):
return sin(.t)*x**.y+sqrt(.z)
'''
Prepro is an otherwise empty class whose metaclass is MetaPrepro.
MetaPrepro's __new__ method gets Example's dict, containing only the
__docstring__ with all the above source. It can change that source
at will, the prepare the real dict with the needed entries and
finally delegate up to type.__new__.
I suggested a similar approach based on bytecodes, but even for
source to source transformation (which is easier and better portable
across different bytecode/VMs) I think it would be the right tack.
Alex
Alex Martelli
...
> shouldn't be using Python. Yes, Python is a nice language, and yes, it
> can be sped up by recoding key portions of an application in C/C++, but
> it is never going to be as fast as an algorithm hand coded in assembly.
It may happen to be just as fast (that would be a rare coincidence)
and it may well be (more likely:-) *much faster*.
That's because more often than not you won't be coding the SAME
algorithm in Python or assembly. In the time it takes you to code
and debug the one algorithm in assembly, you've been able to invent
and try a dozen different ones in Python, and quite likely come up
with one that beats the pants off the one you first thought of.
The fallacy of thinking that "coded in Python" means "slower than
if coded in assembly" comes from the unspoken assumption that the
program will have exactly the same structure in both cases. Most
often, it won't.
A similar fallacy is frequent in laypeople's economics thoughts, and known
for example as "the fallacy of the lump of labor" -- where some people
intuitively think that (e.g.) immigrants "steal jobs from natives". The
fallacy here comes from imagining that there's a predefinite, fixed, total
amount of things to do, a pie of a fixed size, and all that matters is
how the pie is sliced up.
Reality just doesn't work that way. The pie can grow (or shrink due
to restrictive protectionist legislation), better algorithms can be
and are devised and deployed.
Only when you're SURE you have the best possible algorithms (and the
most perfect, developed economy) and you're STILL too slow (or, still
too poor) should coding in lower levels be considered (on the other
hand, protectionist legislation shouldn't be considered even then,
but that's another issue:-).
Alex
Andreas Kostyrka
> So, only the Python secret brotherhood gets to see The List?
Well, it's better this way. The List [of speedup tricks] contains
usually ugly things. :)
So it's quite correct that it is only presented to the truebelievers. :)
There is number of things, but the more important things to keep in
mind:
- function calls are relativly expensive in Python. At least compared to
more static and compiled languages.
- immutable objects need to be copied to be modified.
str1+str2+str3+str4+str5+str6+str7+str8 is usually slower than
"".join([str1,str2,str3,str4,str5,str6,str7,str8])
- every dot access is a dictionary lookup. And python usually cannot do
constant folding, etc., because every call is dynamic:
So instead of
for i in xrange(<huge value>):
x="".join(...)
one might write:
fjoin="".join
for i in xrange(<huge value>):
x=fjoin(...)
There are certainly other rules, but these are the ones that came to my
mind. :)
Andreas
Michael Chermside
...
> shouldn't be using Python. Yes, Python is a nice language, and yes, it
> can be sped up by recoding key portions of an application in C/C++, but
> it is never going to be as fast as an algorithm hand coded in assembly.
Alex Martelli replies:
> It may happen to be just as fast (that would be a rare coincidence)
> and it may well be (more likely:-) *much faster*.
>
> That's because more often than not you won't be coding the SAME
> algorithm in Python or assembly.
...
Alex is (of course) completely correct. My true point was that Python
and a mindset of "do whatever you can to save an extra CPU cycle" don't
fit well together. But Python can be MUCH faster, in a lot of cases.
I'll give an example.
#
# A program to sort lines. Takes, as command-line argument, a
# filename (the file will contain one word or phrase per line),
# and outputs the sorted file to stdout.
#
import sys
filename = sys.argv[1]
lines = file(filename).readlines()
lines.sort()
for line in lines:
print line
Except for some pretty strange circumstances (extremely large file, OS
with really bad virtual memory), this is going to be faster than the
equivalent C program. Unlike the C program, it has to set up the
interpretor, do interpretation of the Python bytecodes, allocation and
reference counting of Python objects, and use OS portable file reading.
But all of these are dwarfed (for large file sizes... let's say 1 GB of
text) by the fact that you are running a REALLY FAST sort algorithm.
Of course, you could code the same sort algorithm in C yourself. Heck,
for that matter, you could copy the sort algorithm (written in C) from
the C source to Python (Python's liscense allows that!). But you
wouldn't. Really... you wouldn't.
But this is all beside the point. The **REAL** reason to use Python for
a problem like this is that the Python script is only 6 lines long, and
those lines are so elegant, so easy to understand, that a computer savy
*middle school student* could understand them, maintain them, and
perhaps even write them. The equivalent C progam would be much, much longer.
Yes, there are times when Python is faster... and usually it's because a
different (better!) algorithm is used (eg: use dicts instead of linear
search through a list). But where it REALLY shines is in its clarity.
And that clarity lets you think at a higher level (notice how I never
had to stop to think about memory allocation or max line lengths?). And
THAT lets you write better programs.
-- Michael Chermside
Terry Reedy
"Michael Chermside" <mch...@destiny.com> wrote in message
news:mailman.1027533340...@python.org...
> # A program to sort lines. Takes, as command-line argument, a
> But all of these are dwarfed (for large file sizes... let's say 1 GB
of
> text) by the fact that you are running a REALLY FAST sort algorithm.
Few of us appreciate the days that Tim Peters spent writing a portable
sort program that is significantly better than standard quicksort for
several classes of input (such as a sorted list with new items
appended to the end). He is currently working on another that may be
even better. (Three horn toots for Tim.)
> Of course, you could code the same sort algorithm in C yourself.
Hmm. How many of us have both the skill and patience to develop and
test such a delicate and complex algorithm tweaked for various special
cases -- and apparently release it bug free (Tim says there has never
been a report of the current sort malfunctioning).
> for that matter, you could copy the sort algorithm (written in C)
from
> the C source to Python (Python's liscense allows that!). But you
> wouldn't. Really... you wouldn't.
Only if really forced to by a crazy employer.
>(notice how I never had to stop to think about memory allocation or
max line lengths?).
Yes -- or number of lines. Merely writing the generic C equivalent of
lines = file('datalines.txt').readlines()
is a bit of a headache. Nice posting.
Terry J. Reedy
Bengt Richter
>Only when you're SURE you have the best possible algorithms (and the
>most perfect, developed economy) and you're STILL too slow (or, still
>too poor) should coding in lower levels be considered (on the other
>hand, protectionist legislation shouldn't be considered even then,
>but that's another issue:-).
>
ISTM everyone believes in protectionism, only differing in their
ideas of what to protect and how to do it ;-)
Regards,
Bengt Richter
Delaney, Timothy
>
> "Michael Chermside" <mch...@destiny.com> wrote in message
> news:mailman.1027533340...@python.org...
> > # A program to sort lines. Takes, as command-line argument, a
> ...
> > But all of these are dwarfed (for large file sizes... let's say 1 GB
> of
> > text) by the fact that you are running a REALLY FAST sort algorithm.
>
> Few of us appreciate the days that Tim Peters spent writing a portable
> sort program that is significantly better than standard quicksort for
> several classes of input (such as a sorted list with new items
> appended to the end). He is currently working on another that may be
> even better. (Three horn toots for Tim.)
That latest figures are *most* impressive. *No* categories of data (random,
partially-ordered, etc) where there is a significant slowdown, most
categories have a performance improvement, and it's a stable sort to boot.
And he says that he can get a bit more improvement in a couple of cases ...
(without special-casing mind you).
I'm really looking forward to timsort replacing samplesort :)
> > Of course, you could code the same sort algorithm in C yourself.
>
> Hmm. How many of us have both the skill and patience to develop and
> test such a delicate and complex algorithm tweaked for various special
> cases -- and apparently release it bug free (Tim says there has never
> been a report of the current sort malfunctioning).
And timsort manages to remove most of the special cases that samplesort was
testing for. Of course, he claims the source of timsort is just as hairy and
convoluted as samplesort ...
Tim Delaney
James Ashley
> On Wed, 24 Jul 2002 08:34:33 GMT, Alex Martelli <al...@aleax.it> wrote:
>>should coding in lower levels be considered (on the other
>>hand, protectionist legislation shouldn't be considered even then,
>>but that's another issue:-).
>>
> What are you protecting with that "shouldn't"? ;-)
Perhaps that was a "generally, any sane person won't, but sometimes
bueracracies insist upon insanity" kind of generalization?
>
> ISTM everyone believes in protectionism, only differing in their
> ideas of what to protect and how to do it ;-)
ROFL...nicely said. I've thought up a few responses to this, and I'm
curious about your answer to them all, but they all seem to whirl around
politics rather than software development, and, therefore, are even more
off-topic than I'm willing to get on this sort of forum.
Regards,
James
holger krekel
Don't bother. This thread is already ridicously long and anyone still
reading it has probably gotten to a very open mental self.state
self.regards()
Andreas Kostyrka
For an example see the page templates in Quixote.
Andreas
Tim Peters
> Few of us appreciate the days that Tim Peters spent writing a portable
> sort program that is significantly better than standard quicksort for
> several classes of input (such as a sorted list with new items
> appended to the end).
The samplesort hybrid that went into Python 1.5.2 is actually significantly
faster than Python's best previous quicksort on all inputs ever tested.
See? I'm more underappreciated than even you realize <wink>. quicksort
isn't really a good way to sort when comparisons are very expensive compared
to the cost of swapping elements (as is true in Python).
> He is currently working on another that may be even better. (Three
> horn toots for Tim.)
That's checked in now (yay!), and will be "the sort" in Python 2.3.
Extensive information about it can be found in the patch report and its
attachments:
http://www.python.org/sf/587076
Finn Bock was so encouraged by the numbers that he already recoded it in
Java for Jython (and did a wonderful porting job -- made me jealous I had to
code it in C!). Jython *was* using Python 1.5's best-ever quicksort
algorithm, not 1.5.2's much hairier samplesort hybrid. To get a feel for
the improvement from 1.5's quicksort (a hybrid of median-of-3 quicksort and
insertion sort) to this one, I'll include the Jython numbers Finn posted to
Python-Dev.
This is output from Python's Lib/test/sortperf.py when run with arguments
"15 19 1". *sort is trying random data, and all the others have *some* kind
of structure in the data. For example, /sort is an already-sorted array,
\sort a reverse-sorted array, %sort takes a sorted array and replaces a
randomly-selected 1% of its entries with new random values, and !sort was
hand-crafted as a bad case for 1.5's quicksort algorithm (BTW, there are no
*known* bad cases for the samplesort hybrid, although in theory some must
exist; the new 2.3 sort is N log N worst case, so has no bad cases even in
theory). For the meaning of the other columns, see the comments in
sortperf.py.
The entries in the table are in seconds, and each line is testing lists with
2**i elements. "timsort" is the name I gave to the new algorithm in a burst
of modesty <wink>:
"""
quicksort/insertionsort:
i *sort \sort /sort 3sort +sort %sort ~sort =sort !sort
15 0.66 0.50 0.30 0.29 0.40 0.42 0.32 0.28 2.53
16 1.36 1.10 0.67 0.67 0.94 1.05 0.75 0.60 6.12
17 3.12 2.38 1.47 1.47 1.88 2.12 1.62 1.28 15.43
18 6.52 5.14 3.22 3.22 4.52 5.56 3.35 2.73 36.04
19 14.32 11.07 6.99 6.99 8.71 11.72 7.33 5.86 87.80
timsort:
i *sort \sort /sort 3sort +sort %sort ~sort =sort !sort
15 0.44 0.05 0.03 0.03 0.04 0.06 0.17 0.02 0.06
16 0.82 0.08 0.06 0.07 0.07 0.11 0.32 0.05 0.11
17 1.76 0.18 0.13 0.13 0.13 0.23 0.64 0.11 0.22
18 3.87 0.34 0.26 0.29 0.27 0.49 1.29 0.21 0.45
19 8.91 0.70 0.53 0.54 0.54 1.07 2.62 0.43 0.90
"""
The C implementations are quite a bit faster than this, but I have to hand
it to Java: if someone ported this algorithm to Python, you might not wait
for the results <wink>. There are lots of timings from different platforms
in the patch report, comparing the C implementations of the samplesort
hybrid and the new algorithm.
highly-ordered-ly y'rs - tim
Terry Reedy
"Tim Peters" <tim...@comcast.net> wrote in message
news:mailman.1028434088...@python.org...
> [Timsort] is checked in now (yay!), and will be "the sort" in Python
2.3.
> Extensive information about it can be found in the patch report and
its
> attachments:
>
> http://www.python.org/sf/587076
The attachments include timsort.txt. Here is a quick summary:
1. View the sequence of N items to be sorted as, and convert it to, a
sequence of K ascending runs (down runs are reversed). To avoid the
overhead of merging small runs, use binary insertsort to bring run
lengths up to a minimum (depending on N) in the range [32,64].
Consequence: a list sorted in ascending order or descending order
without duplicates will be detected as one run and left as is or
simply reversed.
2. For K >= 2, timsort(runs) = timmerge(timsort(runs[:j]),
timesort(runs[j:])). Note: the merge tree is constructed from the
bottom up rather than from the top down as might be implied by the
above description.
3. Merge intelligently (including 'galloping') to minimize compares
and moves and exploit nonrandom structure in the original list.
Terry J. Reedy
Tim Peters
> The attachments include timsort.txt. Here is a quick summary:
Very nice!
> 1. View the sequence of N items to be sorted as, and convert it to, a
> sequence of K ascending runs (down runs are reversed). To avoid the
> overhead of merging small runs, use binary insertsort to bring run
> lengths up to a minimum (depending on N) in the range [32,64].
> Consequence: a list sorted in ascending order or descending order
> without duplicates will be detected as one run and left as is or
> simply reversed.
>
> 2. For K >= 2, timsort(runs) = timmerge(timsort(runs[:j]),
> timesort(runs[j:])). Note: the merge tree is constructed from the
> bottom up rather than from the top down as might be implied by the
> above description.
It's worth noting that there's *some* j for which that's true, but you can't
know which j until the algorithm ends. Merging alternates with finding "the
next" run, to get benefit from that the run just found is still high in the
memory hierarchy. It may or may not get merged into one or more preceding
runs immediately, depending on the unknowable-in-advance natural run
lengths, and that leaves j a mystery until the last run has been found.
> 3. Merge intelligently (including 'galloping') to minimize compares
> and moves and exploit nonrandom structure in the original list.
When you put it that way, I'm surprised the implementation took more than 30
lines <wink>.