My (late) beef with Simple Generator syntax (PEP 255)
Cameron Horn
class foo:
pass
If I want to write an empty function, I write this:
def foo():
pass
-or-
def foo():
return
If I want to write an empty generator, I write this(?):
def foo():
return
yield "never"
Assuming that having an empty generator is as valid as empty functions and classes, what kind of linguistic cruft is that? I expect better out of python, dang it.
What I'd like to write is:
gen foo():
return
Alan Kennedy
> If I want to write an empty generator, I write this(?):
> def foo():
> return
> yield "never"
> Assuming that having an empty generator is as valid as empty functions
> and classes, what kind of linguistic cruft is that? I expect better
> out of python, dang it.
>
> What I'd like to write is:
> gen foo():
> return
What is the semantic of "an empty generator", i.e. what does it mean? A
generator that never generates any values?
Would this fulfill your requirement? (And not upset you so much :-)
def nullgen():
raise StopIteration
yield "never called"
>>> for x in nullgen():
... print x
...
>>>
>>>
It doesn't bother me.
--
alan kennedy
-----------------------------------------------------
check http headers here: http://xhaus.com/headers
email alan: http://xhaus.com/mailto/alan
David Eppstein
Alan Kennedy <ala...@hotmail.com> wrote:
> Cameron Horn wrote:
>
> > If I want to write an empty generator, I write this(?):
> > def foo():
> > return
> > yield "never"
>
> What is the semantic of "an empty generator", i.e. what does it mean? A
> generator that never generates any values?
What I wonder is, why does it need to be a generator? What's wrong with
def foo(): return ( )
?
--
David Eppstein UC Irvine Dept. of Information & Computer Science
epps...@ics.uci.edu http://www.ics.uci.edu/~eppstein/
Cameron Horn
> What is the semantic of "an empty generator", i.e. what does it mean? A
> generator that never generates any values?
Yes, absolutely. I find these empty things to be very useful, both as placeholders for things that I'll implement later and places where I mean to have nothing happen.
> Would this fulfill your requirement? (And not upset you so much :-)
>
> def nullgen():
> raise StopIteration
> yield "never called"
> >>> for x in nullgen():
> ... print x
> ...
> >>>
> >>>
I appreciate your attempt to soothe my delicate sensibilities ;), but no. The fact that a line that I know isn't being executed changes my function, not just in a behavioral way but a structural one, gives me the willies.
> It doesn't bother me.
Syntactic cruft bothers me; that's why I like Python so much.
If generators acted like other functions (i.e., could be used interchangably) I'd have no problem. But they can't, so I'm bothered.
Cameron Horn
On Wednesday, Nov 13, 2002, at 01:55PM, David Eppstein <epps...@ics.uci.edu> wrote:
>In article <3DD2C505...@hotmail.com>,
> Alan Kennedy <ala...@hotmail.com> wrote:
>
>> Cameron Horn wrote:
>>
>> > If I want to write an empty generator, I write this(?):
>> > def foo():
>> > return
>> > yield "never"
>>
>> What is the semantic of "an empty generator", i.e. what does it mean? A
>> generator that never generates any values?
>
>What I wonder is, why does it need to be a generator? What's wrong with
>def foo(): return ( )
>?
You're right if it's a one-time thing. However, if I've got a scaffolding like this, then it matters.
def process_generator(function):
for x in function():
print x
Cameron Horn
On Wednesday, Nov 13, 2002, at 02:26PM, <sism...@hebmex.com> wrote:
>> From: Cameron Horn [mailto:cam...@mac.com]
>> Sent: Wednesday, November 13, 2002 4:26 PM
>>
>> On Wednesday, Nov 13, 2002, at 02:19PM, <sism...@hebmex.com> wrote:
>>
>> >> From: Cameron Horn [mailto:cam...@mac.com]
>> >> Sent: Wednesday, November 13, 2002 4:20 PM
>> >>
>> >> On Wednesday, Nov 13, 2002, at 02:16PM, David Eppstein
>> >> <epps...@ics.uci.edu> wrote:
>> >>
>> >> >On 11/13/02 2:11 PM -0800 Cameron Horn <cam...@mac.com> wrote:
>> >> >>> What I wonder is, why does it need to be a generator?
>> >> What's wrong with
>> >> >>> def foo(): return ( )
>> >> >>> ?
>> >> >>
>> >> >> You're right if it's a one-time thing. However, if I've got a
>> >> >> scaffolding like this, then it matters.
>> >> >>
>> >> >> def process_generator(function):
>> >> >> for x in function():
>> >> >> print x
>> >> >
>> >> >How does the function foo() I defined above fail to work in this
>> >> >scaffolding?
>> >>
>> >> In the "TypeError: iteration over non-sequence" sort of way.
>> >>
>> >
>> >An empty tuple is a sequence also;
>> >
>> >the "return ( )" thing is returning an empty tuple,
>> >not "None". You can make it clearer by using
>> >"return tuple()" rather than "return ()".
>>
>> Oops. Missed those parentheses. Then it is completely
>> useful, unless you end up using the "next" method.
>>
>
>In that case, if you reckon you're going to use .next()
>directly, then you should change 'foo' to something like
>the following:
>
>def foo(): return iter(tuple())
>
>It's going to return a valid iterator which is going to
>try to iterate through an empty tuple, so it's going
>to raise StopIteration the first time you call .next().
>
>
Which is the obvious, clear and consistent way to create a generator that doesn't do anything. Unless you're checking types or doing any other low-level things that you probably oughtn't. I return to my original point. And then, since everything works and I'm just complaining about a tiny bit of syntax, I shut up.
sism...@hebmex.com
Interesting coincidence.
I'm thinking of applying generators for a server application I'm
creating, in which I'd use generators to simulate threading, like
the cooperative multithreading available in Forth.
It seems quite easy to just append all the generators that the
application receives, kind of like tasks, into a list, and
item by item, pop(0) it, do a .next(), and reappend it.
The question is, what is more efficient:
# Case one: enter a for loop and immediately break after
# reappending to the tasks list; if the generator raises
# StopIteration(), the append will never happen.
gen = tasks_list.pop(0)
for _ in gen:
tasks_list.append(gen)
break
# Case two: use try-except.
gen = tasks_list.pop(0)
try:
gen.next()
tasks_list.append(gen)
except StopIteration:
pass
Lets see what happens in a while.
-gustavo
Cameron Horn
Cameron Horn
sism...@hebmex.com
>
> On Wednesday, Nov 13, 2002, at 02:19PM, <sism...@hebmex.com> wrote:
>
> >> From: Cameron Horn [mailto:cam...@mac.com]
> >> Sent: Wednesday, November 13, 2002 4:20 PM
> >>
> >
> >the "return ( )" thing is returning an empty tuple,
> >not "None". You can make it clearer by using
> >"return tuple()" rather than "return ()".
>
> Oops. Missed those parentheses. Then it is completely
> useful, unless you end up using the "next" method.
>
In that case, if you reckon you're going to use .next()
directly, then you should change 'foo' to something like
the following:
def foo(): return iter(tuple())
It's going to return a valid iterator which is going to
try to iterate through an empty tuple, so it's going
to raise StopIteration the first time you call .next().
HTH
-gustavo
sism...@hebmex.com
> Sent: Wednesday, November 13, 2002 4:20 PM
>
> On Wednesday, Nov 13, 2002, at 02:16PM, David Eppstein
> <epps...@ics.uci.edu> wrote:
>
> >On 11/13/02 2:11 PM -0800 Cameron Horn <cam...@mac.com> wrote:
> >>> What I wonder is, why does it need to be a generator?
> What's wrong with
> >>> def foo(): return ( )
> >>> ?
> >>
> >> You're right if it's a one-time thing. However, if I've got a
> >> scaffolding like this, then it matters.
> >>
> >> def process_generator(function):
> >> for x in function():
> >> print x
> >
> >How does the function foo() I defined above fail to work in this
> >scaffolding?
>
> In the "TypeError: iteration over non-sequence" sort of way.
>
An empty tuple is a sequence also;
the "return ( )" thing is returning an empty tuple,
not "None". You can make it clearer by using
"return tuple()" rather than "return ()".
-gustavo
David Eppstein
>> What I wonder is, why does it need to be a generator? What's wrong with
>> def foo(): return ( )
>> ?
>
> You're right if it's a one-time thing. However, if I've got a
> scaffolding like this, then it matters.
>
> def process_generator(function):
> for x in function():
> print x
How does the function foo() I defined above fail to work in this
scaffolding?
François Pinard
> >def foo(): return iter(tuple())
> Which is the obvious, clear and consistent way to create a generator that
> doesn't do anything.
I was to suggest:
def foo(): return iter([])
but using a tuple is probably tinily faster. No need to call tuple(),
however, just write an empty tuple constant:
def foo(): return iter(())
--
François Pinard http://www.iro.umontreal.ca/~pinard
Terry Reedy
"Cameron Horn" <cam...@mac.com> wrote in message
news:mailman.1037219902...@python.org...
> If I want to write an empty generator, I write this(?):
> def foo():
> return
> yield "never"
>
> Assuming that having an empty generator is as valid as empty
functions and classes,
> what kind of linguistic cruft is that? I expect better out of
python, dang it.
I challenge the assumption. A generator, by definition, is a
(nonempty) function (and not a separate generator type) that returns
an iterator with __iter__ and next methods.
TJR
Stuart D. Gathman
>> def nullgen():
>> raise StopIteration
>> yield "never called"
Yuck. How about a built-in null generator that can be assigned to any
desired name? It's not like the implementation needs to be customized or
anything :-)
--
Stuart D. Gathman <stu...@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flamis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.
Cameron Horn
I agree, sort of. The thing is that the implementation seems designed to
let you ignore all those iteration details in most simple cases. I just
find it weird that writing the simplest possible generator, the one that
iterates over no elements, either requires that you write that bit of
nonsense above or know enough about the implementation to return an empty
iterator.
Do you think that a generator that returns an empty iterator (i.e., one
that has no elements, as written above) isn't valid or useful? Or that as
written above doesn't make such a generator?
Terry Reedy
>>> If I want to write an empty generator, I write this(?):
>>> def foo():
>>> return
>>> yield "never"
<my previous comment clipped>
I prefer (see below for why)
def null():
raise StopIteration
yield 1/0
> Do you think that a generator that returns an empty iterator (i.e.,
one
> that has no elements, as written above) isn't valid or useful? Or
that as
> written above doesn't make such a generator?
Yes, an empty iterator and a generator function that produces such
could occasionally be useful just like empty functions and classes.
And yes, I checked: your foo (and my null) are generator functions
that each produce a generator object whose next() method immediately
raises StopIteration when first called.
Directly writing an empty iterator class is easy: make 'raise
StopIteration' the only statement in the next method. (A simple
return does not work here, so I prefer the same in a generator where
return does work.) As you noticed, producing an empty iterator with a
generator function is less graceful even though its less work. The
yield statement does double duty: flag that the function is a
generator and actually yield the value when is is 'converted' to an
iterator. So you need a 'fake' yield after raising the exception to
do the first job even though you do not want to do the second.
Perhaps 'yield 1/0' makes it clearer that nothing is really yielded.
Whether or not a 'generator that returns an empty iterator' (which is
something) is itself 'empty' is a matter of definitional choice.
Because I understand the hidden trick, I see it as not, for the same
reason that [[]] is not null even though [] is.
Part of the problem here is that 'generator' is overloaded with three
related meanings. We write a 'generator'. Python compiles it into a
generator function with code that we did not write. When called, that
function produces a generator object that follows the iterator
protocol and whose next method is a 'resumable function' containing
the code that we did write. The latter two work together with the for
loop machinery to produce the illusion that the generator we write
acts like it seems to. So when you used the phrase 'empty generator',
you may have been thinking most of meaning one while I substituted
meaning two.
Terry J. Reedy
Just
Cameron Horn <cam...@mac.com> wrote:
IMO the best idiom for this currently is
def foo():
while 0:
yield "whatever"
That's much better than the putting the yield after a return...
Just
Alex Martelli
>
> "Cameron Horn" <cam...@mac.com> wrote in message
> news:mailman.103724041...@python.org...
>>>> If I want to write an empty generator, I write this(?):
>>>> def foo():
>>>> return
>>>> yield "never"
>
> <my previous comment clipped>
>
> I prefer (see below for why)
>
> def null():
> raise StopIteration
> yield 1/0
I think that if I ever had to write an empty generator I'd prefer a
single-statement version, such as, for example:
def empty():
yield iter([]).next()
The .next call to the iterator object, constructed on the fly on any
empty iterable, raises StopIteration, and empty()'s yield statement
just cleanly propagates it. More generally, the preference is for
any statement of the form:
yield <any expression that raises StopIteration>
calling .next() on an empty iterator seems the most natural way to
get "an expression that raises StopIteration", but there may be
better ones -- or more natural ways to get an empty iterator than
calling iter([]) -- though none readily comes to mind offhand.
Alex
Alan Kennedy
> Syntactic cruft bothers me; that's why I like Python so much.
>
> If generators acted like other functions (i.e., could be used
> interchangably) I'd have no problem. But they can't, so I'm bothered.
I completely see your point, and on reflection, I'm less certain now
that I'm not bothered by it.
It would be certainly be much nicer to specify a null generator using
something like this
gen null():
pass
However, given that this has proven to be quite a contentious topic
before, and that people who are far more clever and experienced in
language design than I am were unable to reach a clear concensus on the
syntax issue, I think I'm not going to rush in where angels fear to
tread:
I think I'm happy just to live with the slight oddity of the current
syntax, and trust Guido's intuition.
Here's an example of a thread discussing the subject that could have
turned nasty, if pythonistas weren't such nice people ;-)
regards,
Alan Kennedy
> I think that if I ever had to write an empty generator I'd prefer a
> single-statement version, such as, for example:
>
> def empty():
> yield iter([]).next()
Alex,
You have an amazing knack for finding elegant solutions!
I tried to come up with a one-liner that raised StopIteration, but
couldn't arrive at one.
If it was possible to raise an exception by instantiating an exception
class, then it would be even simpler, e.g.
def empty():
yield StopIteration()
But that, of course, doesn't work.
Here is one last stab at it, which has the simplicity of my above
solution, at the expense of having to define a new class:
class emptyGenerator:
def __init__(self):
raise StopIteration()
def emptyGen():
yield emptyGenerator()
Stuart Gathman said "How about a built-in null generator that can be
assigned to any desired name?"
Shouldn't be too hard to include such an "emptyGenerator" class
definition, for example in "site.py".
But I think I prefer the conciseness of Alex' solution.
Niki Spahiev
CH> Assuming that having an empty generator is as valid as empty functions and classes, what kind of linguistic cruft is that? I expect better out of python, dang it.
CH> What I'd like to write is:
CH> gen foo():
CH> return
def foo():
if 0: yield None
optimized to empty generator by compiler
--
Best regards,
Niki Spahiev
Neil Schemenauer
> IMO the best idiom for this currently is
>
> def foo():
> while 0:
> yield "whatever"
This whole discussion is silly, IMHO. The best idiom is:
def foo():
return []
If you really want an interator, then:
def foo():
return iter([])
Requiring that the type returned by foo be <type 'generator'> is
un-Pythonic and silly. If you do want that, the best solution would be
the very clever suggestion of:
def foo()
if 0: yield None
Neil
Alex Martelli
> Just wrote:
>> IMO the best idiom for this currently is
>>
>> def foo():
>> while 0:
>> yield "whatever"
>
> This whole discussion is silly, IMHO. The best idiom is:
>
> def foo():
> return []
Actually, I suspect returning () instead -- empty tuple, not
empty list -- is even better. To wit:
import time
def emptylist(): return []
def emptytuple(): return ()
def timit(func):
repeater = [None] * (100*1000)
start = time.clock()
for x in repeater: func()
stend = time.clock()
return '%s %.2f' % (func.func_name, stend-start)
for f in (emptylist, emptytuple) * 3:
print timit(f)
[alex@lancelot ball]$ python2.2 -O ba.py
emptylist 0.16
emptytuple 0.07
emptylist 0.14
emptytuple 0.08
emptylist 0.15
emptytuple 0.07
[alex@lancelot ball]$ python2.3 -O ba.py
emptylist 0.08
emptytuple 0.07
emptylist 0.07
emptytuple 0.05
emptylist 0.08
emptytuple 0.07
[alex@lancelot ball]$
admittedly the difference is going to be even less in 2.3,
but still, it's about 100 milliseconds per million empties one
returns -- a few billions, and we'd be talking about REAL
performance issues. Surely any implementation of something
as frequent as "returning an empty iterable" cannot aspire
to the title of "the best" with that sort of performance hit.
[Note for the humor-impaired: I'd gladly sprinkle suitable
emoticons throughout the post, but unfortunately my Linux
installation doesn't seem to have /dev/emoticons right now --
must be devfs up to its usual tricks -- sorry].
Alex
Gerhard Häring
> Neil Schemenauer wrote:
>> Just wrote:
>>> IMO the best idiom for this currently is
>>>
>>> def foo():
>>> while 0:
>>> yield "whatever"
>>
>> This whole discussion is silly, IMHO. The best idiom is:
>>
>> def foo():
>> return []
>
> Actually, I suspect returning () instead -- empty tuple, not
> empty list -- is even better. To wit:
>
> admittedly the difference is going to be even less in 2.3,
> but still, it's about 100 milliseconds per million empties one
> returns -- a few billions, and we'd be talking about REAL
> performance issues. Surely any implementation of something
> as frequent as "returning an empty iterable" cannot aspire
> to the title of "the best" with that sort of performance hit.
Yeah, it's quite likely that this will be the bottleneck of your app. That's
why you should squeeze the last percent or so of optimization out and return an
empty string:
[Python 2.2.2/win32]
D:\tmp>python ba.py
emptylist 1.71
emptytuple 1.16
emptystring 1.15
emptylist 1.72
emptytuple 1.16
emptystring 1.15
emptylist 1.72
emptytuple 1.16
emptystring 1.15
No /dev/emoticon on Windows, either.
--
Gerhard Häring
OPUS GmbH München
Tel.: +49 89 - 889 49 7 - 32
http://www.opus-gmbh.net/
Greg Ewing
statement that could be put in front of another
statement to, effectively, comment it out,
e.g.
def foo():
# This function does nothing
don't print "Hello"
Then an empty generator could be written
def g():
don't yield None
Now, since this isn't going to do anything,
it should be permissible to omit any expressions
that would otherwise be required in a statement
covered by a "don't", leaving just
def g():
don't yield
--
Greg Ewing, Computer Science Dept,
University of Canterbury,
Christchurch, New Zealand
http://www.cosc.canterbury.ac.nz/~greg
Greg Ewing
>
> class emptyGenerator:
>
> def __init__(self):
> raise StopIteration()
>
> def emptyGen():
> yield emptyGenerator()
This is quite nice, but it's misnamed slightly.
Something called xxxGenerator should be returning
an iterator when you call it, but this isn't.
It would be nicer if it were called Nothing.
Then you could write
def emptyGen():
yield Nothing()
Skip Montanaro
Greg> be put in front of another statement to, effectively, comment it
Greg> out,
Dunno if your suggestion is serious, but if so, you could extend the pass
statement, yes?
--
Skip Montanaro - sk...@pobox.com
http://www.mojam.com/
http://www.musi-cal.com/
logistix
> Then an empty generator could be written
>
> def g():
> don't yield None
>
> Now, since this isn't going to do anything,
> it should be permissible to omit any expressions
> that would otherwise be required in a statement
> covered by a "don't", leaving just
>
> def g():
> don't yield
Not quite as clean, but it can already be done.
class foo:
def next(self):
pass
def __iter__(self):
return self
Although in either case a for..in loop or list comprehension will repeat infinitely.
Alex Martelli
> Alan Kennedy wrote:
>
>> class emptyGenerator:
>>
>> def __init__(self):
>> raise StopIteration()
>>
>> def emptyGen():
>> yield emptyGenerator()
>
> This is quite nice, but it's misnamed slightly.
> Something called xxxGenerator should be returning
> an iterator when you call it, but this isn't.
>
> It would be nicer if it were called Nothing.
> Then you could write
>
> def emptyGen():
> yield Nothing()
>
It seems to me that there's no real usefulness in making Nothing a class --
it just takes an extra statement or so to come to the key raise statement.
So why not use a function instead:
def Nothing(): raise StopIteration
def emptyGen(): yield Nothing()
Alex
Aahz
Skip Montanaro <sk...@pobox.com> wrote:
> Greg> I have an idea: Suppose there were a "don't" statement that could
> Greg> be put in front of another statement to, effectively, comment it
> Greg> out,
>
>Dunno if your suggestion is serious, but if so, you could extend the pass
>statement, yes?
Re-read the Subject: line. ;-)
--
Aahz (aa...@pythoncraft.com) <*> http://www.pythoncraft.com/
A: No.
Q: Is top-posting okay?
Piet van Oostrum
GE> I have an idea: Suppose there were a "don't"
GE> statement that could be put in front of another
GE> statement to, effectively, comment it out,
GE> e.g.
GE> def foo():
GE> # This function does nothing
GE> don't print "Hello"
GE> Then an empty generator could be written
GE> def g():
GE> don't yield None
def emptyGen():
if 0: yield None
--
Piet van Oostrum <pi...@cs.uu.nl>
URL: http://www.cs.uu.nl/~piet [PGP]
Private email: P.van....@hccnet.nl
Jeremy Bowers
> I have an idea: Suppose there were a "don't"
> statement that could be put in front of another
>...
> that would otherwise be required in a statement
> covered by a "don't", leaving just
>
> def g():
> don't yield
That would interact poorly with the other established constructs one would
expect to see in Pythoncal. What the hell does the compiler do with
def stupid():
don't come from beginning
? You can't call it Pythoncal if it can't "come from".
Still, don't might be worth adding none the less...
don't try:
f = file("something", "w")
except exceptions.DidntTryNonError:
pass
A great way to comment out code that happened to be in a try construct in
the first place.
Alan Kennedy
>> class emptyGenerator:
>> def __init__(self):
>> raise StopIteration()
>>
>> def emptyGen():
>> yield emptyGenerator()
Greg wrote:
> This is quite nice, but it's misnamed slightly.
> Something called xxxGenerator should be returning
> an iterator when you call it, but this isn't.
> It would be nicer if it were called Nothing.
> Then you could write
>
> def emptyGen():
> yield Nothing()
While I agree that it was badly named, on thinking further about it, I'm
not sure that "nothing" (i.e. "no thing") is the right name. The right
name is "no thingS". So that would be
class noThings:
def __init__(self):
raise StopIteration()
def emptyGen:
yield noThings()
hair-splitting-ly yrs :-)
Greg Ewing
> Still, don't might be worth adding none the less...
>
> don't try:
> f = file("something", "w")
> except exceptions.DidntTryNonError:
> pass
And of course it will be interesting deciding on
the semantics of
def f():
don't pass
Terry Reedy
"Greg Ewing" <see_repl...@something.invalid> wrote in message
news:3DDC720E...@something.invalid...
> And of course it will be interesting deciding on
> the semantics of
>
> def f():
> don't pass
In The Science of Programming, David Gries (possibly copying
predecessors) introduced the abort == don't pass statement. In
Python, assert 0 is perhaps closest.
TJR