Re: How to make an empty generator?
Robert Kern
> This has to be a stupid question, but :)
>
> I have some generators that do stuff, then start yielding results. On
> occasion, I don't want them to yield anything ever-- they're only really
> "generators" because I want to call them /as/ a generator as part of a
> generalized system.
>
> The only way I can figure out how to make an empty generator is:
>
> def gen():
> # do my one-time processing here
>
> return
> yield
>
> Is there a better way? The return/yield just makes me flinch slightly. I
> tried just raising StopIteration at the end, but of course that didn't work.
class once(object):
def __init__(self, func, *args, **kwds):
self.func = func
self.args = args
self.kwds = kwds
def __iter__(self):
return self
def next(self):
self.func(*self.args, **self.kwds)
raise StopIteration()
Then write regular functions with the one-time processing code (not
generators!). When you go to pass them into your system that wants an iterator,
just wrap it with once(func).
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
Steven D'Aprano
through to my provider.
On 2010-02-18 16:25 PM, Stephen Hansen wrote:
> This has to be a stupid question, but :)
>
> I have some generators that do stuff, then start yielding results. On
> occasion, I don't want them to yield anything ever-- they're only
> really "generators" because I want to call them /as/ a generator as
> part of a generalized system.
>
> The only way I can figure out how to make an empty generator is:
>
> def gen():
> # do my one-time processing here
>
> return
> yield
If all you want is a generator that doesn't yield anything, then surely
there isn't any one-time processing and you don't need the comment?
>> Is there a better way? The return/yield just makes me flinch slightly.
Meh. An empty generator is a funny thing to do, so it's not bad that it
reads a bit funny. I don't have a problem with it.
If it really annoys you, you could do this:
def empty():
for x in []:
yield
--
Steven
Robert Kern
> On 2010-02-18 16:25 PM, Stephen Hansen wrote:
>
> > This has to be a stupid question, but :)
>
> > I have some generators that do stuff, then start yielding results. On
> > occasion, I don't want them to yield anything ever-- they're only
> > really "generators" because I want to call them /as/ a generator as
> > part of a generalized system.
>
> > The only way I can figure out how to make an empty generator is:
>
> > def gen():
> > # do my one-time processing here
>
> > return
> > yield
>
> If all you want is a generator that doesn't yield anything, then surely
> there isn't any one-time processing and you don't need the comment?
Sure there is. Python doesn't know that nothing gets yielded until it hits the
return statement before the yield. When it calls .next() on the iterator, the
code elided by the comment executes, then the return is hit, a StopIteration
exception is raised, and the iteration is complete.
Ben Finney
> On 2010-02-18 16:25 PM, Stephen Hansen wrote:
> > The only way I can figure out how to make an empty generator is:
> >
> > def gen():
> > # do my one-time processing here
> >
> > return
> > yield
> >
> > Is there a better way? The return/yield just makes me flinch
> > slightly. I tried just raising StopIteration at the end, but of
> > course that didn't work.
No need to define functions or classes; let a generator expression take
care of it for you::
>>> foo = (x for x in list())
>>> foo.next()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration
--
\ “Value your freedom or you will lose it, teaches history. |
`\ “Don't bother us with politics,” respond those who don't want |
_o__) to learn.” —Richard Stallman, 2002 |
Ben Finney
Arnaud Delobelle
[...]
> No need to define functions or classes; let a generator expression take
> care of it for you::
>
> >>> foo = (x for x in list())
> >>> foo.next()
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> StopIteration
What about
foo = iter('')
Then there is the interesting
foo = iter(int, 0)
:)
--
Arnaud
Terry Reedy
> This has to be a stupid question, but :)
>
> I have some generators that do stuff, then start yielding results. On
> occasion, I don't want them to yield anything ever-- they're only really
> "generators" because I want to call them /as/ a generator as part of a
> generalized system.
>
>
> def gen():
> # do my one-time processing here
>
> return
> yield
>
> Is there a better way? The return/yield just makes me flinch slightly. I
> tried just raising StopIteration at the end, but of course that didn't work.
Did you try
def gen():
<stuff>
if 0: yield
?
tjr
Robert Kern
> Robert Kern<rober...@gmail.com> writes:
>
>> On 2010-02-18 16:25 PM, Stephen Hansen wrote:
>>> The only way I can figure out how to make an empty generator is:
>>>
>>> def gen():
>>> # do my one-time processing here
>>>
>>> return
>>> yield
>>>
>>> Is there a better way? The return/yield just makes me flinch
>>> slightly. I tried just raising StopIteration at the end, but of
>>> course that didn't work.
>
> No need to define functions or classes; let a generator expression take
> care of it for you::
>
> >>> foo = (x for x in list())
> >>> foo.next()
> Traceback (most recent call last):
> File "<stdin>", line 1, in<module>
> StopIteration
He doesn't want *any* empty generator. He wants an iterator that executes some
given side-effect-producing code then immediately raises the StopIteration.
Robert Kern
> On Thu, Feb 18, 2010 at 2:56 PM, Robert Kern <rober...@gmail.com
> <mailto:rober...@gmail.com>> wrote:
>
> class once(object):
> def __init__(self, func, *args, **kwds):
> self.func = func
> self.args = args
> self.kwds = kwds
>
> def __iter__(self):
> return self
>
> def next(self):
> self.func(*self.args, **self.kwds)
> raise StopIteration()
>
>
> sideeffect_only or something, but yeah, that's the right approach at least.
Well, with a decorator, you could even use the cringeworthy return/yield syntax
while keeping it hidden from your users (not to mention reducing the amount of
boilerplate people need to write).
from functools import wraps
def sideeffect_only(func):
@wraps(func)
def wrapper(*args, **kwds):
func(*args, **kwds)
# Some helpful comment about why there is a return/yield.
return
yield
return wrapper
But you can also write one where the wrapper() returns a once() instance. It
might be useful to use the once class instead of a generator such that you can
write code that distinguishes side-effect only iterators from other iterators in
your system. It might be useful for debugging if nothing else.
Steven D'Aprano
> > If all you want is a generator that doesn't yield anything, then
> > surely there isn't any one-time processing and you don't need the
> > comment?
>
> Sure there is. Python doesn't know that nothing gets yielded until it
> hits the return statement before the yield. When it calls .next() on the
> iterator, the code elided by the comment executes, then the return is
> hit, a StopIteration exception is raised, and the iteration is complete.
I don't understand why you care about having *any* code before the
StopIteration. That's like:
def empty():
for x in range(1000):
pass # Spin wheels uselessly
return
yield
What's the point of the wheel spinning? Did I miss something?
--
Steven
Ben Finney
> What about
> foo = iter('')
That doesn't return a generator.
>>> foo = iter('')
>>> foo
<listiterator object at 0xf7cd3ed0>
Whether the OP needs to create a generator, or just any iterable type,
isn't clear.
Robert Kern <rober...@gmail.com> writes:
> He doesn't want *any* empty generator. He wants an iterator that
> executes some given side-effect-producing code then immediately raises
> the StopIteration.
Ah, hm. That's a rather perverse use case, but I'm sure the OP has their
reasons. It's going to confuse a reader who isn't expecting it, no
matter how simply it's done.
So, I think the best answer is what has already been suggested, but that
it's perverse enough that the hack *needs* a comment to say why it's
being done.
def make_empty_generator_with_side_effect():
""" Make a generator that does important work, but is empty. """
# Do the important work here.
spam = object()
# Make this function return an empty generator.
if False: yield
--
\ “When cryptography is outlawed, bayl bhgynjf jvyy unir |
`\ cevinpl.” —Anonymous |
_o__) |
Ben Finney
Steven D'Aprano
> He wants an iterator that executes some given side-effect-producing code
> then immediately raises the StopIteration.
Ah, that's the bit I missed!
Yewww. Programming by side-effect... I hope the original poster has a
good reason for such a thing.
--
Steven
Mel
I wonder whether it's for some kind of framework with a main loop like
for it in list_of_iterables:
for x in it:
do_this_or_that (x)
where, every once in a while one wants to throw some arbitrary code into the
process, in the form of an empty iterable with side effects.
Mel.
Steve Holden
> On Thu, Feb 18, 2010 at 2:56 PM, Robert Kern <rober...@gmail.com
> <mailto:rober...@gmail.com>> wrote:
>
> class once(object):
> def __init__(self, func, *args, **kwds):
> self.func = func
> self.args = args
> self.kwds = kwds
>
> def __iter__(self):
> return self
>
> def next(self):
> self.func(*self.args, **self.kwds)
> raise StopIteration()
>
>
> Hmm, yeah. I'd probably tweak it into a decorator and name it
> sideeffect_only or something, but yeah, that's the right approach at least.
>
> asking me, "... what the hell is return / yield?" and although this is
> more expensive, its less clear to me.
>
> Thanks.
> --S
>
So add a comment to the yield statement so it reads
yield # force empty generator
I realise that Jacob Kaplan-Moss calls comments in the code "lies", but
the reader really only needs a small clue.
regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/
Holden Web LLC http://www.holdenweb.com/
UPCOMING EVENTS: http://holdenweb.eventbrite.com/
Robert Kern
Yes. That is exactly what the OP said in his original post:
"""
I have some generators that do stuff, then start yielding results. On occasion,
I don't want them to yield anything ever-- they're only really "generators"
because I want to call them /as/ a generator as part of a generalized system.
"""
--
Robert Kern
> Robert Kern<rober...@gmail.com> writes:
>
>> He doesn't want *any* empty generator. He wants an iterator that
>> executes some given side-effect-producing code then immediately raises
>> the StopIteration.
>
> Ah, hm. That's a rather perverse use case, but I'm sure the OP has their
> reasons.
Which he explained fairly clearly, I thought, in his original post.
Steven D'Aprano
>>> What's the point of the wheel spinning? Did I miss something?
>>
>> I wonder whether it's for some kind of framework with a main loop like
>>
>> for it in list_of_iterables:
>> for x in it:
>> do_this_or_that (x)
>>
>> where, every once in a while one wants to throw some arbitrary code
>> into the process, in the form of an empty iterable with side effects.
>
> Yes. That is exactly what the OP said in his original post:
>
> """
> I have some generators that do stuff, then start yielding results. On
> occasion, I don't want them to yield anything ever-- they're only really
> "generators" because I want to call them /as/ a generator as part of a
> generalized system. """
But he doesn't say anything about side-effects. If you're passing
generators around, sometimes you want an empty generator, just as
sometimes you want an empty string, or list, or whatever.
--
Steven
Ben Finney
> On 2010-02-18 18:33 PM, Ben Finney wrote:
> > Robert Kern<rober...@gmail.com> writes:
> >
> >> He doesn't want *any* empty generator. He wants an iterator that
> >> executes some given side-effect-producing code then immediately
> >> raises the StopIteration.
> >
> > Ah, hm. That's a rather perverse use case, but I'm sure the OP has their
> > reasons.
>
> Which he explained fairly clearly, I thought, in his original post.
(The original post isn't available to me; the reference in your reply
isn't accessible AFAICT.)
In the part of the original that you quoted, he speaks only of empty
generators (easy and clean), not generators that exist only for the
purpose of side-effects without yielding any items.
It's that latter that I describe as perverse, and I would think it worth
some effort to determine if that can be avoided by a different approach.
--
\ “The problem with television is that the people must sit and |
`\ keep their eyes glued on a screen: the average American family |
_o__) hasn't time for it.” —_The New York Times_, 1939 |
Ben Finney
Arnaud Delobelle
> Arnaud Delobelle <arn...@googlemail.com> writes:
>
>> What about
>> foo = iter('')
>
> That doesn't return a generator.
>
> >>> foo = iter('')
> >>> foo
> <listiterator object at 0xf7cd3ed0>
>
> Whether the OP needs to create a generator, or just any iterable type,
> isn't clear.
If it walks and quacks like a duck... Anyway it's not just an iterable
object, it's an iterator. I can't really imagine that there would be
some code which would be happy with generators but not with iterators
(as long as you can't send anything to them, which is always the case
with an empty generator).
--
Arnaud
Ben Finney
> Ben Finney <ben+p...@benfinney.id.au> writes:
> > Whether the OP needs to create a generator, or just any iterable
> > type, isn't clear.
>
> If it walks and quacks like a duck... Anyway it's not just an iterable
> object, it's an iterator. I can't really imagine that there would be
> some code which would be happy with generators but not with iterators
> (as long as you can't send anything to them, which is always the case
> with an empty generator).
I can't imagine that someone would want to create a generator that's
always empty, but has some side-effect that is the *real* purpose for
using the generator.
Clearly, none of us should let our limited imaginations be the guide to
what people actually want to do.
--
\ “Generally speaking, the errors in religion are dangerous; |
`\ those in philosophy only ridiculous.” —David Hume, _A Treatise |
_o__) of Human Nature_, 1739 |
Ben Finney
Robert Kern
> On Fri, 19 Feb 2010 00:15:20 -0600, Robert Kern wrote:
>
>>>> What's the point of the wheel spinning? Did I miss something?
>>>
>>> I wonder whether it's for some kind of framework with a main loop like
>>>
>>> for it in list_of_iterables:
>>> for x in it:
>>> do_this_or_that (x)
>>>
>>> where, every once in a while one wants to throw some arbitrary code
>>> into the process, in the form of an empty iterable with side effects.
>>
>> Yes. That is exactly what the OP said in his original post:
>>
>> """
>> I have some generators that do stuff, then start yielding results. On
>> occasion, I don't want them to yield anything ever-- they're only really
>> "generators" because I want to call them /as/ a generator as part of a
>> generalized system. """
>
> But he doesn't say anything about side-effects.
"I have some generators *that do stuff*, then start yielding results." [emphasis
mine]. Then he gives an example of a generator that does side-effect stuff and
returning before yielding anything.
Robert Kern
> Robert Kern<rober...@gmail.com> writes:
>
>> On 2010-02-18 18:33 PM, Ben Finney wrote:
>>> Robert Kern<rober...@gmail.com> writes:
>>>
>>>> He doesn't want *any* empty generator. He wants an iterator that
>>>> executes some given side-effect-producing code then immediately
>>>> raises the StopIteration.
>>>
>>> Ah, hm. That's a rather perverse use case, but I'm sure the OP has their
>>> reasons.
>>
>> Which he explained fairly clearly, I thought, in his original post.
>
> (The original post isn't available to me; the reference in your reply
> isn't accessible AFAICT.)
You responded to my post which quoted his in full.
> In the part of the original that you quoted, he speaks only of empty
> generators (easy and clean), not generators that exist only for the
> purpose of side-effects without yielding any items.
"""
I have some generators *that do stuff*, then start yielding results. On
occasion, I don't want them to yield anything ever-- they're only really
"generators" because I want to call them /as/ a generator as part of a
generalized system.
...
def gen():
# *do my one-time processing here*
return
yield
"""
[emphasis mine]
Seriously, it's all there. I'm rather appalled at the lack of reading
comprehension demonstrated in this thread.
> It's that latter that I describe as perverse, and I would think it worth
> some effort to determine if that can be avoided by a different approach.
By rearchitecting the system to accept things that aren't iterators, yes. But he
may not be in control of that system. And it may not make things cleaner to do
so if he wants to use itertools to compose the iterables, whether they are for
side effects or not, in various ways.
Steven D'Aprano
> On 2010-02-19 00:21 AM, Steven D'Aprano wrote:
>> On Fri, 19 Feb 2010 00:15:20 -0600, Robert Kern wrote:
>>
>>>>> What's the point of the wheel spinning? Did I miss something?
>>>>
>>>> I wonder whether it's for some kind of framework with a main loop
>>>> like
>>>>
>>>> for it in list_of_iterables:
>>>> for x in it:
>>>> do_this_or_that (x)
>>>>
>>>> where, every once in a while one wants to throw some arbitrary code
>>>> into the process, in the form of an empty iterable with side effects.
>>>
>>> Yes. That is exactly what the OP said in his original post:
>>>
>>> """
>>> I have some generators that do stuff, then start yielding results. On
>>> occasion, I don't want them to yield anything ever-- they're only
>>> really "generators" because I want to call them /as/ a generator as
>>> part of a generalized system. """
>>
>> But he doesn't say anything about side-effects.
>
> "I have some generators *that do stuff*, then start yielding results."
> [emphasis mine].
What does "do stuff" have to do with side-effects? Here's a generator
that does stuff, and it has no side-effects.
def generator_that_does_stuff(x):
y = 3*x**2 - 5*x + 1
yield y
"Do stuff" is ambiguous -- it could mean stuff with side-effects, or
stuff without. The first is potentially harmful, the second is pointless.
> Then he gives an example of a generator that does
> side-effect stuff and returning before yielding anything.
Unfortunately the OP's original post isn't visible to me, so I can only
respond to your post, which may or may not quote the entire original post.
The only example I have seen was an empty generator with a comment saying
"do my one-time processing here", with *no* indication of what that one-
time processing is supposed to be, why it is necessary, and whether it
has side-effects or not.
Since the OP (apparently) hasn't seen fit to comment, we're still all
guessing what he means. It's certainly possible, likely even, that he's
using generators that operate by side-effect: that explanation is
consistent with his request. Personally, I can't imagine that would be
good coding practice, but I could be wrong.
--
Steven
Terry Reedy
> Much to my embarrassment, sometime last night I realized I was being a
> complete idiot, and the 'correct' way to handle this in my scenario is
> really just:
>
> def initialize():
> # do one time processing here
>
> return []
>
> A generator is just a callable that returns an iterator, after all.
Confusing generators and generator functions is, well, confusing.
For future reference, and clarity of communication in Pythonland,
generator function: function that produces a generator when called; if
python coded, its body contains 'yield'.
generator: iterator produced by a generator function; has .__next__ and
self-returning .__init__, like all other iterators.
generator expression: an expression that evaluates to a generator; the
expression is used to create a temporary anonymous generator function
that is called to produce the generator and is then discarded.
>>> def gf(): yield 1
>>> gf
<function gf at 0x00F5F4B0>
>>> g=gf()
>>> g
<generator object gf at 0x00F5BE40>
>>> ge = (1 for i in [1])
>>> ge
<generator object <genexpr> at 0x00F5BFD0>
>>> dir(ge)
[..., '__iter__', ..., '__next__', ...]
So, when you said that you send 'generators' to your core system, when
you meant 'generator functions', we were understanably confused. (Your
core system should accept iterator classes also, unless it specifically
checks to avoid duck typing.)
Terry Jan Reedy
John Posner
> On 2/19/2010 12:44 PM, Stephen Hansen wrote:
>
>> Much to my embarrassment, sometime last night I realized I was being a
>> complete idiot, and the 'correct' way to handle this in my scenario is
>> really just:
>>
>> def initialize():
>> # do one time processing here
>>
>> return []
>>
>> A generator is just a callable that returns an iterator, after all.
>
> Confusing generators and generator functions is, well, confusing.
> For future reference, and clarity of communication in Pythonland,
>
> generator function: function that produces a generator when called; if
> python coded, its body contains 'yield'.
>
> generator: iterator produced by a generator function;
I suggest:
iterator produced by a generator function or a generator expression;
has .__next__ and
> self-returning .__init__, like all other iterators.
>
> generator expression: an expression that evaluates to a generator; the
> expression is used to create a temporary anonymous generator function
> that is called to produce the generator and is then discarded.
Note that the Py2.6.4 documentation is inconsistent. AFAICT, it conforms
to Terry's definitions above in most places. But the Glossary says:
generator
A function which returns an iterator. <... more ...>
generator expression
An expression that returns a generator. <... more ...>
The additional verbiage in these definitions mitigates the damage, but I
think the first entry should be headlined *generator function* instead
of *generator*. And the Glossary should include Terry's additional entry
[ as amended by me :-) ]:
generator
An iterator produced by a generator function or a generator
expression.
-John
Steve Holden
John Posner
> John Posner wrote:
>> Note that the Py2.6.4 documentation is inconsistent. AFAICT, it conforms
>> to Terry's definitions above in most places. But the Glossary says:
>>
>> generator
>> A function which returns an iterator.<... more ...>
>>
>> generator expression
>> An expression that returns a generator.<... more ...>
>>
>> The additional verbiage in these definitions mitigates the damage, but I
>> think the first entry should be headlined *generator function* instead
>> of *generator*. And the Glossary should include Terry's additional entry
>> [ as amended by me :-) ]:
>>
>> generator
>> An iterator produced by a generator function or a generator
>> expression.
>>
>> -John
>>
> +1. Can someone submit a documentation patch, please?
>
Will do. -John
John Posner
>>> An iterator produced by a generator function or a generator
>>> expression.
>>>
>>> -John
>>>
>>
>
> Will do. -John
[sorry if this is a dup]
Done: #8012 "Revise generator-related Glossary entries"
-John
Aahz
Terry Reedy <tjr...@udel.edu> wrote:
>
>Confusing generators and generator functions is, well, confusing.
>For future reference, and clarity of communication in Pythonland,
>
>generator function: function that produces a generator when called; if
>python coded, its body contains 'yield'.
>
>generator: iterator produced by a generator function; has .__next__ and
>self-returning .__init__, like all other iterators.
>
>generator expression: an expression that evaluates to a generator; the
>expression is used to create a temporary anonymous generator function
>that is called to produce the generator and is then discarded.
My preference is to use the terms "generator" and "generator iterator"
(abbreviated "gen iter" or "geniter").
--
Aahz (aa...@pythoncraft.com) <*> http://www.pythoncraft.com/
"Many customs in this life persist because they ease friction and promote
productivity as a result of universal agreement, and whether they are
precisely the optimal choices is much less important." --Henry Spencer
Michael Rudolf
> (abbreviated "gen iter" or "geniter").
lol I don't know why, but this sounds like a sex toy to me ;)
Regards,
Michael
Aahz
Michael Rudolf <spamf...@ch3ka.de> wrote:
>Am 24.02.2010 23:58, schrieb Aahz:
>>
>> (abbreviated "gen iter" or "geniter").
>
>lol I don't know why, but this sounds like a sex toy to me ;)
<smirk> And I thought only smutty Americans would have that twitch.