Calling a generator multiple times
Bruce Eckel
times, but the only thing I've been able to come up with is:
import random
rgen = random.Random()
def itemGenerator(dummy):
return rgen.choice(['one', 'two', 'three', 'four'])
print map(itemGenerator, [None]* 25)
This works, but it requires the 'dummy' argument which seems
inelegant. I'll bet someone has a better idea...
Most current information can be found at:
http://www.mindview.net/Etc/notes.html
===================
Bruce Eckel http://www.BruceEckel.com
Contains free electronic books: "Thinking in Java 2e" & "Thinking
in C++ 2e"
Please subscribe to my free newsletter -- just send any email to:
join-eckel-o...@earth.lyris.net
My schedule can be found at:
http://www.mindview.net/Calendar
===================
Mitch Chapman
>
> I'm trying to create a clever way to call a generator multiple
> times, but the only thing I've been able to come up with is:
>
> import random
> rgen = random.Random()
> def itemGenerator(dummy):
> return rgen.choice(['one', 'two', 'three', 'four'])
>
> print map(itemGenerator, [None]* 25)
>
> This works, but it requires the 'dummy' argument which seems
> inelegant. I'll bet someone has a better idea...
If you're using a recent Python release (>= 2.1?) you could
use list comprehensions:
def itemGenerator():
return rgen.choice(['one', 'two', 'three', 'four'])
print [itemGenerator() for i in range(25)]
--
Mitch Chapman
Mitch....@bioreason.com
Terry Reedy
"Bruce Eckel" <Br...@EckelObjects.com> wrote in message
news:mailman.1007665994...@python.org...
> I'm trying to create a clever way to call a generator multiple
> times, but the only thing I've been able to come up with is:
>
> import random
> rgen = random.Random()
> def itemGenerator(dummy):
> return rgen.choice(['one', 'two', 'three', 'four'])
>
> print map(itemGenerator, [None]* 25)
>
> This works, but it requires the 'dummy' argument which seems
> inelegant. I'll bet someone has a better idea...
Use for i in range(25):
Terry J. Reedy
Bruce Eckel
comprehensions described as:
[i for i in foo where bar]
I didn't realized that the leftmost element in the comprehension
was actually evaluated.
Thanks! New insights into comprehensions...
*********** REPLY SEPARATOR ***********
>--
>http://mail.python.org/mailman/listinfo/python-list
Steven Majewski
On Thu, 6 Dec 2001, Bruce Eckel wrote:
> I'm trying to create a clever way to call a generator multiple
> times, but the only thing I've been able to come up with is:
>
> import random
> rgen = random.Random()
> def itemGenerator(dummy):
> return rgen.choice(['one', 'two', 'three', 'four'])
>
> print map(itemGenerator, [None]* 25)
>
> This works, but it requires the 'dummy' argument which seems
> inelegant. I'll bet someone has a better idea...
Somebody already mentioned list comprehensions.
For clarity, I'ld probably define the function without the dummy
variable, and then use a lambda expression to add a dummy back again:
def itemGenerator(): ...
map( lambda dummy: itemGenerator(), [None]*25 )
Another thing is just use random.Random().choice on copies of the
items:
map( random.Random().choice, [ item-list ]*25 )
or if you need to keep the random generator around for more:
rgen = random.Random()
map( rgen.choice, [ item-list ]*25 )
or if you're doing this alot, maybe just define your function to
take a collection and an optional number:
def itemGenerator( collection, n=1 ):
if n == 1:
return rgen.choice(collection)
else:
return map( rgen.choice, [collection]*n )
>>> itemGenerator(['one','two','three','four'])
'four'
>>> itemGenerator(['one','two','three','four'], 10 )
['two', 'two', 'four', 'one', 'two', 'four', 'one', 'four', 'four', 'two']
-- Steve Majewski
Courageous
>> I'm trying to create a clever way to call a generator multiple
>> times, but the only thing I've been able to come up with is:
I personally think that current generator behavior is all whacked.
The first call to some generator g() should behave like:
f=g()
f.next()
Subsequent calls to g() should behave like
f.next()
The current behavior should be accessible but not the default.
I base this opinion in part upon the way that the generator is
defined. As long as the generator is defined with a signature
which makes it appear as an ordinary function, that's exactly
how it should behave. The semantics of having the function
behave slightly differently upon the second invocation isn't
nearly as surprising as the fact that, under the current
definition, a generator really doesn't behave like a function
at all.
C//
Greg Ewing
>
> The first call to some generator g() should behave like:
>
> f=g()
> f.next()
>
> Subsequent calls to g() should behave like
>
> f.next()
Er... where is this "f" going to be stored? Before
you answer that, make sure you've considered
for i in g():
for j in g():
print i, j
> ... As long as the generator is defined with a signature
> which makes it appear as an ordinary function, that's exactly
> how it should behave.
When generators first appeared, quite a number of
people, including me, argued fervently and passionately
that generators *shouldn't* look like ordinary functions,
for this very reason. Alas, our pleas fell on deaf ears.
--
Greg Ewing, Computer Science Dept, University of Canterbury,
Christchurch, New Zealand
To get my email address, please visit my web page:
http://www.cosc.canterbury.ac.nz/~greg
Paul Jackson
|> [i for i in foo where bar]
Shouldn't that be:
[i for i in spam where eggs]
One must learn to think in Python, mustn't one <grin>?
--
I won't rest till it's the best ...
Manager, Linux Scalability
Paul Jackson <p...@sgi.com> 1.650.933.1373
Courageous
>Er... where is this "f" going to be stored?
After I posted my message, it occured to me that I wasn't
addressing a key part of the semantics, namely the ability
for multiple consumers to generate the generator multiple
times. While I understand that part of the problem is
critical, if you don't mind, I'll politely dodge it*. The
fact remains, as you state that you understand below, the
current implementation of generators is counter-intuitive,
misleading, and incongruous with Python itself.
>When generators first appeared, quite a number of
>people, including me, argued fervently and passionately
>that generators *shouldn't* look like ordinary functions,
>for this very reason.
I agree with you emphatically, and while I am sure that this
comment is a bit snide, let's just say that "language
designer's intuition" should probably needs to smell out
how an average newbie pythonista would react to a function
that isn't.
C//
Bruce Eckel
>(should've been "...incomprehensions") until I discovered that
>the leftmost element was evaluated anew for each element.
>Suddenly it became easy to invoke the same method on a whole
>collection of objects, without lambda:
>
> [foo.asTabSeparatedValue() for foo in selection]
Whoa! (slaps forehead). I've wanted to do this very thing a bunch
of times. And the gods of Python clearly anticipated my needs.
>With list comprehensions many applications of the visitor design
>pattern can be reduced to a single statement.
Based on this, I fooled around with my visitor example until I got
it to work in some fashion with list comprehensions. It cleaned up
nicely. Note the last line, in particular:
# Using list comprehensions to effect visitors.
class Flower:
def pollinate(self, pollinator):
print `self`, "pollinated by", `pollinator`
def eat(self, predator):
print `self`, "eaten by", `predator`
def __repr__(self):
return self.__class__.__name__
class Gladiolus(Flower): pass
class Runuculus(Flower): pass
class Chrysanthemum(Flower): pass
class Bug:
def __repr__(self):
return self.__class__.__name__
class Pollinator(Bug): pass
class Predator(Bug): pass
class Bee(Pollinator): pass
class Fly(Pollinator): pass
class Worm(Predator): pass
import random
rgen = random.Random()
flwrs = [Gladiolus(),Runuculus(),Chrysanthemum()]
flowers = [rgen.choice(flwrs) for i in range(10)]
bee = Bee()
fly = Fly()
worm = Worm()
[(f.pollinate(bee), f.pollinate(fly), f.eat(worm)) for f in
flowers]
At this point I'm trying to figure out
(A) what does multiple dispatching mean in Python?
(B) does Python already, in effect, *do* multiple dispatching, or
perhaps just remove the need for it?
>Sorry for the spewage. It's just that I see many posts from you
>today, and am assuming you're working on "Thinking in Python"
>(or something like it) and am hoping that you plan to cover
>the foregoing therein. If you do, I'll have an *intelligible*
>explanation of list comprehensions :)
I am in fact feverishly translating "Thinking in Patterns" into
"Thinking in Python" to prepare for my tutorial at the Python
conference, which is due monday. I was certain it would be an
enormous job as it was with C++ and Java to write these examples,
but as usual Python changes everything, and I might even hit the
deadline.
Peter Hansen
>
> I am in fact feverishly translating "Thinking in Patterns" into
> "Thinking in Python" to prepare for my tutorial at the Python
> conference, which is due monday. I was certain it would be an
> enormous job as it was with C++ and Java to write these examples,
> but as usual Python changes everything, and I might even hit the
> deadline.
Why not post some of the samples here as a translation exercise?
I'm sure there would be a few people interested in helping
translate some of them over the weekend. People with no lives.
(No, not people like me. I meant *other* people... ;-)
the-c.l.p-community-is-a-large-part-of-python's-advantages-ly yr's,
----------------------
Peter Hansen, P.Eng.
pe...@engcorp.com
Neil Schemenauer
> The first call to some generator g() should behave like:
>
> f=g()
> f.next()
That would reduce the power of generators in Python. There would be no
way to create a generator without starting it.
Neil
Neil Schemenauer
> import random
> rgen = random.Random()
> flowers = [rgen.choice(flwrs) for i in range(10)]
> bee = Bee()
> fly = Fly()
> worm = Worm()
> [(f.pollinate(bee), f.pollinate(fly), f.eat(worm)) for f in
> flowers]
Since you are not using the list produced by the last line it's probably
clearer to use a for loop:
for f in flowers:
f.pollinate(bee)
f.pollinate(fly)
f.eat(worm)
Listcomps are cool but don't go overboard. :-)
Neil
Courageous
>That would reduce the power of generators in Python. There would be no
>way to create a generator without starting it.
I don't think that's necessarily a bad thing, but what I feel
firmly about is that the current semantics versus syntax of
generators in python is obviously wrong.
Just about anything would be better, up to and including the
use of additional keywords. I simply feel strongly that as
long as a generator masquerades as a method, it ought to
behave like one, and that should be its default behavior.
If one wanted to create but not start, one might use "generate,"
e.g.:
f = generate g()
f()
f()
f()
Without "generate," you'd get invocation and generation behavior
at the same time.
C//
Bruce Eckel
"Thinking in Patterns", translated) is going, I'm now thinking that
it might make more sense to build the book in Python first, then
translate it back into Java. For some reason (grin) it seems to be
much easier to work out the ideas and designs using Python... :-)
Not sure when I'll have the initial posting of the book, though.
*********** REPLY SEPARATOR ***********
On 12/7/01 at 9:27 AM Cromwell, Jeremy wrote:
>Peter,
>I thought we weren't supposed to help people with their homework?
>
>I know, you just want <mention in credits> to encourage an author
<free
>book> whose written some good books on other languages <co-author>
to
>complete his Python book for the benefit <ancillary rights> of all
of us.
>
>subliminal-messages-need-never-be-queued-ly yr's,
>**********************************************************
>Jeremy Cromwell
>CIENA Core Switching Division
>jcro...@ciena.com
>--
>http://mail.python.org/mailman/listinfo/python-list
>
>--
>http://mail.python.org/mailman/listinfo/python-list
Cromwell, Jeremy
Just van Rossum
> I don't think that's necessarily a bad thing, but what I feel
> firmly about is that the current semantics versus syntax of
> generators in python is obviously wrong.
>
> Just about anything would be better, up to and including the
> use of additional keywords. I simply feel strongly that as
> long as a generator masquerades as a method, it ought to
> behave like one, and that should be its default behavior.
You really need to dig into the python-iterators list archives.
This has been discussed over and over and over. It's really
not as bad as you think. You need to reset your brain, play
with generators a bit more. It becomes really natural after
a while.
Just
Courageous
>You really need to dig into the python-iterators list archives.
>This has been discussed over and over and over. It's really
>not as bad as you think. You need to reset your brain, play
>with generators a bit more. It becomes really natural after
>a while.
I don't have any personal problem with understanding them;
I've worked extensively with stackless prior to this. There's
no problem.
What I have a problem with is the inconsistency between the
apparent definition and the result. Not even continuations
behave this way.
C//
Just van Rossum
>
> >You really need to dig into the python-iterators list archives.
> >This has been discussed over and over and over. It's really
> >not as bad as you think. You need to reset your brain, play
> >with generators a bit more. It becomes really natural after
> >a while.
>
> I don't have any personal problem with understanding them;
> I've worked extensively with stackless prior to this. There's
> no problem.
I didn't mean generators _in general_, but the "simple generators"
as they are implemented in Python 2.2 _specifically_.
> What I have a problem with is the inconsistency between the
> apparent definition and the result. Not even continuations
> behave this way.
Read the arguments for and against the current implementation
in the python-iterators list archives. It's really not as bad
as you say. A brain-reset might help...
Just
James_...@i2.com
Courageous wrote:
>Thefact remains, as you state that you understand below,
>the current implementation of generators is
>counter-intuitive, misleading, and incongruous with
>Python itself.
As someone who has the vast experience of writing *one* generator in Python
<wink> -- in the Interval/IteratorBounds exercise that I recently posted --
I, on the contrary, found the generator to be not-counter-intuitive,
not-misleading, and not-incongruous-with-Python. I guess it depends on
your point of view. For my exercise, the generator made it easier and more
straightforward for me to create the iterator that I wanted compared with
the alternative of creating an iterator class.
So as to not confuse myself, I think of generators thusly:
A generator is a function or method that -- by virtue of using the yield
statement -- *creates* and *returns* an iterator object when invoked by a
caller. The caller normally takes that returned iterator object and uses
it by invoking its "next" method until StopIteration is raised. The code
inside the generator function or method *runs* when the "next" method of
said iterator is invoked. Each time a caller invokes the *generator*
function or method, a new iterator object is created and returned.
Jim
Courageous
>A generator is a function or method that -- by virtue of using the yield
>statement --
I understand how it works, intuitively. My objection is not that
I don't understand it, but rather that a lone keyword appearing
in the content of the method definition actually changes what is
being defined. That's just plain wrong.
C//
Peter Hansen
>
> > Peter Hansen wrote:
> > > Bruce Eckel wrote:
> > >
> > > I am in fact feverishly translating "Thinking in Patterns" into
> > > "Thinking in Python" to prepare for my tutorial at the Python
> > > conference, which is due monday. I was certain it would be an
> > > enormous job as it was with C++ and Java to write these examples,
> > > but as usual Python changes everything, and I might even hit the
> > > deadline.
> >
> > Why not post some of the samples here as a translation exercise?
>
> I thought we weren't supposed to help people with their homework?
I believe Bruce is no longer in school.
> I know, you just want <mention in credits>
The thought did not occur to me until you mentioned it.
> to encourage an author <free book>
Uh, yes...
> whose written some good books on other languages <co-author>
Yes.
> to complete his Python book for the benefit <ancillary rights>
> of all of us.
And yes.
I'm not sure whether there is sarcasm in your posting, or not,
without a smiley, so I have to assume there was some objection.
Other than the idea of helping somebody who might help the Python
community back, to which part of my suggestion do you object?
(I don't see you posting very often here, so perhaps you are
new and didn't realize that some of us like to post messages in
attempts to help newcomers to Python. It lets us exercise our
Python coding skills, lets us exercise our writing and
explanatory skills, and it gives us personal satisfaction to
help others. Now maybe you'll understand the suggestion.)
Regards,
Paul Jackson
> I'm not sure whether there is sarcasm in your posting, or not,
> without a smiley, so I have to assume there was some objection.
My definite sense in reading Jeremy's post was that
it was tongue in cheek, meant in good humour.
Peter Hansen
>
> Peter Hansen writes:
> > I'm not sure whether there is sarcasm in your posting, or not,
> > without a smiley, so I have to assume there was some objection.
>
> My definite sense in reading Jeremy's post was that
> it was tongue in cheek, meant in good humour.
I'm quite possibly overly sensitive, but go back and try reading
it from my point of view, as the "you" to whom he referred...
--
Paul Jackson
|> I'm quite possibly overly sensitive ...
That, or I'm underly sensitive. Heaven knows.
Oren Tirosh
I agree. It should be immediately apparent that a generator function is
not a regular instance of the 'function' type. I believe that the most
natural way to indicate this is to use the function object's type. A
generator function should be an instance of a 'generator' type, a subclass
of the 'function' type. It should return a 'generator_iterator' object
when called.
Here's what the syntax might look like:
def foo(args)(generator): ...
Why this syntax? Simply following the same rules as class definitions.
A class definition has an optional base class or classes in parens; when
omitted, a default metaclass is used ('class' today, perhaps 'object'
in the future). This syntax is the logical result of extending this rule
to functions and defining 'function' as the default metaclass for new
function definitions. This default may be overridden, with the only
alternative being 'generator', at least for now. There are no new keywords
required, 'generator' is just a built-in type.
The __future__ mechanism gives a clean way to deprecate the current syntax
where a generator function is defined by the mere presence of a 'yield'
statement somewhere within the function's body.
Oren
Just van Rossum
>
> On Sat, Dec 08, 2001 at 12:30:25AM +0000, Courageous wrote:
> >
> > >A generator is a function or method that -- by virtue of using the yield
> > >statement --
> >
> > I understand how it works, intuitively. My objection is not that
> > I don't understand it, but rather that a lone keyword appearing
> > in the content of the method definition actually changes what is
> > being defined. That's just plain wrong.
>
> I agree. It should be immediately apparent that a generator function is
"That's just plain wrong"
"It should be immediately apparent that..."
If it _was_ "plain wrong" it wouldn't be the way it is. I mentioned the
archives of python-iterators list before, but I think the PEP is a better
reference:
http://python.sourceforge.net/peps/pep-0255.html
Just
Oren Tirosh
> After I posted my message, it occured to me that I wasn't
> addressing a key part of the semantics, namely the ability
> for multiple consumers to generate the generator multiple
> times.
The problem with a generic restartable generator is what to do about any
arguments to the generator function. If you want the consumer to be able
to blindly iter() your generator as if it were a list or any other
container you must pack your arguments along.
Here's a functor object for doing that:
class xiter:
def __init__(self, func, *args):
self.func = func
self.args = args
def __iter__(self):
return self.func(*self.args)
Notes:
The resulting object behaves like a container: it has an __iter__ method
with no arguments that may be called multiple times, returning
independent iterators of the same source. If the underlying function
uses only its arguments and does not modify them these iterators should
step through the same sequence.
This is not specific to generator functions. It can work with any function
that returns an iterator.
Why xiter? This is inspired by xrange and xreadlines. They are iterable
objects that act as a deferred-execution version of a function with the
same name sans the 'x'.
In the case of xreadlines the analogy isn't perfect. The function
xreadlines.xreadlines is started with __call__, not __iter__ so it does not
emulate a container. Since it takes no arguments it could have been an
object's __iter__ method and that object would then behave as a container.
An xrange object emulates a container correctly - it can produce
multiple, independent iterators of itself.
Oren
Oren Tirosh
Ok. it's not 'plain wrong'. It's merely 'slightly confusing'.
I fully agree with this particular BDFL pronouncement. Based on the
alternatives presented in PEP 255 the best one has been chosen.
I think that the alternative I have presented might have been preferred,
though, if it were included in the proposals at that time.
Oren
Bruce Eckel
>
yield
invoked by a
>caller. The caller normally takes that returned iterator object
and uses
>it by invoking its "next" method until StopIteration is raised.
The code
>inside the generator function or method *runs* when the "next"
method of
>said iterator is invoked. Each time a caller invokes the
*generator*
>function or method, a new iterator object is created and returned.
I believe that this is too restrictive a definition. My experience
of generators (from C++/STL) is that a generator is simply a
callable entity that returns an object every time you call it; this
call takes no arguments so it is "generating" objects rather than
being a factory -- factory calls take arguments and build the
object based on those arguments while generators create new objects
based on some internal logic. So a generator could be thought of as
a zero-argument factory.
I think it might have been better to say that Python now has
"improved support for generators" in the form of 'yeild.' I suspect
there will be a fair amount of discussion for awhile, every time
the term "generator" is used, if the accompanying code does not
include 'yeild'.
I also think that a generator only returns an iterator if that's
what it promises to generate; otherwise it returns a regular object.
Courageous
>If it _was_ "plain wrong" it wouldn't be the way it is.
One of the objections to what I've written has been, on several
occasions, "After you've worked with them for a while, you'll
not have any problems with them" or some equivalent. What's
wrong-headed about this kind of thinking is that this is true
of original programmers everywhere. They always generally
understand their own code. Embedding a keyword deep in a function
definition and then having this embedding make the function
in effect behave entirely differently isn't just wrong, it's
bizarre.
C//
Steven Majewski
Normally, I'ld agree with you that 'once you get used to them'
isn't a good excuse. But, in the case of generators, I don't
think I've seen any proposals which make what's going on much
more clear. The problem is that there's a lot of syntactic magic
sugar dust being sprinkled here (something that Python usually
avoids) and so it's hard to capture the semantics accurately
with a simple syntax.
Semantically, this:
def g(n):
m = 0
while n < m:
yield n
m += 1
is equivalent to something like this:
class g(generator):
def __init__(self,n):
# make local vars into instance vars
# and do the initial stuff before the loop
self.n = n
self.m = 0
def next():
tmp = self.m
self.m += 1
return tmp
Function g() might be better named 'make_g()' . ( As Bruce Eckel noted,
it's a generator factory function ). In Lisp, we would implement something
like this as a macro 'make-generator, but Python doesn't handle that
type of code manipulation. ( The above is only an approximate translation
of the semantics of something that's implemented in a very different
manner. )
In addition, there's really two levels of indirection going on to
confuse folks: there's the indirection in the generator definition,
which is really the definition of the generator that will be produced
by the syntactically invisible virtual generator factory, and there's
the indirection in the way the iteration protocol invisibly and
automagically calls the generator's 'next' method until StopIteration
is raised.
And when you use these generators in a for loop, they two bits of
confusion seem to cancel each other out. "for i in g(10):" really
behaves just like it's definition. I'm guessing that this fact
was one of the selling points for this syntax -- that if you didn't
look under the hood and just USED them, they would not be confusing.
I'm not sure that supposition has been born out, but they do get
less confusion with use.
Perhaps it would be better to spend another keyword, along with
'def' and 'class' , to distinguish 'generator factories'. Besides
the fact that Guido and Python have both been conservative in
spending their keywords, I think one reason I haven't favored
any of the earlier suggestions was that they seemed to try to
stress a difference between generators and function, which seemed
to be an exaggerated difference. Generators, functions and methods
are just slight variations on each other. The real difference is
between functions and factories.
-- Steve Majewski
Just van Rossum
> I think that the alternative I have presented might have been preferred,
> though, if it were included in the proposals at that time.
I doubt it. I for one don't understand your proposal at all...
Just
Just van Rossum
How can you say that with such absolutely certainty? Have you read
the PEP now?
Just
Courageous
>How can you say that with such absolutely certainty? Have you read
>the PEP now?
Explain to me, very carefully now, why I should have to read
a language designer's doc in order to consider for a moment how
your average user will encounter a language construct?
For the record, yes, I have read this PEP. It's the very first
thing that I did before mucking around with generators and
before posting even my first message regarding generators here.
Pronouncements like the following: "so I have consulted my
language designer's intuition. It tells me that the syntax
proposed in the PEP is exactly right" aren't exactly persuasive,
especially when you consider the paragraph that immediately
preceeds this decree.
C//
Just van Rossum
> Explain to me, very carefully now, why I should have to read
> a language designer's doc in order to consider for a moment how
> your average user will encounter a language construct?
Fair enough. Still, sometimes you need a little background before
new concepts sink in. Look at the problems people coming from
languages like C are having with the concept of name binding.
Sometimes you need to reset your brain.
> For the record, yes, I have read this PEP. It's the very first
> thing that I did before mucking around with generators and
> before posting even my first message regarding generators here.
Great, that's good to know.
> Pronouncements like the following: "so I have consulted my
> language designer's intuition. It tells me that the syntax
> proposed in the PEP is exactly right" aren't exactly persuasive,
> especially when you consider the paragraph that immediately
> preceeds this decree.
I have no idea what you mean with that last sentence. I think all
that Guido tried to say is something like "We could do A or we
could do B. Arguments either way aren't completely convincing,
so I'll just do whatever I think is best." What's so bad about
that? Your attitude of "this is so obviously wrong, what have
they been thinking" however, is pretty insulting, especially
since you don't say anything that hasn't been discussed 1000
times in the iterators mailing list.
Just
Alex Martelli
> I'm trying to create a clever way to call a generator multiple
> times, but the only thing I've been able to come up with is:
>
> import random
> rgen = random.Random()
> def itemGenerator(dummy):
> return rgen.choice(['one', 'two', 'three', 'four'])
>
> print map(itemGenerator, [None]* 25)
>
> This works, but it requires the 'dummy' argument which seems
> inelegant. I'll bet someone has a better idea...
Why not just:
def itemChooser(): return rgen.choice(('one','two','three','four'))
print [itemChooser() for i in range(25)]
(In Python 2.2 and later, 'generator' is a specific term, used in "from
__future__ import" in particular, for a function containing keyword 'yield'
and thus returning an iterator; like other terms that get hijacked by some
programming language for specific meanings, such as "module", "function",
"type", etc, it's thus best avoided in its non-specific wider sense:-).
Alex
Alex Martelli
...
> I believe that this is too restrictive a definition. My experience
> of generators (from C++/STL) is that a generator is simply a
> callable entity that returns an object every time you call it; this
> call takes no arguments so it is "generating" objects rather than
...so a Python generator such as:
def imageneratorhonest(sequence1, sequence2):
for item in sequence1: yield item
for item in sequence2: yield item
isn't a generator...? Unfortunately, clashes with some preexisting
definition are inevitable each time a common term is adopted in a
specifical technical sense in any universe of discourse -- package, module,
function, type, method, class, instance, exception, attribute, item, just
to give a few examples, are all terms which may be usefully used in some
sense or other, but since Python has adopted each of them in one specific
sense (sometimes the same as a few other languages, sometimes not), it's
best to avoid using them in their other senses when talking about Python.
I learned this the hard way when writing about bridge: perfectly reasonable
idioms such as "on the other hand", "the trick is", "a good deal", etc,
cause far too much confusion when used in a universe of language where
"hand", "trick" and "deal" acquire specific, specialized meanings:-).
Alex
Bruce Eckel
>Bruce Eckel wrote:
> ...
>> I believe that this is too restrictive a definition. My
experience
>> of generators (from C++/STL) is that a generator is simply a
>> callable entity that returns an object every time you call it;
this
>> call takes no arguments so it is "generating" objects rather
than
>
>...so a Python generator such as:
>
>def imageneratorhonest(sequence1, sequence2):
> for item in sequence1: yield item
> for item in sequence2: yield item
>
>isn't a generator...?
Exactly! It's actually the factory that creates the generator! The
code makes my point:
from __future__ import generators
def gen():
while 1:
yield 'spam'
g = gen()
print g
for i in range(5):
print g.next()
This produces:
<generator object at 0x00792FE0>
spam
spam
spam
spam
spam
'g' is the generator object, and when you call next() (which takes
no arguments, as I asserted), you get objects popping out. It's a
generator in exactly the sense that I described (perhaps not so
clearly): you don't give it any arguments when you call it, so it's
creating using some internal algorithm. The factory that creates
the generator can take arguments for its initialization. This is
just like creating a generator class that has a constructor that
probably takes arguments, and then using an object of that class
with some kind of argument-less call to create the objects. It's
just slicker using yield.
Courageous
>I have no idea what you mean with that last sentence. I think all
>that Guido tried to say is something like "We could do A or we
>could do B. Arguments either way aren't completely convincing,
>so I'll just do whatever I think is best." What's so bad about
>that?
What's bad about that is pointing people to the PEP and expecting
that it will settle a marked difference of opinion. Furthermore,
I would suggest that if over time I am not alone in raising my
exasperation here in comp.lang.python that the benevolent powers
that be consider that for every poster there exist a hundred in
agreement who don't post, and likewise that perhaps "language
designer's intuition" can sometimes be clouded by the myopathy
that sometimes exists in those who are very close to what it is
they are passing judgement on.
Those who say "work with this some, you'll get used to it" are
far from the point. Python is not supposed to be an environment
where you "get used to it". Unexpected variation in primary
function with identical syntax is begging for trouble. The
principle of least surprise is not being served.
Something that was contentious enough to be discussed "1000
times" in the mailing list is certainly something you are going
to be hearing more about again.
C//
Tim Peters
> What's bad about that is pointing people to the PEP and expecting
> that it will settle a marked difference of opinion.
Nobody (rational) expects that. A BDFL Pronouncement is called for
precisely in those areas where extended debate has proved that consensus
will never be reached. The choices then are eternal paralysis, or, if Guido
wants something enough, he breaks the deadlock by Decree and life moves on.
> Furthermore, I would suggest that if over time I am not alone in
> raising my exasperation here in comp.lang.python
If you feel alone, it's only because you're years late coming to this
debate.
> that the benevolent powers that be consider that for every poster
> there exist a hundred in agreement who don't post,
That cuts both ways, though, and there a hundred who agree with Just who
haven't posted too. That's what "proved that consensus will never be
reached" means, and this incarnation of this debate only confirms that we
got an accurate reading about that from the last N incarnations.
> and likewise that perhaps "language designer's intuition" can sometimes
> be clouded by the myopathy that sometimes exists in those who are very
> close to what it is they are passing judgement on.
Guido isn't an idiot, and generators for Python have been debated since
1991. Trust me: he wasn't close to them at all <wink>.
> ...
> Something that was contentious enough to be discussed "1000
> times" in the mailing list is certainly something you are going
> to be hearing more about again.
The obviousness of that is *why* there's a BDFL Pronouncement: you're not
going to convince anyone, and nobody else is going to convince you. Some
people have convinced *themselves* to switch positions, though, based on
experience with the new feature. I've only seen that switch go in one
direction, though, tending to confirm that Guido's intuition was correct.
Tim Peters
> ...
> And when you use these generators in a for loop, they two bits of
> confusion seem to cancel each other out. "for i in g(10):" really
> behaves just like it's definition. I'm guessing that this fact
> was one of the selling points for this syntax -- that if you didn't
> look under the hood and just USED them, they would not be confusing.
Yes it was, just as the low-level details of __getitem__ and IndexError in
the pre-2.2 iteration protocol remain blissfully unknown to most Python
programmers. The driving force in both cases was to make "for ... in ..."
work smooth as phlegm on a brass doorknob, despite that phlegm and brass
doorknobs aren't all that attractive in isolation <wink>.
> I'm not sure that supposition has been born out, but they do get
> less confusion with use.
All evidence to date says people learn how to use generators in the for-loop
context easily. If they're going to use .next() explicitly, then they have
some conceptual work to do -- even if they're not mucking with generators.
> Perhaps it would be better to spend another keyword, along with
> 'def' and 'class' , to distinguish 'generator factories'.
In practice, starting the docstring with "Generate ..." instead of "Return
..." is clear enough.
> ...
> Generators, functions and methods are just slight variations on each
> other.
Absolutely. People should consider *all* callable objects in 2.2 to be
generators -- it's just that if the body doesn't contain "yield", then the
language helpfully calls .next() once for you by magic <wink>.
Erno Kuusela
<jkr...@san.rr.com> writes:
| Those who say "work with this some, you'll get used to it" are
| far from the point. Python is not supposed to be an environment
| where you "get used to it".
very well said! i so wish the powers that be keep this in mind.
-- erno
Courageous
>Nobody (rational) expects that. A BDFL Pronouncement is called for
>precisely in those areas where extended debate has proved that consensus
>will never be reached. The choices then are eternal paralysis, or, if Guido
>wants something enough, he breaks the deadlock by Decree and life moves on.
This is fair enough, and I can live with it. It's not like generators
are all that hard, or anything. And it's not that I think that the
current _behavior_ of generators is wrong; rather, I'm flummoxed as
to the leaving of the syntax as it is, as it leads to surprises.
The syntax as is predicts other behavior to me.
For example, when I wrote my first generator, I knew _conceptually_
exactly what to expect based on my prior use of stackless
continuations, continuations in lisp, cooperative multi-threading
environments I've written, and esoteric and ill-advised :) use of
setjmp and longjmp in the past.
However, what threw me was the behavior on the first invocation.
Since I was defining a method, I expected that when called the
first time the code prior to first yield would execute. That's
not what happened. The code executed on the first next.
I can see perfectly well why it went the other way, but I certainly
hope you can see how I would state that code appearing in a function
definition probably ought to execute when the function is apparently
called or someone is going to be surprised.
And they will be. I guarantee it. I'm willing to bet that almost
every single new user of generators is going to encountered this
and have the same experience of surprise.
Will this _baffle_ anyone? Hell, no. It's not that hard to say
"what the fuck!?", read the manual a little more carefully, and
come to an understanding. It's not a _difficult_ concept to
grasp, just an unexpected one.
C//
Tim Peters
> A BDFL Pronouncement is called for precisely in those areas where
> extended debate has proved that consensus will never be reached. The
> choices then are eternal paralysis, or, if Guido wants something enough,
> he breaks the deadlock by Decree and life moves on.
[Courageous]
> This is fair enough, and I can live with it.
That's good -- since the alternative appears to be death <wink>.
> It's not like generators are all that hard, or anything. And it's not
> that I think that the current _behavior_ of generators is wrong; rather,
> I'm flummoxed as to the leaving of the syntax as it is, as it leads to
> surprises. The syntax as is predicts other behavior to me.
>
> For example, when I wrote my first generator, I knew _conceptually_
> exactly what to expect based on my prior use of stackless
> continuations, continuations in lisp, cooperative multi-threading
> environments I've written, and esoteric and ill-advised :) use of
> setjmp and longjmp in the past.
This isn't typical. Most people coming to generators won't have any
relevant experience beyond vanilla function calls. The syntax chosen
emphasizes the similarities to functions, not the differences. In this
respect it follows the Icon language, which also makes no syntactic
distinction between generators and functions (although the reason for that
cuts deeper in Icon -- still, the conceptual distinction exists there too,
and the lack of multiple syntactic clues there hasn't been a problem for
them in real life either).
> However, what threw me was the behavior on the first invocation.
> Since I was defining a method, I expected that when called the
> first time the code prior to first yield would execute. That's
> not what happened. The code executed on the first next.
Sure, that's how they work, and it sounds like you also dived in at the deep
end. The expected first use for generators is via:
for whatever in generator():
process whatever
Indeed, Icon and CLU both *restrict* generators to iteration contexts.
Tying it into Python's iteration protocol, and exposing .next(), are
advanced features. Most users won't need to know about that.
> I can see perfectly well why it went the other way, but I certainly
> hope you can see how I would state that code appearing in a function
> definition probably ought to execute when the function is apparently
> called or someone is going to be surprised.
I'd be more sympathetic if it required great effort to get unconfused again;
but it doesn't, so I'm not more concerned about this than that, e.g.,
mutating methods of mutable objects generally return None (another thing you
were suprised by a few days ago).
> And they will be. I guarantee it. I'm willing to bet that almost
> every single new user of generators is going to encountered this
> and have the same experience of surprise.
Some will, some won't. If they stick to generators in for-loops, the
question will never even arise -- "it just works" then.
> Will this _baffle_ anyone? Hell, no. It's not that hard to say
> "what the fuck!?", read the manual a little more carefully, and
> come to an understanding. It's not a _difficult_ concept to
> grasp, just an unexpected one.
Expectations come from prior experience, and you had too much <0.6 wink>.
These isn't Scheme, or even Stackless Python. Most people will get their
expectations about how generators work from the Python docs.
Greg Ewing
>
> I mentioned the
> archives of python-iterators list before, but I think the PEP is a better
> reference:
> http://python.sourceforge.net/peps/pep-0255.html
I don't think reading the list discussions is going to
change anyone's mind. I read them all the first time
around, and they didn't change mine.
The crux of the matter seems to be in this paragraph
from the PEP:
In practice (how you think about them), generators *are*
functions, but with the twist that they're resumable. The mechanics
of
how they're set up is a comparatively minor technical issue, and
introducing a new keyword would unhelpfully overemphasize the
mechanics of how generators get started (a vital but tiny part of a
generator's life).
This could perhaps do with some elaboration. What I
think it's saying is that, the vast majority of the
time, generators will be called in the context of a
for loop:
for x in some_generator(some_args):
...
Now, in that case, if you squint at it and don't
question anything too closely, you can pretend it's
Icon and that the generator proceeds normally until
it hits a yield at which point it starts behaving
something like a couroutine. As long as you're
willing to accept that fiction, you don't need to
know about the magic going on when the generator is
first called -- it's an implementation detail.
But if you stop and think "Hey, wait a minute,
how does that actually *work*?" and start playing
around with the generator on its own, the magic
starts showing through, and generators stop
looking like ordinary functions at all.
It's at *that* point that some of us get the very
strong feeling that there is something wrong with
declaring a generator just like a function. Telling
us to go read the discussions or the PEP does
nothing to dispel that feeling. We've been there,
and have come to a different conclusion.
--
Greg Ewing, Computer Science Dept, University of Canterbury,
Christchurch, New Zealand
To get my email address, please visit my web page:
http://www.cosc.canterbury.ac.nz/~greg
Greg Ewing
>
> And when you use these generators in a for loop, they two bits of
> confusion seem to cancel each other out.
That's a nice way of putting it! Well said.
> "for i in g(10):" really
> behaves just like it's definition. I'm guessing that this fact
> was one of the selling points for this syntax -- that if you didn't
> look under the hood and just USED them, they would not be confusing.
Unfortunately, when you *do* come to look under the
hood, there's twice as much confusion as there would
have been if the confusion had been spread around
more evenly.
So, looked at that way, Guido has chosen to put all
the confusion in one spot rather than distributing
it. :-)
David Goodger
Tim Peters wrote:
> The obviousness of that is *why* there's a BDFL Pronouncement: you're not
> going to convince anyone, and nobody else is going to convince you. Some
> people have convinced *themselves* to switch positions, though, based on
> experience with the new feature. I've only seen that switch go in one
> direction, though, tending to confirm that Guido's intuition was correct.
That last seems almost a truism. Which person seems more likely?
1. Someone who, having adopted a position, finds their position
vindicated by a BDFL Pronouncement, resulting in the implementation
of a feature in their favorite flavor, then turns around and says
"no, sorry, was wrong, the other way was better".
2. Someone who, having adopted a position, finds their position
rejected by the same BDFL Pronouncement, but swallows their pride
and stops tilting at windmills, gets used to the way things are,
and finds that it's not so bad after all.
I'd say just the fact that the feature is implemented and available for all
to use, in itself, virtually *guarantees* that there will be a preponderance
of converts to that feature. In the rare case that I don't agree with a
Pronouncement, I'm definitely in the same, pragmatic camp as person 2.
Guido designed a wonderful language: tastes great, *and* less filling.
--
David Goodger goo...@users.sourceforge.net Open-source projects:
- Python Docstring Processing System: http://docstring.sourceforge.net
- reStructuredText: http://structuredtext.sourceforge.net
- The Go Tools Project: http://gotools.sourceforge.net
Courageous
> In practice (how you think about them), generators *are*
> functions, but with the twist that they're resumable.
>This could perhaps do with some elaboration.
Maybe more like a revision. Thinking about them as functions
isn't right. When you call a function, it's code executes.
When you call a generator, something else executes which prepares
the code the programmer wrote for execution.
While people will get to used to it just fine, let's just say
I happen to agree with the listing of this as a "wart".
>It's at *that* point that some of us get the very
>strong feeling that there is something wrong with
>declaring a generator just like a function. Telling
>us to go read the discussions or the PEP does
>nothing to dispel that feeling. We've been there,
>and have come to a different conclusion.
Word.
C//
Steven Majewski
On Sun, 9 Dec 2001, Tim Peters wrote:
> [Steven Majewski]
> > ...
> > Generators, functions and methods are just slight variations on each
> > other.
>
> Absolutely. People should consider *all* callable objects in 2.2 to be
> generators -- it's just that if the body doesn't contain "yield", then the
> language helpfully calls .next() once for you by magic <wink>.
>
Well, I know I don't have to tell Tim that, semantically speaking,
functions and subroutines are just a specialized subcase of
semi-coroutines (which are a specialized case of coroutines).
-- Steve Majewski
James_...@i2.com
<Jim> responding to <Bruce> responding to <Jim>
I was describing only Python's current "simple generators"
(as that is what I thought the discussion was about).
Jim
===============================================
>So as to not confuse myself, I think of generators thusly:
>
>A generator is a function or method that -- by virtue of using the
yield
>statement -- *creates* and *returns* an iterator object when
invoked by a
>caller. The caller normally takes that returned iterator object
and uses
>it by invoking its "next" method until StopIteration is raised.
The code
>inside the generator function or method *runs* when the "next"
method of
>said iterator is invoked. Each time a caller invokes the
*generator*
>function or method, a new iterator object is created and returned.
I believe that this is too restrictive a definition. My experience
of generators (from C++/STL) is that a generator is simply a
callable entity that returns an object every time you call it; this
call takes no arguments so it is "generating" objects rather than
being a factory -- factory calls take arguments and build the
object based on those arguments while generators create new objects
based on some internal logic. So a generator could be thought of as
a zero-argument factory.
I think it might have been better to say that Python now has
"improved support for generators" in the form of 'yeild.' I suspect
there will be a fair amount of discussion for awhile, every time
the term "generator" is used, if the accompanying code does not
include 'yeild'.
I also think that a generator only returns an iterator if that's
what it promises to generate; otherwise it returns a regular object.
Most current information can be found at:
http://www.mindview.net/Etc/notes.html
===================
Bruce Eckel http://www.BruceEckel.com
Contains free electronic books: "Thinking in Java 2e" & "Thinking
in C++ 2e"
Please subscribe to my free newsletter -- just send any email to:
join-eckel-o...@earth.lyris.net
My schedule can be found at:
http://www.mindview.net/Calendar
===================