wrapping yield ?
Michael Sparks
I'd like to be able to wrap "yield" in a function - is that
possible, and if not (my eading of stuff I've seen so far
says not) are there any plans to make it possible?
Whilst it may seem odd, I'd like to be able to do this:
def my_generator(arg):
dostuff....
while(1):
waitForSomethingCheckingPeriodically()
processit
where
def waitForSomethingCheckingPeriodically():
found = 0
while(not found):
x = checkTheThing()
if foundRightThing(x):
found = 1
else:
yield "foo"
return x
The reason for wanting to wrap this up is due to the
waitForSomethingCheckingPeriodically is a sequence of
code that I'd like to have in several different generators.
The problem of course here is that "my_generator"
doesn't create a generator due to not containing a "yield"
keyword, whereas waitForSomethingCheckingPeriodically
generates a generator, even though I don't want it to be
one...
The context here is of a piece of network server code,
and wanting to be able to enforce relinquishing of control
in the pieces of code that perform reads/writes, as well as
connection acceptance etc. (basically I want to wrap up
the yield call into the piece of code that should yield control,
rather than rely on the user of the code being relied upon
to yield in the right places)
Having read: http://www.python.org/peps/pep-0255.html
I suspect that what I'd like to do here wouldn't work, and
what I really want is something more like:
def_generator my_generator(arg):
...
def waitForSomethingCheckingPeriodically():
...
And have the keyword (or something other than "yield")
indicate that the definition is a generator, since that would
solve this completely - which I not has been ruled out in that
document, and I'm hoping things may've changed since
then :-)
If this is the wrong place to ask this q - apologies, I'd
appreciate being pointed in the right direction :-)
Regards & thanks in advance,
Michael.
Terry Reedy
"Michael Sparks" <Michael...@rd.bbc.co.uk> wrote in message
news:alqehl$9qv$1...@nntp0.reith.bbc.co.uk...
> Hi,
>
> I'd like to be able to wrap "yield" in a function - is that
> possible, and if not (my eading of stuff I've seen so far
> says not) are there any plans to make it possible?
>
> Whilst it may seem odd, I'd like to be able to do this:
>
> def my_generator(arg):
> dostuff....
> while(1):
> waitForSomethingCheckingPeriodically()
> processit
You have to put in a yield x *somewhere* in order to yield anything to
the caller.
Hard to see why this is a problem. Something like
x = wait....
yield process(x)
TJR
You are
Duncan Booth
> I'd like to be able to wrap "yield" in a function - is that
> possible, and if not (my eading of stuff I've seen so far
> says not) are there any plans to make it possible?
>
> Whilst it may seem odd, I'd like to be able to do this:
>
> def my_generator(arg):
> dostuff....
> while(1):
> waitForSomethingCheckingPeriodically()
> processit
>
> where
>
> def waitForSomethingCheckingPeriodically():
> found = 0
> while(not found):
> x = checkTheThing()
> if foundRightThing(x):
> found = 1
> else:
> yield "foo"
> return x
>
You have to propogate the yield up to the top level by yielding from each
intermediate generator, and your generator cannot return a value so you
should yield the returned results. So something like this should work:
def my_generator(arg):
dostuff....
for item in waitForSomethingCheckingPeriodically():
if item is None:
yield None
else:
processit(item)
def waitForSomethingCheckingPeriodically():
while 1:
x = checkTheThing()
if foundRightThing(x):
yield x
else:
yield None
--
Duncan Booth dun...@rcp.co.uk
int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3"
"\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?
Michael Sparks
news:Xns92885F082A2...@127.0.0.1...
...
> So something like this should work:
>
> def my_generator(arg):
> dostuff....
> for item in waitForSomethingCheckingPeriodically():
> if item is None:
> yield None
> else:
> processit(item)
Whereas if I could wrap up yield, I simply have to do:
item = waitForSomethingCheckingPeriodically()
process(item)
Which if I have lots of this sort of thing going on, results in much
clearer (and hence maintainable) code.
If it can't (putting yield in a function, and not turning the function into
a
generator) be done, and AFAICT it can't, and this is a suggested work
around - many thanks :-)
Best Regards,
Michael.
Duncan Booth
> Whereas if I could wrap up yield, I simply have to do:
>
> item = waitForSomethingCheckingPeriodically()
> process(item)
>
> Which if I have lots of this sort of thing going on, results in much
> clearer (and hence maintainable) code.
>
> If it can't (putting yield in a function, and not turning the function
> into a
> generator) be done, and AFAICT it can't, and this is a suggested work
> around - many thanks :-)
The syntax wouldn't look quite so bad if your function wasn't returning a
value. At present 'for x in gen(): yield x' is the only way to nest
generators, but there were discussions here recently suggesting a new
syntactic sugar:
yield every gen()
as an equivalent for the for loop.
There definitely isn't any way to put a yield in a function and not have it
turn into a generator. When you yield from a generator, only a single frame
is saved, not a stack, so you can only yield from the outer loop. By
yielding at each level you effectively simulate the stack since each frame
chains to the others.
Michael Sparks
> The syntax wouldn't look quite so bad if your function wasn't returning a
> value.
Problem is, it would in most situations, also I'm trying to keep the number
of generators to a minimum whilst a lower overhead than things like threads
and processes if I had several 10s of thousands going, keeping throughput
up may become an issue. (Hence the reason for wanting the functions called
part of the same generator rather than a separate, extra one)
> There definitely isn't any way to put a yield in a function and not have
it
> turn into a generator. When you yield from a generator, only a single
frame
> is saved, not a stack, so you can only yield from the outer loop.
Ah... that makes sense then...Pity - after all, "all you'd need" (looking at
it
simplistically) is a marker in the stack frame to tell the system how much
of the
stack to save. Amiga E had something very similar (from a use perspective)
for
determining how far back to throw exceptions without requiring the
equivalent
of try/(except|catch) blocks.
Thanks!
Michael.
Duncan Booth
> "Duncan Booth" <dun...@NOSPAMrcp.co.uk> wrote in message
> news:Xns92887594497...@127.0.0.1...
>> The syntax wouldn't look quite so bad if your function wasn't
>> returning a value.
>
> Problem is, it would in most situations, also I'm trying to keep the
> number of generators to a minimum whilst a lower overhead than things
> like threads and processes if I had several 10s of thousands going,
> keeping throughput up may become an issue. (Hence the reason for
> wanting the functions called part of the same generator rather than a
> separate, extra one)
Don't forget that iterating with a generator is *much* more efficient than
calling a Python functions thousands of times.
Python has a big overhead on each function call, but you only pay that
overhead when you first call a generator and the subsequent iterations have
virtually no overhead at all.
Michael Sparks
> calling a Python functions thousands of times.
> the subsequent iterations have virtually no overhead at all.
Can't forget something you don't already know - very much appreciated...
Best Regards,
Michael.
Oren Tirosh
> Don't forget that iterating with a generator is *much* more efficient than
> calling a Python functions thousands of times.
>
> Python has a big overhead on each function call, but you only pay that
> overhead when you first call a generator and the subsequent iterations have
> virtually no overhead at all.
In most programming environments function calls are fast and context
switches are slow. Python is quite unusual in having it the other way
around - a yield statement causes a context switch that is much faster
than a function call.
Oren