Question regarding implicit yeiilding

[RJ]

My program:
import eventlet
from eventlet.green import urllib2 

def fetch(url):
  print "opening", url
  body = urllib2.urlopen(url).read()
  print "done with", url
  return url, body

pool = eventlet.GreenPool()
for url, body in pool.imap(fetch, my_list):
    print body
 
pool.waitall()

Output:

opening URL1
opening URL2
opening URL3
done with URL1
done with URL2
URL1 OUTPUT
URL2 OUTPUT
done with URL3
URL3 OUTPUT

But since its green threads and its cooperative. I was expecting it will implicit yield only on I/O points not any other points like here in case of return function. Therefore, I was expecting this:

opening URL1
opening URL2
opening URL3
done with URL1
URL1 OUTPUT
done with URL2
URL2 OUTPUT
done with URL3
URL3 OUTPUT

May be I misunderstood how implicit yielding works? Can someone please explain? Thanks in advance.

[Sean Talts]

What version are you using? The current docs say this about
pool.imap:
"""
imap(func, iterable)
An equivalent of itertools.imap()

TODO: Fix this.
"""

Seems like maybe it's broken right now in the version the docs refer
to (0.13.6?)

-Sean

On Jan 22, 5:14 pm, RJ <jainwolver...@gmail.com> wrote:
> *My program:*

> import eventlet
> from eventlet.green import urllib2
>
> def fetch(url):
>   print "opening", url
>   body = urllib2.urlopen(url).read()
>   print "done with", url
>   return url, body
>
> pool = eventlet.GreenPool()
> for url, body in pool.imap(fetch, my_list):
>     print body
>
> pool.waitall()
>

> *Output:*

[Denis Bilenko]

Well, after one greenlet has finished another has to run. Which one is
unspecified though. You're assuming that after URL1 greenlet has
finished, MAIN greenlet will run. But the underlying scheduler picks
the "URL2" greenlet instead. This does not affect the correctness of
your program.

BTW, this mailing list is about gevent, not eventlet. (Some comparison
here: http://blog.gevent.org/2010/02/27/why-gevent/ although a big
chunk of it is out of date now).

Here's how your script would look translated to gevent:

from gevent import monkey; monkey.patch_all()
import urllib2
from gevent.pool import Pool

def fetch(url):
print "opening", url
body = urllib2.urlopen(url).read()
print "done with", url
return url, body

pool = Pool()

for url, body in pool.imap(fetch, my_list):
print body

pool.join()

Use gevent 1.0b1 rather than the latest stable from PyPI (0.13.6). It
has better imap() implementation. You can get it from
http://code.google.com/p/gevent/downloads/list

On Tue, Jan 24, 2012 at 4:29 AM, Sean Talts <xit...@gmail.com> wrote:
> What version are you using? The current docs say this about
> pool.imap:
> """
> imap(func, iterable)
> An equivalent of itertools.imap()
>
> TODO: Fix this.
> """
>
> Seems like maybe it's broken right now in the version the docs refer
> to (0.13.6?)

Yes, this is fixed in 1.0.