Slowness while calling wait() method of gevent.event
Manish Shrestha
Hi all,
We have encountered a problem with the wait() method of gevent.event. Previously till gevent version 0.13.8 (OS: Ubuntu 12.04), it used to take very small amount of time (23 microseconds) to wait with timeout value of 0, so we could use it in our main processing loop of a service. Now as we upgrade to event version 1.0 ( OS: Ubuntu 14.04), its taking a long time to return from the method call, approximately around 1.23 milliseconds.
Basically here is the representation of our service code:
import gevent.event
event = None
event = event or gevent.event.Event()
timeout=0 #most of the cases timeout=0 is used, in some case #timeout of 1 is also used
..
#event would be set when config was changed.
… ..
#In the while loop, that should be fast enough, we called event.wait(timeout),
.. .. ..
while True
data = sock.recv()
if (event.wait(timeout)) :
reload_config()
do_process(data)
Please note that, we usually set timeout=0, but in some instances use timeout =1 in some services to prevent the race conditions.
So, am I missing to call something or is there any way to achieve faster speed in event v1.0 without large amount of change?
I had benchmarked in my actual code using various logging statements. So have I been mislead during my poor man benchmarking?
If there is any better approach/alternative also, the suggestions are most welcome.
The code to benchmark the performance is:
import timeit
setup="""import gevent.event
event = None
event = event or gevent.event.Event()
timeout=0
event.clear()
"""
timeit.Timer('event.wait(timeout)', setup=setup).repeat(7, 1000)
In gevent v1.0 (Ubuntu 14.04, Python 2.7.6), I got the following results of the benchmarking:
[1.2477359771728516, 1.2388949394226074, 1.255936861038208, 1.2381761074066162, 1.2475359439849854, 1.2562599182128906, 1.2880918979644775]
In event v0.13.8 (Ubuntu 12.04, Python 2.7.3), I get the following results of the benchmarking:
[0.023181915283203125, 0.04351806640625, 0.0211789608001709, 0.02157902717590332, 0.021185874938964844, 0.021312952041625977, 0.021281003952026367]
Jason Madden
Version | Time/wait
--------------------
0.13.8 | 15.9us
1.0.2 | 17.3us
1.1.1 | 18.6us
These were captured using ipython's %timeit magic, but I get similar results using the example script.
(Comparing 0.13.8 to 1.x is a bit apples-and-organes because 0.13.8 was based on libevent whereas later versions use libev. I'm not actually that familiar with the way libevent implemented timers, so other that the experiment presented above, I'm not totally sure what to expect with it.)
I notice you're comparing across Python versions. A lot changed between 2.7.3 and 2.7.6, so that could account for some of it. Try to compare using the same Python version. (Comparing on the same kernel version is also a good idea when benchmarking timing like this.)
Also, if you're installing Ubuntu-provided gevent distributions, I suggest either building gevent yourself or installing the manylinux wheels distributed on PyPI. Not only is installing from PyPI going to get you a more up-to-date version of gevent (the current 1.0 series is 1.0.2, but the current---and *recommended* version---is 1.1.1), it's also going to get you a more up-to-date and statically linked libev event loop.
Finally, depending on your need, you may want to just check `event.ready()` instead of waiting with a timeout of 0. The difference is that ready() won't yield and require a full trip around the event loop whereas wait() does (which, despite a timeout of 0, could actually end up taking much longer if there are pending callbacks to run).
Jason
> You received this message because you are subscribed to the Google Groups "gevent: coroutine-based Python network library" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to gevent+un...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.