why dose gevent create thousands of threads automatically?

209 views
Skip to first unread message

Ziyi Zhao

unread,
Sep 2, 2016, 3:44:41 AM9/2/16
to gevent: coroutine-based Python network library
        I'm writing a server,    when receiving  thousands of requests, I create a thread( using gevent.threadpool, size of the poll is 100) and in the thread I create a greenlet(using gevent.spawn) for every request。    
        when I use command "pstree -p pid",   I found the server creates thousands of linux thread。
        why so many threads?    would gevent create threads used to do some helper jobs on the background? 
        thx

Ziyi Zhao

unread,
Sep 2, 2016, 9:57:04 AM9/2/16
to gevent: coroutine-based Python network library
by the way,   in every spawned greenlet, I use urllib2(which has been monkey-patched) to send requests.

在 2016年9月2日星期五 UTC+8下午3:44:41,Ziyi Zhao写道:

Jason Madden

unread,
Sep 2, 2016, 10:04:53 AM9/2/16
to gev...@googlegroups.com
Every native thread (that you use gevent in) has an event loop and a Hub. Every Hub has a threadpool. The threadpool is used by default to do operations like network name resolving and (on windows) certain types of IO. So you can see how threads can multiply.

Unless you are doing something that releases the GIL, and is *not* something that is handled by gevent (for example, you call into a C extension that releases the GIL and spend most of your time there), you probably get very little benefit from trying to use a native thread for each request. (Part of the point of gevent is that you don't need to use native threads for most things.) Consider using greenlets and a greenlet pool instead; see StreamServer and BaseServer.
> --
> You received this message because you are subscribed to the Google Groups "gevent: coroutine-based Python network library" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to gevent+un...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

Reply all
Reply to author
Forward
0 new messages