gevent + gipc bounded pool?

Skip to first unread message

James Addison

Feb 9, 2017, 11:00:39 AM2/9/17
to gevent: coroutine-based Python network library
Looking for a reply to - pasting the question here for gipc familiar folks:

Is gipc compatible with a pool scenario? I want to take web requests to resize images - the resizing of images likely fits well into a separate process, given the GIL.

That said, I do not want uncontrolled process spawning. Is it possible to achieve something like gevent.pool.ThreadPool like upper limit pool sizing? Along the order of:

- On initialization, create a process pool size of 20. 
- Handle incoming web request, ask the pool to perform image resizing function and cooperatively wait for the result
- Once the result is returned from the process, return it to the web client
- Handle many incoming requests, but do not perform more than 20 image resizes at a time because the process pool is bounded

If this is possible, can the processes be reused to save on process initialization overhead? It would be Queue-based, I assume? Could I still wait cooperatively for the result?

The docs make it seem like gipc is great for spawning processes and waiting, but in an uncontrolled fashion? Or am I misunderstanding?

Many thanks.

Apr 8, 2020, 12:23:14 PM4/8/20
to gevent: coroutine-based Python network library
Did you ever get an answer to this question? I can't see the bitbucket issue.

Matt Billenstein

Apr 8, 2020, 1:48:43 PM4/8/20
Looks like you found the github issue:

Couldn't you use a regular greenlet pool where each greenlet manages a
single long-running process?

> --
> You received this message because you are subscribed to the Google Groups
> "gevent: coroutine-based Python network library" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to
> To view this discussion on the web visit
> gevent/

Matt Billenstein
Reply all
Reply to author
0 new messages