how to make use of multi-core in gevent just like goroutine?

[luaj...@gmail.com]

As known, the goroutine makes use of multi-core implicitly by running/switching goroutine among native threads. The developer does not have to manage threading stuff explicitly.

How to archive the same goal with gevent?

The coroutine in gevent is limited to run in the same native thread, right?

When it comes to handle block device IO or CPU intensive task, the only way is to use gevent.threadpool or gevent.fork? However, the goroutine handles such tasks automatically.

The server in gevent supports multiprocessing, right?

[Grady Player]

in short you kind of can, kind of can't...

the naive attempt to just spawn gevent greenlets will end up with everything running on one core, that is by design; a strength and a weakness.

if you spawned multiple native threads on a OS that has native thread support and spawned multiple gevent greenlets on each of those threads, 

you might get some additional parallelism, but if you are using an implementation with a Global Interpreter Lock (like C Python, the reference standard), you have limited concurrency.

you can start additional actual processes... then communicate with them through queues, much like a goroutine... you may want to look at gipc.

https://pypi.python.org/pypi/gipc

which is a related project and allows interprocess communication with gevent.

-grady

--
You received this message because you are subscribed to the Google Groups "gevent: coroutine-based Python network library" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gevent+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Matt Billenstein]

Greenthreads and goroutines are not really intended to be the same thing -- the
architecture of your application will be different using different languages
because of this probably -- for servers I put a load balancer in front of a
bunch of discreet processes something like:

Nginx (terminate ssl, serve statics, etc)
- gunicorn
- pool of wsgi workers

Typically one nginx and gunicorn process per box, and one or two wsgi workers
per core on that box. With Go, that gunicorn and wsgi worker pool could
probably just be replaced with a single process -- you probably still want a
load balancer to help with failover in the case of an unhealthy host.

For I/O bound jobs, I use task queues with a dedicated python process per job.
In Go, maybe you don't need that, or maybe you still use a task queue to put
the work on another set of hosts or something. Depends on your needs/tastes.

I want to write more code in Go, but it's simply just not as nice and concise
of a language to program in as Python. And for most of the things I do, Python
is fast enough -- I do leverage PyPy where it makes sense as well.

m

> --
> You received this message because you are subscribed to the Google Groups
> "gevent: coroutine-based Python network library" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to gevent+un...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--
Matt Billenstein
ma...@vazor.com
http://www.vazor.com/