Hi,

I'd like to ask a you a few questions that would be interesting to me
and to other users who consider using gevent.

a. what are you using gevent for?

b. what other options did you consider and why have chosen gevent? /
if you switched from another network library, why?

c. are you planning to use gevent in future projects, if so what kind
of tasks will you use it for?

Thanks,
Denis.

- - - - -

my answers:

a. I use it for my startup, where it currently helps to turn a C
library into a web-service. The library is powered is libevent so it
shares the event loop with gevent.

In the past, I had successfully
 - served a web-app with Django and gevent.wsgi.
 - polled a number of rss feeds with gevent
 - used amqplib on gevent's monkey patched socket. (Worked quite well,
although I had a separate connections for producer and consumers
because monkey patching the socket is not enough to make multiplexing
work properly, you have to dig into the library)

b. I considered twisted, eventlet. Some of the reasons why I wrote
gevent are in my blog (http://blog.gevent.org/2010/02/27/why-gevent/)

I've also looked at Concurrence, which seems great but does not
support standard socket interface and required Stackless (greenlet
support was a bit weak 9 months ago, not sure how it's now).

c. Everything that touches sockets, client- and server-side. Also
scripts. Unlike twisted, where you have to reorganize your code
accordingly, with gevent it's easy to get started. Just spawn() a few
background jobs when you find that you need to do things concurrently.

Hello,

a. I'm currently using gevent as a fast http crawler. (>80mbps)

b. I tried pycurl, twisted, urllib2+threads and was not satisfied with the
performance. I was probably. doing something wrong. I also prefer gevent's
simple way of doing things compared to twisted.

c. I will definitely use gevent in future projects where concurrency &
networking is needed.

Thanks.

On Tue, Mar 2, 2010 at 3:01 AM, Denis Bilenko <denis.bile...@gmail.com>wrote:

Hello.

At my main work we are using gevent for developing payment processing
system and related services. We had started with Twisted, but soon
realized that with amount of business logic we have to implement our
code turning to be huge heap of callbacks. That led us to choose
between gevent, eventlet and concurrence (all are make use of
greenlet). Final reasons to choose gevent were focus to libevent
library and small code base. Now we have almost complete system in
preproduction which use along with gevent:

* SQLAlchemy for async connectivity with MySQL databases (we are using
pure python mysql-connector library as DB-API adapter).
* Custom binary protocol for intercommunicating between services.
* Gevent patched stdlib's httplib.

Also I am working on backends for vkontakte/facebook applications.
Gevent helps a lot:

* Patched httplib again for querying facebook/vkontakte APIs.
* Uploading files to Amazon S3 with patched boto library.
* Fast evhttp-based WSGI server for frontends based on Werkzeug WSGI library.

Yes, for example, now I'm planning to use gevent to provide complete
implementation of Erlang distributed node in Python.

Andrey Popp.

How's your experience with mysql-connector (
http://forge.mysql.com/projects/project.php?id=302) been? I'd love to be
able to use mysql asynchronously (as I'm sure a lot of people would), but
the project looks young. Have you experienced many bugs? Do you maintain a
pool of connections so multiple queries can run in parallel?

Thanks,
Ted

we are using it to implement a custom job queue server. It's part of
mwlib, our mediawiki parsing/rendering library.

some parts of mwlib still use twisted. We've been using twisted for a
long time. I knew about eventlet for some time and was lurking on the
mailing list when you first posted about gevent there. The docs were
quite good and I was really sick of twisted.

We are using eventlet's spawning to serve some wsgi apps.

I will use it for a wsgi server, which will power wikipedia's pdf
generation and talk to the above queue server to dispatch jobs. It will
also have to serve generated pdf files to some http clients (will write
a mail about issue 4 in a few moments)

Some shameless advertising:

http://pediapress.com/ sells printed books on demand, i.e. you select
articles from a mediawiki installation and can either download them as a
pdf file or order a (unique) printed book. We are already live on some
wikipedia installations. en.wikipedia.org is yet to come. You can try
the feature however, if you create an account there.

The mwlib git repo can be found here:
http://code.pediapress.com/git/mwlib
The job queue server code resides mwib/async.

- Ralf

We haven't experienced any problems with mysql-connector yet. I am
forgot to say that we are using trunk of SQLAlchemy, which have
support for mysql-connector. Connection pooling is also provided by
SQLAlchemy.

Of course we have strong isolation level, that means we do not share
nor session object neither mapped classes' instances between
greenlets. It is not a constraint, because spawning new greenlet means
starting new "unit of work" and so creating new session.

--
С уважением, Андрей Попп.
+7 911 740 24 91

On Mar 2, 2010, at 2:01 AM, Denis Bilenko wrote:

Online game servers and a WebSocket server.

I played around with Stackless and Twisted first. Twisted was too complicated and bloated, and cluttered the code flow enough to drive me crazy. I moved to Eventlet, which I used until very recently. Eventlet worked well, but I encountered issues with duplex sockets that I couldn't resolve, even after Eventlet seemed to support them. Eventlet was also in the process of overhauling its API, which is a good thing, but I played around with gevent during my troubles, and haven't looked back. The speed boost in gevent is a welcome addition, and its API is intuitive and easy to grok.

The only major downside I encountered switching from eventlet to gevent was the current lack of support for threading/subprocesses. But I pulled together a small wrapper for the one use case I had for subprocess, which has worked fine for me so far.

Wherever I need lightweight threads. I've often marveled at Erlang's ability to scale, and I would really like to see a Python equivalent to that. Right now, some combination of multiprocessing/gevent/eventlet/spawning/stackless does a lot for concurrency, but it still isn't the same as a full-fledged runtime like Erlang. But that's a pipe dream.

gevent is fast, stable, and it just works. And that's what I need in a coroutine library.

Marcus

Our startup (www.ellerdale.com) does semantic analysis of the real-
time web. As part of that, we processes a large amount of data
(millions of items per hour) in real-time from a variety of sources.
Many of these messages contain short URLs, and as part of our data
cannonicalization process we resolve these back to the original URLs.
I used Gevent to implement a service that resolves short URLs back to
the original URL. The system receives the short URL via a socket,
queues the request, and sends the response back to the caller (via a
socket) when it's ready. Each of the many of hundreds of short URL
services gets it own queue and manages parallel requests (some
services are OK with many at the same time, others need to be
throttled back).

I looked at Tornado, Concurrence, Gevent, and Twisted. I prototyped
the system in the first three, and quickly discovered I didn't like
the callback programming pattern. Gevent let me develop in a very
straightforward and easy to understand manner. I also implemented the
same system using a Threaded model, but switched to reduce CPU load /
increase scalability. In addition to the code, I also looked at the
communities around each of the systems.

Very likely.

-- David

I'm using gevent to load test an app deployed on Google App Engine
(http). Originally the tester was written with eventlet but with
gevent it looks like we're getting more simultaneous connections and
fewer errors. App Engine can soak up whatever you throw at it so
that's helpful. Switching was a snap.

I'm also using gevent for a web crawler.

I tried Stackless and urllib2 with threads. Stackless was ok, but I
don't like the fact that I have to modify the distribution. I got
decent performance with urllib2 and threading, but the code was messy.
I thought about using twisted, but with twisted's documentation, I
couldn't even figure out where to start.

I've modified the code significantly since I first started (with
Stackless), but using gevent has resulted in much cleaner code, easier
to follow, and better performance.

I have a few other projects with which gevent may come in handy, like
a python-based eggdrop replacement. I will definitely also use it for
times when I need to put together a webserver quickly.

On 2 mar, 09:01, Denis Bilenko <denis.bile...@gmail.com> wrote:

http://groups.google.com/group/gevent/browse_thread/thread/36f8dd594b...

I don't usually write GUI applications, but some weeks ago I needed a
pygtk frontend for an existing  library (full of urllib2.urlopen())
and it bothered me that I'd have to re-implement part of it with
callbacks (or use threads) to use it. So I started looking for
alternatives and found eventlet and gevent. Both seem to be a good
approach to the problem (yet not still documented) to re-use blocking
sequential code in a event-based GUI.

I am not sure if I'll finally use gevent for the app, but I've
definitely learned some valuable concepts exploring it.

arnau

On Mar 2, 4:01 am, Denis Bilenko <denis.bile...@gmail.com> wrote:

My Web site Omegle.com, which is a fairly popular chat site.

Omegle was originally using Twisted. To be completely honest, I like
working with Twisted's API better, but Twisted's performance became
unacceptably poor as the site grew. I only switched a few days ago,
but so far, gevent's performance has been very good.

I don't have any specific plans at the moment, but I will definitely
consider using gevent for future projects with similar requirements.

Leif

I've created a wiki page that lists the open source projects that use
gevent: http://code.google.com/p/gevent/wiki/ProjectsUsingGevent

If you'd like to be added, mention it here.