Tornado benchmark on PyPy

[Didip Kerabat]

I randomly came across this on Reddit (7 month old post). I believe kingkilr is PyPy dev. http://www.reddit.com/r/Python/comments/fr4w9/benchmarking_go_and_python_web_servers/c1i2of9

Copy pasted below for convenience:

I ran the same benchmark (same code, I used ab), using CPython (2.6.5) and PyPy (fresh from trunk). Here were my results (first one is PyPy, second is CPython, both truncated to show relevant values):

alex@alex-laptop:/tmp$ ab -c200 -n50000 http://localhost:8000/Hello/100
Concurrency Level:      200
Time taken for tests:   20.114 seconds
Complete requests:      50000
Failed requests:        0
Write errors:           0
Total transferred:      57350000 bytes
HTML transferred:       49500000 bytes
Requests per second:    2485.80 [#/sec] (mean)
Time per request:       80.457 [ms] (mean)
Time per request:       0.402 [ms] (mean, across all concurrent requests)
Transfer rate:          2784.38 [Kbytes/sec] received


alex@alex-laptop:/tmp$ ab -c200 -n50000 http://localhost:8001/Hello/100
Concurrency Level:      200
Time taken for tests:   42.483 seconds
Complete requests:      50000
Failed requests:        0
Write errors:           0
Total transferred:      57350000 bytes
HTML transferred:       49500000 bytes
Requests per second:    1176.94 [#/sec] (mean)
Time per request:       169.932 [ms] (mean)
Time per request:       0.850 [ms] (mean, across all concurrent requests)
Transfer rate:          1318.31 [Kbytes/sec] received

- Didip -

[Peter Bengtsson]

That's a 100% improvement. Not a small deal! Why aren't all of my tornado sites running on pypy???

[Nabil Servais]

Because some tornado sites are using other modules incompatible or instable with pypy...

[Frank Smit]

The Psycopg2 PyPy port doesn't have support for asynchronous
connections as far as I can see. :(

[Ovidiu Predescu]

Unfortunately this module doesn't seem to be supported anymore, so it
may be a challenge to use anyway. I remember trying to get it running
a while ago with no success.

Ovidiu

[Nabil Servais]

There is a ctypes module for use postgresql, it's based on libpq maybe it could work with pypy. http://pypi.python.org/pypi/pypq/0.1.3

[Ovidiu Predescu]

Nice, thanks for the pointer!

[Didip Kerabat]

I feel you, oursql makes heavy use of Cython and it's not fully supported by PyPy.

- Didip -

[Carlo Cabanilla]

It'd be nice to be able to compare the memory usage, since that's the main tradeoff of pypy.

--


.Carlo

syntacticbayleaves.com

[Fabio[mbutubuntu]Buda]

Very useful benchmark... thank you Didip!

On Oct 7, 8:15 pm, Carlo Cabanilla <carlo.cabani...@gmail.com> wrote:
> It'd be nice to be able to compare the memory usage, since that's the main
> tradeoff of pypy.
>
>
>
>
>
>
>
>
>
> On Wed, Oct 5, 2011 at 12:43 PM, Didip Kerabat <did...@gmail.com> wrote:
> > I randomly came across this on Reddit (7 month old post). I believe
> > kingkilr is PyPy dev.

> >http://www.reddit.com/r/Python/comments/fr4w9/benchmarking_go_and_pyt...

>
> > Copy pasted below for convenience:
>
> > I ran the same benchmark (same code, I used ab), using CPython (2.6.5) and
> > PyPy (fresh from trunk). Here were my results (first one is PyPy, second is
> > CPython, both truncated to show relevant values):
>

> > alex@alex-laptop:/tmp$ ab -c200 -n50000http://localhost:8000/Hello/100

> > Concurrency Level:      200
> > Time taken for tests:   20.114 seconds
> > Complete requests:      50000
> > Failed requests:        0
> > Write errors:           0
> > Total transferred:      57350000 bytes
> > HTML transferred:       49500000 bytes
> > Requests per second:    2485.80 [#/sec] (mean)
> > Time per request:       80.457 [ms] (mean)
> > Time per request:       0.402 [ms] (mean, across all concurrent requests)
> > Transfer rate:          2784.38 [Kbytes/sec] received
>

> > alex@alex-laptop:/tmp$ ab -c200 -n50000http://localhost:8001/Hello/100

[Frank Smit]

I noticed that psycopg2-ctypes[1] support async connections now and
did some small test with Momoko and PyPy. It works but not 100%.

[1]: https://github.com/mvantellingen/psycopg2-ctypes

[Ben Darnell]

In addition to the module compatibility issues mentioned in this
thread (FWIW, MySQLdb is also incompatible with pypy), pypy 1.6 has
some issues with memory and garbage collection:
http://groups.google.com/group/python-tornado/browse_thread/thread/2921b407da0aaf89/42ab9821339321c9

My current understanding of the issue is that it's not quite a leak
and has a disproportionate effect on "hello world" workloads.
Basically, memory allocated by C extensions is "invisible" to the pypy
garbage collector. It'll still be freed when GC runs, but it has no
effect on when GC happens, so there may not be a GC until pypy has
used much more memory than it normally would. In tornado's case, the
main culprit appears to be openssl objects used by the hashlib module
(used for computing etags). This should be fixed in the upcoming pypy
1.7 (http://mail.python.org/pipermail/pypy-dev/2011-October/008524.html),
although the underlying problem may crop up with other modules as
well.

-Ben