[Performance testing question.] How fast cat web.py be?

[biran]

hi, all:
I am running a web server using lighttpd and web.py.
I am just at the beginning.

But I have some problem , relating to the performance issue.
the running machine is a old pc with a PIII 667 CPU and 256MB
memory.
harddisk is 15GB.
OS is ubuntu 5.10

I've install lighttpd and web.py on it.
using the document found at
http://webpy.org/tutorial

I've configured in the lighttpd.conf to run code.py via fastcgi( as
the tutorial teach.)

But i found that the fastcgi code.py runs slowly.

this is the following related test.

# ------------------------------
1. -- test the static request.
# -----------------------------
that means : lighttpd directly fetch webpy.html
from filesystem and send it to the client.
the result is very fast. it's faster than apache2 in this round.
brian@ropzsu:~$ ab -c 300 -n 3000 http://ropzsu:81/static/webpy.html
Total transferred: 405373074 bytes
HTML transferred: 404600217 bytes
Requests per second: 484.52 [#/sec] (mean)
Time per request: 619.175 [ms] (mean)
Time per request: 2.064 [ms] (mean, across all concurrent
requests)
Transfer rate: 63935.42 [Kbytes/sec] received

# ------------------------------
2. -- test the web.py served request.
# -----------------------------
this time, i use code.py to generate a hello + username webpage,
by using the code in http://webpy.org/.

-- well, I found it runs slowly, I am worry about it. why does it run
so slowly?
the speed is only a tenth of the static request.

brian@ropzsu:~$ ab -c 4 -n 300 http://ropzsu:81/hello
Concurrency Level: 4
Time taken for tests: 6.269751 seconds
Complete requests: 300
Failed requests: 0
Write errors: 0
Total transferred: 42000 bytes
HTML transferred: 6000 bytes
Requests per second: 47.85 [#/sec] (mean)
Time per request: 83.597 [ms] (mean)
Time per request: 20.899 [ms] (mean, across all concurrent
requests)
Transfer rate: 6.54 [Kbytes/sec] received

compare these:
static request: Requests per second: 484.52 [#/sec]
(mean)
web.py served request: Requests per second: 47.85 [#/sec] (mean)

---
what do you think of this results. How fast do web.py serve at yours'
server?
thanks a lot.

[biran]

I did another 2 tests.

# ------------------------------------------------
3. test request using web.render('view.html')
#------------------------------------------------
brian@ropzsu:/www/pages$ ab -c 4 -n 300 http://ropzsu:81/user/tim

Concurrency Level: 4
Time taken for tests: 11.223492 seconds

Complete requests: 300
Failed requests: 0
Write errors: 0

Total transferred: 64500 bytes
HTML transferred: 16500 bytes
Requests per second: 26.73 [#/sec] (mean)
Time per request: 149.647 [ms] (mean)
Time per request: 37.412 [ms] (mean, across all concurrent
requests)
Transfer rate: 5.52 [Kbytes/sec] received

# -------------------------------
test request using web.render('view.html') and
perform some mysql operations in view.html
# -------------------------------
Concurrency Level: 4
Time taken for tests: 17.627011 seconds

Complete requests: 300
Failed requests: 0
Write errors: 0

Total transferred: 204300 bytes
HTML transferred: 156000 bytes
Requests per second: 17.02 [#/sec] (mean)
Time per request: 235.027 [ms] (mean)
Time per request: 58.757 [ms] (mean, across all concurrent
requests)
Transfer rate: 11.29 [Kbytes/sec] received

the results is : these are even slower.
Requests per second: 26.73 [#/sec] (mean) web.render.
Requests per second: 17.02 [#/sec] (mean) web.render + mysql

[Aaron Swartz]

This doesn't seem too unreasonable -- on my server I get:

static: Requests per second: 4957.69 [#/sec] (mean)
web.py: Requests per second: 40.11 [#/sec] (mean)

I don't think this is a practical problem. reddit.com runs using
web.py on a single small server, is approximately the 2500th most
popular site on the Internet, and still is extremely fast (around
500ms). web.py has never been the performance bottleneck.

[Kamal Mustafa]

On 4/3/06, Aaron Swartz <m...@aaronsw.com> wrote:
>
> This doesn't seem too unreasonable -- on my server I get:
>
> static: Requests per second: 4957.69 [#/sec] (mean)
> web.py: Requests per second: 40.11 [#/sec] (mean)

wow ! that's a lot of different with mine. the same test (-c 4 n 300)
on my server just got:-

/static/prototype.js - Requests per second: 51.72 [#/sec] (mean)
/customers - Requests per second: 21.62 [#/sec] (mean)

the page just grab few rows from postgresql db and put it in html
table (using jinja template engine). this is using max-proc = 1 in
lighttpd.conf. I changed the max-process to 5 but didn't spot any
difference. I thought that would be faster since now it can handle the
4 concurrent request at the same time, right ?

The server is Pentium 4 2.4G with 512MB RAM, running FreeBSD 5.4 and
I'm running the test from my laptop.
--
kamal, www.k4ml.com

[Aaron Swartz]

> > static: Requests per second: 4957.69 [#/sec] (mean)
> > web.py: Requests per second: 40.11 [#/sec] (mean)
> wow ! that's a lot of different with mine. the same test (-c 4 n 300)
> on my server just got:-
>
> /static/prototype.js - Requests per second: 51.72 [#/sec] (mean)
> /customers - Requests per second: 21.62 [#/sec] (mean)

If we're to take the proportions seriously, yours is much faster (40%
static instead of .8%). That said, I suspect the difference is the
database hit -- the page I was testing on was cached.

> lighttpd.conf. I changed the max-process to 5 but didn't spot any
> difference. I thought that would be faster since now it can handle the
> 4 concurrent request at the same time, right ?

Because web.py is multithreaded, it can handle concurrent requests in
a single process, so max-procs=1 is just fine.

[biran]

The result share some common points.

while using web.py to serve the request, the mean speed is abount 30
Requests per second.
The hardware, Mem, Operating system do not make much difference.

A. Requests per second: 40.11 [#/sec] (mean)
B. Requests per second: 17.02 [#/sec] (mean) web.render + mysql

C. Requests per second: 21.62 [#/sec] (mean)

A: Aaron Swartz
B: brian
C: Kamal Mustafa

I've found that http://reddit.com use web.py to serve the users up to 1
million request per day.
Well, that speed is fast. but not fast enough.
if the server can serve 30 requests per second, the maximum requests it
can serve per day is:
>>> 30 * 3600 * 24
2592000
>>>

that is
2.5 million per day.

----
As I was once be a perl fast-cgi user.
I found that when generating simple pages like 'Hello world' page, the
perl fast-cgi is really fast.
It can serve at 80% of the static's speed .
And this pages should be fast.
But web.py seems not to be so good. May it be python's problem?

---
when a dynamic page needs to operate a database monipulation, the
database really become
the bottleneck. so,
C. Requests per second: 21.62 [#/sec] (mean) will be
acceptable,
but simple pages like 'hello world' should run much faster.

[Aaron Swartz]

> But web.py seems not to be so good. May it be python's problem?

To see whether it was web.py's fault, I wrote a script that returns
"Hello, world!" in web.py and simply using Python and flup (the
library that web.py uses to speak FastCGI).

static: Requests per second: 2373.87 [#/sec] (mean)
flup/py: Requests per second: 277.18 [#/sec] (mean)
web.py: Requests per second: 243.39 [#/sec] (mean)

Considering that web.py does a whole bunch of stuff -- set up basic
environment variables, filter by applying regular expressions to the
URL, etc. -- it's awfully fast. So I don't think that web.py is the
problem here; it's either lighttpd, FastCGI, Python, or flup. However,
250 requests/sec. seems plenty fast for the time being so I don't have
any plans to investigate further. (However, if you do figure it out,
let us know.)

[eliott]

On a test machine, I get the following results with lighttpd, using
fcgi with 3 processes each of ruby and python, with ab run over a
nearly unused network:
ab -q -c 10 -n 1000
static html file: Requests per second: 754.17 [#/sec] (mean)
web.py hello world: Requests per second: 133.68 [#/sec] (mean)
rails hello world: Requests per second: 36.91 [#/sec] (mean)

This seems to be expected. Rails does more, thus is a bit slower.

With ab -kq -c 10 -n 1000 (keepalives)
static html file: Requests per second: 1074.01 [#/sec] (mean)
web.py hello world: Requests per second: 140.34 [#/sec] (mean)
rails hello world: ....

For some reason, rails with lighttpd was working properly with
keepalives. I have had issue with keepalives on lighttpd on occasion
with php while running the 'ab' utility (ab uses http/1.0). This is
more of an issue with 'ab' than what is being tested.

Note: This was just a quick test, and I didn't run the tests more than
a few times each, and chose the median scores.

[eliott]

My last message should have read "For some reason, rails with lighttpd
was *not* working properly with keepalives"

[eliott]

After a recommendation to try scgi as well, I reran some of my tests.
Results:

ab -q -c 10 -n 1000

* static html file: Requests per second: 782.01 [#/sec] (mean)
* web.py (scgi): Requests per second: 180.00 [#/sec] (mean)
* web.py (fcgi): Requests per second: 133.68 [#/sec] (mean)

With ab -kq -c 10 -n 1000 (keepalives)

* static html file: Requests per second: 1468.25 [#/sec] (mean)
* web.py (scgi): Requests per second: 230.30 [#/sec] (mean)
* web.py (fcgi): Requests per second: 140.34 [#/sec] (mean)

I was quite amazed at the different that scgi made. Not only did it
perform faster, but the load was more evenly distributed across the
spawned web.py app instances (3 of them).