simple benchmark

65 views
Skip to first unread message

vince

unread,
Dec 21, 2008, 10:33:34 PM12/21/08
to web2py Web Framework

i've done a simple benchmark between pylons and web2py both testing
page have one single query one inherit template

pylons: use the quickwiki demo, sqlite sqlalchemy mako
web2py: made a similiar wiki using sqlite





web2py:
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 10.8.8.18 (be patient)
Completed 100 requests
Completed 200 requests
Finished 200 requests


Server Software: Apache/2.2.3
Server Hostname: 10.8.8.18
Server Port: 80

Document Path: /welcometest/wiki
Document Length: 1277 bytes

Concurrency Level: 1
Time taken for tests: 19.245 seconds
Complete requests: 200
Failed requests: 0
Write errors: 0
Total transferred: 334600 bytes
HTML transferred: 255400 bytes
Requests per second: 10.39 [#/sec] (mean)
Time per request: 96.224 [ms] (mean)
Time per request: 96.224 [ms] (mean, across all concurrent
requests)
Transfer rate: 16.98 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 3.0 0 30
Processing: 75 96 22.7 87 194
Waiting: 74 95 22.7 87 193
Total: 75 96 22.9 88 194

Percentage of the requests served within a certain time (ms)
50% 88
66% 94
75% 103
80% 112
90% 126
95% 141
98% 171
99% 187
100% 194 (longest request)



pylons:

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 10.8.8.17 (be patient)
Completed 100 requests
Completed 200 requests
Finished 200 requests


Server Software: Apache/2.2.3
Server Hostname: 10.8.8.17
Server Port: 80

Document Path: /wiki/
Document Length: 511 bytes

Concurrency Level: 1
Time taken for tests: 19.706 seconds
Complete requests: 200
Failed requests: 0
Write errors: 0
Total transferred: 142394 bytes
HTML transferred: 102200 bytes
Requests per second: 10.15 [#/sec] (mean)
Time per request: 98.530 [ms] (mean)
Time per request: 98.530 [ms] (mean, across all concurrent
requests)
Transfer rate: 7.06 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 1.0 0 13
Processing: 78 98 20.0 90 193
Waiting: 77 98 20.0 90 193
Total: 78 98 20.0 91 193

Percentage of the requests served within a certain time (ms)
50% 91
66% 98
75% 107
80% 112
90% 127
95% 138
98% 161
99% 190
100% 193 (longest request)
Message has been deleted

vince

unread,
Dec 22, 2008, 12:27:05 AM12/22/08
to web2py Web Framework
another one with totally identical setup

web2py:
model:
-------
db=SQLDB('sqlite://storage.db')
db.define_table('page',
SQLField('title'),
SQLField('body','text'))
-------
controller
-------
def index():
mypages=db().select
(db.page.id,db.page.title,orderby=db.page.title)
return dict(pages=mypages)
-------
view
-------
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html>
<head>
<title>QuickWiki 6</title>
</head>
<body>
<div class="content">
{{include}}
</div>
</body>
</html>
-------
{{extend 'layout.html'}}
<h1 class="main">Title List</h1>
<ul id="titles">
{{for page in pages:}}
<li>
<span>{{=page.title}}</span>
&nbsp;[{{=A("visit",_href=URL(r=request,f='show',args=[page.id]))}}]
</li>
{{pass}}
</ul>

pylons:
model:
-----
import quickwiki.lib.helpers as h

from pylons import config
from sqlalchemy import Column, MetaData, Table, types
from sqlalchemy.orm import mapper
from sqlalchemy.orm import scoped_session, sessionmaker
Session = scoped_session(sessionmaker(autoflush=True,
transactional=True,
bind=config
['pylons.g'].sa_engine))
metadata = MetaData()
pages_table = Table('pages', metadata,
Column('title', types.Unicode(40), primary_key=True),
Column('content', types.Unicode(), default='')
)
class Page(object):
content = None

def __str__(self):
return self.title
mapper(Page, pages_table)
-------
controller
-------
from quickwiki.model import Page
from quickwiki.lib.base import *
def list(self):
c.titles = [page.title for page in Session.query(Page).all()]
return render('/list.mako')
--------
template
--------
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html>
<head>
<title>QuickWiki 6</title>
</head>
<body>
<div class="content">
${next.body()}
</div>
</body>
</html>
----------------
<%inherit file="base.mako"/>
<h1 class="main">Title List</h1>
<ul id="titles">
% for title in c.titles:
<li>
<span>${title}</span>
&nbsp;[${h.link_to('visit', h.url_for(title=title, action="index"))}]
</li>
% endfor
</ul>
-----------------

benchmark

web2py:
Server Software: Apache/2.2.3
Server Hostname: 10.8.8.18
Server Port: 80

Document Path: /welcometest/wiki
Document Length: 895 bytes

Concurrency Level: 1
Time taken for tests: 4.428 seconds
Complete requests: 100
Failed requests: 0
Write errors: 0
Total transferred: 129000 bytes
HTML transferred: 89500 bytes
Requests per second: 22.58 [#/sec] (mean)
Time per request: 44.278 [ms] (mean)
Time per request: 44.278 [ms] (mean, across all concurrent
requests)
Transfer rate: 28.45 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 1.8 0 18
Processing: 38 44 6.1 42 72
Waiting: 38 44 6.0 41 70
Total: 39 44 6.4 42 72

Percentage of the requests served within a certain time (ms)
50% 42
66% 43
75% 44
80% 48
90% 54
95% 57
98% 66
99% 72
100% 72 (longest request)


pylons:
Server Software: Apache/2.2.3
Server Hostname: 10.8.8.17
Server Port: 80

Document Path: /wiki/page/list
Document Length: 895 bytes

Concurrency Level: 1
Time taken for tests: 5.191 seconds
Complete requests: 100
Failed requests: 0
Write errors: 0
Total transferred: 109219 bytes
HTML transferred: 89500 bytes
Requests per second: 19.27 [#/sec] (mean)
Time per request: 51.906 [ms] (mean)
Time per request: 51.906 [ms] (mean, across all concurrent
requests)
Transfer rate: 20.55 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 1.8 0 19
Processing: 47 51 3.1 51 64
Waiting: 47 51 3.1 51 64
Total: 47 52 3.6 51 70

Percentage of the requests served within a certain time (ms)
50% 51
66% 52
75% 54
80% 54
90% 57
95% 58
98% 64
99% 70
100% 70 (longest request)


mdipierro

unread,
Dec 22, 2008, 2:59:53 AM12/22/08
to web2py Web Framework
Fantastic.

could you make one more benchmark. Same web2py code but bytecode
compile the app.

Massimo
Message has been deleted

vince

unread,
Dec 22, 2008, 4:19:24 AM12/22/08
to web2py Web Framework
oh actually pylons one was compiled as when it's running under
mod_wsgi u need to touch the wsgihandler or it will use pyc directly.
i don't think packing pylons project into compiled egg will increase
performance.

compiled version, stock centos5.2 x86_64, python2.5.2 and mod_wsgi 2.3
(self compiled x86_64), cpu underclocked at 1G
model name : AMD Athlon(tm) 64 X2 Dual Core Processor 5000+
cpu MHz : 1000.000

# ab -n 500

web2py

Server Software: Apache/2.2.3
Server Hostname: 10.8.8.18
Server Port: 80

Document Path: /welcometest/wiki
Document Length: 895 bytes

Concurrency Level: 1
Time taken for tests: 20.865 seconds
Complete requests: 500
Failed requests: 0
Write errors: 0
Total transferred: 645000 bytes
HTML transferred: 447500 bytes
Requests per second: 23.96 [#/sec] (mean)
Time per request: 41.730 [ms] (mean)
Time per request: 41.730 [ms] (mean, across all concurrent
requests)
Transfer rate: 30.19 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 3.9 0 60
Processing: 36 41 6.4 39 116
Waiting: 36 41 6.4 39 115
Total: 36 42 7.4 39 116

Percentage of the requests served within a certain time (ms)
50% 39
66% 40
75% 41
80% 43
90% 50
95% 53
98% 64
99% 71
100% 116 (longest request)


pylons

Server Software: Apache/2.2.3
Server Hostname: 10.8.8.17
Server Port: 80

Document Path: /wiki/page/list
Document Length: 895 bytes

Concurrency Level: 1
Time taken for tests: 26.368 seconds
Complete requests: 500
Failed requests: 0
Write errors: 0
Total transferred: 546221 bytes
HTML transferred: 447500 bytes
Requests per second: 18.96 [#/sec] (mean)
Time per request: 52.736 [ms] (mean)
Time per request: 52.736 [ms] (mean, across all concurrent
requests)
Transfer rate: 20.23 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 2.8 0 60
Processing: 46 52 6.6 51 152
Waiting: 46 52 6.7 51 152
Total: 47 53 7.2 51 153

Percentage of the requests served within a certain time (ms)
50% 51
66% 52
75% 54
80% 54
90% 58
95% 61
98% 70
99% 78
100% 153 (longest request)

vince

unread,
Dec 22, 2008, 4:31:19 AM12/22/08
to web2py Web Framework
ab -c 5 -n 500

web2py

Server Software: Apache/2.2.3
Server Hostname: 10.8.8.18
Server Port: 80

Document Path: /welcometest/wiki
Document Length: 895 bytes

Concurrency Level: 5
Time taken for tests: 22.284 seconds
Complete requests: 500
Failed requests: 0
Write errors: 0
Total transferred: 645000 bytes
HTML transferred: 447500 bytes
Requests per second: 22.44 [#/sec] (mean)
Time per request: 222.836 [ms] (mean)
Time per request: 44.567 [ms] (mean, across all concurrent
requests)
Transfer rate: 28.27 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.4 0 6
Processing: 69 222 61.0 215 596
Waiting: 68 218 60.4 212 574
Total: 69 222 61.0 215 596

Percentage of the requests served within a certain time (ms)
50% 215
66% 233
75% 249
80% 259
90% 283
95% 319
98% 410
99% 492
100% 596 (longest request)


pylons

Server Software: Apache/2.2.3
Server Hostname: 10.8.8.17
Server Port: 80

Document Path: /wiki/page/list
Document Length: 895 bytes

Concurrency Level: 5
Time taken for tests: 32.494 seconds
Complete requests: 500
Failed requests: 0
Write errors: 0
Total transferred: 544961 bytes
HTML transferred: 447500 bytes
Requests per second: 15.39 [#/sec] (mean)
Time per request: 324.936 [ms] (mean)
Time per request: 64.987 [ms] (mean, across all concurrent
requests)
Transfer rate: 16.38 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.6 0 7
Processing: 131 324 53.4 321 771
Waiting: 129 321 52.7 318 771
Total: 131 324 53.3 321 772

Percentage of the requests served within a certain time (ms)
50% 321
66% 338
75% 350
80% 357
90% 382
95% 403
98% 447
99% 508
100% 772 (longest request)


vince

unread,
Dec 22, 2008, 4:36:20 AM12/22/08
to web2py Web Framework
ab -c 10 -n 500

web2py

Server Software: Apache/2.2.3
Server Hostname: 10.8.8.18
Server Port: 80

Document Path: /welcometest/wiki
Document Length: 895 bytes

Concurrency Level: 10
Time taken for tests: 22.178 seconds
Complete requests: 500
Failed requests: 0
Write errors: 0
Total transferred: 645000 bytes
HTML transferred: 447500 bytes
Requests per second: 22.54 [#/sec] (mean)
Time per request: 443.567 [ms] (mean)
Time per request: 44.357 [ms] (mean, across all concurrent
requests)
Transfer rate: 28.40 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 5.3 0 73
Processing: 87 441 111.0 428 1064
Waiting: 77 437 110.7 425 1049
Total: 88 442 111.9 428 1064

Percentage of the requests served within a certain time (ms)
50% 428
66% 473
75% 501
80% 522
90% 573
95% 623
98% 711
99% 832
100% 1064 (longest request)



pylons

Server Software: Apache/2.2.3
Server Hostname: 10.8.8.17
Server Port: 80

Document Path: /wiki/page/list
Document Length: 895 bytes

Concurrency Level: 10
Time taken for tests: 33.341 seconds
Complete requests: 500
Failed requests: 0
Write errors: 0
Total transferred: 544604 bytes
HTML transferred: 447500 bytes
Requests per second: 15.00 [#/sec] (mean)
Time per request: 666.826 [ms] (mean)
Time per request: 66.683 [ms] (mean, across all concurrent
requests)
Transfer rate: 15.95 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.1 0 2
Processing: 151 664 105.0 661 1088
Waiting: 151 660 104.9 658 1088
Total: 151 664 105.0 661 1088

Percentage of the requests served within a certain time (ms)
50% 661
66% 695
75% 720
80% 733
90% 780
95% 830
98% 944
99% 1007
100% 1088 (longest request)

mdipierro

unread,
Dec 22, 2008, 4:50:29 AM12/22/08
to web2py Web Framework
Bottom line: we are +50% faster (22-23 rq/sec vs 15-18)!

Thank you vince.

Would you add an entry in the http://mdp.cti.depaul.edu/web2py_wiki
summarzing your results and posting sample code?

Massimo

vince

unread,
Dec 22, 2008, 5:18:30 AM12/22/08
to web2py Web Framework
sure will do.

-vince
On Dec 22, 5:50 pm, mdipierro <mdipie...@cs.depaul.edu> wrote:
> Bottom line: we are +50% faster (22-23 rq/sec vs 15-18)!
>
> Thank you vince.
>
> Would you add an entry in thehttp://mdp.cti.depaul.edu/web2py_wiki

mmstud

unread,
Dec 22, 2008, 5:26:19 AM12/22/08
to web2py Web Framework
Have you ever considered using libevent based server (FAPWS2)? I
realized the excellence of libevent (what comes to performance and
concurrency) year or so ago and to my glad just found, that there has
been python projects, that uses it.

-Marko

vince

unread,
Dec 22, 2008, 10:56:56 AM12/22/08
to web2py Web Framework
mod_wsgi on apache doing quite well. i'll sure take a look it.

-vince
Reply all
Reply to author
Forward
0 new messages