web2py binary contest
210 views
Skip to first unread message
Niphlod
Oct 12, 2013, 2:57:08 PM10/12/13
to web...@googlegroups.com
Hi all, in the near future web2py will build binaries using bbfreeze instead of py2exe. We noticed a few shortcomings of py2exe (it's kinda old, latest release is dated november 2008) and start wondering if there were better tools for the job.
In the latest two weeks, I tested bbfreeze and managed to build successfully web2py with no issues.
bbfreeze allows to package some libraries that are not compatible with py2exe, and in a more streamlined way.
First things first, we need you to test the binary release made with bbfreeze ^_^
Second, as a nice performance booster, we'll ship another executable inside the archive that will use gevent to serve web2py instead of the default rocket webserver.
And here's the deal: we want you to build it, so you get more comfortable with the process of building windows binaries for web2py.
Here there is a slice to make a nice build out of a fresh windows system: http://www.web2pyslices.com/slice/show/1726/build-windows-binaries
Beware: you can of course make the guide better contributing with your own ideas and your patches to setup_exe.py .
Long story short: the first one who will send the correct setup_exe.py (via email to niphlod [at] the big g) to build the binary with bbfreeze AND including the new executable for the gevented webserver gets to be acknowledged on the official CHANGELOG for the next stable release (don't worry, it's just a few lines away).
We will then put the slice on the official book to let everybody know how to build a working web2py binary.
Good luck to everybody
In the latest two weeks, I tested bbfreeze and managed to build successfully web2py with no issues.
bbfreeze allows to package some libraries that are not compatible with py2exe, and in a more streamlined way.
First things first, we need you to test the binary release made with bbfreeze ^_^
Second, as a nice performance booster, we'll ship another executable inside the archive that will use gevent to serve web2py instead of the default rocket webserver.
And here's the deal: we want you to build it, so you get more comfortable with the process of building windows binaries for web2py.
Here there is a slice to make a nice build out of a fresh windows system: http://www.web2pyslices.com/slice/show/1726/build-windows-binaries
Beware: you can of course make the guide better contributing with your own ideas and your patches to setup_exe.py .
Long story short: the first one who will send the correct setup_exe.py (via email to niphlod [at] the big g) to build the binary with bbfreeze AND including the new executable for the gevented webserver gets to be acknowledged on the official CHANGELOG for the next stable release (don't worry, it's just a few lines away).
We will then put the slice on the official book to let everybody know how to build a working web2py binary.
Good luck to everybody
Niphlod
Oct 14, 2013, 3:04:31 PM10/14/13
to web...@googlegroups.com
bump (@moderators, this is just to keep it in the first posts in this week)
Joe Barnhart
Oct 15, 2013, 4:16:53 AM10/15/13
to web...@googlegroups.com
Is the new process cross-platform? I looked it up on the Python site and it didn't say anything specific about OSX except it's compatible with "unix like" operating systems.
Or course, I have the weirdest requirement of all -- I'd like to automate the build of a website (with data) on a Linux server and download it for either Windows or Mac to run locally. I know, not your usual requirement. My idea is to customize a site that gets downloaded by a user and run locally on his laptop.
-- Joe B.
Niphlod
Oct 16, 2013, 8:34:07 AM10/16/13
to web...@googlegroups.com
there are (ATM) no packagers being able to produce binaries for all platforms using a single platform (i.e., you can't build a win binary on linux).
BTW, you can easily redistribute the official binaries with your application in it, and I really don't see why your requirement needs to be able to build from sources web2py.
Niphlod
Oct 17, 2013, 6:15:15 PM10/17/13
to web...@googlegroups.com
bump. Contest will close in 1 week, 25 Oct 2013.
Brian M
Oct 18, 2013, 10:40:33 PM10/18/13
to web...@googlegroups.com
OK, since nobody else seems to be trying and I do use the windows binaries for a work app I gave this a try and didn't seem to have any trouble building the default binary and one with gevent. Niphlod I'll send you an email off-list to see if I did it right.
~Brian
Brian M
Oct 19, 2013, 12:34:57 AM10/19/13
to web...@googlegroups.com
Basic speed comparison between the two suggests that gevent may indeed be noticeably faster. (I wouldn't jump to too many conclusions - this is totally unscientific and being run on a relatively low powered laptop - Win8, 1.7GHz AMD E2, 8GB RAM.)
Summary
500 requests @ concurrency of 20
Rocket: 7.38 requests/second (67.7sec total)
Gevent: 10.92 requests/second (45.8 sec total)
Additional observations - Rocket consumed between 34-40MB RAM while Gevent used just 23-24MB (Private Memory). Rocket was using 15 threads while Gevent had only 4 according to the Resource Monitor.
With standard Rocket version
C:\Users\Brian\Downloads\python\httpd-2.4.6-win64-VC11\Apache24\bin>ab.exe -n 500 -c 20 http://127.0.0.1:8000/welcome/default/index
This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Finished 500 requests
Server Software: Rocket
Server Hostname: 127.0.0.1
Server Port: 8000
Document Path: /welcome/default/index
Document Length: 14093 bytes
Concurrency Level: 20
Time taken for tests: 67.736 seconds
Complete requests: 500
Failed requests: 0
Write errors: 0
Total transferred: 7256500 bytes
HTML transferred: 7046500 bytes
Requests per second: 7.38 [#/sec] (mean)
Time per request: 2709.424 [ms] (mean)
Time per request: 135.471 [ms] (mean, across all concurrent requests)
Transfer rate: 104.62 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 3 31.2 1 495
Processing: 1002 2656 297.5 2686 3114
Waiting: 998 2654 297.5 2684 3113
Total: 1003 2659 290.9 2686 3114
Percentage of the requests served within a certain time (ms)
50% 2686
66% 2753
75% 2792
80% 2816
90% 2909
95% 2959
98% 3002
99% 3043
100% 3114 (longest request)
With Gevent version (Not sure why ab's output doesn't give anything for Server Software)
C:\Users\Brian\Downloads\python\httpd-2.4.6-win64-VC11\Apache24\bin>ab.exe -n 500 -c 20 http://127.0.0.1:8000/welcome/default/index
This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Finished 500 requests
Server Software:
Server Hostname: 127.0.0.1
Server Port: 8000
Document Path: /welcome/default/index
Document Length: 14093 bytes
Concurrency Level: 20
Time taken for tests: 45.806 seconds
Complete requests: 500
Failed requests: 0
Write errors: 0
Total transferred: 7239000 bytes
HTML transferred: 7046500 bytes
Requests per second: 10.92 [#/sec] (mean)
Time per request: 1832.232 [ms] (mean)
Time per request: 91.612 [ms] (mean, across all concurrent requests)
Transfer rate: 154.33 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 0.5 1 2
Processing: 405 1796 158.9 1813 2142
Waiting: 405 1795 158.9 1812 2142
Total: 406 1796 158.9 1813 2142
Percentage of the requests served within a certain time (ms)
50% 1813
66% 1839
75% 1854
80% 1863
90% 1886
95% 1901
98% 1917
99% 1921
100% 2142 (longest request)
Niphlod
Oct 19, 2013, 8:08:40 AM10/19/13
to web...@googlegroups.com
yep. retry that test pointing to a static file, or a page that doesn't needs session and/or database queries, and you'll see much more performance gain.
BTW, I'm going to inspect your submission later :-P
BTW, I'm going to inspect your submission later :-P
Brian M
Oct 19, 2013, 2:41:25 PM10/19/13
to web...@googlegroups.com
Yeah bit of difference there when hitting a static file Rocket not quite 50/sec Gevent just over 300/sec :D
On Rocket
C:\Users\Brian\Downloads\python\httpd-2.4.6-win64-VC11\Apache24\bin>ab.exe -n 50
This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Finished 500 requests
Server Software: Rocket
Server Hostname: 127.0.0.1
Server Port: 8000
Document Path: /welcome/static/images/twitter.png
Document Length: 1120 bytes
Concurrency Level: 20
Time taken for tests: 10.064 seconds
Complete requests: 500
Failed requests: 0
Write errors: 0
Total transferred: 692000 bytes
HTML transferred: 560000 bytes
Requests per second: 49.68 [#/sec] (mean)
Time per request: 402.580 [ms] (mean)
Time per request: 20.129 [ms] (mean, across all concurrent requests)
Transfer rate: 67.15 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 17 88.9 0 541
Processing: 23 380 291.6 518 1081
Waiting: 20 325 221.9 499 664
Total: 27 396 298.0 519 1081
Percentage of the requests served within a certain time (ms)
50% 519
66% 532
75% 562
80% 580
90% 659
95% 1040
98% 1058
99% 1069
100% 1081 (longest request)
On Gevent
C:\Users\Brian\Downloads\python\httpd-2.4.6-win64-VC11\Apache24\bin>ab.exe -n 50
This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Finished 500 requests
Server Software:
Server Hostname: 127.0.0.1
Server Port: 8000
Document Path: /welcome/static/images/twitter.png
Document Length: 1120 bytes
Concurrency Level: 20
Time taken for tests: 1.645 seconds
Complete requests: 500
Failed requests: 0
Write errors: 0
Total transferred: 674500 bytes
HTML transferred: 560000 bytes
Requests per second: 303.91 [#/sec] (mean)
Time per request: 65.808 [ms] (mean)
Time per request: 3.290 [ms] (mean, across all concurrent requests)
Transfer rate: 400.37 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 0.5 1 1
Processing: 16 63 9.1 63 94
Waiting: 15 63 9.1 62 94
Total: 17 64 9.1 63 95
Percentage of the requests served within a certain time (ms)
50% 63
66% 64
75% 65
80% 66
90% 71
95% 82
98% 86
99% 93
100% 95 (longest request)
Niphlod
Oct 20, 2013, 3:22:29 PM10/20/13
to web...@googlegroups.com
@Brian M won the contest. Congrats to him for committing to test out the recipe and fiddle a little with the source code.
@everyone else: feel free to test the recipe...next binary release will be done following those guides.
@everyone else: feel free to test the recipe...next binary release will be done following those guides.
Brian M
Oct 20, 2013, 4:52:23 PM10/20/13
to web...@googlegroups.com
WhooHoo! :D And thank you Niphlod for all you contribute to the web2py community!
Massimo Di Pierro
Oct 20, 2013, 6:50:34 PM10/20/13
to web...@googlegroups.com
Congratulations!
Reply all
Reply to author
Forward
0 new messages