Please check out the site at http://speed.rubyspec.org and look around. There isn't much data posted yet. The EC2 instance is churning through a few commits I cherry-picked from early May through the present day. It is working on today's commit(s) first and will eventually chug through the rest of the backlog.
When clicking on the Timeline tab, it is going to load up a *lot* of benchmarks. The Ruby world has a lot of low-level benchmarks so we can track the perf of each method in the core (and eventually the standard library). We even have separate benchmarks for the same method but with different arguments (e.g. "a sample string".split(' ') and "a sample string".split('|')) so we can test average and worst case scenarios.
As you can see from that page, it really stresses your browser's javascript engine. :)
Again, thanks!
cr
> Hi:
>
> On Jun 21, 5:08 am, Chuck Remes <cremes.devl...@mac.com> wrote:
>> Please check out the site at http://speed.rubyspec.organd look around. There isn't much data posted yet. The EC2 instance is churning through a few commits I cherry-picked from early May through the present day. It is working on today's commit(s) first and will eventually chug through the rest of the backlog.
>>
> Interesting approach with EC2. What are the guarantees you get there?
> Are you guaranteed to have your VM image to be scheduled always on the
> same machine? And always access to the same resources i.e. memory and
> CPU time?
The EC2 instance was donated by a member of the community. I don't know what kinds of guarantees it offers, if any. I've heard EC2 performance can fluctuate as much as 20% over time, but for the purposes of tracking perf trends across runtimes I still think it's sufficient. If any major change shows up, interested developers can zero in on it and see if it's a real problem or just an artifact of EC2 usage.
>> When clicking on the Timeline tab, it is going to load up a *lot* of benchmarks.
> Indeed, Chrome asked my twice and then I canceled. Safari actually
> showed the webpage.
> Think, there is some performance tuning necessary, otherwise many of
> your users will just not be patient enough.
I don't even know where to begin. When I click on that tab, I see that python goes to 100% for a period of time. I don't know how to improve that. It looks to me like a *browser* performance issue.
We are using the built-in web server for sending out the results. I have run out of time to figure out how to do a deployment with apache/nginx/lighttpd/etc so it will have to do for now.
If people who are interested in the data cancel the operation before it completes, then perhaps they aren't *really interested* in the data. I don't imagine it will be a high traffic site except for an initial flurry of activity now that it has launched.
> In case you are doing something in that direction, I would be
> interested, since I also have a lot benchmarks and performance
> troubles.
In a few weeks I hope to have time to fix up some of the things where I had to take a short-cut. I am also hoping that our experiences and results will turn into a few issues that can be tackled by members of the codespeed community who are familiar with the code base. In particular, I think it would be neat to have "collapsable" views for the Timeline tab where we can define a hierarchy for the benchmarks and by default they are hidden unless someone clicks on them to expand (think of the List view in MacOSX Finder).
e.g.
core/array/bench_append
core/array/bench_dup
core/array/bench_pack
core/array/bench_map
core/array/bench_sort
core/hash/bench_dup
core/hash/bench_sort
these would collapse to
> core
then expand to
>core->array
>core->hash
and again to
>core->array
bench_append
bench_dup
etc
>core->hash
bench_dup
bench_sort
Like I said, I'll start to file issues on these kinds of things in a few weeks. I want to get a little distance from the problem so that I can see the forest for the trees before I start suggesting a bunch of enhancements/changes to the UI.
cr
Hi,
> > On Jun 21, 5:08 am, Chuck Remes <cremes.devl...@mac.com> wrote:
> >> Please check out the site at http://speed.rubyspec.organd look around.
> >> There isn't much data posted yet. The EC2 instance is churning through
> >> a few commits I cherry-picked from early May through the present day.
> >> It is working on today's commit(s) first and will eventually chug
> >> through the rest of the backlog.
> >>
[...]
> >> When clicking on the Timeline tab, it is going to load up a *lot* of
> >> benchmarks.
> > Indeed, Chrome asked my twice and then I canceled. Safari actually
> > showed the webpage. Think, there is some performance tuning necessary,
> > otherwise many of your users will just not be patient enough.
>
> I don't even know where to begin. When I click on that tab, I see that
> python goes to 100% for a period of time. I don't know how to improve
> that. It looks to me like a *browser* performance issue.
>
> We are using the built-in web server for sending out the results. I have
That is a built-in development server. It is single threaded and *only* for
development! IOW it is not able to serve more than one client at a time.
> run out of time to figure out how to do a deployment with
> apache/nginx/lighttpd/etc so it will have to do for now.
I'm happy to help w/ Apache or nginx. Tell me what you prefer I explain how
to configure it using WSGI.
> > In case you are doing something in that direction, I would be
> > interested, since I also have a lot benchmarks and performance
> > troubles.
Can you please provide a little more details? Thanks.
Bye,
Frank
--
Frank Becker <f...@alien8.de> (jabber|mail) | http://twitter.com/41i3n8
GnuPG: 0xADC29ECD | F01B 5E9C 1D09 981B 5B40 50D3 C80F 7459 ADC2 9ECD
SILC-Net: a8 | Home: http://www.alien8.de | <<</>> http://www.c3d2.de
"> Freedom is just chaos, with better lighting. <" Alan Dean Foster
> Chuck Remes [2011-06-21, 13:31 +0200]:
>> On Jun 21, 2011, at 4:13 AM, Stefan Marr wrote:
>>
>
> Hi,
>
>>> On Jun 21, 5:08 am, Chuck Remes <cremes.devl...@mac.com> wrote:
>>>> Please check out the site at http://speed.rubyspec.organd look around.
>>>> There isn't much data posted yet. The EC2 instance is churning through
>>>> a few commits I cherry-picked from early May through the present day.
>>>> It is working on today's commit(s) first and will eventually chug
>>>> through the rest of the backlog.
>>>>
> [...]
>>>> When clicking on the Timeline tab, it is going to load up a *lot* of
>>>> benchmarks.
>>> Indeed, Chrome asked my twice and then I canceled. Safari actually
>>> showed the webpage. Think, there is some performance tuning necessary,
>>> otherwise many of your users will just not be patient enough.
>>
>> I don't even know where to begin. When I click on that tab, I see that
>> python goes to 100% for a period of time. I don't know how to improve
>> that. It looks to me like a *browser* performance issue.
>
>>
>> We are using the built-in web server for sending out the results. I have
> That is a built-in development server. It is single threaded and *only* for
> development! IOW it is not able to serve more than one client at a time.
It hasn't been a problem yet! :)
>> run out of time to figure out how to do a deployment with
>> apache/nginx/lighttpd/etc so it will have to do for now.
> I'm happy to help w/ Apache or nginx. Tell me what you prefer I explain how
> to configure it using WSGI.
I would prefer nginx. Telling me all of the right components to 'apt-get' would be great, plus which files I should touch for codespeed/django to use it.
>>> In case you are doing something in that direction, I would be
>>> interested, since I also have a lot benchmarks and performance
>>> troubles.
> Can you please provide a little more details? Thanks.
That isn't my quote; it was from Stefan. Hopefully he will elaborate.
cr
great to see you got that Speed center up and running. Congrats!
First of all, I think it is terrific that Chuck is bringing up all
this issues (instead of just giving up and not providing feedback),
and that you all are helping out in finding solutions.
It is clear that Codespeed wasn't designed to do well beyond a number
of parameters (exes*envs*benchs < 50 maybe), neither UI-wise, nor
performance-wise. With enough users and contributors now, I think we
have the chance to rethink how we could improve on that so that it
covers most use-cases. Chuck's "benchmark grouping" idea and Stefan
progressive rendering are going in the right direction, IMHO.
I'll release 0.8 shortly and use the chance to start the 0.9 release
cycle discussion.
Cheers!
PS:@Frank: and if you have any questions while helping out Chuck, just ask!
2011/6/21 Stefan Marr <goo...@stefan-marr.de>:
Can you please have a look whether a8's Django template stuff
refactoring discussed in
https://github.com/tobami/codespeed/issues/25
can be merged before 0.8?
I haven't seen a pull request for that, or maybe it already is merged,
not sure, anyway, at least I'm interested in those for my setup.
Thanks.
Kind regards,
Steffen
--
Steffen Schwigon <s...@renormalist.net>
Ha, I should read all emails before answer. :-)
I just read it's not yet in 0.8 but planned for 0.9.
Hi,
>> Can you please have a look whether a8's Django template stuff
>> refactoring discussed in
>>
>> https://github.com/tobami/codespeed/issues/25
>>
>> can be merged before 0.8?
>
> Ha, I should read all emails before answer. :-)
> I just read it's not yet in 0.8 but planned for 0.9.
And, as the ticket reads I keep that branch
https://github.com/a8/codespeed-a8/tree/separation
rebased on top of tobami's master.
Thanks,
Hi,
>>> run out of time to figure out how to do a deployment with
>>> apache/nginx/lighttpd/etc so it will have to do for now.
>> I'm happy to help w/ Apache or nginx. Tell me what you prefer I explain how
>> to configure it using WSGI.
>
> I would prefer nginx. Telling me all of the right components to 'apt-get' would be great, plus which files I should touch for codespeed/django to use it.
Please have a look at:
https://github.com/a8/codespeed-a8/blob/separation/example/README.md
section: Installing for production
and tell me at which step you might fail or what has to be documented
differently/better.
Thanks,
> On 6/21/11 3:00 PM, Chuck Remes wrote:
>
> Hi,
>
>>>> run out of time to figure out how to do a deployment with
>>>> apache/nginx/lighttpd/etc so it will have to do for now.
>>> I'm happy to help w/ Apache or nginx. Tell me what you prefer I explain how
>>> to configure it using WSGI.
>>
>> I would prefer nginx. Telling me all of the right components to 'apt-get' would be great, plus which files I should touch for codespeed/django to use it.
> Please have a look at:
> https://github.com/a8/codespeed-a8/blob/separation/example/README.md
> section: Installing for production
> and tell me at which step you might fail or what has to be documented
> differently/better.
Frank,
I've gotten stuck right away.
I can't accomplish step #3 because I cannot find any directory 'deploy' with the nginx.default-site.conf file. Also, my codespeed directory doesn't have a directory called sitestatic (I can't find that anywhere).
Step #5 doesn't work. There is no command 'collectstatic' that is accepted by manage.py.
Step #6 doesn't work unless 'gunicorn' is added to the INSTALLED_APPS array in settings.py.
Perhaps your fork has diverged a bit more from the codespeed master than you recalled?
cr
I see that a number of these things only exist on the 'separation' branch. Going through it now.
cr
Hi,
>>>>> run out of time to figure out how to do a deployment with
>>>>> apache/nginx/lighttpd/etc so it will have to do for now.
>>>> I'm happy to help w/ Apache or nginx. Tell me what you prefer I explain how
>>>> to configure it using WSGI.
>>>
>>> I would prefer nginx. Telling me all of the right components to 'apt-get' would be great, plus which files I should touch for codespeed/django to use it.
>> Please have a look at:
>> https://github.com/a8/codespeed-a8/blob/separation/example/README.md
>> section: Installing for production
>> and tell me at which step you might fail or what has to be documented
>> differently/better.
>
> Frank,
>
> I've gotten stuck right away.
Sorry, right. My fault. All those files are in that branch [0] on
github. The purpose of that branch is to "decouple" the codespeed app
(Django wording for [extension|plugin|module|...]) That makes it way
more simple to integrate codespeed into other Django projects - among
other advantages. History of that is in ticket #25 [1]. One concern was
that it may be more difficult to set up a new speedcenter site from
scratch. Hence, I tried to write up some install docu.
The part that describes how to configure Nginx or Apache should work on
the current v0.8 tobmai/master.
To get the files please check out my tree:
git clone g...@github.com:a8/codespeed-a8.git
cd codespeed-a8
git checkout -b separation origin/separation
cd example/deploy
OR
find the files on github via the Web:
https://github.com/a8/codespeed-a8/tree/separation/example/deploy
[0] https://github.com/a8/codespeed-a8/tree/separation
[1] https://github.com/tobami/codespeed/issues/25
Good luck,
Hi,
> The part that describes how to configure Nginx or Apache should work on
> the current v0.8 tobmai/master.
After a 2nd thought that is not right. Django v1.3 has a former Django
app called staticfiles [0] integrated. That deals w/ serving static
files (done by e. g. ngnix and not Django). One part of my branch is to
switch to use that.
You will need an alias for admin_media e. g. in Apache speak:
Alias /admin_media/ /usr/share/pyshared/django/contrib/admin/media/
and please ignore step:
5 Prepare static files cd /path/to/speedcenter/ python ./manage.py \
collectstatic
[0] https://docs.djangoproject.com/en/1.3/howto/static-files/
Good luck again,
> On 6/22/11 4:29 PM, Frank Becker wrote:
>> On 6/22/11 3:58 PM, Chuck Remes wrote:
>>> On Jun 22, 2011, at 8:00 AM, Frank Becker wrote:
>>>
>>>> On 6/21/11 3:00 PM, Chuck Remes wrote:
>
> Hi,
>
>> The part that describes how to configure Nginx or Apache should work on
>> the current v0.8 tobmai/master.
> After a 2nd thought that is not right. Django v1.3 has a former Django
> app called staticfiles [0] integrated. That deals w/ serving static
> files (done by e. g. ngnix and not Django). One part of my branch is to
> switch to use that.
>
> You will need an alias for admin_media e. g. in Apache speak:
>
> Alias /admin_media/ /usr/share/pyshared/django/contrib/admin/media/
>
> and please ignore step:
> 5 Prepare static files cd /path/to/speedcenter/ python ./manage.py \
> collectstatic
>
>
> [0] https://docs.djangoproject.com/en/1.3/howto/static-files/
Frank, I got allof it working with nginx. The hardest part of the instructions to understand was the static file stuff.
In my /etc/nginx/site-enabled/default file I made the path to my static files equal to /path/to/speedcenter/media/$1 and it's working okay. The site is a *lot* faster with this option enabled.
I ran into a bug with some python library (python_meld?) and supervisor, so I switched over to using a shell script and upstart to start/stop it. It's working great. I now have 3 gunicorn workers to help spread the load.
Also, I found this blog post very helpful in understanding what I was doing:
http://davidpoblador.com/run-django-apps-using-gunicorn-and-nginx/
cr
Hi,
> Frank, I got allof it working with nginx. The hardest part of the instructions to understand was the static file stuff.
Congratulations. OK, the static file stuff will come back.
I'll try to explain that a little more.
> In my /etc/nginx/site-enabled/default file I made the path to my static files equal to /path/to/speedcenter/media/$1 and it's working okay. The site is a *lot* faster with this option enabled.
>
> I ran into a bug with some python library (python_meld?) and supervisor, so I switched over to using a shell script and upstart to start/stop it. It's working great. I now have 3 gunicorn workers to help spread the load.
Debian or Ubuntu? Oh I see, Ubuntu 11.04.
The problem is in the xml module. I will see if there is a Ubuntu bug
open already tomorrow.
> Also, I found this blog post very helpful in understanding what I was doing:
>
> http://davidpoblador.com/run-django-apps-using-gunicorn-and-nginx/
:-) I've learned about supervisor from that page. Upstart is not
available everywhere.
The root directive in the ngnix config did not work for me though.
Bye,
Hi,
> I ran into a bug with some python library (python_meld?) and supervisor, so I switched over to using a shell script and upstart to start/stop it. It's working great. I now have 3 gunicorn workers to help spread the load.
Just for completeness here is the Ubuntu bug for supervisor incl. some
work arounds:
https://bugs.launchpad.net/ubuntu/+source/supervisor/+bug/777862
Bye,