Website benchmarking
18 views
Skip to first unread message
Jeremy Morse
Sep 5, 2015, 7:55:37 AM9/5/15
to sr...@googlegroups.com
Hi all,
As mentioned in a couple of threads there are some media things going
on. This is going to mean an uptick in website traffic when it happens;
exactly how much is impossible to predict. I'd quite like to put some
effort into keeping the website up.
Several people I've asked have recommended putting cloudflare in front
of the website to cache a bunch of things, and generally use their CDN
to offload work. This makes sense, is free [0, 1], and we can just turn
it on and off. The questions to my mind is "how much more traffic can we
take with cloudflare?", and "are there any rough edges we can optimise?".
The best way to answer those questions is actually doing some
benchmarking of srweb [2]. That requires knowing a representative
distribution of pages on the website that your average visitor hits. We
can then hit the website [3] with / without cloudflare using that
distribution, and measure the difference. There are also a wealth of web
benchmarking tools out there we can use to optimise Stuff.
I have several days of free time booked up; would someone have spare
time for generating such a benchmarking situation? The data available
would be a few weeks worth of webserver logs that can be processed;
there's also the contents of piwik (srobo.org/piwik) that might yield
useful information. I can also generate a VM + subdomains to be
benchmarked at.
[0] Mild concern about SSL: they'd be using their own CA / certificate
for studentrobotics.org. If we're at all worried about security though
we can keep a subdomain pointing at the real webserver and access that
directly.
[1] Cloudflare use SNI to disambiguate the destination domain of https
requests. This rules out several old browsers; however AFAIK our SSL
configuration (i.e. banning RC4) already rules all those old browsers out.
[2] The unwashed masses will not be using the IDE, forum, trac etc.
[3] Or a copy of it
--
Thanks,
Jeremy
As mentioned in a couple of threads there are some media things going
on. This is going to mean an uptick in website traffic when it happens;
exactly how much is impossible to predict. I'd quite like to put some
effort into keeping the website up.
Several people I've asked have recommended putting cloudflare in front
of the website to cache a bunch of things, and generally use their CDN
to offload work. This makes sense, is free [0, 1], and we can just turn
it on and off. The questions to my mind is "how much more traffic can we
take with cloudflare?", and "are there any rough edges we can optimise?".
The best way to answer those questions is actually doing some
benchmarking of srweb [2]. That requires knowing a representative
distribution of pages on the website that your average visitor hits. We
can then hit the website [3] with / without cloudflare using that
distribution, and measure the difference. There are also a wealth of web
benchmarking tools out there we can use to optimise Stuff.
I have several days of free time booked up; would someone have spare
time for generating such a benchmarking situation? The data available
would be a few weeks worth of webserver logs that can be processed;
there's also the contents of piwik (srobo.org/piwik) that might yield
useful information. I can also generate a VM + subdomains to be
benchmarked at.
[0] Mild concern about SSL: they'd be using their own CA / certificate
for studentrobotics.org. If we're at all worried about security though
we can keep a subdomain pointing at the real webserver and access that
directly.
[1] Cloudflare use SNI to disambiguate the destination domain of https
requests. This rules out several old browsers; however AFAIK our SSL
configuration (i.e. banning RC4) already rules all those old browsers out.
[2] The unwashed masses will not be using the IDE, forum, trac etc.
[3] Or a copy of it
--
Thanks,
Jeremy
Harry Cutts
Sep 7, 2015, 7:08:14 AM9/7/15
to sr...@googlegroups.com
Hi Jeremy,
On 5 September 2015 at 12:55, Jeremy Morse <jeremy...@gmail.com> wrote:
The best way to answer those questions is actually doing some
benchmarking of srweb [2].
Last time I looked, the work being done on porting srweb to Jekyll was nearly complete. (Tom, maybe you can give us a status report.) This would make nearly everything into static content, removing what (I imagine) is a rather large performance hit. Might it be worth investing time into getting srweb-jekyll ready before September 18th, rather than in benchmarking the 'old' system?
Harry Cutts
Tom Leese
Sep 7, 2015, 8:32:59 AM9/7/15
to sr...@googlegroups.com
Hi,
> On 7 Sep 2015, at 12:07, Harry Cutts <eterna...@gmail.com> wrote:
>
> Last time I looked, the work being done on porting srweb to Jekyll was nearly complete. (Tom, maybe you can give us a status report.) This would make nearly everything into static content, removing what (I imagine) is a rather large performance hit. Might it be worth investing time into getting srweb-jekyll ready before September 18th, rather than in benchmarking the 'old' system?
I do agree it would be great if we could aim for this.
The last remaining bit of srweb-jekyll is the teams status subsystem. I suspect this will require Peter’s input if we wanted this done quickly.
Tom
> On 7 Sep 2015, at 12:07, Harry Cutts <eterna...@gmail.com> wrote:
>
> Last time I looked, the work being done on porting srweb to Jekyll was nearly complete. (Tom, maybe you can give us a status report.) This would make nearly everything into static content, removing what (I imagine) is a rather large performance hit. Might it be worth investing time into getting srweb-jekyll ready before September 18th, rather than in benchmarking the 'old' system?
The last remaining bit of srweb-jekyll is the teams status subsystem. I suspect this will require Peter’s input if we wanted this done quickly.
Tom
Jeremy Morse
Sep 8, 2015, 8:39:12 AM9/8/15
to sr...@googlegroups.com
Hi,
On 07/09/15 12:07, Harry Cutts wrote:
> Last time I looked, the work being done on porting srweb to Jekyll was
> nearly complete. (Tom, maybe you can give us a status report.) This
> would make nearly everything into static content, removing what (I
> imagine) is a rather large performance hit. Might it be worth investing
> time into getting srweb-jekyll ready before September 18th, rather than
> in benchmarking the 'old' system?
My understanding is that cloudflare will cache anything that doesn't
contain a query string in the URL, which captures the entirety of srweb.
If that's the case, there wouldn't be any improvement here.
In either case, it'd be possible to apply the website benchmarking
against both projects with trivial effort (assuming the same url scheme
has been kept), so this is something we can actually determine if a
benchmark suite existed.
(We also wouldn't be able to correctly characterise how people use a
jekyll website as we don't have any data on that).
--
Thanks,
Jeremy
On 07/09/15 12:07, Harry Cutts wrote:
> Last time I looked, the work being done on porting srweb to Jekyll was
> nearly complete. (Tom, maybe you can give us a status report.) This
> would make nearly everything into static content, removing what (I
> imagine) is a rather large performance hit. Might it be worth investing
> time into getting srweb-jekyll ready before September 18th, rather than
> in benchmarking the 'old' system?
contain a query string in the URL, which captures the entirety of srweb.
If that's the case, there wouldn't be any improvement here.
In either case, it'd be possible to apply the website benchmarking
against both projects with trivial effort (assuming the same url scheme
has been kept), so this is something we can actually determine if a
benchmark suite existed.
(We also wouldn't be able to correctly characterise how people use a
jekyll website as we don't have any data on that).
--
Thanks,
Jeremy
Rich Barlow
Sep 8, 2015, 8:56:26 PM9/8/15
to sr...@googlegroups.com
Hi,
On Sat, 2015-09-05 at 12:55 +0100, Jeremy Morse wrote:
> The best way to answer those questions is actually doing some
> benchmarking of srweb [2]. That requires knowing a representative
> distribution of pages on the website that your average visitor hits. We
> can then hit the website [3] with / without cloudflare using that
> distribution, and measure the difference. There are also a wealth of web
> benchmarking tools out there we can use to optimise Stuff.
I've just done some benchmarking using Tsung[1]. I'll publish more
details and configs/scripts at some point soon. I've created a Linode
with the same spec as saffron and have been testing with a user story of
home->15s->top news article->7s->how_to_enter. The story includes all
external resources (js, css, etc.) and assumes that browser caches work.
For now, I can safely say that we can sustain 18 new users per second
(following the above user story) for an indefinite period. Adding users
at this rate results in an average of 400 concurrent users. The 98th
percentile request time is 167ms, the mean is 28ms. The mean load time
of the home page (including external resources) is 290ms.
Rich
[1] http://tsung.erlang-projects.org/
On Sat, 2015-09-05 at 12:55 +0100, Jeremy Morse wrote:
> The best way to answer those questions is actually doing some
> benchmarking of srweb [2]. That requires knowing a representative
> distribution of pages on the website that your average visitor hits. We
> can then hit the website [3] with / without cloudflare using that
> distribution, and measure the difference. There are also a wealth of web
> benchmarking tools out there we can use to optimise Stuff.
details and configs/scripts at some point soon. I've created a Linode
with the same spec as saffron and have been testing with a user story of
home->15s->top news article->7s->how_to_enter. The story includes all
external resources (js, css, etc.) and assumes that browser caches work.
For now, I can safely say that we can sustain 18 new users per second
(following the above user story) for an indefinite period. Adding users
at this rate results in an average of 400 concurrent users. The 98th
percentile request time is 167ms, the mean is 28ms. The mean load time
of the home page (including external resources) is 290ms.
Rich
[1] http://tsung.erlang-projects.org/
Jeremy Morse
Sep 9, 2015, 5:28:17 AM9/9/15
to sr...@googlegroups.com
Hi,
On 09/09/15 01:56, Rich Barlow wrote:
> I've just done some benchmarking using Tsung[1]. I'll publish more
> details and configs/scripts at some point soon. I've created a Linode
> with the same spec as saffron and have been testing with a user story of
> home->15s->top news article->7s->how_to_enter. The story includes all
> external resources (js, css, etc.) and assumes that browser caches work.
/me 's eye twitches at the term "user story",
> For now, I can safely say that we can sustain 18 new users per second
> (following the above user story) for an indefinite period. Adding users
> at this rate results in an average of 400 concurrent users. The 98th
> percentile request time is 167ms, the mean is 28ms. The mean load time
> of the home page (including external resources) is 290ms.
OK, that's actually sounding pretty good: obviously we have no way of
knowing how many people are actually going to visit the website. If
possible, it'd be great to know if the number of new-users scales
nearly-linearly with the size of the VM: if we can get away with just
getting the largest linode available for a day, we'd need very little
other technical work.
--
Thanks,
Jeremy
On 09/09/15 01:56, Rich Barlow wrote:
> I've just done some benchmarking using Tsung[1]. I'll publish more
> details and configs/scripts at some point soon. I've created a Linode
> with the same spec as saffron and have been testing with a user story of
> home->15s->top news article->7s->how_to_enter. The story includes all
> external resources (js, css, etc.) and assumes that browser caches work.
> For now, I can safely say that we can sustain 18 new users per second
> (following the above user story) for an indefinite period. Adding users
> at this rate results in an average of 400 concurrent users. The 98th
> percentile request time is 167ms, the mean is 28ms. The mean load time
> of the home page (including external resources) is 290ms.
knowing how many people are actually going to visit the website. If
possible, it'd be great to know if the number of new-users scales
nearly-linearly with the size of the VM: if we can get away with just
getting the largest linode available for a day, we'd need very little
other technical work.
--
Thanks,
Jeremy
0 new messages