> A handler that according to tornado finishes in 10ms is getting half
> the RPS as a handler that is generated in 30ms, and I am pretty
> clueless atm.

> On Dec 2, 12:14 pm, keztoo <kez...@yahoo.com> wrote:

> > On that same box I have 200 open at the same time but I am sure the
> > number is much higher as that was a simple test using a single python
> > app per request and we were running from swap almost immediately. I
> > would guess that 500-1000 (if they were not very busy) would be
> > possible.

> > On Dec 2, 3:04 pm, Stanislav <vishnevs...@gmail.com> wrote:

> > > While we are at it, out of curiosity how many open long polling
> > > connections could one Tornado hold?

> > > On Dec 2, 10:24 am, Stanislav <vishnevs...@gmail.com> wrote:

> > > > That is a wonderful response, I was just hoping for a target point to
> > > > aim for when benchmarking our site. I benchmarked various pages of our
> > > > site from pages that use 1 SQL query to more, and and one instance of
> > > > tornado goes from 25-30 to 70 RPS depending on page, so I am guessing
> > > > based on you reply that our RPS is in good standing.

> > > > On Dec 2, 9:12 am, keztoo <kez...@yahoo.com> wrote:

> > > > > There are 86,400 seconds in a day. If you handle 1 request per second
> > > > > you are handling 86,400 requests per day. If you handle 10 requests
> > > > > per second you are handling 864,000 requests per day (or nearly 1
> > > > > million requests a day) and if you can handle 100 requests per second
> > > > > you are handling 8,640,000 (over 8 million), so 100 requests per
> > > > > second is usually a good starting point. Unfortunately, usage is not
> > > > > evenly distributed like this and you tend to get bursts of requests
> > > > > which can impact your server.

> > > > > In your case, another factor to consider is how much data you deliver
> > > > > (a 1k page, a 10k page, a 100k page, etc) and how long your sql
> > > > > requests take. I tend to use a service oriented architecture where the
> > > > > sql requests go over the wire to a farm of services which do the
> > > > > actual sql access. I use async call backs on the front end and using
> > > > > this approach I have noticed a communication overhead of about 2ms per
> > > > > request. The actual sql takes about 20ms and misc overhead contributes
> > > > > enough for us to say a typical request will take about 25ms.

> > > > > Assuming a single threaded approach where requests happen serially
> > > > > (which is not the case, but just for performance analysis) this means
> > > > > you can get about 40 requests per second (1,000 ms per second divided
> > > > > by 25ms per request) out of a single single threaded instance. Now in
> > > > > the case of tornado you can get more than one request going at a time,
> > > > > so you will get more than 40 requests per second out of a single
> > > > > tornado instance.

> > > > > Usually the limiting factor is the page payload and front-end
> > > > > bandwidth. Assuming you deliver 100k per page and you have a 10mbs
> > > > > link to your website, you can deliver about 10 pages a second
> > > > > (remember 100k is in bytes, mbs is in bits; there is about a 10:1
> > > > > ratio here) or put another way; you can support 10 simultaneous
> > > > > users.

> > > > > But lets forget about front-end bandwidth concerns (assume you are on
> > > > > a cloud with unlimited bandwidth) and concentrate on server
> > > > > performance. Now lets say each tornado instance as described above can
> > > > > deliver roughly 40-100 requests per second (which is what we see on
> > > > > our small cloud machine which is a 1Ghz Athelon with about 380Mb of
> > > > > memory) then you can easily increase this by using nginx to load
> > > > > balance several of the front-end (we call them MVCs or on-lines)
> > > > > machines in parallel. To keep this post brief I will summarize what we
> > > > > currently see ...

> > > > > Using 1 Front-ends and 1 service instance we get about 25 rps.
> > > > > Using 2 Front-ends and 2 service instances we get about 40 rps.
> > > > > Using 4 Front-ends and 16 service instances we get about 80 rps.
> > > > > Using 8 Front-ends and 32 service instances we get about 125 rps.

> > > > > Not what we were expecting but we are investigating what is wrong as I
> > > > > speak (for example, we also have couch db running on that box, a rails
> > > > > app and some other stuff). Now keep in mind when I say "Using 8 Front-
> > > > > ends and 32 service instances we get about 125 rps" what I really mean
> > > > > is on that single 1Ghz machine (with about 380Mb of memory) we start
> > > > > one instance of nginx (to load balance both the front and back ends)
> > > > > 32 instances of our back end service and 8 tornados (each on its on
> > > > > port obviously) so we are probably overloading our poor 1Ghz cloud
> > > > > machine but like I said, we are currently investigating. Still, we get
> > > > > about 100rps for a single very small machine. I know that when I did
> > > > > some tests at work (the small cloud machine is for a home start-up we
> > > > > are working on) on dedicated hardware I was seeing closer to
> > > > > 500-700rps which is about what I expected and is not bad for a single
> > > > > server.

> > > > > So to summarize; if you want to support about a million requests a day
> > > > > you need to be able to handle at least 10 requests per second
> > > > > sustained. This should only require one small server.

> > > > > Hope that helps a bit.

> > > > > On Dec 2, 6:29 am, Stanislav <vishnevs...@gmail.com> wrote:

> > > > > > We are developing a large site and we love Tornado, however we were
> > > > > > wondering how many requests per second is a good number for a single
> > > > > > Tornado instance? This instance is of course doing queries to SQL, and
> > > > > > rendering a template.

> > > > > > I would greatly appreciate a reply.