Beginner's question about tomcat and USL

[Mohit Chawla]

Hello folks, 

I am trying to apply the principles from Dr Neil's books to our tomcat based application, but I have hit a blocker. 

So, from tomcat JMX, we have metrics like request count and processing time, which are total requests tomcat has processed and total processing time for all the requests. If we take the rate of request count per second, we get the throughput. And if we divide the processing time by request count for a time interval, we get the response time. 

Now, if I think of applying little's law in tomcat's context, I have: 

lambda = Throughput X Response Time

lambda = req/s X (processing time / req/s)

lambda = processing time

I am not sure if this is right, can someone take a look please ?  

Thanks, 
Mohit

[DrQ]

Hi Mohit,

In the email request your sent originally there was a text attachment, but it didn't seem to be more than what appears here so, I'm not sure what you mean by "take a look" (at what?) other than the below.

That said and just looking at what you've written here, I see a problem. Your definition of Little's law is not correct.

Your 'lambda' (the arrival rate, λ, I assume) is the same as the departure rate (or throughput, X) in steady state. So, lambda is a throughput.  In my book notation: 

λ = X. Depending on which version of Little's law you are using (see "A Little Triplet") you can calculate other perf metrics, like Q (queue length) or response time (R) or ρ (utilization) or service time (S). For example:

• Q = λ * R
  
• ρ = λ * S

In other words, if you have any 2 of the LL metrics measured, you can calculate the third.

[Mohit Chawla]

Hello Dr. Neil, 

I was just gonna reply with a correction to my email. Thanks for your response. 

Regarding taking a look, I am sorry, I meant just seeing if the way I am applying Little's Law makes sense in tomcat's context, with the metrics it has. 

So, to correctly phrase the usage: 

Q = λ * R

In tomcat's context: 

Q = Requests per second * ( Total processing time for all requests in milliseconds per second / Requests per second )

Q = Requests per second * Average processing time per request

I am thinking what's a good (or just correct !)  interpretation of this, as well as the next set of interpretations that need to be done.

I think, this closely resembles the description of Little's macroscopic law from your book, in which case this number simply means total number of requests in the system. 

However, if I think in terms of the microscopic law, I can't figure out the difference. Do I perhaps need to consider a different set of variables, say, the cpu utilization, number of threads, number or servers for the microscopic / utilization law ? And then to tie these with the universal scalability law, will be the next step.  

Any pointers for moving in the right direction would be appreciated !

Thanks, 
Mohit

--
You received this message because you are subscribed to the Google Groups "Guerrilla Capacity Planning" group.
To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-...@googlegroups.com.
To post to this group, send email to guerrilla-cap...@googlegroups.com.
Visit this group at https://groups.google.com/group/guerrilla-capacity-planning.
For more options, visit https://groups.google.com/d/optout.

[DrQ]

Maybe still some confusion.

From JMX, you can probably get something like the following metrics as inputs:

A: the number of requests in a given time bucket

T: the duration of the time bucket (measurement interval)

R: total processing time

Then, using your below version of LL, we have:

Q = λ * R

But λ = A/T by definition (see my book) so, we can rewrite the above as:

Q = (A/T) * R

Assuming you do have all the numbers on the RHS of this equation, you can calculate Q. But what is Q? (I'm guessing you'd like to know)

For a single queueing facility it's just the queue length (i.e., the average number of requests waiting + those requests in service). Think of a checkout lane in your fave grocery store. More generally, you can consider it to be the level of concurrency in the system of interest. In other words, Q is the total number of requests that are capable of impressing load onto the computing resources.

In your case, it's the number of active java threads (not the thread pool), but I don't know off hand if JMX measures that number correctly.

To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-planning+unsub...@googlegroups.com.
To post to this group, send email to guerrilla-capacity-planning@googlegroups.com.

[Mohit Chawla]

Thanks for your reply and explanation.

Yes, jmx does provide the current number of active threads. 

And to confirm what you meant, Q is the number of (http) requests capable of putting load on the computing resources, that is, they will use up some threads, correct ? 

To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-...@googlegroups.com.
To post to this group, send email to guerrilla-cap...@googlegroups.com.

Visit this group at https://groups.google.com/group/guerrilla-capacity-planning.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Guerrilla Capacity Planning" group.

To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-...@googlegroups.com.
To post to this group, send email to guerrilla-cap...@googlegroups.com.

[DrQ]

I would put it this way: Q is the number of whatevers that are executing, together with the number that will execute. JMX will likely tell you what it thinks those things are.

On Thursday, June 16, 2016 at 12:42:34 PM UTC-7, Mohit Chawla wrote:

Thanks for your reply and explanation.

Yes, jmx does provide the current number of active threads. 

And to confirm what you meant, Q is the number of (http) requests capable of putting load on the computing resources, that is, they will use up some threads, correct ? 

To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-planning+unsub...@googlegroups.com.
To post to this group, send email to guerrilla-capacity-planning@googlegroups.com.

Visit this group at https://groups.google.com/group/guerrilla-capacity-planning.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Guerrilla Capacity Planning" group.

To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-planning+unsub...@googlegroups.com.
To post to this group, send email to guerrilla-capacity-planning@googlegroups.com.

[Mohit Chawla]

Hehe. I think it does give all the numbers, but by now, I have probably confused you as well. Jmx gives the number of requests processed, as well as the current number of threads executing those requests.

In the bigger picture, for doing capacity planning, I need to think in terms of  how many threads would be needed to handle the daily traffic requests for our site, by combining the above numbers in the universal scalability law. That is, if I am still on the right path. :)

To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-...@googlegroups.com.
To post to this group, send email to guerrilla-cap...@googlegroups.com.

Visit this group at https://groups.google.com/group/guerrilla-capacity-planning.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Guerrilla Capacity Planning" group.

To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-...@googlegroups.com.
To post to this group, send email to guerrilla-cap...@googlegroups.com.

Visit this group at https://groups.google.com/group/guerrilla-capacity-planning.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Guerrilla Capacity Planning" group.

To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-...@googlegroups.com.
To post to this group, send email to guerrilla-cap...@googlegroups.com.

[DrQ]

On Thursday, June 16, 2016 at 1:24:37 PM UTC-7, Mohit Chawla wrote:

Hehe. I think it does give all the numbers, but by now, I have probably confused you as well.

That wouldn't be hard. :)

 

Jmx gives the number of requests processed,

That would be more like throughput.

 

as well as the current number of threads executing those requests.

You should verify that statement using the measured JMX numbers. 

Only LL comes from God. Everything else (measured) is the work of the Devil. 

 

In the bigger picture, for doing capacity planning, I need to think in terms of  how many threads would be needed to handle the daily traffic requests for our site, by combining the above numbers in the universal scalability law. That is, if I am still on the right path. :)

That's right. There, Q would become N in the USL and you'd also have the corresponding X(N) from JMX.

 

To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-planning+unsub...@googlegroups.com.
To post to this group, send email to guerrilla-capacity-planning@googlegroups.com.

Visit this group at https://groups.google.com/group/guerrilla-capacity-planning.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Guerrilla Capacity Planning" group.

To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-planning+unsub...@googlegroups.com.
To post to this group, send email to guerrilla-capacity-planning@googlegroups.com.

Visit this group at https://groups.google.com/group/guerrilla-capacity-planning.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Guerrilla Capacity Planning" group.

To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-planning+unsub...@googlegroups.com.
To post to this group, send email to guerrilla-capacity-planning@googlegroups.com.

[DrQ]

FYI, in case you're not aware of developments in the USL world: How to Quantify Scalability

[Mohit Chawla]

Thanks a lot. I bought your books 4-5 years ago. Unfortunately no dog ears on them. But by the weekend I am sure there will be enough for the rest of of my life.  

To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-...@googlegroups.com.
To post to this group, send email to guerrilla-cap...@googlegroups.com.

Visit this group at https://groups.google.com/group/guerrilla-capacity-planning.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Guerrilla Capacity Planning" group.

To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-...@googlegroups.com.
To post to this group, send email to guerrilla-cap...@googlegroups.com.

Visit this group at https://groups.google.com/group/guerrilla-capacity-planning.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Guerrilla Capacity Planning" group.

To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-...@googlegroups.com.
To post to this group, send email to guerrilla-cap...@googlegroups.com.

Visit this group at https://groups.google.com/group/guerrilla-capacity-planning.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Guerrilla Capacity Planning" group.

To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-...@googlegroups.com.
To post to this group, send email to guerrilla-cap...@googlegroups.com.

[Alex Bagehot]

For completeness, as tomcat has a thread per connection the active thread count is affected by http keep alive if enabled. 

Threads are kept 'active' by keep alive connections after they finished servicing a request, not consuming compute resource.

There are many references that discuss it eg 
http://techblog.netflix.com/2015/07/tuning-tomcat-for-high-throughput-fail.html .

Thanks

Alex

[DrQ]

Exactly, Alex. That's the kind of thing I was hinting at. 

I know in the case of Dynatrace, what it counts as "active" threads is bigger than the actual number of executing threads. That's why I suggest checking the reported JMX numbers against LL, to find out. On the other hand, the O/S should know what's executing on CPU, but then the question becomes: does it report it?

On Thursday, June 16, 2016 at 4:38:16 PM UTC-7, ceeaspb wrote:

For completeness, as tomcat has a thread per connection the active thread count is affected by http keep alive if enabled. 

Threads are kept 'active' by keep alive connections after they finished servicing a request, not consuming compute resource.

There are many references that discuss it eg 
http://techblog.netflix.com/2015/07/tuning-tomcat-for-high-throughput-fail.html .

Thanks

Alex

On Thursday, June 16, 2016, Mohit Chawla wrote:

Hehe. I think it does give all the numbers, but by now, I have probably confused you as well. Jmx gives the number of requests processed, as well as the current number of threads executing those requests.

In the bigger picture, for doing capacity planning, I need to think in terms of  how many threads would be needed to handle the daily traffic requests for our site, by combining the above numbers in the universal scalability law. That is, if I am still on the right path. :)

To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-planning+unsub...@googlegroups.com.
To post to this group, send email to guerrilla-capacity-planning@googlegroups.com.

Visit this group at https://groups.google.com/group/guerrilla-capacity-planning.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Guerrilla Capacity Planning" group.

To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-planning+unsub...@googlegroups.com.
To post to this group, send email to guerrilla-capacity-planning@googlegroups.com.

Visit this group at https://groups.google.com/group/guerrilla-capacity-planning.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Guerrilla Capacity Planning" group.

To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-planning+unsub...@googlegroups.com.
To post to this group, send email to guerrilla-capacity-planning@googlegroups.com.

Visit this group at https://groups.google.com/group/guerrilla-capacity-planning.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Guerrilla Capacity Planning" group.

To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-planning+unsub...@googlegroups.com.
To post to this group, send email to guerrilla-capacity-planning@googlegroups.com.

[Mohit Chawla]

Thanks folks.

On Jun 17, 2016 1:47 AM, "'DrQ' via Guerrilla Capacity Planning" <guerrilla-cap...@googlegroups.com> wrote:

Exactly, Alex. That's the kind of thing I was hinting at. 

I know in the case of Dynatrace, what it counts as "active" threads is bigger than the actual number of executing threads. That's why I suggest checking the reported JMX numbers against LL, to find out. On the other hand, the O/S should know what's executing on CPU, but then the question becomes: does it report it?

On Thursday, June 16, 2016 at 4:38:16 PM UTC-7, ceeaspb wrote:

For completeness, as tomcat has a thread per connection the active thread count is affected by http keep alive if enabled. 

Threads are kept 'active' by keep alive connections after they finished servicing a request, not consuming compute resource.

There are many references that discuss it eg 
http://techblog.netflix.com/2015/07/tuning-tomcat-for-high-throughput-fail.html .

Thanks

Alex

On Thursday, June 16, 2016, Mohit Chawla wrote:

Hehe. I think it does give all the numbers, but by now, I have probably confused you as well. Jmx gives the number of requests processed, as well as the current number of threads executing those requests.

In the bigger picture, for doing capacity planning, I need to think in terms of  how many threads would be needed to handle the daily traffic requests for our site, by combining the above numbers in the universal scalability law. That is, if I am still on the right path. :)

To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-...@googlegroups.com.
To post to this group, send email to guerrilla-cap...@googlegroups.com.

Visit this group at https://groups.google.com/group/guerrilla-capacity-planning.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Guerrilla Capacity Planning" group.

To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-...@googlegroups.com.
To post to this group, send email to guerrilla-cap...@googlegroups.com.

Visit this group at https://groups.google.com/group/guerrilla-capacity-planning.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Guerrilla Capacity Planning" group.

To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-...@googlegroups.com.
To post to this group, send email to guerrilla-cap...@googlegroups.com.

Visit this group at https://groups.google.com/group/guerrilla-capacity-planning.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Guerrilla Capacity Planning" group.

To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-...@googlegroups.com.
To post to this group, send email to guerrilla-cap...@googlegroups.com.

Visit this group at https://groups.google.com/group/guerrilla-capacity-planning.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Guerrilla Capacity Planning" group.

To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-...@googlegroups.com.
To post to this group, send email to guerrilla-cap...@googlegroups.com.