Re: myFirstPDQ, its about time

[DrQ]

It would help if I could see your PDQ model and run it myself.

"Using 8 service nodes, can I say that service time will be reduced by a factor of 8? " you could do that in your current M/M/1 type model, but that carries with it certain intrinsic assumptions that may not apply. Why not use CreateMultiNode? http://www.perfdynamics.com/Tools/PDQman.html#tth_sEc3.2

On Monday, May 20, 2013 10:30:14 AM UTC-7, steve1951 wrote:

Yes, pun intended.  Starting to use the PDQ.  To keep things simple, I am using the same code as test, changing service time and arr rate.  The system I'm investigating is a database server with a known business transaction completions and a measured utilization.  I use the standard equations to get to arr rate and service time. The user base is over 1000 controlling re-do type work and more than 5000 outside points initiating applications.  The server itself is a multiprocessor 2 socket, 4 core per socket machine meaning it has a total of 8 core with 8 threads processing at a time (no hyper threading, DOP=8).  When I did this as an excel exercise, using Little's law, it worked out within 2% of the average utilization.  Comparing Little to a wide range of actual utilization levels was very successful as well, mostly staying around +/- 3% .  I also found a M/M/1 engine on the web.  The results for the average workload and service time hit the mark for utilization.  When I extrapolate the demand, the M/M/1 model hits close to another server doing the same work for different clients at threshold demand, showing a response time at the server similar to the actual second system measured response time and utilization.  I think this is very cool, but I want to be able to hit this with an understanding of the underlying number of core.  In this way, develop a model that will help estimate response times for varying levels of core.

My next step was to install the PDQ.  I ran the test script without issue.  I created my own 'MyFirstPDQ.pl' basically just copying the test.pl and changing the arr rate and service time parameters - to run a M/M/1 model.  This of course did not work.  The echo back was 'don't know how to make 'myfirstpdq'.  Ok, didn't let that bother me.  I ran 'test' with the same changes.  Output matched the web q model output very closely.  

Now the hard part.  looking at the server in a more microscopic lens.  Using 8 service nodes, can I say that service time will be reduced by a factor of 8?  That actually severely understates the work being done by the server.  Contrarily, increasing it by a factor of 8, gets to an approx utilization but completely overstates the actual residence; which I would expect because service time becomes the major component of residence time.  Here are the basic measures

service time 2.978 seconds

arr. rate        .169 trans/sec

Thanks for any feedback/discussion,

Steve

             

[stephen marksamer]

That is exactly what I did; I used the CreateMultiNode function.  Here's my script: attached

--
You received this message because you are subscribed to the Google Groups "Guerrilla Capacity Planning" group.
To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-...@googlegroups.com.
To post to this group, send email to guerrilla-cap...@googlegroups.com.
Visit this group at http://groups.google.com/group/guerrilla-capacity-planning?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[DrQ]

From what you described, I thought you meant you were dividing the service time by 8: S -> S/8 to make the M/M/1 model go 8x faster. I see you used pdq::CreateMultiNode(8, ...) but you also seem to have increased S -> S*8. Why? That implies a different workload.

The general idea is to add more server capacity with the same S to reduce the residence time R. I also see that your arrival rate is $arrivRate = 0.147; and not the 0.169 trans/sec you originally stated. Why?

With 8x server capacity, you can potentially drive the request rate up to 8 times higher.

So, using $arrivRate = 0.169; with 8 servers, this is what I get. But you didn't supply any values to compare against, so I don't  know if it's meaningful.

               ********   RESOURCE Performance   ********

Metric          Resource     Work              Value   Unit

------          --------     ----              -----   ----

Capacity        server       work                  8   Servers

Throughput      server       work             0.1690   Customers/Seconds

In service      server       work             0.5033   Customers

Utilization     server       work             6.2910   Percent

Queue length    server       work             0.5033   Customers

Waiting line    server       work             0.0000   Customers

Waiting time    server       work             0.0000   Seconds

Residence time  server       work             2.9780   Seconds

To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-planning+unsub...@googlegroups.com.
To post to this group, send email to guerrilla-capacity-planning@googlegroups.com.

[stephen marksamer]

thanks.  The change in arr rate was targeting just a different level in the demand.  I looked at both the .147 and the .169.  Expecting a 20% utilization for a service time of 2.978 and 243 loan applications per hour (,067 arr).  The 20% and 243 are actual measured.  Service time is derived from U*3600 to get seconds in an hour, then B/C.  I have a regression (R2=.83) that uses both the number of loan applications and number of users (346 for avg hour) that also points to a 20% utilization (20.72 expected value).  The .147 arr rate represents a new level of demand with first new client coming on board.of 528/hr (total aggregated).  Then the .169 represents second new client aggregating total workload at 608/hr.

Here's the base run.  I should have given you this first.

COMMENT: A simple M/M/1 queue

               ==========================================

               ********    PDQ Model INPUTS      ********

               ==========================================

WORKLOAD Parameters:

Node Sched Resource   Workload   Class     Demand

---- ----- --------   --------   -----     ------

  1  FCFS  server     work       Open      2.9780

Queueing Circuit Totals

Streams:   1

Nodes:     1

Arrivals       per Seconds     Demand

--------       --------     -------

work           0.0676        2.9780

               ==========================================

               ********   PDQ Model OUTPUTS      ********

               ==========================================

Solution Method: CANON

               ********   SYSTEM Performance     ********

Metric                     Value    Unit

------                     -----    ----

Workload: "work"

Number in system          0.2521    Customers

Mean throughput           0.0676    Customers/Seconds

Response time             3.7286    Seconds

Stretch factor            1.2521

Bounds Analysis:

Max throughput            0.3358    Customers/Seconds

Min response              2.9780    Seconds

               ********   RESOURCE Performance   ********

Metric          Resource     Work              Value   Unit

------          --------     ----              -----   ----

Capacity        server       work                  1   Servers

Throughput      server       work             0.0676   Customers/Seconds

In service      server       work             0.2013   Customers

Utilization     server       work            20.1313   Percent

Queue length    server       work             0.2521   Customers

Waiting line    server       work             0.0507   Customers

Waiting time    server       work             0.7506   Seconds

Residence time  server       work             3.7286   Seconds

Utilization and Residence time validate to what I can measure (there is a measured workflow time by the application which measures beyond just the SQL server, however, other elements are much less, allowing for the approximation). 

The question still remains, how to translate this to an 8 engine model correctly.

Here is the 8 engine run:

                            PRETTY DAMN QUICK REPORT

               ==========================================

               ***    of: Tue May 21 10:12:50 2013    ***

               ***   for: OpenCenter                  ***

               ***   Ver: PDQ Analyzer 6.1.1 011013   ***

               ==========================================

COMMENT: A not as simple M/M/8 queue

               ==========================================

               ********    PDQ Model INPUTS      ********

               ==========================================

WORKLOAD Parameters:

Node Sched Resource   Workload   Class     Demand

---- ----- --------   --------   -----     ------

  8  MSQ   server     work       Open      2.9780

Queueing Circuit Totals

Streams:   1

Nodes:     1

Arrivals       per Seconds     Demand

--------       --------     -------

work           0.0676        2.9780

               ==========================================

               ********   PDQ Model OUTPUTS      ********

               ==========================================

Solution Method: CANON

               ********   SYSTEM Performance     ********

Metric                     Value    Unit

------                     -----    ----

Workload: "work"

Number in system          0.2013    Customers

Mean throughput           0.0676    Customers/Seconds

Response time             2.9780    Seconds

Stretch factor            1.0000

Bounds Analysis:

Max throughput            2.6864    Customers/Seconds

Min response              2.9780    Seconds

               ********   RESOURCE Performance   ********

Metric          Resource     Work              Value   Unit

------          --------     ----              -----   ----

Capacity        server       work                  8   Servers

Throughput      server       work             0.0676   Customers/Seconds

In service      server       work             0.2013   Customers

Utilization     server       work             2.5164   Percent

Queue length    server       work             0.2013   Customers

To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-...@googlegroups.com.
To post to this group, send email to guerrilla-cap...@googlegroups.com.

[DrQ]

First off, I've de-cluttered the Perl code...

#!/usr/bin/perl

# Edited by NJG on Monday, May 20, 2013

# Edited by NJG on Tuesday, May 21, 2013

use pdq;

$arate    = 243/3600; # TPS = 243 loan applications per hour

$stime    = 2.978;    # from SM's email description

$engines  = 8;

pdq::Init("Marksamer Model");

pdq::CreateOpen("work", $arate);

pdq::CreateMultiNode($engines, "server", $pdq::CEN, $pdq::FCFS);

pdq::SetDemand("server", "work", $stime);

pdq::SetWUnit("Cust");

pdq::SetTUnit("Secs");

pdq::Solve($pdq::CANON);

pdq::Report();

If I understand your original question, you are happy with the M/M/1 results but that doesn't reflect the reality of there being 8 engines instead of 1 in the system, and you'd like to make your CaP projections as a function of the number engines. 

Using your parameters, as you've explained them below, I would say there is no significant difference b/w the PDQ output for m=1 and m=8 engines. In other words, you can take the m=8 model (above) as your baseline and move forward from there. 

So now, I'm not sure what the problem is.

To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-planning+unsubs...@googlegroups.com.

To post to this group, send email to guerrilla-capacity-planning@googlegroups.com.
Visit this group at http://groups.google.com/group/guerrilla-capacity-planning?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "Guerrilla Capacity Planning" group.
To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-planning+unsub...@googlegroups.com.

[stephen marksamer]

The problem is, as a M/M/1 model utilization is 20% (which compares favorably to the measured utilization of the production system).  With my M/M/8 model it is 2.5% (as with your cleaned code).  You stated what I wanted to do precisely.  If I can understand the model with 8 engines, I can do upgrade analysis based on that.  Or should I understand that the Utilization output is on a per engine basis.  So the total utilization is 2.5% * 8?  Would that be true for the other output results?  That doesn't flow for me.

To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-...@googlegroups.com.

[DrQ]

> So the total utilization is 2.5% * 8?

Precisely. The only parametric difference b/w M/M/1 and M/M/8 is the m=8 value. You've simply distributed the utilization across 8 engines.  Hence, the other PDQ output metrics are more or less the same in both cases. 

The technical reason for any differences comes from the fact that an M/M/8 queueing facility enables the HOL request to get the next available engine, so there is some advantage in a kind of parallelism coming from the M/M/8 configuration. However, you are not really seeing that advantage yet b/c your arrival rate is still small relative to what M/M/8 can handle (i.e., ~8x bigger than what you have now).

Having said all that, the other consideration is the error margin in your measured performance metrics. If you feel the residence time should be closer to 3.7272 secs (+/- ???) than 2.9780 (+/- ???) secs, then you can tweak up your service time parameter "stime" (the input metric to your PDQ model), for example, to get a better match. It all depends on what the +/- margin is on your measurements and your derived input metric values. All data is wrong, by definition. The only question is, how wrong is it? And how much do you need to account for that wrongness in your PDQ model. 

 

To post to this group, send email to guerrilla-cap...@googlegroups.com.

Visit this group at http://groups.google.com/group/guerrilla-capacity-planning?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[stephen marksamer]

OK, that does make sense.  The residence time could very well be the 2.978.  I don't really have a good way of measuring directly.The application response time includes an application layer not part of this model.  Because most of the processing and delays are in the sql server, I attribute negligible time at the application server.  'True' residence time would be somewhere in between the two.  This has been a big help.  

Now, how do I get pdq to run a different named pl program?  Only test.pl runs.  I'm sure its something simple.  

Thanks,

Steve

To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-...@googlegroups.com.

[DrQ]

Since you want to predict R (appln response time) as a function of m engines, you'll need to have some idea of how well the analytic values represent the measured values (now and in the future). I think you'll eventually be able to sort that out as you play around with it over time.

This has been a good demonstration for other GCaP members to see what it takes in practice for a newbie to put together a PDQ model of a real system and get it calibrated. Not that difficult. The biggest struggle (as it is for all of us) is to get your various perceptions and intuitions to jive with the structure of your chosen PDQ model. The good news is that you worked pretty hard to keep things simple, viz., M/M/1 in the first instance, rather than trying to throw the kitchen sink at it and having no idea which end is up. Very well done and it will be interesting for us to hear how things progress, if you are allowed to tell us.

I should add that you may find eventually that certain details need to be added into the current baseline model. That may mean adding a bit more complexity in to the model, e.g., other queues, other workloads. More likely, when things don't jive, it will turn out to be something in the system: poor DB indexing, new binary deployed, etc.

Since your test.pl works, any PDQ .pl should now run under perl. You'll have to provide more diagnostic info to get more detailed help.