Percentile analysis

[Jamie Tabone]

Hi,

In my dissertation I am trying to compare two web technology stacks. I am currently trying to assess which one is more scalable using the graphs generated by Gatling. I know that percentiles and average are two different things, but can you tell me which percentile value (50, 80, 100 ..) is best to use in order to make conclusions on which stack scales better as load increases?

Thank you. 

Jamie Tabone 

[Abhinav Gogna]

On applications I test, I am more interested in 90th percentiles. That's the trend I have noticed on most of the applications I have worked on.

[Jamie Tabone]

Thanks :) I will take that into consideration.

--
You received this message because you are subscribed to a topic in the Google Groups "Gatling User Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gatling/6p5cH97IttA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gatling+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[John Arrowwood]

Honestly, if you try to point to a single number and use that as justification for making a value judgment, you are doing yourself and your stated purpose a disservice.  You want to look at everything.  You want to MEASURE everything.  And you need to be able to tell the story of what that particular measurement means.  For example:

* number of requests per second that can be serviced at a particular resource utilization level - say, 75% of peak.  If one server can process twice as many in the same time period at the same resource level, clearly, it can scale higher

* ability of the server to recover after being maxed out for an extended period

* error rates while at peak load

* ability of the technology to scale by adding hardware resources, such as in the cloud - not every technology can 

[Jamie Tabone]

Thank you for your response.

--

[Jamie Tabone]

I really appreciate

[Alex Bagehot]

you need to look at the requirements, which can be difficult if you are doing a project or benchmarking one system against another without a specific boss/client/customer guiding what is important.

Other than that Latency percentile measurement is an active area in performance engineering:

 https://github.com/brianfrankcooper/YCSB/pull/214

 https://github.com/TechEmpower/FrameworkBenchmarks/issues/1227

Good luck!

--
You received this message because you are subscribed to the Google Groups "Gatling User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gatling+u...@googlegroups.com.