How to Emulate Web Traffic Using Standard Load Testing Tools

[DrQ]

The following abstract has been submitted to CMG 2016.

James Brady (State of Nevada) and Neil Gunther (Performance Dynamics)

Conventional load-testing tools are based on a fifty year old time-share computer paradigm where a finite number of users submit requests and respond in a synchronized fashion. Conversely, modern web traffic is essentially asynchronous and driven by an unknown number of users. This difference presents a conundrum for testing the performance of modern web applications. Even when the difference is recognized, performance engineers often introduce virtual-user script modifications based on hearsay; much of which leads to wrong results. We present a coherent methodology for emulating web traffic that can be applied to existing test tools.

[DrQ]

This just in from Twitter:

I don't know what version of JMeter was used in our 2016 paper: How to Emulate Web Traffic Using Standard Load Testing Tools.

It's not mentioned explicitly in the text (which is a bit of an oversight). Some of the screenshots suggest it was v 2.11. 

[James Brady]

Our 2016 paper is based on JMeter 2.13 which is long before the February 2018 release of JMeter 4.0 when the Precise Throughput Timer was introduced. I have not used that timer yet but it appears to force a specific number of samples to be produced in a fixed time interval so I question how independent the times between samples are for a given user thread. The basic delay formula for Poisson Arrivals is "t = -u * ln(r)" where t = time to delay, u = mean delay time, and ln(r) is the natural log of a random number between 0 and 1. A derivation of this formula can be found on the perfdynamics.com web site under the "Supplements" link. It is contained in Sections 5 of the paper I wrote labeled  "Traffic Generation Concepts" under the "7 Modeling Internet/Web Traffic" heading.

One of the concerns people have using this formula is that it may draw a very long delay time, keeping the user thread alive way past the test run time. As a result they put procedures in place to manage the thread's behavior. I don't know how the "Target Throughput" mechanism in the Precise Throughput Timer works, but it likely impacts delay time independence.

One way to manage the long delay time event is to check if the time to delay drawn causes the thread to wait longer than the test run time and if it does, terminate the thread. I don't think JMeter has that option.

--
You received this message because you are subscribed to the Google Groups "Guerrilla Capacity Planning" group.
To unsubscribe from this group and stop receiving emails from it, send an email to guerrilla-capacity-...@googlegroups.com.
To post to this group, send email to guerrilla-cap...@googlegroups.com.
Visit this group at https://groups.google.com/group/guerrilla-capacity-planning.
For more options, visit https://groups.google.com/d/optout.

[Vladimir Sitnikov]

>I don't know how the "Target Throughput" mechanism in the Precise Throughput Timer works, but it likely impacts delay time independence.

Precise Throughput Timer uses -Math.log(1 - u) / throughput where u is [0, 1)

The timer creates schedule in advance, so it just retries in case it got too few or too many sample points.

That enables to have "exact number of samples for given test duration".

Here's the code: https://github.com/apache/jmeter/blob/9810100c70bc27efa8dfb0e1a01e126a53f4a169/src/components/org/apache/jmeter/timers/poissonarrivals/ConstantPoissonProcessGenerator.java#L98-L109

Vladimir