Throughput shaping time together with Concurrency Thread Group

[maurizio...@gmail.com]

Hi,

I have to load test a web application (calling URL with GET/POST with specific parameters) composed by the following steps

1. open a reference (test) application
2. call a /Login endpoin
3. call 3 times an /authenticate endpoint (username and password)
4. call twice an /authorize endpoint
5. call a logout endpoint

I put these 5 steps in a Transaction Controller called "Load Test Controller", this one under a Concurrency Thread Group and having a Throughput Shaping Timer.

The goal of the load test is to have at least 8 concurrent users executing the 5 steps above.

Fact is: if I set in the Concurrency Thread Group to have

• target concurrency = 30
• ramp up time (s) = 10
• ramp uip steps count = 5
• hold target rate time (s) = 180

I got between 1 to 9 requests per second, with an average more or less of 4 RPS:

Ok, now if I want to use the Throughput Shaping Timer with Concurrency Thread Group, then I configure them to have 2 RPS and using the callback function ${__tstFeedback(throughputShapingTimerDev, 1, 50, 50)}

Well, here the results are epic bad: threads jump from 1 to 52 in 3 seconds, and then everything is stuck. I mean, the "00-ref-app" (picture 03, red line) is loaded only 2 times per second, and the first thread terminating the whole procedure "Load test controller" (green line in picture 03) takes more than 3 minutes (and is not even displayed below!)

I'm pretty sure I did not understand properly how these 2 items can be related, but I do not have any precise idea where the error(s) can be.

Any suggestion is appreciated.

thank you in advance.

[Andrey Pokhilko]

Hi,

What do you want to achieve in your test? Is it certain level of parallel VUs? Or is it certain level of hits/s?

It seems you have configured everything fine for testing certain level of hits/s. But your app seems to be unable to handle it. Looks like your app responds quite slow and 2 hits/s is too much. So maybe your result is right and you found that you need to request less hits/s.

Also, I'd suggest to use ramp-up time in TST, so any request rate you request would be added by small increments.

--

Andrey Pokhilko

11.09.2018 14:06, maurizio...@gmail.com пишет:

--
You received this message because you are subscribed to the Google Groups "jmeter-plugins" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jmeter-plugin...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Maurizio Lattuada]

Hi Andrey,

thank you for your quick answer.

this is a good question that I have already on my todo list to be clarified with the business. The requirement is to have "8 calls per second for 60 minutes". Considering that this login process, for which we have to guarantee 8 calls per second, is composed by 4 steps (I excluded the 5th one "Logout"), I intended it as "8 users concurrently involved in the 4-steps-long login process"., so hits/s.

My big misunderstanding is

• I use only the concurrency thread group (150 threads, ramp up 60s, 5 steps) without any relationship with throughputShapingTimerDev (the latter is disabled too) --> I can have between 1 and 9 hits/s, that is between 1 and 9 users involved in the 4-steps-long login process: 

• I use the concurrency thread group *with* relationship with throughputShapingTimerDev having a fixed target of 2 RPS and using the tstFeedback function and ramp up time of 60s (${__tstFeedback(throughputShapingTimerDev, 10, 150, 50)}, note the max amount of thread = 150, like above) --> I have only 1 RPS and the 1st one is after 2 minutes..., plus having different warning messages "WARN k.a.j.t.f.TSTFeedback: Got concurrency less than zero: -7"

why there is such a huge different in the results in terms of RPS? :(

Which concept is not correctly understood?

[Andrey Pokhilko]

Are you sure you are using latest version of TST plugin?

--

Andrey Pokhilko

11.09.2018 16:56, Maurizio Lattuada пишет:

--

[Maurizio Lattuada]

Yes, version 2.5

is it correct, right?

[Maurizio Lattuada]

Just to mention, JMeter 4.0 and JDK 10.0.2

Here the setup with some more comments:

[Andrey Pokhilko]

Then it's quite strage that you get message about negative rates...

Anyhow, it seems that request rates you work with, need very slow start. I advice you to configure the ramp-up of TST into very small levels, like ramping from 0.1 to 1 within 1 min. So it will be more forgiving the slow responses.

Also, cap the thread count in your tst function with also value ~15-20, so it won't kill you server as bad. Or even 10.

--

Andrey Pokhilko

11.09.2018 17:06, Maurizio Lattuada пишет:

[Maurizio Lattuada]

To avoid misunderstanding, with "cap the thread count" do you mean to set e.g. equal to 15 the parameter B of function ${__tstFeedback(throughputShapingTimerDev, A, B, C)}, right? --> ${__tstFeedback(throughputShapingTimerDev, 5, 15, 5)}

Plus a step up like 

Sounds good?

[Andrey Pokhilko]

Function cap is good.

For TST schedule, start with smaller value. 0.1 is ok IMO. And use longer ramp-up, like 2 minutes or even 5.

Andrey Pokhilko

11.09.2018 17:43, Maurizio Lattuada пишет:

--

[Maurizio Lattuada]

Tried right now with this TST setup (throughputShapingTimerDev):

And the concurrency thread group having target concurrency equal to ${__tstFeedback(throughputShapingTimerDev, 3, 15, 1)}

Right at the beginning of the test, 6 threads are launched:

but, according to what I read and understand from the hits per second and transactions per second charts, it seems these thread are sleepy/dead. I mean, platform is not loaded at all.

On the other hand, by using e,g, an ultimate thread group instead of concurrency thread group, and with the TST deactivated

the hits per seconds and transactions per second are way higher

So, this difference in the results makes me asking myself whether I understood correctly the meaning and how to use the ultimate thread group + TST elements together.

[Andrey Pokhilko]

UTG is no way better than CTG for your case. Maybe you need to just remove TST from your test and use CTG to load the server and see if TPS you get is higher or equal to your SLA.

--

Andrey Pokhilko

12.09.2018 11:30, Maurizio Lattuada пишет: