Charts "Transaction Throughput vs Threads" vs "Transactions per Second": semantic of numbers

[Maurizio Lattuada]

Hi everybody.

I have a question related to the semantic of the numbers shown in the "Transaction Throughput vs Threads" and "Transactions per Second" charts.

According to the plugin page:

• Transaction Throughput vs Threads --> "... it shows total server's transaction throughput for active test threads"
  
• Transactions per Second --> "... shows the number of transactions per second for each sampler"
  

My test is composed by HTTP GET and POST requests.

Given i.e. the graphs related to a specific transaction controller (I have many of them):

transactions per second: 

• here I read that I had mainly 1 TPS with one peak to 6 TPS, one to 5, one to 4, five to 3 and the remaining to 2 TPS

transaction throughput vs thread

• here I read that, with 6 threads, I have an estimated 6.7 TPS

Well, at the end of the test, what is the real number of transactions per second? Where I can read it?

I cannot honestly understand the semantics of the numbers shown in "transactions per second" chart and in the "transaction throughput vs thread".

Thank you in advance,

Maurizio

[Maurizio Lattuada]

Ok,

I can reply to myself.

The chart "transaction throughput vs thread" shows that with 6 threads I can obtain 6.7 transactions per second. As a consequence, I have, for each thread, about 1.12 transaction per second. This is roughly aligned in what is shown in the "transaction per second" chart.

[Maurizio Lattuada]

At this point I come back to the original question.

Said that

• "transaction throughput vs thread" chart tells me that I can obtain 6.7 transaction per second with 6 threads
• "transaction per thread" chart tells me I have about 1.12 transaction per second

well, which is really the real value I should consider to reply to the question "how many transaction per second I generated"?

[Andrey Pokhilko]

Hi,

Answer to question "how many transaction per second I generated?" is inside Aggregate Report, column named "Throughput".

If you want to ask different question like "How my TPS change when I change threads?" - that's the question answered by "transaction throughput vs thread" listener.

"transaction per thread" - I don't know listener with this name, sorry.

Andrey Pokhilko

28.09.2018 10:22, Maurizio Lattuada пишет:

--
You received this message because you are subscribed to the Google Groups "jmeter-plugins" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jmeter-plugin...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Maurizio Lattuada]

My fault, is "transaction per second" chart

[Andrey Pokhilko]

"Transaction per second" answers question "how my TPS change over time", and God knows what was happening with threads over that time, it depends on your test configuration (eg Ultimate thread Group could be generating spiky load).

So what is your real question for TPS?

Andrey Pokhilko

28.09.2018 10:37, Maurizio Lattuada пишет:

My fault, is "transaction per second" chart

Il giorno venerdì 28 settembre 2018 09:29:16 UTC+2, Andrey Pokhilko ha scritto:

"transaction per thread" - I don't know listener with this name, sorry.

--

[Maurizio Lattuada]

Oh, thanks for the answer.

I have a requirement like: "I need to support 8 TPS for 5 minutes. Is the system able to keep this load?"

Then I need, clearly, to find out the right amount of threads able to hold that number of TPS.

So I used a concurrent thread group calling the tstFeedback function on a throughput shaping timer configured to have this kind of load with small ramp-ups like "1->2->4->6->8". Ramp up "1->2->4->6->8" is done in 3 minutes, the it remains to 8 for the remaining 2 minutes..

The results of a reduced test are in the 1st post.

So resuming:

• transaction throughput vs thread chart tells me that with 6 threads I can have 6.7 TPS (note here: in this test I had always 6 threads)
• transaction per second chart (sampling = 1000ms) tells me that, over time, I had mainly 1 and 2 TPS (some peaks to 3, 5 and 6 TPS)

Well, I do not know where to look to answer the question "Am I able to support 8 TPS?".

• transaction throughput vs thread chart tells me "about yes". Answer could be "no" if the number of threads increases, but the number of TPS drastically goes down (= system under too heavy load)
• transaction per second chart tells me "definitely no".
  

Thank you

[Andrey Pokhilko]

"support 8 TPS for 5 minutes" - this has no threads mention. So your end goal is "Transactions Per Second" listener.

Especially if you use TSTFeedback function - it is meant to spawn more threads when you need more to suffice your TPS requirements. So if you see graph of TPS that differs from one in TST significantly - you fail your requirement.

Once again: Graph in TST is what you expect over time. TPS is what you get over time. They should match more or less, otherwise you don't fulfill the requirement.

Andrey Pokhilko

28.09.2018 11:15, Maurizio Lattuada пишет:

--

[Maurizio Lattuada]

My answers are inline.

Il giorno venerdì 28 settembre 2018 10:51:22 UTC+2, Andrey Pokhilko ha scritto:

"support 8 TPS for 5 minutes" - this has no threads mention. So your end goal is "Transactions Per Second" listener.

 

Exactly, this is why I used a concurrent thread group together with a throughput shaping timer and the tstFeedback function. They are doing the dirty job to compute the right amount of thread to fulfill the TPS stated in the throughput shaping timer :)

 

Especially if you use TSTFeedback function - it is meant to spawn more threads when you need more to suffice your TPS requirements. So if you see graph of TPS that differs from one in TST significantly - you fail your requirement.

Then this is my case. I had a curve 1->2->4->6->8 in 3 minutes, then hold 8 TPS for 2 minutes. The transaction per second chart tells me "ehy man, you have 1 to 2 TPS, some spike to 3, 5 and 6 TPS --> no way, your system is not able to keep that load"

 

Once again: Graph in TST is what you expect over time. TPS is what you get over time. They should match more or less, otherwise you don't fulfill the requirement.

Totally agree

Thank you