Throughput Shaping Time - No free threads left in worker pool

[NV]

I am using the Throughput Shaping Timer in JMeter 2.5.1 to test
against Mule 3.2 ESB. I am sending ~600 byte SOAP requests from
JMeter. When I reach ~350/tps I start seeing the error message below
in the jmeter-server.log file:

2011/10/21 16:07:32 WARN -
kg.apc.jmeter.timers.VariableThroughputTimer: No free threads left in
worker pool, made 357/359 samples

I am not able to get more than 350 tps out of a jmeter server instance
with this plugin. I have looked at jmeter with VisualVM; no issues
with CPU, memory, thread count in the VM and we are no where near the
limits on the network bandwidth. I tried this on several jmeter server
(Linux and Windows) and I get the same behavior. Also, the ESB is just
purring along (no resource issues).

Increasing the number of threads on the Thread Group does not help.

Any ideas?


btw, great product just need it to get to 500 tps and I will be happy.

[Andrey Pohilko]

Hi,

Am I right that you see increasing Response Times when you close to 350rps?

This may be caused by some limit in your system like workes threads/processes, database connections etc.

Try disabling the Shaper and repeat your test - can you get more than 350 rps from your server at all?

[NV]

Thanks. I tried using the JMeter Constant Throughput Timer and I am
hitting the same 350 tps limit. There must be a max threads limit in
Mule.

[ora...@gmail.com]

NV,

I have exactly the same problem you had, I cannot get more than 250 TPS using HTTP sampler on Elasticsearch . 

I noticed that changes in num_sample_threshold parameters affect the limit. 

I have 80 TPS limit at  num_sample_threshold=80 , 250 TPS at num_sample_threshold=250,

Unfortunately, above num_sample_threshold=250, performance becomes unstable and spiky. 

Did you figure out what was the reason for the throughput limit in your case?

Thanks

Igor

[DanyC]

Is this still the case in the latest version?

 

Cheers,

Dani

[Mark Tomlinson]

NV - you should note the error message: "357/359 samples"

You are actually producing more than 350 samples per second, if that error message is generated for a one second sampling interval.

Typically with the jp@gc - Console Status Logger, you see output about every 1 second; and this # of samples seems to correspond.

[nvi...@gmail.com]

A colleague of mine is also running in to this limitation for some AMQP/RabbitMQ tests he is running. Once again, monitors indicate CPU, Disk, Network, Memory resources are available, however, test start logging this warning and Std Devs go high. We are going to did into the code a bit to see if we can find an issue. I really like this plugin, but need to be able to push more load out of it.

[Andrey Pohilko]

You won't be able to push more from it, because you likely have a bottleneck in your app. It may not necessarily look like exhausted CPU or other resource. For example, if you use mutexes in your app, you'll spend all response times waiting for that mutex. 

I suggest you using some profiler/tracer to find exact bottleneck in your app. If you trust my experience, of course :)

четверг, 24 октября 2013 г., 20:28:15 UTC+4 пользователь nvi...@gmail.com написал: