Assert Percentiles from aggregate report

[Sinan YILMAZ]

Hi all,

I would like to assert the percentiles from aggerate report when I run the Jmeter. It should be self checked by itself. I dont want to test the result manually?  And it must be work at gitlab pipeline without needing new docker image?

Is there a way to do that?

[DT]

Neither latest stable JMeter 5.5 nor any of existing plugins I'm aware of provides this functionality. 

So the options are in:

• Calculate the percentiles from the .jtl results file using a shell script and compare it to the expected value. Return a non-zero exit status code if the percentile is higher than expected
• Use Taurus tool as a wrapper for your test, Taurus provides flexible Pass/Fail Criteria subsystem which can check percentile and again give non-zero exit status code if the defined criteria are not met.
  

Both solutions are CI/scripting friendly

[Vincent Daburon]

Hi,

Look at

https://github.com/tilln/jmeter-junit-reporter

Generates CI/CD-compatible JUnit Reports in XML format based on custom KPIs: Comparing aggregate metrics with threshold values determines pass/fail.
Supports filtering of sample labels by regular expressions.
Supported aggregates:

• Response time percentiles (any)
• Maximum and minimum response time
• Response time average and standard deviation
• Sample error rate
• Total number of samples
• Custom metrics

Regards.

Vincent DAB.

On Saturday, January 28, 2023 at 2:22:27 PM UTC+1 cakime...@gmail.com wrote:

[Vincent Daburon]

Hi,
JUnit XML file is a standard for tests result
For gitlab look at :

https://docs.gitlab.com/ee/ci/testing/unit_test_reports.html

Regards
Vincent DAB.