JMeter : Download embedded resources and hits

550 views
Skip to first unread message

philippe...@gmail.com

unread,
Aug 5, 2018, 9:45:28 AM8/5/18
to codename-taurus
Hello,
I have a test plan that uses "Download Embedded resources" on HTTP Request Defaults.
In this mode a Parent SampleResults is created for the page and subresults for download of every embedded resource of the page.

When I run the plan, the hits metric is wrong as it ignores the hits made by subresults, even if I force subresults
Looking into kpi.jtl , I notice that only the parent SampleResult is present and not subresults.

Can this be modified :

execution:
- scenario: existing
  concurrency: 3
  hold-for: 5m
  ramp-up: 30s
  locations:
    us-east-1: 1

scenarios:
  existing:
    script: embedded.jmx

modules:
  jmeter:
    path: ~/.bzt/jmeter-taurus/bin/jmeter
    version: r1837207
    download-link: https://ci.apache.org/projects/jmeter/nightlies/r1837207/apache-jmeter-r1837207.zip
    detect-plugins: false
    properties:
      jmeter.save.saveservice.subresults: true
      jmeter.save.saveservice.output_format: csv
      saveservice_properties: saveservice.properties
      jmeter.reportgenerator.report_title: JMeter Embedded Demo
      jmeter.reportgenerator.overall_granularity: 1000

Andrey Pokhilko

unread,
Aug 7, 2018, 4:46:37 AM8/7/18
to codenam...@googlegroups.com

Hi,

Registering embedded resource hits is a quite controversial topic. I know that some users prefer to include embedded resources as hits/s, others prefer to look at top-level transactions. Moreover, in practice there are some of embedded resources that you want to include and some that not, because they are stored on thirdparty (CDN) and not hit your service. There can't be single "right" way of registering this KPI.

In Taurus, we have chosen the way of using top-level transactions only, as it is more universal and takes less resources on framework side to process results.

I can also give example of JMeter - in Aggregate Report there is Throughput column that displays top-level hits per interval. And I never heard of "include subsamples into hits" somewhere in JMeter as common practice. All I used to see is "top-level samples are our hits", including mind-blowing Transaction Controller with no "generate parent sample" set.

--

Andrey Pokhilko
Open Source Initiatives Leader
CA BlazeMeter

05.08.2018 16:45, philippe...@gmail.com пишет:
CAUTION: This email originated from outside of CA. Do not click links or open attachments unless you recognize the sender and know the content is safe.

--
You received this message because you are subscribed to the Google Groups "codename-taurus" group.
To unsubscribe from this group and stop receiving emails from it, send an email to codename-taur...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/codename-taurus/d5cab2ab-ede2-4b32-932a-e6a8ca74ca4d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Philippe Mouawad

unread,
Aug 7, 2018, 4:15:25 PM8/7/18
to Andrey Pokhilko, codenam...@googlegroups.com
Hits concerns requests going to server, with jmeter you can select which requests you want to be include to only include the ones hitting your servers.

There was a limitation in JMeter when outputting csv that only retained parent sample result and not subresults and it was explicitly mentioned in the generated html report.

This limitation will be lifted in upcoming 5.0 .

My opinion in the load tests I have made using embedded resources is that it was annoying not to have it, as some performance issues could have this number of hits (static + dynamic) as root cause when an apache httpd misconfiguration would saturate the worker pool.

Also, the concept of subresults is not only for embedded http load test, it can appear in other tests like jms.

But I guess that Taurus being the intersection of multiple oss load test products requires you to make some choices.

Regards 
To unsubscribe from this group and stop receiving emails from it, send an email to codename-taurus+unsubscribe@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "codename-taurus" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/codename-taurus/EiSC3658fLo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to codename-taurus+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/codename-taurus/218d5825-ea68-6c5e-c362-d3232d734904%40contractor.ca.com.

For more options, visit https://groups.google.com/d/optout.


--
Cordialement.
Philippe Mouawad.



Andrey Pokhilko

unread,
Aug 8, 2018, 4:08:09 AM8/8/18
to codenam...@googlegroups.com

Hmm, I took the latest snapshot of JMeter and I'm shocked by the change in JTL writing. It is strange that it writes subsamples now with no respect to "Save Sub Results" flag. I simply can't turn new behavior off!

It will break all the automated scripts written in the ecosystem. Even without Taurus, if I run the test with Aggregate Report, I see one number. If I save it into JTL and re-load it in Aggregate Report, I see completely different picture. All my response times are counted twice now!

I believe something really breaking has happened, and it will hurt users. Just because it changes the math of reading results the same way Transaction Controller breaks it...

I will send copy of this into JMeter developer list for further discussion.

--

Andrey Pokhilko
Open Source Initiatives Leader
CA BlazeMeter

07.08.2018 23:15, Philippe Mouawad пишет:
To unsubscribe from this group and stop receiving emails from it, send an email to codename-taur...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/codename-taurus/CAH9fUpbVRqihJiDNR99Z1tt4RgRZuEoQhVE8EzQuBK-qAJXu5A%40mail.gmail.com.

Philippe Mouawad

unread,
Aug 8, 2018, 4:53:21 AM8/8/18
to Andrey Pokhilko, codenam...@googlegroups.com


On Wednesday, August 8, 2018, Andrey Pokhilko <andrey....@contractor.ca.com> wrote:

Hmm, I took the latest snapshot of JMeter and I'm shocked by the change in JTL writing. It is strange that it writes subsamples now with no respect to "Save Sub Results" flag. I simply can't turn new behavior off!


I’ll double check, it surely is intended to be turned off.
Did you try using this property or in a different way:
jmeter.save.saveservice.subresults=false

This one works as some integration tests use this way.

It will break all the automated scripts written in the ecosystem. Even without Taurus, if I run the test with Aggregate Report, I see one number. If I save it into JTL and re-load it in Aggregate Report, I see completely different picture. All my response times are counted twice now!

I believe something really breaking has happened, and it will hurt users. Just because it changes the math of reading results the same way Transaction Controller breaks it...

I will send copy of this into JMeter developer list for further discussion.

Thanks 

--

Andrey Pokhilko
Open Source Initiatives Leader
CA BlazeMeter

07.08.2018 23:15, Philippe Mouawad пишет:
CAUTION: This email originated from outside of CA. Do not click links or open attachments unless you recognize the sender and know the content is safe.

Hits concerns requests going to server, with jmeter you can select which requests you want to be include to only include the ones hitting your servers.

There was a limitation in JMeter when outputting csv that only retained parent sample result and not subresults and it was explicitly mentioned in the generated html report.

This limitation will be lifted in upcoming 5.0 .

My opinion in the load tests I have made using embedded resources is that it was annoying not to have it, as some performance issues could have this number of hits (static + dynamic) as root cause when an apache httpd misconfiguration would saturate the worker pool.

Also, the concept of subresults is not only for embedded http load test, it can appear in other tests like jms.

But I guess that Taurus being the intersection of multiple oss load test products requires you to make some choices.

Regards 

--
You received this message because you are subscribed to a topic in the Google Groups "codename-taurus" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/codename-taurus/EiSC3658fLo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to codename-taurus+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/codename-taurus/bf00e5e0-6293-64c5-484e-fdcf3b8c789b%40contractor.ca.com.

For more options, visit https://groups.google.com/d/optout.


--
Cordialement.
Philippe Mouawad.



Philippe Mouawad

unread,
Aug 20, 2018, 1:02:43 PM8/20/18
to Andrey Pokhilko, codenam...@googlegroups.com
I m

On Wed, Aug 8, 2018 at 10:53 AM Philippe Mouawad <philippe...@gmail.com> wrote:


On Wednesday, August 8, 2018, Andrey Pokhilko <andrey....@contractor.ca.com> wrote:

Hmm, I took the latest snapshot of JMeter and I'm shocked by the change in JTL writing. It is strange that it writes subsamples now with no respect to "Save Sub Results" flag. I simply can't turn new behavior off!


I’ll double check, it surely is intended to be turned off.
Did you try using this property or in a different way:
jmeter.save.saveservice.subresults=false

This one works as some integration tests use this way.

It will break all the automated scripts written in the ecosystem. Even without Taurus, if I run the test with Aggregate Report, I see one number. If I save it into JTL and re-load it in Aggregate Report, I see completely different picture. All my response times are counted twice now!

I believe something really breaking has happened, and it will hurt users. Just because it changes the math of reading results the same way Transaction Controller breaks it...

I will send copy of this into JMeter developer list for further discussion.

Thanks 

--

Andrey Pokhilko
Open Source Initiatives Leader
CA BlazeMeter

07.08.2018 23:15, Philippe Mouawad пишет:
CAUTION: This email originated from outside of CA. Do not click links or open attachments unless you recognize the sender and know the content is safe.

Hits concerns requests going to server, with jmeter you can select which requests you want to be include to only include the ones hitting your servers.

There was a limitation in JMeter when outputting csv that only retained parent sample result and not subresults and it was explicitly mentioned in the generated html report.

This limitation will be lifted in upcoming 5.0 .

My opinion in the load tests I have made using embedded resources is that it was annoying not to have it, as some performance issues could have this number of hits (static + dynamic) as root cause when an apache httpd misconfiguration would saturate the worker pool.

Also, the concept of subresults is not only for embedded http load test, it can appear in other tests like jms.

But I guess that Taurus being the intersection of multiple oss load test products requires you to make some choices.

Regards 
To unsubscribe from this group and stop receiving emails from it, send an email to codename-taur...@googlegroups.com.
--
You received this message because you are subscribed to a topic in the Google Groups "codename-taurus" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/codename-taurus/EiSC3658fLo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to codename-taur...@googlegroups.com.


--
Cordialement.
Philippe Mouawad.



--
You received this message because you are subscribed to the Google Groups "codename-taurus" group.
To unsubscribe from this group and stop receiving emails from it, send an email to codename-taur...@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "codename-taurus" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/codename-taurus/EiSC3658fLo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to codename-taur...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/codename-taurus/bf00e5e0-6293-64c5-484e-fdcf3b8c789b%40contractor.ca.com.
For more options, visit https://groups.google.com/d/optout.


--
Cordialement.
Philippe Mouawad.





--
Cordialement.
Philippe Mouawad.


Philippe Mouawad

unread,
Aug 20, 2018, 1:03:36 PM8/20/18
to Andrey Pokhilko, codenam...@googlegroups.com
Hello,
I made the tests in GUI and non GUI mode configuring this using Configure also and the setting is taken into account.

I completed the answer on jmeter dev mailing list.
Regards

On Wed, Aug 8, 2018 at 10:53 AM Philippe Mouawad <philippe...@gmail.com> wrote:


On Wednesday, August 8, 2018, Andrey Pokhilko <andrey....@contractor.ca.com> wrote:

Hmm, I took the latest snapshot of JMeter and I'm shocked by the change in JTL writing. It is strange that it writes subsamples now with no respect to "Save Sub Results" flag. I simply can't turn new behavior off!


I’ll double check, it surely is intended to be turned off.
Did you try using this property or in a different way:
jmeter.save.saveservice.subresults=false

This one works as some integration tests use this way.

It will break all the automated scripts written in the ecosystem. Even without Taurus, if I run the test with Aggregate Report, I see one number. If I save it into JTL and re-load it in Aggregate Report, I see completely different picture. All my response times are counted twice now!

I believe something really breaking has happened, and it will hurt users. Just because it changes the math of reading results the same way Transaction Controller breaks it...

I will send copy of this into JMeter developer list for further discussion.

Thanks 

--

Andrey Pokhilko
Open Source Initiatives Leader
CA BlazeMeter

07.08.2018 23:15, Philippe Mouawad пишет:
CAUTION: This email originated from outside of CA. Do not click links or open attachments unless you recognize the sender and know the content is safe.

Hits concerns requests going to server, with jmeter you can select which requests you want to be include to only include the ones hitting your servers.

There was a limitation in JMeter when outputting csv that only retained parent sample result and not subresults and it was explicitly mentioned in the generated html report.

This limitation will be lifted in upcoming 5.0 .

My opinion in the load tests I have made using embedded resources is that it was annoying not to have it, as some performance issues could have this number of hits (static + dynamic) as root cause when an apache httpd misconfiguration would saturate the worker pool.

Also, the concept of subresults is not only for embedded http load test, it can appear in other tests like jms.

But I guess that Taurus being the intersection of multiple oss load test products requires you to make some choices.

Regards 
To unsubscribe from this group and stop receiving emails from it, send an email to codename-taur...@googlegroups.com.
--
You received this message because you are subscribed to a topic in the Google Groups "codename-taurus" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/codename-taurus/EiSC3658fLo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to codename-taur...@googlegroups.com.


--
Cordialement.
Philippe Mouawad.



--
You received this message because you are subscribed to the Google Groups "codename-taurus" group.
To unsubscribe from this group and stop receiving emails from it, send an email to codename-taur...@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "codename-taurus" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/codename-taurus/EiSC3658fLo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to codename-taur...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/codename-taurus/bf00e5e0-6293-64c5-484e-fdcf3b8c789b%40contractor.ca.com.
For more options, visit https://groups.google.com/d/optout.


--
Cordialement.
Philippe Mouawad.





--
Cordialement.
Philippe Mouawad.


Reply all
Reply to author
Forward
0 new messages