Gatling Simulation - How to simulate static resource loading similar to normal browser

[Raja]

Hello Gatling Experts,

I have created sample test to load webpage along with all static resources listed in the same page.

Code Snippet:

   val scn = scenario("Scenario Name")

.group("Home Page") {

      exec(http("Homepage")

      .get("/")

      .headers(headers_1)

       )

     .exec(http("request_2")

     .get("/scripts/vendor/tags/jquery.tagsinput.css") .headers(headers_2)

      )

    .exec(http("request_3")

    .get("/assets/styles/plugins.css") .headers(headers_2)

      )

         ..........so on ....

   .exec(http("request_57")

      .get("/scripts/vendor/jquery.matchHeight-min.js")

      )

}

    setUp(scn.users(1).protocolConfig(httpConf))

 

After running the test, the home page load time shown as 5872 ms, higher than normal browser loading time which is around 400-500 ms (Used Chrome - Developer Tools).

Could you please advise, is there a way to trigger Parallel downloads for few of static resources. 

Basically I'm trying to ensure Gatling test is simulating same as that of normal browser.

Please refer to the attachment for complete script reference & Gatling Report for the same.

Thanks & Regards,

Raja

[Raja]

Sorry, Forgot to Mention. I am using Gatling 1.5.6.

[Pierre DAL-PRA]

Hi Raja, 

Unfortunately, there is no way to fetch ressources in parallel with Gatling 1.5.6.

This is however a built-in in Gatling2, called "inferHtmlResources", which you can enable by setting it up on the protocol.

The documentation for "inferHtmlResources" is located here.

I strongly advise that you migrate your Gatling 1.5.x simulation to Gatling 2 : Gatling 2.0.0-RC1 has been released last Friday on Central, and it's quite stable. 

Also, Gatling 1 will no longer be maintained after Gatling 2 stable release, scheduled for September.

Along "inferHtmlResources", Gatling 2 offers a lot more features than Gatling 1 ;)

Cheers, 

Pierre

[Pierre DAL-PRA]

Sorry completely forgot the link to "inferHtmlResources" documentation... it's here : http://gatling.io/docs/2.0.0-RC1/http/http_protocol.html#resource-inferring

[Raja]

Hello Pierre,

Many thanks for quick response. 

I have started using Gatling2.0.0-RC1. 

My testsite has most of static resources of type .js, .css, .png, .woff files. 

Documentation says css images are not covered under .inferHtmlResources.  is there any approach to trigger paralell downloading for above resource types (.png, .woff, .css)

Thanks & Regards,

Raja

On Monday, 4 August 2014 18:07:51 UTC+1, Raja wrote:

[Alex Bagehot]

The parallel resources that are not covered by infer can be simulated by

http://gatling.io/docs/2.0.0-RC1/http/http_request.html#resources

Combinations of sequential blocking .exec() and parallel .resources() should cover most needs if you need to get closer to how a real browser makes requests.

As the doc states: gatling isn't a real browser. So ymmv.

There's scope to do more work on this in 2.1, we have been discussing it. Would be useful to hear who needs to more closely simulate page loading behaviour to determine if the work needed to simulate/record more complicated blocking/parallel resource loading is a priority.
See
https://github.com/gatling/gatling/issues/1943
For the kind of consideration needed.

--
You received this message because you are subscribed to the Google Groups "Gatling User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gatling+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Raja]

Hi Alex or any other gatling experts,
Many thanks for your response.
Is there any best way to display page load time in gatling reports.

Currently using exec, resources able to run the test. However page load time shown in report using group function is not correct, does not seem to be aggregate as well. The report
Looks good only when loading page without resources.

Kindly brief me if any other approach is there to display page min, max,995 response time.

Thanks,
Raja

[Alex Bagehot]

Hi Raja,

[Pierre, Stephane,

correct me if I am wrong but,]

there is no concept of "web page load time" yet/currently in Gatling. Some good work has been done to provide support towards simplifying page scripting, but there is no feature to measure the elapsed time afaik.

Furthermore with reference to issue 1943 it is non trivial to get a reasonable approximation of page load time outside of a real browser as the load time is often impacted by CPU and bandwidth constraints, blocking and nonblocking resources, etc etc.

For completeness, the other possibility is where gatling controls a real browser via webdriver for example. There are tradeoffs though obviously.

I searched back for "gatling transaction" in my inbox and found a couple of previous discussions. Previously it was noted that both JMeter and LR call a custom measured elapsed durations a "transaction". Stephane was not happy with this naming as it implies ACID etc as an already used term in eg database systems.

Since then I don't see any (repeated) requests for this even though it would be reasonable to expect it given other tool's support.

So we need:

1) a feature request in github issue tracker - https://github.com/gatling/gatling/issues/2085

2) people to vote for this (if they want) for it to be prioritised (reply "+1" for example in the Github issue on this )

3) need an agreed naming/DSL. One possible source of inspiration may be: 

http://www.w3.org/TR/user-timing/

here a custom timing is scripted as "mark(name-of-timing)"

so maybe we call it "mark()"? seems like a possible candidate which is consistent with usage elsewhere.

else propose other names for this

thanks,

Alex

Raja

[Alex Bagehot]

btw I wouldn't suggest this goes in 2.0.

[Raja]

Hello Alex, 

Many thanks for your response.

Sorry, I meant to say Response Time (not page load time) shown in Gatling report is incorrect when trying to simulate launching page with all its resources using  exec and resources . Response time in report is correct only when using inferhtmlResources , without any further static resources.

Thanks for raising feature request, Looks like till then I need to use scripts with .inferhtmlResources option only or try to find alternative/workaround for Load Testing. 

Thanks,

Raja

On Monday, 4 August 2014 18:07:51 UTC+1, Raja wrote:

[Pierre DAL-PRA]

Currently, what approach best your notions of "transactions" are Gatling's groups.

When you set up a group, both individual requests timings and cumulated timings for all requests inside this group are computed and shown up in Gatling's reports.

[Alex Bagehot]

Hi Pierre, 

apologies, I was being blinded by (what I think is) a latency measurement bug in inferResources yesterday when I looked at this, making it look like the group didn't measure the elapsed time from the start to end of the group -> defect will be coming.

from a usability point of view there are a few challenges with groups if you want to focus for example on the response time of the groups rather than individual requests. I'll document this to make it clear and see if it can be improved.

finally when navigating to a group's details page in the report, there are some defects in the reporting of groups latency. I'll raise defects for these so they can be looked into and fixed.

sorry, possibly a lot of defects, but best know about them before the release so you can decide whether they can be fixed or not...

[Pierre DAL-PRA]

No worries, and never hesitate to bring up anything that you consider being a defect in groups timings computation, so that we investigate and fix it, should it be necessary :)

Cheers,

Pierre

[Raja]

Hello Gatling Experts,

Many Thanks for prompt response.  

Would you please advise if any workaround to find aggregate of Page Response time other than report, for eg: from logs.  So I could project it in my test report.

  

As mentioned earlier, current report does not display correct Page response time (Attached screenshot).

Thanks,
Raja  

On Monday, 4 August 2014 18:07:51 UTC+1, Raja wrote:

[Alex Bagehot]

Hi Raja, can you send the report bundle as a zip? 

or show screen shot of the details page for the home page group?

thanks Alex

[Raja]

Hello Alex,

PFA "TestSimulationBasic" Test Script along with report generated.

Also attaching Test report "TestSimulation.tar.gz", when Home page loaded with additional resources using exec and resources for sequential and parallel resource download.

Regards,

Raja

On Monday, 4 August 2014 18:07:51 UTC+1, Raja wrote:

[Raja]

Hello Alex,

Sorry to hassle, Would it be possible to review my test script/logs and share your thoughts about Gatling reporting errors by Monday.

It would great, if there is any workaround for this.

Thanks in advance

Best Regards,

Raja

 

On Monday, 4 August 2014 18:07:51 UTC+1, Raja wrote:

[Alex Bagehot]

Hi Raja,

your thread is very similar to Srujana's - may be worth keeping track of that one also.

0) we started with:

"After running the test, the home page load time shown as 5872 ms, higher than normal browser loading time which is around 400-500 ms (Used Chrome - Developer Tools)."

You provided a screenshot where it said "60 resources 4.8KB".

The count there includes requests from cache (the "size content" column will say "from cache" if so) which is evident from the small size of the download bytes. ---> so that measurement is from a partial or full cached page.

the gatling report above it shows that gatling was making a request starting from an empty cache as all the resources are requested and so unsurprisingly (given a fresh look!) the response time is accordingly longer.

therefore not a valid comparison.

Your original comment was that Gatling overstated the page response time compared with Chrome dev tools.

The most recent multiuser reports you have attached show a response time in that ball park 500ms. the comparison there is valid, as both measurements are from full caches. valid as a comparison only though as the user injection is likely wrong (for you to determine yourself).

1)

"Sorry, I meant to say Response Time (not page load time) shown in Gatling report is incorrect when trying to simulate launching page with all its resources using  exec and resources . Response time in report is correct only when using inferhtmlResources , without any further static resources."

-> see https://github.com/gatling/gatling/issues/2090

Looks like there is a defect here as you have identified.

2)

From the reports you attached it looks like, as Pierre suspected in the other thread, page resources are being cached after a certain time.

What to do depends on your users (your workload [model]) who access the site:

a) if there is a fixed number of identifiable (ie. "Bob", and "Jane", "employee number 12") users who sit in front of a computer and run these scenarios over and over again, then

leave the simulation as is. the response time will be correct (ignoring outstanding reporting issues) as the resources will be cached up quickly and those users ("Joe" and colleagues) will enjoy good performance from their browser caches as they repeatedly do the same thing. Maybe the site is providing some admin or call center features.

b) else, if your users come from the general population of people (typically called "the general public", a set of independent people) who want to access your site's features, and they are all independent of each other, and they typically only access the site once, and they do their business on the site then leave to get on with their busy lives, then...

you leave caching on,

but remove any looping in the scenarios and change the user injection as follows.

You need to use the 

http://gatling.io/docs/2.0.0-RC2/general/simulation_setup.html#injection

constantUsersPerSec(20) during(15 seconds), // 4
    constantUsersPerSec(20) during(15 seconds) randomized, // 5
    rampUsersPerSec(10) to(20) during(10 minutes), // 6
    rampUsersPerSec(10) to(20) during(10 minutes) randomized, // 7

constructs to inject independent users into the test, and apply them to the SUT.

As they are all independent, each new user will start with an empty cache, so the caching issue should not be so strong, if present at all.

As a side note, when measuring the page load time in a tool like firebug/chrome dev tools/webpagetest you need to take 3 measurements (tools may provide these measurements also, likely not the 3rd though):

1) first hit, empty cache

2) repeat hit of exactly the same page, will serve the most possible from cache

3) if applicable, repeat hit of 2 different pages of the same type, the different pages will have a partially full cache as there will be some common cached up files, but may have other http calls which are different possibly due to different query parameters which won't be in the cache.

Thanks

Alex

[Raja]

Hello Alex,

Many thanks for detailing on improvements for user simulation using constantUsersPerSec/RampUsersPerSec.  

Regarding following defect, this issue present even when using only inferHtmlResources with Group, and no resources are included (As per report in TestSimulationBasic.tar.gz attached in previous mail.) 

 see https://github.com/gatling/gatling/issues/2090

Would it be possible to prioritise this issue and schedule fix in Gatling 2.0.x , as we can't report correct response time for each page (When loading with resources) upon completion of load testing. 

Till this issue is resolved, is it a good approach to project aggregate of all embedded resources (.html, .gif,.jpg,.png,.js etc) as overall page response time.  Kindly let me know if any better approach, to report at end of Load Testing.

Thanks & Regards,

Raja

 

On Monday, 4 August 2014 18:07:51 UTC+1, Raja wrote:

[Alex Bagehot]

Hi Raja, I have added your comment to the issue for now

Thanks Alex