Difference max response time between Gatling and Jmeter

[JESSICA VIDAL]

Hi, 

I new in gatling and I have a problem with the max response time. I run a script with gatling and jmeter.

I had this script 

class Tierra extends Simulation{

  object Tierra{

    val r = scala.util.Random

    val geoCodeIpAddress = scenario("By Ip Address")
      .feed(ipAddress)
      .exec(http("By Ip Address")
      .get("/v1.3/geocode")
      .queryParam( """ip_address""", "${ip_address}")
      .queryParam( """client_id""", clientId)
      .queryParam("""loadRequests""", loadRequests)
      )
  }

  val ipAddress = csv("ipaddress.csv").random


  val httpConf = http
    .baseURL("http://xxxx.xxx")
    .connectionHeader("keep-alive")
    .extraInfoExtractor(extraInfo => List(extraInfo.request))

  setUp(
    Tierra.geoCodeIpAddress.inject(constantUsersPerSec(5000) over (2 seconds))

  ).protocols(httpConf)

}

I saw the following numbers:

Simulation finished
Parsing log file(s)...
Parsing log file(s) done
Generating reports...

================================================================================
---- Global Information --------------------------------------------------------
> request count                                      10000 (OK=9999   KO=1     )
> min response time                                      2 (OK=2      KO=366   )
> max response time                                    950 (OK=950    KO=366   )
> mean response time                                   161 (OK=161    KO=366   )
> std deviation                                        142 (OK=142    KO=0     )
> response time 50th percentile                        122 (OK=122    KO=366   )
> response time 75th percentile                        210 (OK=210    KO=366   )
> mean requests/sec                                1353.546 (OK=1353.411 KO=0.135 )
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                          9939 ( 99%)
> 800 ms < t < 1200 ms                                  60 (  1%)
> t > 1200 ms                                            0 (  0%)
> failed                                                 1 (  0%)
---- Errors --------------------------------------------------------------------
> status.find.in(200,304,201,202,203,204,205,206,207,208,209), b      1 (100.0%)
ut actually found 404
================================================================================


With Jmeter

IpAddress	10000	1	1	97	1.0718442004321338	1.0E-4	506.7139599695971	176.55314756460604	356.7899
TOTAL		10000	1	1	97	1.0718442004321338	1.0E-4	506.7139599695971	176.55314756460604	356.7899


As you can see the difference between the max response time is huge: 950 vs 97. 
I don't If my gatling script is wrong or there is another problem

Also I checked the numbers with splunk and I saw another number. In splunk shows less than gatling report. 

Could someone help me?

I am using the last version of gatling: 2.1.7

[Stéphane LANDELLE]

Gatling doesn't only account for the HTTP request to response time, but also:

* possible DNS resolution

* TCP connection opening

Maybe the other tools don't.

But then, you have to run each tools many times to get statistically significant results.

Stéphane Landelle

Lead developer

slan...@gatling.io

--
You received this message because you are subscribed to the Google Groups "Gatling User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gatling+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Incognito]

I observed same. I evaluated Jmeter and CURL against Gatling. The result in Jmeter and CURL was quite similar but in Gatling the max response time was much higher (7times) than both of them.

I did this test at least 10 times and all the time Gatling shown different result than Jmeter & Curl.

All tests were having a plain post request without any assertion, feeder etc. Gatling's request was feeded-in as as ELFBody json file.

[slan...@gatling.io]

The only possible explanation for such gap is that you’re not running the same test with the different tools and you’re comparing apples and oranges: just comparing boot time, not opening the same number of sockets, etc.

Le 19 juin 2020 à 07:24, Incognito <incogn...@gmail.com> a écrit :



--
You received this message because you are subscribed to the Google Groups "Gatling User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gatling+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gatling/32574dff-ac16-4033-bc0a-7f1bb7232bf1o%40googlegroups.com.