Gatling Vs Simple Curl Based Script.

[Vikram Godbole]

Hello,

I am doing performance benchmarking of an application service, and I notice Gatling performance is very poor compared to simple shell script with curl command.

Following is the load profile.

I have 30 concurrent users doing a HTTP POST with a large json body read from a circular/random feeder.

1. GATLING Scenario.

...

        val nbUsers = Integer.getInteger("users", 1)

         val myRamp = java.lang.Long.getLong("ramp", 0L)

        val myIter = Integer.getInteger("iter", 1)

        val myUrl = System.getProperty("url")

        val pgReqFeeder = ssv("requestBody.ssv").circular

        val httpConf = httpConfig

.baseURL(myUrl)

.acceptHeader("*/*")

val scn = scenario("Performance Testing")

                .repeat(myIter) {

                   feed(RequestFile)

                   .exec(

                   http("POST")

.post("/something.do")

.header("Content-Type", "application/json")

.body("${REQUESTBODY}")

.asJSON

.check(status.is(200)))

          }

...

To Execute:

JAVA_OPTS="-Dusers=30 -Diter=100 -Durl=http://<ServiceUrl>:9000 -Xms4096m -Xmx4096m" ./bin/gatling.sh -s basic.Perfloadfeeder -nr

2. CURL BASED SCRIPT: I have a simple shell script making a curl call in a loop.

#!/usr/bin

# ./loadtest.sh

# random.sh returns a random line from a file (HTTP POST Payload)

for i in {1..100 }

do 

./random.sh | \   

    curl -i \

        -k \

        -s -w "%{http_code}  %{http_connect}  %{time_namelookup}  %{time_connect}  %{time_appconnect}  %{time_pretransfer}  %{time_starttransfer}  %{num_connects}  %{speed_upload}  %{speed_download}  %{time_total}\n" \

        -o /dev/null \

        -H 'Accept: */*' \

        -H "Content-Type: application/json" \

        "http://<Service Url>:9000/something.do"  \

        -d @-

done

I have a wrapper shell script that just calls this curl command in a script 30 times and 

Something like

#!/bin/sh

...

./loadtest.sh &

...

...

./loadtest.sh &

Here are the comparison numbers.

Tool:                   Curl Script                GATLING          GATLING

#Concurrent Users:          30                          30                       1

#Iterations                     100                        100                      10

Start Time:        1404644781            1404648014

End Time:         1404644867            1404648454

Total Requests:           3000                       3000                      10

MIN(s):                       0.129                       0.08                  0.240

MAX(s):                      2.390                       7.75                  0.380

MEAN(s):               0.735419                     4.204                  0.318

STD DEV:              0.319655                     2.221                  0.048

Avg. TPS:                        34                           7                        3

Both tests run on from the same m/c.

H/W: AMD 4 Core Cpu (2301 MHz each), 8G RAM.

ulimit -n --> 524288

I don't think the test needs too many ports (Only 30 concurrent users).

I would appreciate your help in figuring why i have such poor performance with gatling.

Thanks,

Vikram

[Stéphane Landelle]

Hi,

I would need a reproducer.

Regards,

Stéphane

--
You received this message because you are subscribed to the Google Groups "Gatling User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gatling+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Vikram Godbole]

Stephane,

Thanks for the quick response.

I am attaching the files for your reference.

Gatling scenario uses feeder to read from file pgRequest.ssv (unfortunately i wont be able to supply the data). Both files have the same JSON body data with the difference in escaping the quotes and header for ssv file. The format is as follows.

head -2 user-files/data/pgRequest.ssv (Total Records: 96, Total No of chars in file: 1022406, Approx size of 1 request body = 8K)

REQUESTBODY;NONE  

"{""Field"":""Value"", ""Field1"":""Value"", ""Field3"": {""F4"": ""V4"", ""F5"": ""V5"" ....} ...}"; 

head -1 pgRequest.txt  # -- 1 JSON BODY per line (Total Records: 95, Total No of chars in file: 923781C, Approx. size of 1 request body = 8K)

{"Field":"Value", "Field1":"Value", "Field3": {"F4": "V4", "F5": "V5" ....} ...}

Let me know your findings.

Thanks and Regards,

Vikram

[Vikram Godbole]

BTW

Gatling ver: gatling-charts-highcharts-1.5.5-bundle.tar.gz

-Vikram

[Stéphane Landelle]

I don't see anything wrong with your simulation...

Could you try upgrading to latest snapshot, please?

https://github.com/excilys/gatling/wiki/Continuous-Integration

Here's your migrated simulation:

package basic

import io.gatling.core.Predef._

import io.gatling.http.Predef._

import io.gatling.jdbc.Predef._

import io.gatling.http.HeaderNames._

import scala.concurrent.duration._

class LoadGen extends Simulation {

val nbUsers = Integer.getInteger("users", 1)

val myRamp = java.lang.Long.getLong("ramp", 0).toInt

val myIter = Integer.getInteger("iter", 1).toInt

val myUrl = System.getProperty("url")

val pgReqFeeder = ssv("pgRequest.ssv").circular

println(myUrl)

val httpConf = http

.baseURL(myUrl)

.acceptHeader("*/*")

val scn = scenario("Enrichment Demo")

.repeat(myIter) {

feed(pgReqFeeder)

.exec(

http("Http Post")

.post("/something.do")

.header("Content-Type", "application/json")

.body(StringBody("${REQUESTBODY}"))

.asJSON

.check(status.is(200)))

}

setUp(scn.inject(rampUsers(nbUsers) over myRamp)).protocols(httpConf)

}

[Nicolas Rémond]

How do you get the Start and End time with Gatling?

You are not looking at the beginning and the end of the process, right?

[Vikram Godbole]

@Nicolas: I am using start time and end time at beginning and end of the process. But simulation prints out elapsed time, and reports do show RPS. Start time and end time are not used in any calculations.

@Stéphane: I will try using the latest snapshot today and report results.