Issue with throttling scenarios individually

[John Arrowwood]

We have a simulation that simulates a lot of different transactions.  Some of those transactions need to be capped at a certain incoming rate because they push data to an upstream system that can not handle heavy load.  Most of the rest can be scaled up quite high (by comparison) before there is an issue.

I'm trying to set a target requests per second, and then do the math to ensure that the capped transactions stay appropriately capped, and scale up everything else to fill the difference.

What I did was divided the scenario into two parts, two scenarios, and I put a .throttle on each.  The reachRps() on the one is hard-coded to the value that corresponds to the maximum we should allow through that scenario, and the other scenario is set to reachRps() of the target rate minus the rate of the first scenario.  

To make things concrete, one scenario wants to reach 1 rps, the other wants to reach 9 rps.  Both scenarios are launching 100 users (each) and have a .during() loop in the middle of 10 minutes plus the ramp time.

My expectation is that the throttling will result in an average of 10 requests per second, one of which will belong to one scenario, the rest will belong to the other scenario.  In actuality, the requests from the smaller throttled scenario did not even start happening until approximately after the holdFor() was completed (give or take).  

For reference, I am running Gatling 2.0.1.  I have attached the simulation.log, so you can generate the reports and see what happened.  Here is my scenario setup:

  val numUsers    = java.lang.Integer.getInteger("users",1).toInt
  val rampTime    = if ( numUsers == 1 ) 0 else numUsers / 2
  val duration    = Duration( java.lang.Long.getLong("duration",30L), "seconds" ) + rampTime
  val maxDuration = duration + rampTime + 300
 
  val typicalSurveyRate = 1 // requests per second
  val peakSurveyRate = 3
  val targetSurveyRate = typicalSurveyRate
 
  val typicalTotalRate = 2
  val peakTotalRate = 30
  val targetRate = 10
 
  println( "NUM_USERS = " + numUsers )       // 100
  println( "RAMP_TIME = " + rampTime )       // 50
  println( "DURATION  = " + duration )       // 650
  println( "MAX_DURATION = " + maxDuration ) // 1000
 
  setUp(
    prescn.inject(atOnceUsers(1)),
    surveys.inject( rampUsers( numUsers ) over ( rampTime ) )
        .throttle(
                reachRps( targetSurveyRate ) in ( rampTime ),
                holdFor( maxDuration )
        ),
    everythingElse.inject( rampUsers( numUsers ) over ( rampTime ) )
        .throttle(
                reachRps( targetRate - targetSurveyRate ) in ( rampTime ),
                holdFor( maxDuration )
        )
  )
    .maxDuration( maxDuration )
    .protocols( httpConf )

  

[Stéphane LANDELLE]

Can you reproduce with Gatling 2.1.3?

Stéphane Landelle

Lead developer

slan...@gatling.io

--
You received this message because you are subscribed to the Google Groups "Gatling User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gatling+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[John Arrowwood]

I can't use 2.1.3 right now, because of all the refactoring which broke my library pimping.  It will be a while before I have the chance to update.

In the mean time, the test case is a 9 to 1 ratio of scenario 1 to scenario 2, but with relatively low RPS rates (9 and 1, to be precise).  It should be easy enough to test using something like this:

val foo =
  scenario( "foo" )
  .during( 1000 seconds ) {
    exec( http("foo").get("http://www.google.com/?q=foo") )
  }

val bar =
  scenario( "bar" )
  .during( 1000 seconds ) {
    exec( http("bar").get("http://www.google.com/?q=bar") )
  }

setUp(
  foo
    .inject( rampUsers( 100 ) over ( 50 seconds ) )
    .throttle( reachRps( 1 ) in ( 50 seconds ) )
    .holdFor( 1000 seconds ),

  bar
    .inject( rampUsers( 100 ) over ( 50 seconds ) )
    .throttle( reachRps( 9 ) in ( 50 seconds ) )
    .holdFor( 1000 seconds ),
)
.maxDuration( 1100 seconds )
.protocols( ... )

The results SHOULD show around 9,000 calls to bar, and about 1,000 calls to foo, if the feature is behaving as advertised, right?

Does anyone who has 2.1.3 installed have time to test it?  For that matter, is anyone on 2.0 that can confirm that they see the same problem (or not)?

[Stéphane LANDELLE]

I did find a small glitch: https://github.com/gatling/gatling/issues/2525

But still, I get pretty fine results with current master:

val httpProtocol = http

  .baseURL("http://www.ebusinessinformation.fr")

  .acceptCharsetHeader("ISO-8859-1,utf-8;q=0.7,*;q=0.7")

  .acceptHeader("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")

  .acceptLanguageHeader("fr,fr")

  .userAgentHeader("Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:8.0.1) Gecko/20100101 Firefox/8.0.1")

  .disableCaching

  

val foo =

  scenario("foo")

    .during(1000 seconds) {

      exec(http("foo").get("/"))

    }

val bar =

  scenario("bar")

    .during(1000 seconds) {

      exec(http("bar").get("/"))

    }

setUp(

  foo

    .inject(atOnceUsers(10))

    .throttle(jumpToRps(1), holdFor(30 seconds)),

  bar

    .inject(atOnceUsers(10))

    .throttle(jumpToRps(9), holdFor(30 seconds)))

  .protocols(httpProtocol)

---- Requests ------------------------------------------------------------------

> Global                                                   (OK=285    KO=0     )

> foo                                                      (OK=30     KO=0     )

> bar                                                      (OK=255    KO=0     )

================================================================================

bar/foo is 8.5 which is pretty close to the expected 9 (it's probably not exactly 9 because of the fast shutdown at 30 sec, before getting the last round responses).

Stéphane Landelle

Lead developer

slan...@gatling.io

[John Arrowwood]

Good.  More motivation to find the time to upgrade... :)

[Stéphane LANDELLE]

What results do you get with my sample and 2.0.2?

Stéphane Landelle

Lead developer

slan...@gatling.io

[John Arrowwood]

When I get a chance, I will let you know.

--
You received this message because you are subscribed to a topic in the Google Groups "Gatling User Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gatling/7ZHgjsulmX4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gatling+u...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

John Arrowwood,

Consulting Hypnotist / Hypnotherapist
http://www.JohnArrowwood.com
503.863.4823