Use Session attributes inside randomSwitch

[Kan Wu]

Hey guys,

I want to read a url and weight from a csv file, then execute the url by percent of weight. so this is how I do it:

val req = feed(Properties.urls)

.exec(session => {

randomSwitch(

any2ArrowAssoc(Integer.getInteger(session("weight").as[String]).toDouble) -> exec(http("${url}").get("${url}"))

)

session.set("","")}

)

but keep getting errors:

[ERROR] [11/05/2014 16:05:51.017] [GatlingSystem-akka.actor.default-dispatcher-22] [akka://GatlingSystem/user/$d] null

java.lang.NullPointerException

Ideas? or a better way of doing that? csv file contains 2 columns: url, weight

I tried not put randomSwitch inside session but in that way I could not read attribute value out from session, script could not compile and it complains about session is not defined.

[John Arrowwood]

You can't put an exec or other chain builder inside of a session function.  So where you have randomSwitch(...) will not work.  

Without looking at the code, I can't say if you can build a randomSwitch that can vary at run time.  I suspect you can't.  So to get that behavior, you will probably have to simulate it. 

[Stéphane Landelle]

Not supported atm, weights are static values.

But then? Why do you want to do that? Weights are expected to be a statistically computation on a given population, i.e. a scenario.

--
You received this message because you are subscribed to the Google Groups "Gatling User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gatling+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Kan Wu]

My goal is to create a generic request template so I can read url and weight from csv then execute it. urls and weights can change over time, but we just need to update the csv, not the actual script. randomSwitch supports use var as it's percent value and I have tried that. Now I just need to update the var every time based the value read from csv. I will move randomSwitch out of session but use session to set a variable and randomSwitch read from that variable. Will this work?

[Stéphane Landelle]

That's not how you should be doing things. I'm still convinced that there's no reason to define weigths per user, but per population.

If you don't want to hard code them, you can pass them as variables to the Simulation (System properties, config file, etc...).

[Kan Wu]

I dont define weight for each user but for all. However, in execution, if every user is executing a request by certain weight then for sure the overall percentage is also correct. Defining hundreds of urls in system properties or configs, will that be too messy?

[Kan Wu]

this is my code, and this works fine:

object Web {

var percent = 0

val setVar = feed(Properties.urls)

.exec(session => {

percent = session("weight").as[String].toInt

session.set("percent",percent)

})

val req = randomSwitch(

any2ArrowAssoc(percent.toDouble) -> exec(http("${url}").get("${url}"))

)

val scn = setVar.exec(req)

[Kan Wu]

this piece of code didnt work as expected, though I did see some requests went through but later it became extremely slow and taking 100% CPU and had memory leak, used more than 5GB.

object Web {

var percent = 0.0

val req = feed(Properties.urls)

.exec(session => {

percent = session("weight").as[String].toDouble

session.remove("weight")

})

.randomSwitch(

any2ArrowAssoc(percent) -> exec(http("${url}-"+percent+"%").get("${url}"))

)

}

Which part might be wrong? If not this approach then I have to read the whole csv and make up the Map[%,Chain] to use in randomSwitch? OR any chance we dont use randomSwitch?

[Kan Wu]

very interesting that after I added a print line, the memory leak stopped and the test could sustain. this is very interesting. without the print, test can only run for 50 seconds with 100 sessions and memory could go over 3 GB. but with the print, test can just keep running though the final weight was not as good as expected, more like round robin. ideas?

object Web {

var percent = 0.0

val req = feed(Properties.urls)

.exec(session => {

percent = session("weight").as[String].toDouble

print(percent)

session.set("percent",percent)

})

.randomSwitch(

any2ArrowAssoc(percent) -> exec(http("${url}-${percent}%").get("${url}"))

)

}

================================================================================

2014-11-06 17:30:07                                         315s elapsed

---- BEAS ----------------------------------------------------------------------

[--------------------------------------------------------------------------]  0%

          waiting: 0      / active: 100    / done:0     

---- Requests ------------------------------------------------------------------

> Global                                                   (OK=722    KO=0     )

> /a-20.0%                                               (OK=149    KO=0     )

> /b-30.0%                                               (OK=151    KO=0     )

> /c-20.0%                                               (OK=155    KO=0     )

> /d-10.0%                                               (OK=167    KO=0     )

> /e-20.0%                                               (OK=100    KO=0     )

================================================================================

[Stéphane Landelle]

Of course!

Your code is super non threadsafe! You're mutating a global variable from concurrent threads.

[Kan Wu]

hmm, then why add a print line will help improve memory leak situation? I guess I will still need to read the whole csv and then make the Map[%,Chain]. Or do you have a better idea how I can read url and weight from csv and execute by the weight?

[Kan Wu]

I am thinking of use feed to get every line and then form a map[Double, io.gatling.core.structure.ChainBuilder], then use the map in randomSwitch. But the script could not compile, could be my scala issue. Code:

object CSV2Map {

val process = feed(Properties.urls)

.exec(session => {

var percent = session("weight").as[String].toDouble

Properties.executeMap += (percent -> exec(http("${url}-${percent}%").get("${url}")))

session.remove("weight")

})

}

object Web {

val req = randomSwitch(

Properties.executeMap                  //  type mismatch; found : Map[Double,io.gatling.core.structure.ChainBuilder] required: (Double, io.gatling.core.structure.ChainBuilder)

)

[John Arrowwood]

Your best bet is probably to implement your own logic, something like this:

.exec( session => {

  // inspect your map and implement your own selection logic to select a url, and then

  session.set( "theURL", theUrl )

  })

.exec( http( desc ).get( "${theURL}" ) )

[Kan Wu]

but in this way how do i control the weight? I have to stick with randomSwitch because I dont see any other ways in Gatling to set the execution percentage.

[John Arrowwood]

Gatling is not magic.  Anything Gatling can do, you can do yourself.  :)

When you load the urls and counts, sum the counts to get a total.  Store those for later use.

When you need to pick a url, pick a random number between 1 and total.  Step through the list of urls and weights and add the current weight to the accumulator.  Return the first URL that the accumulator is >= the random number.  

[Kan Wu]

I see your proposal but that is essential as same as set weight for each user because every time a session goes to the map and pick up a pair of url and weight, then bases on the weight to execute url. When session number is numerous, the process would be inefficient and get random number for every time every session could be expensive too.

[John Arrowwood]

What I described, and what you are describing, is essentially what Gatling is doing under the covers.  It is no more inefficient than what you are trying to do using randomSwitch, and it will let you do what you are shooting for in a more flexible way.

--
You received this message because you are subscribed to a topic in the Google Groups "Gatling User Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gatling/1OmkcnLK-nY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gatling+u...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

John Arrowwood,

Consulting Hypnotist / Hypnotherapist
http://www.JohnArrowwood.com
503.863.4823

[Kan Wu]

something interesting is going with my code or Gatling:

object Web {

var rnd = new scala.util.Random(System.currentTimeMillis)

val scn = exec(_.set("urlSeq",Properties.executeSeq))

.exec(_.set("random", rnd.nextInt(Properties.total.toInt)))

.foreach("${urlSeq}", "currentTuple", "counter") {

doIf(session => session("counter").as[Int] == 0) {

exec(session => {

println("random: "+session("random").as[Int])

session.set("lower", 0)

  .set("upper", 0)

})

}

.exec(session => {

val tuple = session("currentTuple").as[Tuple2[Double,String]]

println("lower: "+session("upper").as[Int])

println("upper: "+(session("upper").as[Int]+tuple._1.toInt))

session.set("lower", session("upper").as[Int])

  .set("upper", session("upper").as[Int]+tuple._1.toInt)

  .set("percent", tuple._1)

  .set("exeUrl", tuple._2)

})

.doIf(session => (session("upper").as[Int] >= session("random").as[Int] && session("random").as[Int] > session("lower").as[Int])){

exec(session => {

println("Inside IF LOOP\n")

session.set("", "")

})

.exec(http("${exeUrl}-${percent}%").get("${exeUrl}"))

}

}

}

it could run for first 10 seconds, then memory kept going up and there was no Gatling summary table. I added some extra print to find out which part might be wrong but then i found it actually went into the final if loop but just the last http exec didnt get executed. A potential Gatling bug?

console output is lengthy but this is part of that. Clearly it went into the random number check but it didnt execute the http request. Ideas?

random: 88

lower: 0

upper: 30

lower: 30

upper: 50

lower: 50

upper: 60

lower: 60

upper: 80

lower: 80

upper: 100

Inside IF LOOP

random: 95

lower: 0

upper: 30

lower: 30

upper: 50

lower: 50

upper: 60

lower: 60

upper: 80

lower: 80

upper: 100

Inside IF LOOP

[Stéphane Landelle]

The only reason for a request to not be sent is that the name couldn't be resolved ("${exeUrl}-${percent}%" here). If so, you'd get a debug log message.

[Kan Wu]

I dont see that. And at beginning some requests could get through:

================================================================================

2014-11-12 16:25:23                                          15s elapsed

---- ProcessCSV ----------------------------------------------------------------

[##########################################################################]100%

          waiting: 0      / active: 0      / done:1     

---- BEAS ----------------------------------------------------------------------

[--------------------------------------------------------------------------]  0%

          waiting: 0      / active: 1      / done:0     

---- Requests ------------------------------------------------------------------

> Global                                                   (OK=3      KO=0     )

> /a-20.0%                                               (OK=1      KO=0     )

> /b-30.0%                                               (OK=1      KO=0     )

> /c-10.0%                                               (OK=1      KO=0     )

================================================================================

[Kan Wu]

I found out why. It was caching playing the game with me. In debug log I could see skip request because of caching. After disable that everything is normal now. But when caching was not disabled, I did have a memory leak problem. Is that something Gatling should be aware of?

Thanks for all your help!