All permutations of two feeders

[Henri Tremblay]

Hello,

I have two feeders. I would like to do all the permutation of the lines in both of them. If I use circular, my understanding is that it will loop around both feeders simultaneously but won't do my matrix kinda thing.

Appart from merging the two feeders in one? (what's the scala code for that?), it there a solution?

thanks

Henri 

[Stéphane Landelle]

Salut,

What do you want exactly? Do you want each virtual user to play them all, or just build a common pool? Do you want to make sure all the permutations are only played once and only once, or it doesn't matter?

--
You received this message because you are subscribed to the Google Groups "Gatling User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gatling+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Henri Tremblay]

It doesn't matter if it is called more than once. But I want every permutations to be done. It could be by a single user.

Let's say I want to load a cache with a key using two parameters passed to the query.

So if one feeder contain a, b, c and the other 1,2. I want to call the server with

a1, a2, b1, b2, c1, c2

-

Henri

[John Arrowwood]

Do you need to use a Feeder, or is there another way of solving this problem?  

And is the problem really that you need to exercise all permutations, or just that you don't always exercise the same values?

For example, if you have two feeders, and both are set to Random, then over time you would exercise many of the different combinations.  But it would be a long time before you exercised all of them.  If your goal is to exercise them each exactly once, this would not be a good solution.  If your real goal is to throw lots of different traffic at it, then it will work great.

If you really need to do every permutation of two sets, one easy thing to do is to just merge them into a single feed, outside of gatling.  

Another option is nested loops.  At the start of the script, record the unique ID of the first record of each feeder.  Set the second feed to be circular.  Let the outer loop terminate when the first feeder is empty.  In the inner loop, stop once the second feed is back to the first record.  That should get you exactly one of each permutation.

HOWEVER, that will only work with a single user.  If you want to be able to spread it over multiple users, and you don't want to merge them into a single feed, then you will need to build a custom feed.

If we understood why you want the permutations, we might be able to give more useful advice.

[Stéphane Landelle]

I guess Henri want to load a cache.

The most simple solution is with one user: a repeat loop the size of the first file, then the first feed, then repeat on the second file content (not a feeder). See http://gatling.io/docs/2.0.0-RC6/session/feeder.html#non-shared-data

[Henri Tremblay]

Yep.

My final solution is:

 val aRecords = tsv("a.tab").records
 val bRecords = tsv("b.tab").records

val scn = scenario("MySimulation")
   .foreach(aRecords, "aRecord") {
     exec(flattenMapIntoAttributes("${aRecord}"))
     .foreach(bRecords, "bRecord") {
       exec(flattenMapIntoAttributes("${bRecord}"))
         .exec(http("call")
         .get("""/api?a=${a}&b=${b}""")
     }
   }

That works. However, indeed, I could gain from being in parallel on multiple users. But I'm not sure I read the documentation right. Does all users play all the records or is each records played by only one user?

I will also need to remove duplicates from aRecords. But for that I guess I just need to do a distinct on the entries based on one key on the map underneath. As soon as I figure out how to scala that, I should be fine :-)

For those how haven't a clue of what I'm talking about, I mean that in my file I have:

1 a

1 b

2 a

2 c

...

and I only use the first column. So I want my final record to contain

1

2

--
You received this message because you are subscribed to a topic in the Google Groups "Gatling User Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gatling/dNnIGUhUXxA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gatling+u...@googlegroups.com.

[John Arrowwood]

The solution you have will not work as expected if you use two feeders and more than one user.  If the inside loop is an iterator over a list, then you could use multiple users, and each user is handling one subset of the whole.  Which begs the question:

 

Is this source file something that is going to change?  Or is it something that you could do in Scala code?  

If it is two files that may (in theory) be different every time you run, then your best, most scalable solution (no pun intended) is to write Scala code that reads both files, and builds an in-memory data structure of all of the permutations (if there are not too many), and then serves as a single iterator that returns the data.  Then you could ramp as many users as you like using that feeder, and no two of them would ever pull the same combination, and every combination would be covered.

But if you have something that works, and it is fast enough...

[Henri Tremblay]

With some help from a friend, the final version is

 val aRecords = tsv("a.tab").records.map { _.get("key")}.distinct

 val bRecords = tsv("b.tab").records

 val mixed = for {
   a <- aRecords
   b <- bRecords
 } yield b + ("key" -> a)

val scn = scenario("MySimulation")

   .foreach(mixed.records, "record") {
     exec(flattenMapIntoAttributes("${record}"))
     .exec(http("query")
     .get( """/?a=${key}&b=${id}""")
 }

[John Arrowwood]

Well done. :)