How to test multiple URLs from a csv file

[phy...@gmail.com]

Hi,

Could I ask about testing multiple URLs from a csv file using Gatling? Thanks for your help. I have spent tens of hours on this performance testing, but still cannot get it right.

My urls.csv file is like this:
query=1
query=1
query=1
...

And my scala script is:
    val url = csv("urls.csv").circular
    val scn = scenario("urls_test").feed(url).during(10 seconds) {
                    exec(
                        http("Performance Test for URLs")
                        .get("/search?${url}")
                        .check(status.is(200)))
    }

Best,
Phy.

[Stéphane Landelle]

The way you wrote it, each virtual user will be only fed once. Is this what you really want?

Otherwise, you have to move feed inside the during loop.

Then, note that the queries inside your csv file have to be proper queries (keys and values properly urlencoded).

--
You received this message because you are subscribed to the Google Groups "Gatling User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gatling+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Stéphane Landelle]

You have to define a first line that contains the headers. Those will be used for naming the session attributes ("url" in your case).

Please properly read the documentation: https://github.com/excilys/gatling/wiki/Feeders#fileParsers

2014-05-13 20:08 GMT+02:00 <phy...@gmail.com>:

Hi Stéphane,

Thanks for your help, especially correcting my understanding in the "feed" functions.

May I ask the details for the csv file format? Most url files I usually tested, based on RestFul, are only separated by the line breaks, which are not in the format of the keys and values. Do you have any idea about how to handle it? Thank you very much.

url file:
url1
url2
url3
......

Best Regards,
Phy.

[alex bagehot]

Hi, maybe I misunderstood but It looks like csv will work for you as you have 1 record per line and one column only resulting in no commas.

You could try it out with a limited data set to see if it produces the output you expect?

Thanks
Alex

On 13 May 2014 22:56, <phy...@gmail.com> wrote:

Hi Stéphane,

I really appreciate your invaluable advice. May I pls ask more deeply about this multiple-URLs testing which is quite common case for perf-testing? If my URLs.txt file contains commas, and each line is separated by "\n" (so cannot be treated as csv, tsv, ssv), is it possible to use the function of "feed" for loading these urls from the external file? Thank you very much.

Best Regards,
Phy.

[Stéphane Landelle]

@Phy What's your problem exactly.

I think the only thing you have to change is to just add a first header line in your file with the string "url".

[Stéphane Landelle]

OK, so as I said earlier, "the queries inside your csv file have to be proper queries (keys and values properly urlencoded)".

pipe, comma and question mark are definitively NOT valid and have to be urlencoded.

http://en.wikipedia.org/wiki/Percent-encoding

Best would be that you have properly encoded queries in your csv file. 

2014-05-14 21:05 GMT+02:00 <phy...@gmail.com>:

Hi Stéphane,

I find if we have a file with 1 record per line, and each line contains commas, csv will not work.

I tried tsv, and seems it works, but I donot really understand its mechanism. Thank you very much.

Best,
Phy.

[Stéphane Landelle]

You only have 1 column, so you don't even have to care about the column separator.

The other solution is to encode on the fly, but there's of course an overhead:

.get(session => "search?" + java.net.URLEncoder.encode(session("url").as[String]))

This assumes that url is one single query parameter value that can be urlencoded as a whole.

2014-05-14 21:18 GMT+02:00 Yuchuan Jian <phy...@gmail.com>:

Hi Stéphane,

Thanks for your prompt reply and invaluable suggestions. I do think it is necessary to process the input file before testing. Currently I use tsv instead of csv, and interestingly it works, which maybe "/t" is similiar to "\n", I guess?

Thank you,

Phy.

--
You received this message because you are subscribed to a topic in the Google Groups "Gatling User Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gatling/zMtJEzzKXhM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gatling+u...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Stéphane Landelle]

Actually, that's  java.net.URLEncoder.encode(session("url").as[String], "UTF-8") or whatever encoding you use.

[Stéphane Landelle]

I assumed Gatling 2M3a. If not please checkout Session API documentation and adapt.

https://github.com/excilys/gatling/wiki/Session#members

2014-05-14 21:59 GMT+02:00 Yuchuan Jian <phy...@gmail.com>:

There is some error for session("url"). Here is my script:
 val url = csv("urls.txt").circular
 val scn = scenario("PerfTest").during(10 seconds) {
                    feed(url)
                    .exec(
                        http("URLs")
                        .get(session => "/search/query?" +  java.net.URLEncoder.encode(session("url").as[String], "UTF-8") )
                        .check(status.is(200)))
}

Thank you,
Phy.