Chunked Encoding

[Javier Arrieta]

Hi,

I've spent some time to figure out how to configure the Http Client to use chunked encoding. I've tried setting the header Transfer-Encoding: chunked

My code is 

  val scn = scenario("My scenario")

            .exec(http("Upload")

              .post("http://192.168.212.71:9080/file")

              .header("Transfer-Encoding", "chunked")

              .fileBody("payload.bin")

            )

But the requests are not chunked.

Any way to set the request to use chunked encoding?

Thanks!

Javier

[Stéphane Landelle]

Hi,

Strange, it should work.

What's your file's size? AHC chunks size is hardcoded with a 8ko value.

Stéphane

2013/3/17 Javier Arrieta <javier....@gmail.com>

Javier

--
You received this message because you are subscribed to the Google Groups "Gatling User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gatling+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Javier Arrieta]

Hi Stéphane,

the file is 50 MB,I've also tried with 30MB with the same results. If I use curl I get a correct chunked encoding transfer:

javier@gatling:~/gatling-charts-highcharts-1.4.4$ curl  -v -X POST -H "Transfer-Encoding: chunked" -i --data-binary @user-files/request-bodies/payload.bin http://192.168.212.71:9080/file

* About to connect() to 192.168.212.71 port 9080 (#0)

*   Trying 192.168.212.71... connected

> POST /file HTTP/1.1

> User-Agent: curl/7.22.0 (i686-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3

> Host: 192.168.212.71:9080

> Accept: */*

> Transfer-Encoding: chunked

> Content-Type: application/x-www-form-urlencoded

> Expect: 100-continue

> 

< HTTP/1.1 100 Continue

HTTP/1.1 100 Continue

< HTTP/1.1 201 Created

HTTP/1.1 201 Created

< Server: spray-can/1.1-M7

Server: spray-can/1.1-M7

< Date: Tue, 19 Mar 2013 20:01:25 GMT

Date: Tue, 19 Mar 2013 20:01:25 GMT

< Content-Length: 0

Content-Length: 0

< 

* Connection #0 to host 192.168.212.71 left intact

* Closing connection #0

Thanks!

Javier

[Stéphane Landelle]

Hi Javier,

Would you by any chance a sample to share so I could reproduce, please?

This would save me a lot of time...

Stéphane

2013/3/19 Javier Arrieta <javier....@gmail.com>

[Javier Arrieta]

Hi

Do you mean the scenario? Because it is in the first post. Adding it again, it is very simple. The file is generated randomly with dd (dd if=/dev/urandom of=payload.bin bs=50M count=1)

The server is in github (also very simple spray can server):

https://github.com/javierarrieta/spray-spikes/tree/master/examples/http-server-example

The scenario:

import com.excilys.ebi.gatling.core.Predef._

import com.excilys.ebi.gatling.http.Predef._

import com.excilys.ebi.gatling.jdbc.Predef._

import akka.util.duration._

import bootstrap._

import assertions._

class FileUploadSimulation extends Simulation {

  // your code starts here

  val scn = scenario("My scenario")

            .exec(http("Upload")

              .post("http://192.168.212.71:9080/file")

              .header("Transfer-Encoding", "chunked")

              .fileBody("payload.bin")

            )

                

  setUp(scn.users(18))

  // your code ends here

}

Thanks for your help!

--
You received this message because you are subscribed to a topic in the Google Groups "Gatling User Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gatling/rDdH7nY3QbQ/unsubscribe?hl=en.
To unsubscribe from this group and all its topics, send an email to gatling+u...@googlegroups.com.

[Stéphane Landelle]

Perfect! Will try tomorrow.

Thanks!

2013/3/19 Javier Arrieta <jav...@arrieta.nom.es>

[Stéphane Landelle]

OK, got it.

First, chunked transfer encoding is to be used when the Content-Length is unknown. This is available in AHC when providing a body in the form of an InputStream. We don't currently expose it in Gatling as we haven't seen a use case for it. In the case of a file, the Content-Length is perfectly known and AHC will perform a zero-copy and directly write on the Socket from the file.

Then, I think your spray-can sample uses the default BufferingRequestStreamActor that buffers the chunks before handling a full HttpRequest to your HttpServiceActor, so you'll never match chunk events there. See https://github.com/spray/spray-can#receiving-chunked-requests

Cheers,

Stéphane

2013/3/19 Stéphane Landelle <slan...@excilys.com>

[Javier Arrieta]

Hi 

Thanks for the info. So for this to work I should modify gatling to accept an InputStream.

The spray-can is working for chunked-encoded requests, it is not aggregating them but creating the correct messages (ChunkedRequestStart, MessageChunk and ChunkedMessageEnd) because of the configuration:

request-chunk-aggregation-limit = 0

I wish I had more time to get a bit deeper in gatling internals. I will try to explore the options for the inputStream

Thanks Stéphane,

Javier

[Stéphane Landelle]

My bad, so that might not be the problem.

I will dig deeper as soon as I find time...

2013/3/20 Javier Arrieta <jav...@arrieta.nom.es>

[Stéphane Landelle]

Ah, I realized that src/main/resources wasn't in my classpath (don't ask me why...).

Will check again ASAP.

2013/3/20 Stéphane Landelle <slan...@excilys.com>

[Stéphane Landelle]

I've introduced an inputStreamBody in Gatling 2/master, and I confirm that the body gets indeed chunked and that Spray actor gets the Chunk messages.

Stéphane

2013/3/20 Stéphane Landelle <slan...@excilys.com>

[Javier Arrieta]

Thanks!

I will try it as soon as I can and post the results

[Stéphane Landelle]

Beware that at this point, the feature is only for Gatling 2, and there's quite a few things that have changed and are not documented yet...

2013/3/20 Javier Arrieta <jav...@arrieta.nom.es>