java.lang.OutOfMemoryError: Required array size too large

[Daniil Z]

Hello everyone,

I have an issue when trying to upload a large file (5 GB and larger) as it tries to load the whole content to the memory instead of streaming it.

According to the documentation RawFileBody doesn't support streaming when HTTPS protocol is used. 

Is there a way how to stream this large file?

I'm using gatling-gradle-plugin of the version 3.6.0,

Please find below the code I use:

def uploadFile: ChainBuilder = exec(
http("Upload file")
.post("/data-api/v4/datasets/testDataset")
.header("Content-Type", "multipart/form-data")
.header("Authorization", "Bearer ${access_token}")
.formUpload("datasetFile", s"5_gb.json")
.requestTimeout(30.minutes)
.check(status.is(201))
)

val httpProtocol: HttpProtocolBuilder = http.baseUrl(s"$baseURL")

val scn: ScenarioBuilder = scenario("Upload big file")
.exec(
authenticateUser,
uploadFile
)

setUp(
scn.inject(
atOnceUsers(1),
)
)
.protocols(httpProtocol)
.assertions(global.responseTime.percentile(10).lte(800))

Thank you for any suggestions/corrections,

Daniil

[Daniil Z]

I completely forgot to attach stacktrace, so here it is:

14:55:41.032 [ERROR] i.g.a.Gatling$ - Run crashed

java.lang.OutOfMemoryError: Required array size too large

        at java.base/java.nio.file.Files.readAllBytes(Files.java:3212)

        at io.gatling.core.util.Resource.bytes(Resource.scala:92)

        at io.gatling.core.util.Resource.bytes$(Resource.scala:92)

        at io.gatling.core.util.FilesystemResource.bytes(Resource.scala:119)

        at io.gatling.core.body.RawFileBodies.$anonfun$asResourceAndCachedBytes$1(RawFileBodies.scala:48)

        at io.gatling.commons.validation.Success.map(Validation.scala:37)

        at io.gatling.core.body.RawFileBodies.asResourceAndCachedBytes(RawFileBodies.scala:47)

        at io.gatling.http.request.BodyPart$.rawFileBodyPart(BodyPart.scala:37)

        at io.gatling.http.request.BodyPartSupport.RawFileBodyPart(BodyPartSupport.scala:41)

        at io.gatling.http.request.BodyPartSupport.RawFileBodyPart$(BodyPartSupport.scala:40)

        at io.gatling.http.Predef$.RawFileBodyPart(Predef.scala:19)

        at Scala.uploadFile(Scala.scala:60)

        at Scala.<init>(Scala.scala:77)

        ... 10 common frames omitted

Wrapped by: java.lang.reflect.InvocationTargetException: null

        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)

        at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

        at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)

        at io.gatling.app.Runner.run0(Runner.scala:62)

        at io.gatling.app.Runner.run(Runner.scala:49)

        at io.gatling.app.Gatling$.start(Gatling.scala:83)

        at io.gatling.app.Gatling$.fromArgs(Gatling.scala:45)

        at io.gatling.app.Gatling$.main(Gatling.scala:37)

        at io.gatling.app.Gatling.main(Gatling.scala)

Exception in thread "main" java.lang.reflect.InvocationTargetException

        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)

        at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

        at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)

        at io.gatling.app.Runner.run0(Runner.scala:62)

        at io.gatling.app.Runner.run(Runner.scala:49)

        at io.gatling.app.Gatling$.start(Gatling.scala:83)

        at io.gatling.app.Gatling$.fromArgs(Gatling.scala:45)

        at io.gatling.app.Gatling$.main(Gatling.scala:37)

        at io.gatling.app.Gatling.main(Gatling.scala)

Caused by: java.lang.OutOfMemoryError: Required array size too large

        at java.base/java.nio.file.Files.readAllBytes(Files.java:3212)

        at io.gatling.core.util.Resource.bytes(Resource.scala:92)

        at io.gatling.core.util.Resource.bytes$(Resource.scala:92)

        at io.gatling.core.util.FilesystemResource.bytes(Resource.scala:119)

        at io.gatling.core.body.RawFileBodies.$anonfun$asResourceAndCachedBytes$1(RawFileBodies.scala:48)

        at io.gatling.commons.validation.Success.map(Validation.scala:37)

        at io.gatling.core.body.RawFileBodies.asResourceAndCachedBytes(RawFileBodies.scala:47)

        at io.gatling.http.request.BodyPart$.rawFileBodyPart(BodyPart.scala:37)

        at io.gatling.http.request.BodyPartSupport.RawFileBodyPart(BodyPartSupport.scala:41)

        at io.gatling.http.request.BodyPartSupport.RawFileBodyPart$(BodyPartSupport.scala:40)

        at io.gatling.http.Predef$.RawFileBodyPart(Predef.scala:19)

        at Scala.uploadFile(Scala.scala:60)

        at Scala.<init>(Scala.scala:77)

        ... 10 more

[Stéphane LANDELLE]

https://github.com/gatling/gatling/issues/4134

--

Stéphane Landelle Chief Technical Officer

slan...@gatling.io gatling.io

   

--
You received this message because you are subscribed to the Google Groups "Gatling User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gatling+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gatling/a388e3e1-06d0-4012-bcaf-8895dbdb48d8n%40googlegroups.com.