[2.x] Uploading files bigger than approx. 2GB does not work

[Florian Heinisch]

Hello everyone,

uploading large files (approx. 2GB or bigger) does not seem to work with Play 2.x.

I tested it both with Play 2.0.4 and Play 2.1.

My use case:

I have a simple upload form that uploads files via multipart POST request to the Play server.

The browser does not seem to be the limiting part. I tested the file upload with Safari, Firefox and Chrome. The current versions of these browsers support file uploads > 2 GB (Chrome and Safari even up to 20GB).

To be precise: In Safari, the upload works when I upload a file with 2,147,483,442 bytes (that is 2GB minus 206 bytes). All uploads with files greater than 2,147,483,442 bytes are failing with following stacktrace (resulted from a test with Play 2.1):

java.lang.NegativeArraySizeException

at org.jboss.netty.buffer.HeapChannelBuffer.<init>(HeapChannelBuffer.java:42)

at org.jboss.netty.buffer.BigEndianHeapChannelBuffer.<init>(BigEndianHeapChannelBuffer.java:34)

at org.jboss.netty.buffer.ChannelBuffers.buffer(ChannelBuffers.java:134)

at org.jboss.netty.buffer.HeapChannelBufferFactory.getBuffer(HeapChannelBufferFactory.java:69)

at org.jboss.netty.buffer.AbstractChannelBuffer.readBytes(AbstractChannelBuffer.java:307)

at org.jboss.netty.handler.codec.replay.ReplayingDecoderBuffer.readBytes(ReplayingDecoderBuffer.java:359)

at org.jboss.netty.handler.codec.http.HttpMessageDecoder.decode(HttpMessageDecoder.java:295)

at org.jboss.netty.handler.codec.http.HttpMessageDecoder.decode(HttpMessageDecoder.java:101)

at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:502)

at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:437)

at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)

at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)

at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:84)

at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:472)

at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:333)

at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)

at java.lang.Thread.run(Thread.java:722)

java.lang.NegativeArraySizeException

at org.jboss.netty.buffer.HeapChannelBuffer.<init>(HeapChannelBuffer.java:42)

at org.jboss.netty.buffer.BigEndianHeapChannelBuffer.<init>(BigEndianHeapChannelBuffer.java:34)

at org.jboss.netty.buffer.ChannelBuffers.buffer(ChannelBuffers.java:134)

at org.jboss.netty.buffer.HeapChannelBufferFactory.getBuffer(HeapChannelBufferFactory.java:69)

at org.jboss.netty.buffer.AbstractChannelBuffer.readBytes(AbstractChannelBuffer.java:307)

at org.jboss.netty.handler.codec.replay.ReplayingDecoderBuffer.readBytes(ReplayingDecoderBuffer.java:359)

at org.jboss.netty.handler.codec.http.HttpMessageDecoder.decode(HttpMessageDecoder.java:295)

at org.jboss.netty.handler.codec.http.HttpMessageDecoder.decode(HttpMessageDecoder.java:101)

at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:502)

at org.jboss.netty.handler.codec.replay.ReplayingDecoder.cleanup(ReplayingDecoder.java:557)

at org.jboss.netty.handler.codec.frame.FrameDecoder.channelDisconnected(FrameDecoder.java:366)

at org.jboss.netty.channel.Channels.fireChannelDisconnected(Channels.java:399)

at org.jboss.netty.channel.socket.nio.AbstractNioWorker.close(AbstractNioWorker.java:716)

at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.handleAcceptedSocket(NioServerSocketPipelineSink.java:111)

at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.eventSunk(NioServerSocketPipelineSink.java:66)

at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:55)

at org.jboss.netty.channel.Channels.close(Channels.java:820)

at org.jboss.netty.channel.AbstractChannel.close(AbstractChannel.java:197)

at play.core.server.netty.RequestBodyHandler$$anon$1.exceptionCaught(RequestBodyHandler.scala:103)

at org.jboss.netty.handler.codec.frame.FrameDecoder.exceptionCaught(FrameDecoder.java:378)

at org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:533)

at org.jboss.netty.channel.AbstractChannelSink.exceptionCaught(AbstractChannelSink.java:49)

at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)

at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)

at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:84)

at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:472)

at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:333)

at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)

at java.lang.Thread.run(Thread.java:722)

Is it possible to configure Play to support file uploads for files > 2GB?

If so, I'd appreciate if anyone could point me into the right direction.

Cheers,

Florian

[Isaias Barroso]

Hi, 

I think you can try to use a custom Body Parser, see it https://github.com/mslinn/play21-file-upload-streaming

Best regards

[Florian Heinisch]

Thanks Isaias for your suggestion. I am already using a custom body parser. How ironic you are linking to Mike's repo which actually is a fork of my custom body parser example I put on Github:

https://github.com/heiflo/play21-file-upload-streaming

[James Roper]

Hi,

This looks like a Play 2.1 regression.  The bug appears to be this one in Netty:

https://github.com/netty/netty/issues/845

It's been fixed in Netty 3.6.1.  If you could raise a bug in GitHub, then I'll schedule this to be fixed in Play 2.1.2.

Cheers,

James

--
You received this message because you are subscribed to the Google Groups "play-framework" group.
To unsubscribe from this group and stop receiving emails from it, send an email to play-framewor...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
James Roper
Software Engineer

Typesafe - The software stack for applications that scale
Twitter: @jroper

[Florian Heinisch]

James,

thanks for taking care of this. I raised a bug in GitHub:

https://github.com/playframework/Play20/issues/884

Is there a planned release date for Play 2.1.2?

Cheers,

Florian

[James Roper]

On Thursday, March 21, 2013 10:42:48 AM UTC+11, Florian Heinisch wrote:

James,

thanks for taking care of this. I raised a bug in GitHub:

https://github.com/playframework/Play20/issues/884

Is there a planned release date for Play 2.1.2?

No. But you could try updating your dependencies to depend on Netty 3.6.1.  Not sure if it'll work, but there's no harm in trying.

[Isaias Barroso]

Hi Florian,

So your code is a great reference for other play developers :-)

Best regards