Processing large request bodies
684 views
Skip to first unread message
Nathan
Jul 1, 2015, 4:40:14 PM7/1/15
to ve...@googlegroups.com
Hello,
I'm looking into Vert.x 3 for a project that needs to support uploading large files (GBs) via a REST interface. I started with the highest level component, Vert.x-Web. I have a few questions regarding some limitations I've run into.
The payload will come via a POST or PUT. In the case of a POST, the file will be one attribute in a multipart form. I have set the BodyHandler handler on the route to receive the entire request body for now.
1) The payload should not be written to a local file system (no file uploads). I would like to stream the payload to a different system. BodyHandle buffers the entire body prior to routing handlers being called. Do I need to provide my own implementation of a steaming BodyHandler that would operate similar to Vert.x-Core's HttpServerRequest handlers?
2) I would like REST clients to use HTTP 100-Continue protocol to allow the server to verify the headers and authenticate the request before receiving the request body. I have not found where Vert.x-Web has this capability.
3) The multipart form attributes provided via HttpServerRequest::getFormAttribute() returns all attribute values as Strings regardless of the content type. For application/octet-stream types, the data gets corrupted due to the various character translations. Is it possible to have form attributes processed based on their content-type?
The Vert.x-Core component exposes lower level API's to control how HTTP requests are processed. Perhaps I would need to utilize that instead of Vert.x-Web. However, I did not see any server side processing for HTTP 100-Continue in Vert.x-Core.
Lastly, congrats to the team for getting v3 released. Nice work.
Nathan
Tim Fox
Jul 1, 2015, 4:55:42 PM7/1/15
to ve...@googlegroups.com
On 01/07/15 21:40, Nathan wrote:
Hello,
I'm looking into Vert.x 3 for a project that needs to support uploading large files (GBs) via a REST interface. I started with the highest level component, Vert.x-Web. I have a few questions regarding some limitations I've run into.
The payload will come via a POST or PUT. In the case of a POST, the file will be one attribute in a multipart form. I have set the BodyHandler handler on the route to receive the entire request body for now.
1) The payload should not be written to a local file system (no file uploads). I would like to stream the payload to a different system. BodyHandle buffers the entire body prior to routing handlers being called. Do I need to provide my own implementation of a steaming BodyHandler that would operate similar to Vert.x-Core's HttpServerRequest handlers?
Right, Bodyhandler is for the case where you want the body up front.
If you want to stream then just use the core API for file uploads
(of course this is accessible in Vert.x web too).
2) I would like REST clients to use HTTP 100-Continue protocol to allow the server to verify the headers and authenticate the request before receiving the request body. I have not found where Vert.x-Web has this capability.
You can enable _automatic_ 100-Continue handling by setting this
property
https://github.com/eclipse/vert.x/blob/master/src/main/java/io/vertx/core/http/impl/ServerConnection.java#L73
But this will simply send the interim response to continue immediately. There is no functionality right now verify the headers etc before sending the 100 continue. This would require a change in vertx-core I believe.
But this will simply send the interim response to continue immediately. There is no functionality right now verify the headers etc before sending the 100 continue. This would require a change in vertx-core I believe.
3) The multipart form attributes provided via HttpServerRequest::getFormAttribute() returns all attribute values as Strings regardless of the content type. For application/octet-stream types, the data gets corrupted due to the various character translations. Is it possible to have form attributes processed based on their content-type?
I don't think this is currently supported, but it's something we
could add.
Please feel free to add feature requests for these :)
Please feel free to add feature requests for these :)
The Vert.x-Core component exposes lower level API's to control how HTTP requests are processed. Perhaps I would need to utilize that instead of Vert.x-Web. However, I did not see any server side processing for HTTP 100-Continue in Vert.x-Core.
Lastly, congrats to the team for getting v3 released. Nice work.
Nathan
--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Nathan
Jul 1, 2015, 7:50:27 PM7/1/15
to ve...@googlegroups.com
On Wednesday, July 1, 2015 at 4:55:42 PM UTC-4, Tim Fox wrote:
Please feel free to add feature requests for these :)
Happy to. I'll add one for the server side 100-Continue support and one for honoring multipart form content-type in the request attributes. I assume request body streaming is working as designed, that is its a core feature.
Are feature requests managed with GitHub issues for the vertx-web repository?
Nathan
Tim Fox
Jul 2, 2015, 1:32:43 PM7/2/15
to ve...@googlegroups.com
Nathan
Jul 3, 2015, 2:26:37 PM7/3/15
to ve...@googlegroups.com
On Thursday, July 2, 2015 at 1:32:43 PM UTC-4, Tim Fox wrote:
Great! Thanks for the quick turn around. This will be a big help. I'll try to run some tests as soon as I can.
Nathan
Reply all
Reply to author
Forward
0 new messages