Vert.x 2.1 File Upload - how to limit upload size on server side?

[Jarkko Viinamäki]

Let's suppose we have a vert.x HTTP server with the following handler:

// handles HTTP POST of multipart/form-data html form with input type="file"

public void handleRequest(final HttpServerRequest req)

{

req.expectMultiPart(true);

req.uploadHandler(new Handler<HttpServerFileUpload>() {

@Override

public void handle(final HttpServerFileUpload upload) {

long size = upload.size(); // always returns 0 

upload.exceptionHandler(new Handler<Throwable>() {

@Override

public void handle(Throwable event) {

req.response().end("Upload failed");

}

});

upload.endHandler(new Handler<Void>() {

@Override

public void handle(Void event) {

req.response().end("Upload successful, you should see the file in the server directory");

}

});

upload.streamToFileSystem("upload/" + upload.filename());

}

});

}

Questions:

1. How do I set the maximum size for the uploaded file on the server side?

Note that we cannot trust "Content-Length" header sent by the client since that value can be forged. Also it seems that upload.size() always returns 0 (although this bug has been marked as fixed). We also cannot first stream the whole file to the disk since some evil hacker might use a script that just sends an endless stream of data to fill the disk. So we need to detect "on-the-fly" when X number of bytes have been received and if more is received, upload is interrupted and possibly an error message is sent to the client.

I took a look at https://github.com/eclipse/vert.x/blob/master/vertx-core/src/main/java/io/vertx/core/streams/Pump.java but wasn't able to figure out how to create a custom Pump (confused about those lambdas) which interrupts the pumping and gracefully throws exception/stops the upload so that it can be handled by the HttpServerFileUpload handler or some other way.

What would be the best way to implement such upload limit?

2. If I need to dynamically create a directory for the file to be uploaded before the actual upload happens, how should it be created?

Within "public void handle(final HttpServerFileUpload upload)" I tried to do something like:

_verticle.getVertx().fileSystem().mkdir(dir, new Handler<AsyncResult<Void>> () {

@Override

public void handle(AsyncResult<Void> arg0) {

if( arg0.succeeded() )

{

upload.streamToFileSystem(filepath + upload.filename());

}

}

});

but the size of the uploaded file was always 0. Traditional new java.io.File(dir).mkdirs() works but I guess that's blocking IO and should be avoided? Then again, isn't the "cost" of creating a directory always the same and operation is atomic (cannot be sliced) so it doesn't really matter even if we use java.io.File(dir).mkdirs()?

3. What happens to verticle event loop performance if there are e.g. 50 clients each concurrently uploading a file of size 200 MB? Does it slow down other http requests significantly? In other words, can vert.x upload handler slice the upload events so that uploads are processed incrementally and other incoming requests get a chance to get processed before uploads are complete?

Thanks in advance for any tips you can offer!

BR,

Jarkko

[Paulo Lopes]

Hello,

please look at Yoke BodyParser source code:

https://github.com/pmlopes/yoke/blob/develop/framework/src/main/java/com/jetdrone/vertx/yoke/middleware/BodyParser.java

Yoke does support limiting the upload sizes, the vert.x specific code you might be interested in is here:

https://github.com/pmlopes/yoke/blob/develop/framework/src/main/java/com/jetdrone/vertx/yoke/middleware/BodyParser.java#L122-L147

Cheers,
Paulo