Vert.x Streaming file directly From http request to object Store(Openstack Swift)

[mhima...@gmail.com]

Hi,

I have specific requirement to stream file from http request directly to object store(OpenStack swift) on Vert.x

Has anyone tried this or any ideas how I can implement this??

[Alexander Lehmann]

Vert.x will send you chunks of the http body data via the data handler, so you will receive the data as it is being send, if you can write that to the store in chunks, that should be easy to implement (haven't used swift yet, so I'm not sure if there is a vert.x module for that).

[mhima...@gmail.com]

Thanks for your reply.

The problem is swift doesn't allow file to be uploaded in parts.

[Tim Fox]

I'm not sure I really understand the question, but as you receive the body just pass it on to "object store(OpenStack swift)" (whatever that is)

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Alexander Lehmann]

does this mean you have to buffer the complete request body in memory before sending it to the store api?

That can be done easily by using RxJava since the map function you can use is just buffer.append in that case. This will give you a Buffer with the complete post body, which should be ok for files up to a few megs, but it's note really streaming of course

[Tim Fox]

On 22/07/14 12:06, Alexander Lehmann wrote:

does this mean you have to buffer the complete request body in memory before sending it to the store api?

That can be done easily by using RxJava since the map function you can use is just buffer.append in that case.

or just use bodyHandler ;)

This will give you a Buffer with the complete post body, which should be ok for files up to a few megs, but it's note really streaming of course

+1

On Tuesday, July 22, 2014 2:44:53 AM UTC+2, mhima...@gmail.com wrote:

Thanks for your reply.

The problem is swift doesn't allow file to be uploaded in parts.

On Sunday, July 20, 2014 6:21:49 AM UTC-4, Alexander Lehmann wrote:

Vert.x will send you chunks of the http body data via the data handler, so you will receive the data as it is being send, if you can write that to the store in chunks, that should be easy to implement (haven't used swift yet, so I'm not sure if there is a vert.x module for that).


On Sunday, July 20, 2014 2:52:08 AM UTC+2, mhima...@gmail.com wrote:

Hi,

I have specific requirement to stream file from http request directly to object store(OpenStack swift) on Vert.x

Has anyone tried this or any ideas how I can implement this??

--

[Alexander Lehmann]

While on the subject of uploads, does vert.x have a multipart/form-data decoder for POST requests?

[Alexander Lehmann]

Ok, after looking up the api documentation for swift, the create operation is a PUT request that supports chunked encoding. This means that you can pass the buffer chunks you get via the data handler from your http request handler to the HttpClientRequest you use to write the object to your storage, without doing any length calculation (assuming you have a PUT request as well), if the request has a Content-Length field, you can use that and turn off chunked encoding.

If you are processing a POST request (multipart/form-data), you will have to do the parsing of the individual parts and extract the data part you need and pass these chunks to the PUT operation. Not sure if there is a parser available for this yes.

On Tuesday, July 22, 2014 1:18:05 PM UTC+2, Tim Fox wrote: