Possible approaches to download a medium-sized file with Dispatch 0.11.0

[Matteo De Martino]

Hi all

I am using Dispatch reboot (0.11.0) to download several videos from an URL.
I managed already to download the file "as a whole", but what I would like to be able to do is to download the file "chunk-by-chunk", tracking the progress of the download.

I found the following syntax which I could potentially use:

Http(url(<url> + i) OK as.Bytes)
     .map{b => 
          /* save bytes in buffered fileoutputstream */
     }

Only, it feels very convoluted and unidiomatic.
Also, I am not sure it would actually work as I expect: would it call the "map" multiple times, every N bytes? Or only once, at the end of the download (in which case, it would be useless to me)?

Can anybody suggest a better approach?
How would you download a file of, say, 20-50MB?

Thanks

[Nathan]

Right, loading the full response body into memory as a byte array is not
going to be good for larger entities.

You might want to take a look at the Strings AsyncHandler:
https://github.com/dispatch/reboot/blob/master/core/src/main/scala/stream/strings.scala

And how it's used to make the as.stream.Lines verb:
https://github.com/dispatch/reboot/blob/master/core/src/main/scala/as/stream/lines.scala

Since you're dealing with binary data you can't use these as a
foundation, but you could implement a similar extension of AsyncHandler.

Nathan

On 03/21/2014 01:35 PM, Matteo De Martino wrote:
> Hi all
>
> I am using Dispatch reboot (0.11.0) to download several videos from an URL.
> I managed already to download the file "as a whole", but what I would
> like to be able to do is to download the file "chunk-by-chunk",

> *tracking the progress* of the download.

>
> I found the following syntax which I could potentially use:
>
> |Http(url(<url> + i) OK as.Bytes)
> .map{b=>
> /* save bytes in buffered fileoutputstream */
> }|
>
> Only, it feels very convoluted and unidiomatic.
> Also, I am not sure it would actually work as I expect: would it call
> the "map" multiple times, every N bytes? Or only once, at the end of the
> download (in which case, it would be useless to me)?
>
> Can anybody suggest a better approach?
> How would you download a file of, say, 20-50MB?
>
> Thanks
>

> --
> You received this message because you are subscribed to the Google
> Groups "Dispatch" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to dispatch-scal...@googlegroups.com
> <mailto:dispatch-scal...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.