Large File Transfer Help

[Mathew Tasalloti]

Hi Christoph and team,

I'm trying to use Fusio to relay large video file streams (1GB) from our internal API server to external clients.   I'm using a simple HTTP Get to initiate this download and it seems fine for smaller files.  However, for real large files the server is coming back with a 500 code.

All the logs seem fine but I just can't figure out a way around this.

Are there any settings that I can set to just let the stream through until completion?   Also, is there a better way to do this rather than the HTTP Processor method I'm currently using?

Many thanks for any tips!

Mathew

[Christoph Kappestein]

Hi Mathew,

could you show me the 500 error which gets returned in this case, I would assume that it is probably something like an out of memory error?

A better way to serve such large files would be probably to move those files to a storage service i.e. like AWS S3 and then you pass only the bucket url of the file to Fusio.

Maybe it would be also possible to move the storage service process into Fusio, so that Fusio gets the file from the API and uploads it and then returns the url to your user,

but for this you would probably need to build a custom action.

best regards
Christoph

[Mathew Tasalloti]

Hi Christoph,

This is perfect and thank you for your guidance on this.  

We are definitely going the way you've suggested now and it makes too much sense and works perfectly now.

Thank you for your insight and recommendation and I just wanted to close the circle back and send you a big thanks !

Best Regards,

Mathew