Large file downloads fail

[Kevin Bowrin]

Has anyone else experienced a problem when downloading large files?

We have a 3 GB file which, when downloaded, stops downloading at a random number of bytes after 400 MB

Downloading the file directly from Fedora works fine.

Can't reproduce on Nurax, since the file upload size limit is too low.

No obvious errors in Hyrax application logs.

I'm going to try and get to the bottom of it. Some combination of wireshark, ruby debugging, and nginx debugging. Wanted to ask before I go down that rabbit hole.

[Kevin Bowrin]

In our puma+nginx production deployment, a combination of increasing the puma worker_timeout (might not be necessary) and

proxy_http_version 1.1;

proxy_buffering off;

In nginx config seemed to fix the issue. proxy_buffering might also not be necessary.

[Jeremy Friesen]

We talked about this in today's Samvera Tech call.

One thought we had was to look at the `client_max_body_size` setting in Nginx.

[Mike Korcynski]

We had a similar problem as well, we never quite got to the root cause.  Our infrastructure team was reluctant to pull out wireshark to track it down because we'd found a workaround.  We had a load balancer in front of Fedora even though we only have a single fedora node to do ssl offloading and so infrastructure would manage the cert for us rather than having the cert on the server.  

In our case, the large download directly from Fedora through the load balanced service name would work fine through browser and curl but when fronted with Hyrax it would regularly fail when the load balanced service name was in confg/fedora.yml.  We've now configured Hyrax's fedora.yml to use the hostname not the load balanced service name bypassing the load balancer and have not had the problem reoccur since.  Note we are using apache not nginx.

Mike

(Tufts University)

--
You received this message because you are subscribed to the Google Groups "samvera-tech" group.
To unsubscribe from this group and stop receiving emails from it, send an email to samvera-tech...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/samvera-tech/d66a1b3d-087e-45c3-8158-a8f762492d5cn%40googlegroups.com.