Using Cowboy static file handler in Nitrogen module for dynamically generated files.

[Ramkrishna Kulkarni]

Hello,

I have a file which is dynamically generated upon user request. I need to handover this file to Cowboy static file handler in Nitrogen module as response. Can this be done?

I see a case for file in build_response function of cowboy_simple_bridge. Is there a way to specify the file in Nitrogen module so that this case is matched?

I'm looking for this solution because writing file contents as response in Nitrogen module leads into partial downloads especially on slow connections and large files.

Thanks.

[Jesse Gumm]

Hi Ram,

For the past few hours, I've been experimenting with the partial
download problem you've previously reported, and I cannot for the life
of me reproduce the early-terminating downloads (using throttling
software from another machine on the same network, and throttling it
at 5KB/s).

As a gut move, I'm going to try to convert the cowboy simple_bridge to
use a chunking response for larger responses, and see if that helps.

As for passing the buck to the cowboy static_handler, there is not a
way to do it directly from within a nitrogen-handled request, but
you've got two options here:

For both:

1) Add a "downloads/" to the static_paths in simple_bridge.config.
2) Save the generated file somewhere to the disk in "site/static/downloads"

Then you're two options are (assuming you saved your file as
"site/static/downloads/my_large_file.whatever")

A) wf:redirect("/downloads/my_large_file.whatever")
B) Return {file, "downloads/my_large_file.whatever"} from the main()
function in your module.

I'm going to spend the next half-hour or so experimenting with the
cowboy handler to see what I can come up. But it's quite odd to me
that the connection would die after 30 seconds.

-Jesse

> --
> You received this message because you are subscribed to the Google Groups
> "Nitrogen Project / The Nitrogen Web Framework for Erlang" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to nitrogenweb...@googlegroups.com.
> To post to this group, send email to nitro...@googlegroups.com.
> Visit this group at http://groups.google.com/group/nitrogenweb.
> For more options, visit https://groups.google.com/d/optout.



--
Jesse Gumm
Owner, Sigma Star Systems
414.940.4866 || sigma-star.com || @jessegumm

[Jesse Gumm]

Scratch that: (B), that doesn't work, so (A) remains the current real
suggestion.

I also have a potential fix I'll be posting in a few minutes..

-Jesse

[Jesse Gumm]

Alrighty,

Here's a simple proof-of-concept for enabling streaming returning from
a nitrogen page module:

If this works, this is *extremely* likely to be changed, but I'll let
you know when and in what form:

But use the following commits:

nitrogen_core (branch called: stream_test):
https://github.com/choptastic/nitrogen_core/commit/1c27833d257dd68c9a10ddda5d451bc7eb13e56e

simple_bridge (branch called: stream_cowboy):
https://github.com/choptastic/simple_bridge/commit/183cd7b0629d054a8b077bfeda93fa153e3696d3

Please let me know how these work for you. They worked for me, but
again, I was unable to reproduce the dying connections.

-Jesse

[Jesse Gumm]

Oh, I completely forgot this detail.

To use the above commits, have your main() function in your module
return the following:

cowboy_simple_bridge:stream_body(FullPathToYourFile).

So for example, to stream some file from my Downloads directory, I did:

main() ->
cowboy_simple_bridge:stream_body("/home/gumm/Downloads/file.bin").

And, of course, you can use the wf:header functions as you'd expect.

-Jesse

[Ramkrishna Kulkarni]

Hi Jesse,

Both these solutions work perfectly well. I chose the second one (code changes) as it allows download without leaving the page.

Really appreciate all the quick responses and solving my problems. 

Thanks a lot!

[Jesse Gumm]

Great.

So as I understand it, you pulled on my updated Nitrogen_core and simple_bridge and that fix with the streaming worked?

If so, that helps me out! Thanks!

--
Jesse Gumm
Owner, Sigma Star Systems
414.940.4866 || sigma-star.com || @jessegumm

[Ramkrishna Kulkarni]

Yes, exactly. I pulled those changes to nitrogen_core and simple_bridge. It worked really well.

[Jesse Gumm]

Great.

Like I said, I'll very likely be updating this interface a bit before
merging it into master, as adding a universal streaming option to
simple_bridge probably isn't a bad idea.

-Jesse

On Thu, Dec 10, 2015 at 12:32 AM, Ramkrishna Kulkarni