Direct access to response.outputStream?
Jeppe Nejsum Madsen
I need to create a response that downloads a file. The file is
generated by a 3rd party api that takes an output stream as the target
for the file. As the file can be rather large, I would like to stream
this file directly to the client.
I've looked at StreamingResponse, but this seem to require something
akin to an input stream.
As there a way to generate a response with direct access to the output stream?
/Jeppe
Marius
No. IMO this would bring serious problems related with "committed
response" if people will start writing directly into servlet's
response output stream. But you should be able to bridge the
outputstream needed by that library and the "inputstream" from the
StreamingResponse ( which takes a structural type not really an
InputStream) through mechanisms similar with Pipes. You could also
build your own OutputStream that also has def read(buf: Array[Byte]):
Int. Thus your library will write stuff in your OutputStream, you
would then buffer the data and wait for that data to be drained by the
servlet's input stream. A simple producer/consumer approach.
>
> /Jeppe
Jeppe Nejsum Madsen
> On Feb 26, 11:50 am, Jeppe Nejsum Madsen <je...@ingolfs.dk> wrote:
>> Hi,
>>
>> I need to create a response that downloads a file. The file is
>> generated by a 3rd party api that takes an output stream as the target
>> for the file. As the file can be rather large, I would like to stream
>> this file directly to the client.
>>
>> I've looked at StreamingResponse, but this seem to require something
>> akin to an input stream.
>>
>> As there a way to generate a response with direct access to the output stream?
>
>
> No. IMO this would bring serious problems related with "committed
> response" if people will start writing directly into servlet's
> response output stream.
Normally, yes. But I think there are special cases that may warrant this
and I tend to think this is one of them :-)
> But you should be able to bridge the outputstream needed by that
> library and the "inputstream" from the StreamingResponse ( which takes
> a structural type not really an InputStream) through mechanisms
> similar with Pipes.
But unless I create a new thread, I'll still end up with the entire file
in memory. Ie the call to the library's write method must return before
the streaming response can be created and thus the file contents must
be stored somewhere.
> You could also build your own OutputStream that also has def read(buf:
> Array[Byte]): Int. Thus your library will write stuff in your
> OutputStream, you would then buffer the data and wait for that data to
> be drained by the servlet's input stream. A simple producer/consumer
> approach.
If I create a new thread I could probably use PipedOutputStream which
does this already. But still the file contents will be copied twice: 1)
From pipe output to pipe input, 2) from pipe input to response output
So the thread creation/scheduling and the two times file copying could be
avoided by writing directly to the output stream. But I agree this would
have to be special cased somehow. It's not an urgent issue atm, but I'll
try to see if some clean solution can be implemented.
/Jeppe
Jeppe Nejsum Madsen
> So the thread creation/scheduling and the two times file copying could be
> avoided by writing directly to the output stream. But I agree this would
> have to be special cased somehow. It's not an urgent issue atm, but I'll
> try to see if some clean solution can be implemented.
An idea just struck :-) Could this be handled in much the same way as
a redirect? Something like
S.sendFile("application/pdf", "myfilename.pdf", outputStream =>
mylibrary.write(outputStream))
this would throw an exception like ResponseShortcutException, lift
would intercept it, write the appropriate headers and execute the
passed function with the response outputstream as parameter.
This would prohibit accidental output to response.
Thoughts?
/Jeppe
Marius
I would prefer something like :
final case class OutputStreamingResponse(data: (OutputStream) => Unit,
size: Long, headers: List[(String, String)], cookies:
List[HTTPCookie], code: Int) extends BasicResponse {
..
}
hence remain consistent with Lift's response paradigm.
We *COULD* provide the OutputStrem from the servlet response but that
would be ok since we are in a LiftResponse and NOT inside the
rendering pipeline.
>
> /Jeppe
Jeppe Nejsum Madsen
Agreed, a cleaner solution. And this could also be used outside of a
stateful response. The size should probably be Box[Long] as it may be
unknown.
Should I create a ticket for this?
/Jeppe
Marius
On Feb 26, 3:05 pm, Jeppe Nejsum Madsen <je...@ingolfs.dk> wrote:
Sure it can be stateless since at the end of the day it is just a
LiftResponse.
> The size should probably be Box[Long] as it may be
> unknown.
>
> Should I create a ticket for this?
Sure, but I'm not sure who/when will take care of this. We'll see.
>
> /Jeppe
Marius
Jeppe Nejsum Madsen
> I just added this in master.
Cool, thanks! I noticed the discussion on the committers list wrt
Content-Length. Is this header added if I don't specify it in the
"headers" parameter?
/Jeppe
Marius
On Apr 14, 11:26 am, Jeppe Nejsum Madsen <je...@ingolfs.dk> wrote:
> On Tue, Apr 13, 2010 at 6:59 PM, Marius <marius.dan...@gmail.com> wrote:
> > I just added this in master.
>
> Cool, thanks! I noticed the discussion on the committers list wrt
> Content-Length. Is this header added if I don't specify it in the
> "headers" parameter?
Yes if you don't specify it in the companion, then -1 is assumed, but
Lift will put Content-Length only for values >= 0. Thus you can safely
use HTTP chunking when you write to the output stream.
>
> /Jeppe