Best method for streaming bytes out to the servlet response.

[westaj]

I'm generating PDF's using a compojure route handler.

The PDF library I'm using uses an OutputStream to flush its content.
Since compojure builds the response upstream based on the return body
of the route handler, I don't have direct access to the servlet
response output stream. I can't return a string because the header
gets stamped with html/text as the content type.

Is there a better way of doing the following?

(defn gen-pdf-response
[request title col-titles server-result]
(compojure.http.response/create-response request
(let [doc (new Document)
stream (new
ByteArrayOutputStream)
writer (. PdfWriter
(getInstance doc stream))
]
(do
(. doc open)
(. doc (add (new
Paragraph title)))
(. doc (add (new
Paragraph "")))
(. doc (add (gen-pdf-
table (server-result :results) col-titles)))
(. doc close)
(new
ByteArrayInputStream (. stream toByteArray))))))

Cheers,

Aaron...

[Luke Renn]

On Aug 2, 8:16 pm, westaj <wes...@e5networks.com> wrote:
> Is there a better way of doing the following?
>

> ...

You could have still set the content type even if you returned a
String, but in this case I don't think you want to.

How about something like:

http://gist.github.com/160995

(import 'java.io.ByteArrayOutputStream)
(import 'java.io.ByteArrayInputStream)
(import 'com.lowagie.text.Document)
(import 'com.lowagie.text.DocumentException)
(import 'com.lowagie.text.Paragraph)
(import 'com.lowagie.text.pdf.PdfWriter)

(defn gen-pdf-stream [title]
(let [doc (new Document)
stream (ByteArrayOutputStream.)
writer (PdfWriter/getInstance doc stream)]
(.open doc)
(.add doc (Paragraph. title))
(.close doc)
(ByteArrayInputStream. (.toByteArray stream))))


(defroutes example-app
(GET "/pdf" [{:headers {"Content-Type" "application/pdf"}}
(gen-pdf-stream "Lorum Ipsum")]))

(run-server {:port 8080} "/*" (servlet example-app))

Requires iText in the classpath. Note that even if you return a
String the content-type is still set to PDF (but you don't want to do
that).

Luke

[Vagif Verdi]

On Aug 3, 6:49 pm, Luke Renn <luke.r...@gmail.com> wrote:
> (import 'java.io.ByteArrayOutputStream)
> (import 'java.io.ByteArrayInputStream)
> (import 'com.lowagie.text.Document)
> (import 'com.lowagie.text.DocumentException)
> (import 'com.lowagie.text.Paragraph)
> (import 'com.lowagie.text.pdf.PdfWriter)
>
> (defn gen-pdf-stream [title]
>   (let [doc (new Document)
>         stream (ByteArrayOutputStream.)
>         writer (PdfWriter/getInstance doc stream)]
>     (.open doc)
>     (.add doc (Paragraph. title))
>     (.close doc)
>     (ByteArrayInputStream. (.toByteArray stream))))
>
> (defroutes example-app
>   (GET "/pdf" [{:headers {"Content-Type" "application/pdf"}}
>                (gen-pdf-stream "Lorum Ipsum")]))
>
> (run-server {:port 8080} "/*" (servlet example-app))
>

I do exactly the same thing with pdfs and zip archives created on the
fly. The only difference is, i use

(import [org.apache.commons.io.output ByteArrayOutputStream])

I do not even know why. I just read on the commons API documentation
that their implementation of ByteArrayOutputStream is supposedly "more
robust".

Having said that i have a question.

It looks to me that

(ByteArrayInputStream. (.toByteArray stream))

is a bottleneck, especialy for big multi-megabyte files.

(.toByteArray stream) copies the data into new object, doubling memory
usage, and possibly taking some time in case of big files.

Is there a library that exposes InputStream and OutputStream
interfaces on the same object, sharing the internal buffer, so no
unnecessary copying happens ?

[Korny Sietsma]

I don't know about the clojure side (I'm still a learner, and one with
no spare time either) - but I can highly recommend the Jakarta
commons-io libraries for this sort of thing.
Mostly I use FileUtils for buffered file copies, but for your example
you probably want IOUtils:
http://commons.apache.org/io/api-release/org/apache/commons/io/IOUtils.html

Check out the copy(InputStream, OutputStream) method:
http://commons.apache.org/io/api-release/org/apache/commons/io/IOUtils.html#copy%28java.io.InputStream,%20java.io.OutputStream%29
it should do precisely what you want, via an internal buffer.

- Korny (always vaguely astonished that this sort of thing isn't
anywhere in the core Java libraries...)

--
Kornelis Sietsma korny at my surname dot com
kornys on twitter/fb / korny on wave sandbox
"Every jumbled pile of person has a thinking part
that wonders what the part that isn't thinking
isn't thinking of"

[Vagif Verdi]

On Aug 4, 12:01 am, Korny Sietsma <ko...@sietsma.com> wrote:
> Check out the copy(InputStream, OutputStream) method:http://commons.apache.org/io/api-release/org/apache/commons/io/IOUtil...

> it should do precisely what you want, via an internal buffer.

Unfortunately it copies from InputStream to OutputStream. I need from
OutputStream to InputStream.

Luckily i figured out how to use PipedStreams for that:

(defn #^PipedInputStream create-pdf-stream [pdf-info]
(let [in-stream (new PipedInputStream)
out-stream (PipedOutputStream. in-stream)
thread-fn (Thread. (fn [] (; Here you write into out-stream
)))]
(.start thread-fn)
in-stream))