Long running requests flushing response periodically

[Vagif Verdi]

I do not see how can i implement it in compojure.
I need to notify a user on some long running process progress.
It is easy to do in java servlet. You just send chunks of text via
response and flush them. So the user sees messages apearing on the
screen.

Process File 1...
Process File 2...
Process File 3...
Creating a Zip file...

And then finally he gets the download popup.

How do i pull this off in compojure ?

[James Reeves]

On Jul 9, 7:30 pm, Vagif Verdi <Vagif.Ve...@gmail.com> wrote:
> I do not see how can i implement it in compojure.
> I need to notify a user on some long running process progress.
> It is easy to do in java servlet. You just send chunks of text via
> response and flush them.
>

> How do i pull this off in compojure ?

Hi Vagif,

Sorry for the late reply, but I got back home too late yesterday to go
through my emails.

To do this in Compojure, you can use a lazy seq. I've just pushed a
change to the Compojure output that will flush the response stream
after each item in the seq, so you'll need to pull the latest commit
from github for this to work.

Here's a quick example of how you'd use streaming:

(use 'compojure)

(defn repeat-with-delay [x delay]
(repeatedly
#(do (Thread/sleep delay) x)))

(defroutes site
(GET "/:message"
[(content-type "text/plain")
(repeat-with-delay (params :message) 1000)]))

(run-server {:port 8080}
"/*" (servlet site))

If you go to http://localhost:8080/baa then you should get a
continuous stream of "baa"s.

- James

[Vagif Verdi]

On Jul 10, 10:12 am, James Reeves <weavejes...@googlemail.com> wrote:

> To do this in Compojure, you can use a lazy seq. I've just pushed a
> change to the Compojure output that will flush the response stream
> after each item in the seq, so you'll need to pull the latest commit
> from github for this to work.
>

Great work! Thx!

[Mark Triggs]

James Reeves <weave...@googlemail.com> writes:

> On Jul 9, 7:30 pm, Vagif Verdi <Vagif.Ve...@gmail.com> wrote:
>> I do not see how can i implement it in compojure.
>> I need to notify a user on some long running process progress.
>> It is easy to do in java servlet. You just send chunks of text via
>> response and flush them.
>>
>> How do i pull this off in compojure ?
>

[...]

> To do this in Compojure, you can use a lazy seq. I've just pushed a
> change to the Compojure output that will flush the response stream
> after each item in the seq, so you'll need to pull the latest commit
> from github for this to work.

That works well for me too. I just thought I'd mention that I was
thinking about this issue a week or so ago and wondered if returning
an InputStream could be made to do the same thing. Modifying your
example slightly, I initially tried something like:

(defn repeat-with-delay [x delay]
(let [out (PipedOutputStream.)
in (PipedInputStream. out)
wr (PrintWriter. out)]
(.start (Thread. #(while true
(.println wr x)
(.flush wr)
(Thread/sleep delay))))
in))

But found this didn't work because the IOUtils/copy method buffers its
input stream. I had to modify servlet.clj to use unbuffered IO and
flush after every character instead:

(instance? InputStream body)
(with-open [out (.getOutputStream response)]
(doseq [ch (take-while #(not= % -1)
(repeatedly #(.read body)))]
(.write out ch)
(.flush out))
(.close body))

Maybe this sort of behaviour with InputStreams isn't useful anyway,
but I just thought I'd chime in with the thought.

Cheers,

Mark