twitter example (async) missing

[Jim Klucar]

Hello everyone,

I'm new to Aleph and Lamina. I'm writing my own HTTP/1.1 server to stream data using the chunked transfer encoding, just like Twitter's streaming API does. More succinctly, I'd like to asynchronously stream a lazy infinite sequence to a client. To my delight I found the Twitter example on the wiki but it doesn't work with aleph-0.3.0-alpha2 because the async command isn't found. This seems to no longer be part of Lamina, so what should be done instead of the async command? I searched some discussions on here but couldn't find why it was removed or what the new proper way to asynchronously stream a sequence using aleph.

Any feedback is greatly appreciated!

Jim

[Andrew Cholakian]

Hey Jim, you don't need the async macro at all. If you look here ( https://github.com/ztellman/aleph/wiki/HTTP ), you'll notice see that you basically return a ring response with a lamina channel as a body. Enqueuing messages on this channel chunks the response if it's plain HTTP, or sends a message if it's a websocket.

You can also try out noir-async, which I'm the author of. It merges noir and lamina together into a small concise DSL, letting you do this very thing in only a few lines of code. You can find it at: https://github.com/andrewvc/noir-async

[Jim Klucar]

Andrew,

Thanks for the info. Looks like your noir-async was what I'm after. I followed your example with (async-push "chunk one") but it's throwing an error from inside the (close conn) call:

clojure.lang.ArityException: Wrong number of args (1) passed to: core$async-push

at clojure.lang.AFn.throwArity(AFn.java:437)

at clojure.lang.AFn.invoke(AFn.java:39)

at noir_async.core$close.invoke(core.clj:85)

at data_spew.views.spew$fn__11465$na_aleph_handle_inner__11468.invoke(spew.clj:8)

at aleph.http$wrap_aleph_handler$fn__9691.invoke(http.clj:71)

at compojure.response$fn__10118.invoke(response.clj:26)

at compojure.response$fn__10095$G__10090__10102.invoke(response.clj:10)

at data_spew.views.spew$fn__11465.invoke(spew.clj:6)

at compojure.core$if_route$fn__10197.invoke(core.clj:39)

at compojure.core$if_method$fn__10190.invoke(core.clj:24)

at compojure.core$routing$fn__10212.invoke(core.clj:98)

at clojure.core$some.invoke(core.clj:2388)

at compojure.core$routing.doInvoke(core.clj:98)

at clojure.lang.RestFn.applyTo(RestFn.java:139)

at clojure.core$apply.invoke(core.clj:602)

at compojure.core$routes$fn__10216.invoke(core.clj:103)

at noir.server.handler$init_routes$fn__11376.invoke(handler.clj:93)

at noir.request$wrap_request_map$fn__11267.invoke(request.clj:14)

at hiccup.middleware$wrap_base_url$fn__11027.invoke(middleware.clj:12)

at noir.session$noir_flash$fn__11292.invoke(session.clj:104) ....

Looking at the close function, it calls async-push without a conn, but that's just my dumb analysis.

[Jim Klucar]

To follow-up on my own response, I was able to do what I wanted, and close the connection with this:

(defpage-async "/spew" [] conn 

               (async-push conn {:status 200                                                                                                                                                    

                                 :headers {"Content-Type" "application/json"}

                                 :chunked true } )

               (async-push conn "chunk one")

               (async-push conn "chunk two")

               (async-push conn "") ; seems like this closes the connection

                                    ; which makes sense, a chunked connection is

                                    ; finished when it passes a 0 for the size

               ) 

not sure what state that leaves the lamina channel in.

[Andrew Cholakian]

Just to update the thread, I pushed a fix for this as noir-async 1.1.0-beta2