State of Ring CPS

[James Reeves]

As some may know, I've been experimenting with using continuation-passing-style to support asynchronous HTTP in Ring. An asynchronous handler accepts two arguments, the request, and a continuation function.

  (defn handler [request cont]

    (cont {:status 200, :headers {}, :body "Hello World"}))

The primary advantage of this approach is that it's backward compatible. We can support both the current synchronous design and the new asynchronous design at once:

  (defn handler

    ([request]

      {:status 200, :headers {}, :body "Hello World"})

    ([request cont]

     (cont {:status 200, :headers {}, :body "Hello World"})))

This is currently being developed in the cps branch of the Ring repository.

I've decided not to support non-blocking I/O for the body for the initial release of this feature. Instead the body will be written via a blocking OutputStream as before. While this is not ideal, most of the advantages of asynchronous HTTP remain.

The second problem is something I'd like to get feedback on, which is how to handle errors that occur outside the handler's thread. In the current system, we can simply throw an exception, but once we allow the response to be processed asynchronously, the error may occur outside the handler function's scope.

As I see it, there are a couple of possible solutions.

The first solution is to consider errors as just another response. We could return a response with a status code of 500, or return a raw exception instead of a map that can be later processed by middleware like wrap-stacktrace.

The second solution, and the one I'm currently favouring, is to pass a third argument to the handler that represents an error callback:

  (defn handler [request cont raise]

    (raise (Exception. "Something went wrong")))

The advantage of this approach is it provides a more straightforward way of short-circuiting response middleware. For instance, consider middleware that adds a header to the response:

  (defn wrap-foo [handler]

    ([request]

      (-> (handler request) (assoc-in [:headers "X-Foo"] "bar")))

    ([request cont]

      (handler request #(assoc-in % [:headers "X-Foo"] "bar"))))

If we need to check the value of the response is not an error, we need to do so for each middleware:

  (defn wrap-foo [handler]

    ([request]

      (-> (handler request) (assoc-in [:headers "X-Foo"] "bar")))

    ([request cont]

      (handler request #(if (map? %) (assoc-in % [:headers "X-Foo"] "bar") %))))

On the other hand, if we use an error callback, we can avoid repeating that logic in each wrapping of middleware:

  (defn wrap-foo [handler]

    ([request]

      (-> (handler request) (assoc-in [:headers "X-Foo"] "bar")))

    ([request cont raise]

      (handler request #(assoc-in % [:headers "X-Foo"] "bar") raise)))

I believe this approach also dovetails well with Manifold. Unless I'm very much mistaken, a function that returns a deferred from Manifold could be converted into a CPS handler just by passing the two callbacks to manifold.deferred/on-realized.

  (defn ->handler [deferred-handler]

    (fn [request cont raise]

      (let [deferred-response (deferred-handler request)]

        (d/on-realized deferred-response cont raise))))

I'd be very interested in hearing people's feedback, particularly regarding how to handle errors.

- James

[Timothy Baldridge]

It really seems that this design is hindered by the desire to remain backwards compatible, and perhaps also the attempt to keep Ring from acquiring new dependencies. While both of these are valid design decisions I can't help but think there is a better way. 

If you remove core.async, interceptors, or any sort of data-driven approach from the equation, then the only valid remaining option is cps. And callbacks are always going to present a sub-par user interface in a JVM based language where we lack tail-calls, delimited continuations or any sort of platform support for tracking call stacks across asynchronous suspensions. 

Having worked with async code quite a bit, I'd rather not use a library that gives me a call stack that terminates at a thread pool, I want to trace my web request call stack all the way back to the socket that initiated the request. I need introspection into a failed request...I can't get that with comp'ed functions, let alone callbacks. The original Ring model was a pain to debug at times, this design will be 10x worse. 

So that's my feedback. The same as it was last year. Stop using opaque functions, develop something higher level with decent introspection. Doesn't have to be interceptors, doesn't have to be core.async, but please don't let it be this. 

[kovas boguta]

On Mon, Jun 27, 2016 at 4:48 PM, Timothy Baldridge 

So that's my feedback. The same as it was last year. Stop using opaque functions, develop something higher level with decent introspection. Doesn't have to be interceptors, doesn't have to be core.async,

As a user who doesn't understand all the details / tradeoffs, this is a sentiment I can get behind. 

Since there seems to be renewed interest in making clojure's web story better, I'll throw in several cents here that are not particularly related to the async question, but are related to the above sentiment and my experience as a user, if anyone is reading this thread needs a datapoint :)

Me using ring:

1. Track down and load all the relevant handler-builders 

2. Compose them 

3. Painfully figure out what's actually happening on the inside until it works

The composition part is nice but all the effort is in steps 1 and 3. My middleware disappearing into an opaque blob means I have to manually tear apart my logic (or make sure I set it up a certain way in the first place) to figure out what's going on. Importing handlers from random libs means I never quite understand or trust what any individual one is doing. 

I like yada (despite the grief its gotten in some quarters), I don't care at all about 100% http compliance, but the bells-and-whistles-included, data-centric approach is easier for me.

Still its not ideal for me (yet). 

Ideal for me is: No painful tracking down of modules. Data-oriented descriptions decoupled from implementation. Obvious descriptions of inputs/outputs (clojure.spec?). Easy to sample behavior of any stage of transformation without making an effort to write my code upfront to be able to do that. 

[Dmitri Sotnikov]

I think the problem is that async fundamentally makes it more difficult to reason about what's happening. I think that one big benefit of this proposal is that async becomes an opt-in feature. Since you likely won't need it in majority of situations, you can avoid the problem altogether. On the other hand, using an an async all the way down approach incurs the mental overhead whether you need the async functionality or not.

[James Reeves]

On 27 June 2016 at 21:48, Timothy Baldridge <tbald...@gmail.com> wrote:

So that's my feedback. The same as it was last year. Stop using opaque functions, develop something higher level with decent introspection. Doesn't have to be interceptors, doesn't have to be core.async, but please don't let it be this. 

I recognise that the debugging asynchronous code on the JVM is not a pleasant experience, but what's the alternative?

My response to your argument is also the same as last year; if someone comes up with a design that fulfils your criteria and provides the same functionality as CPS functions, I'd be happy to consider it. In the meantime, should we stop work because someone might come up with a better solution down the line?

The problems you describe are ones that affects not just asynchronous Ring, but any function in Clojure that is executed outside its original scope. The issues with the callstack occur when trying to debug lazy seqs that throw an exception. I think it's outside the scope of a HTTP abstraction library to address problems in the language or in the underlying VM.

I also don't think this design precludes the use of better instrumentation. For example, you could wrap middleware in a tracing function that records the middleware being applied.

  (defn trace [middleware key]

    (fn [handler]

      (let [handler' (middleware handler)]

        (fn [request cont raise]

          (log [:enter key])

          (handler'

           request

           (fn [response] (log [:exit key]) (cont response))

           (fn [error] (log [:error key]) (raise error)))))))

Or instead of composing middleware directly, you could keep middleware in an ordered collection that could be dynamically inspected or altered.

While CPS introduces no new dependencies, it still integrates well with more sophisticated async solutions like Manifold and core.async. It's not inherently data driven, but it doesn't prevent the use of defining middleware via data, either. It's backward compatible with existing middleware, and we can support asynchronous and synchronous solutions simultaneously. If we want a better callstack, middleware can be wrapped to keep track of what middleware has been applied.

- James

[Paul deGrandis]

I also don't think this design precludes the use of better instrumentation. For example, you could wrap middleware in a tracing function that records the middleware being applied.

...

Or instead of composing middleware directly, you could keep middleware in an ordered collection that could be dynamically inspected or altered.

While I think additional wrapping only aggravates the issue, if you end up composing via a data structure, then you're just shy of the core interceptor chain in Pedestal.  The next logical jump one would make would be to program the composition against a protocol, and then you exactly have the interceptor chain.  No sense in duplicating that work now that the interceptor chain is a standalone module.

 

While CPS introduces no new dependencies, it still integrates well with more sophisticated async solutions like Manifold and core.async. It's not inherently data driven, but it doesn't prevent the use of defining middleware via data, either.

Without placing the CPS pieces within some well-defined container/record, there's nothing to which one could pin/ground a protocol.  Without a protocol in place, all data-described efforts will be ad-hoc.
 

 

It's backward compatible with existing middleware,

This is I think the strongest case for CPS...

 

and we can support asynchronous and synchronous solutions simultaneously. If we want a better callstack, middleware can be wrapped to keep track of what middleware has been applied.

Again, this is exactly what the interceptor chain does (both in terms of keeping track of middleware/interceptors executed and in terms of allowing both synchronous and asynchronous solutions at the same time).

As I said in the other thread, there is a world in which the API Ring exposes is CPS, but the execution of middleware happens on an interceptor chain.  While this wouldn't let Middleware's be data-described, it would create a foundation in data-description is first-class.
This would also address the issues around debugging and introspectability.  But would bring in additional dependencies (the chain itself, core.async).

Cheers,
Paul