Proposal: Asynchronous Ring with Continuations

[James Reeves]

Hello all,

I have a proposal for extending Ring with support for asynchronous I/O, and I'm interested in getting feedback on it.

If I haven't made any glaring mistakes, this proposal will first begin life as an experimental library (Ring-CPS), with the eventual goal of incorporating it into Ring core once it's been sufficiently tested in real world environments. Ring will remain backward compatible even with the changes.

Overview

The central idea is to use continuation-passing style (CPS) to add a two-argument arity to handlers. So currently in Ring we might write:

  (defn app [req]

    {:status 200, :headers {}, :body "foo"})

In CPS, we'd write:

  (defn app [req cont]

    (cont {:status 200, :headers {}, :body "foo"}))

Middleware can also be adapted to this style. In Ring currently:

  (defn wrap-foo [handler foo]

    (fn [req]

      (assoc-in (handler req) [:headers "X-Foo"] foo)))

In CPS:

  (defn wrap-foo [handler foo]

    (fn [req cont]

      (handler req (fn [resp] (assoc-in resp [:headers "X-Foo"] foo)))))

By dispatching on arity, we can write handlers and middleware that transparently support both a synchronous and asynchronous approach:

  (defn wrap-foo [handler foo]

    (fn

      ([req]

       (assoc-in (handler req) [:headers "X-Foo"] foo))

      ([req cont]

       (handler req (fn [resp] (assoc-in resp [:headers "X-Foo"] foo))))))

Ring-CPS will need also to differ slightly in terms of its request and response maps.

In Ring, the request body is a InputStream object, which blocks on read. In Ring-CPS, it makes sense to use the non-blocking ReadableByteChannel instead.

The Ring response body can currently be a String, ISeq, File or InputStream. These hard-coded values made sense when Ring was first designed, but now we have protocols, so we can instead write:

  (defprotocol ChannelWritable

    (write-channel [x ^WritableByteChannel ch]))

Types such as strings, byte arrays and so forth can be extended with this protocol. More complex streaming can be achieves by reifying the protocol and writing custom code to push to the channel. The user will be responsible for closing the channel when they are done.

This also suggests an equivalent feature for core Ring, which may be added to Ring 1.5.0:

  (defprotocol StreamWritable

    (write-stream [x ^OutputStream st]))

Advantages

It's simple.

It doesn't depend on any external libraries (e.g. core.async, manifold, etc.).

But it can be used with external libraries (e.g. you can extend core.async's ManyToManyChannel with ChannelWritable).

The same middleware can transparently have synchronous and asynchronous versions.

Middleware and handlers can be programmatically upgraded to have a two-arity version for compatibility.

The request body can be read asynchronously.

The response can be sent asynchronously.

The response body can be streamed asynchronously.

Disadvantages

Headers cannot be streamed.

If a middleware function or handler without a two-arity dispatch is used with an async adapter, it will raise an exception.

No opinions on how to asynchronously handle errors.

Doesn't include WebSockets, just HTTP.

Feedback

I'll be glad to answer any questions, or receive any feedback or criticism of this approach.

- James

[James Reeves]

A small correction to the CPS middleware examples (I had forgotten to call cont):

  (defn wrap-foo [handler foo]

    (fn [req cont]

      (handler req (fn [resp] (cont (assoc-in resp [:headers "X-Foo"] foo))))))

  (defn wrap-foo [handler foo]

    (fn

      ([req]

       (assoc-in (handler req) [:headers "X-Foo"] foo))

      ([req cont]

       (handler req (fn [resp] (cont (assoc-in resp [:headers "X-Foo"] foo)))))))

- James

[Arnout Roemers]

Nice that discussion is (re)started. For completeness sake, this pull request [1] reactivated this discussion, and discusses above CPS approach and alternative "reactor" approach. There is also an older discussion from 2010 about this. [2]

[1] https://github.com/ring-clojure/ring/pull/206

[2] https://groups.google.com/forum/#!topic/ring-clojure/JD9FLJFTVsg

[Timothy Baldridge]

I've recently started to view CPS as a poor substitute for another concept, something I call "reified call stacks". By this I mean a call stack that is no longer an abstract invisible concept in a program, but instead is a very concrete object that can be manipulated and queried by a program, just as any other data structure would be. 

So let's say we were creating a HTTP server and wanted to see what the call stack would look like, we would end up seeing it as something like this:

Servlet -> Header Parser -> Param Parser -> Handler

But of course the handler has to return something so our diagram is incorrect, as after we get to the handler, the stack unwinds back up:

Servlet -> Header Parser -> Param Parser -> Handler

Servlet <- Header Parser <- Param Parser <-   

So this is what I think about when I think of a reified call stack. Now, I would like to say "hey, when you get to Param Parser, hand me back the rest of the stack, I'll finish calling that later". So in a way, that starts to look a little like delimited continuations.

To me, this is the holy grail. I could later, decide to go and modify the stack I've been handed. I could call it later on a different thread, or not...it doesn't matter. 

So how do we model this in Clojure? Well we could quite easily model it as a vector of functions, perhaps even un-wrapping the two parts of the functions:

[Servlet-enter HeaderParser-enter ParamParser-enter Handler ParamParser-exit HeaderParser-exit Servlet-exit]

We could then create a simple interpreter that walks this data structure executing each function along the way. A normal execution then becomes a sort of reduce:

(reduce 
   (fn [ctx f]

      (f ctx))
   (new-context)
  stack)

It's fast and efficient. However we can still easily terminate the reduce at any time, and resume it whenever wherever we want, as long as we have a idx in the stack, the stack, and the context. Going async is then as simple as wrapping up those three parameters into a function that can be handed to another thread. In essence you can turn any stack and index into a continuation. 

Two awesome features of this approach are that implementation is fairly simple. Handlers now take a context, and return a context. The runner becomes a reduce over a list of context manipulating functions. At any time, part of your application could completely modify the stack, inserting new items, removing items, creating trees of handlers, etc. The power comes from the reification of the call stack. You're no longer locked into a single calling model, because the stack can be modified as needed by the application. 

Secondly, debugging just got a lot easier. Now you can do things like get a trace of all contexts up till the point of an exception, You can step through the stack step-by-step. No longer are the guts of your handlers hidden inside anonymous functions where you can't get at the state hidden there. No longer is a continuation an opaque thing. A continuation is a stack, a pointer into that stack, and a context, all of those are just normal data structures or named vars. 

If all this sounds familiar it's because this is pretty much what Pedestal's interceptors are. I think the design of Pedestal's interceptor library could be simplified quite a bit, so I'm not against someone rewriting/rethinking it from scratch. But the core ideas are extremely powerful, and probably deserve a bit of thought. 


Thanks, 

Timothy

--
You received this message because you are subscribed to the Google Groups "Ring" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ring-clojure...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

“One of the main causes of the fall of the Roman Empire was that–lacking zero–they had no way to indicate successful termination of their C programs.”
(Robert Firth)

[Zach Tellman]

The CPS model, in general, seems like to sit at a similar level of abstraction as the synchronous Ring model.  I agree that using something like Manifold or core.async should be avoided if at all possible.

However, Readable/Writable ByteChannels are not really asynchronous.  SelectableChannels allow for multiplexing, so it's at least not one thread per connection, but at the cost of significant implementation complexity.  Every server implementation would need a thread set aside to run the selection loop, and for streaming requests some sort of execution model outside of that loop would be needed to handle the reads.  In other words, every Ring-CPS implementation would asymptotically approach Netty.  Given that efficient interaction with WritableByteChannels also requires selectors, this probably is also true of any attempt to extend ChannelWritable over core.async, etc.

So while I think the CPS approach is adequate, pretty much everything relating to the bodies of the requests and responses needs some more thought, and definitely one or more reference implementations before the spec is finalized.

Zach

[James Reeves]

On 27 May 2015 at 17:01, Zach Tellman <ztel...@gmail.com> wrote:

However, Readable/Writable ByteChannels are not really asynchronous.  SelectableChannels allow for multiplexing, so it's at least not one thread per connection, but at the cost of significant implementation complexity.  Every server implementation would need a thread set aside to run the selection loop, and for streaming requests some sort of execution model outside of that loop would be needed to handle the reads.  In other words, every Ring-CPS implementation would asymptotically approach Netty.  Given that efficient interaction with WritableByteChannels also requires selectors, this probably is also true of any attempt to extend ChannelWritable over core.async, etc.

Thanks for your insight, Zach. Would you then favour abstracting the request/response body reader and writer through protocols? If there's no suitable abstraction in Java, then that would seem to be the next obvious step.

So while I think the CPS approach is adequate, pretty much everything relating to the bodies of the requests and responses needs some more thought, and definitely one or more reference implementations before the spec is finalized.

I plan on releasing it as an experimental library for people to play around with. I don't expect a stable version to come out any time soon, and it won't be integrated into Ring core for quite some time, if indeed it ever is.

- James

[Zach Tellman]

Yes, I think I'd favor protocols that resemble the read and write halves of http://docs.oracle.com/javase/7/docs/api/java/nio/channels/AsynchronousByteChannel.html.  Note that we can't just use AsynchronousByteChannel, since it's only a way of representing sockets, not arbitrary flows of bytes in a process.  Note that you'd probably want synchronous AND asynchronous read/write methods, and some way for the stream object to indicate which is native for it.  Otherwise, you'd end up pushing every "asynchronous" read from an InputStream onto another thread so it doesn't block, rather than just spinning up a thread to do a series of synchronous reads.  So something like https://gist.github.com/ztellman/2e2a266076f34e3f5892.

Note that if you go down this path, you can do something like core.async to allow reads and writes that complete immediately to not invoke the callback, but instead just return the value.  This can definitely improve performance, at the cost of some implementation complexity.  I'd weakly assert that it'd be worth it, but I could be persuaded otherwise.

Zach

--

[Timothy Baldridge]

>> Does this concept have another name besides "reified call stacks"?

I'm not sure that it does. And it's more of a general term I've created for any program that represents a call stack as something I can manipulate. So in that case, I guess Eff might qualify (http://math.andrej.com/eff/). In addition, many interpreted languages like Python allow the callstack to be queried and manipulated at runtime (https://docs.python.org/2/library/inspect.html).

But I would like to ask the following of James and Zach. Why keep the complexity of the ring call model? Ring handlers combine the modification of a context with the calling of a new context, but for what purpose?

Why do this:

(defn parse-params [handler]

  (fn [ctx]
     (handler (assoc ctx :params (do-param-parsing (:url ctx))))))

When I can do this:

(defn parse-params []
  (fn [ctx]
     (assoc ctx :params (do-param-parsing (:url ctx))))

It seems that the latter is much cleaner, simpler, and supports both request/response and push semantics. So you could use the latter model for HTTP, JMS, an asynchronous dataflow, or really anything that needs some sort of composable middleware. Why restrict yourself by baking the semantics into the handlers themselves?

I guess I don't see the advantage of requiring handlers to call the next handler at all, regardless of if it's CPS or a normal function invocation.

Timothy

[Zach Tellman]

Well, one reason is that the Ring model gives an easy way to *not* call the next handler (due to a cache hit, authentication failure, etc).  That can be emulated by some sort of `reduced` value, but more generally you get to explicitly decide how the inner handler is executed, whether on another thread or not at all.  With that said, I don't think the nested-function middleware is the best approach, just the familiar one.  I think the CPS approach has a nice symmetry with it, and will be straightforward for people to understand.  Something similar to what you describe may be a better approach overall, but I think we'd need a more concrete spec to do an actual comparison.

[James Reeves]

On 27 May 2015 at 18:02, Timothy Baldridge <tbald...@gmail.com> wrote:

But I would like to ask the following of James and Zach. Why keep the complexity of the ring call model? Ring handlers combine the modification of a context with the calling of a new context, but for what purpose?

I'm somewhat leery about abstractions that seek to improve upon the runtime environment, because it tends to lead to a situation where you need to reimplement at least part of the language's toolchain. This isn't necessarily a bad thing, but it is something that's rarely done seamlessly. Trying to avoid the "complexity" of closures may result in a system that exhibits complexity in a different way.

I'll also suggest that "reified call stacks" is a solution that doesn't need to specific to Ring. I'd rather see it developed as a standalone library, then then see how it performs and where the abstraction leaks, giving us a better idea of its merits in practice, before integrating it into a Ring library.

(Incidentally, this is one of my biggest bugbears with Pedestal; it has a lot of interesting stuff that would be extremely useful on its own, but it's all packaged together as one inseparable bundle.)

If it sounds like I'm being dismissive, that's not my intention. I'm planning on writing a Ring-CPS library as an experiment, and it'll likely remain as such for at least a year. There's no reason why you or someone else couldn't write a "reified call stack" library as well, and the approaches could be compared.

- James

[James Reeves]

On 27 May 2015 at 17:41, Zach Tellman <ztel...@gmail.com> wrote:

Yes, I think I'd favor protocols that resemble the read and write halves of http://docs.oracle.com/javase/7/docs/api/java/nio/channels/AsynchronousByteChannel.html.  Note that we can't just use AsynchronousByteChannel, since it's only a way of representing sockets, not arbitrary flows of bytes in a process.  Note that you'd probably want synchronous AND asynchronous read/write methods, and some way for the stream object to indicate which is native for it.  Otherwise, you'd end up pushing every "asynchronous" read from an InputStream onto another thread so it doesn't block, rather than just spinning up a thread to do a series of synchronous reads.  So something like https://gist.github.com/ztellman/2e2a266076f34e3f5892.

 I was thinking something more like:

  (defprotocol Closeable

    (close! [_])

  (defprotocol AsyncReadable

    (read! [_ buf cb])

  (defprotocol AsyncWriteable

    (write! [_ buf cb])

Protocols for synchronous reads and writes can be deferred for now, since we won't need them for a purely async library.

The reason for a buffer on reads is to allow buffer reuse.

- James

[James Reeves]

On 28 May 2015 at 14:38, James Reeves <ja...@booleanknot.com> wrote:

The reason for a buffer on reads is to allow buffer reuse.

In retrospect this is a bad idea, since buffer pools are a better solution to this.

- James

[Julien Fantin]

Hi James,

Have you considered the fact that this would allow for spec-compliant
HTTP clients in both Clojure and ClojureScript ?

I've in fact been using the exact design you're proposing in a number
of client-side middlewares and it's worked very well for us.

My contention was that all the existing client libraries are very
"complected"; they re-implement the same functionality in ad-hoc,
subtly incompatible ways, and most don't provide clean extension
points like middlewares do. Were they to adopt this spec, most of
their features could be packaged as compatible middlewares, and it
would leave them as simple adapters in Ring parlance.

While this might seem loosely related to Ring and Ring-CPS, I'd like
to argue that, assuming an existing Ring-CPS compliant HTTP client:

1. Reader conditionals make it possible to ship portable client
   counterparts to some existing Rnig middlewares. All the params
   handling and payload coercion ones come to mind.

2. There would be an incentive to extract all the request/response
   helpers into a portable library (.cljc)

3. Last but not least, client-code generation starts to look like a
   worthwhile undertaking.

Curious to hear your thoughts this.

-- Julien

[Timothy Baldridge]

"I'm somewhat leery about abstractions that seek to improve upon the runtime environment, because it tends to lead to a situation where you need to reimplement at least part of the language's toolchain. "

This is true of CPS as well, things like exceptions and stack traces will be quite unusable in CPS as well, and proper handling of them will need to be re-implemented in a continuation style. 

Timothy

--

You received this message because you are subscribed to the Google Groups "Ring" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ring-clojure...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[James Reeves]

On 31 May 2015 at 15:00, Timothy Baldridge <tbald...@gmail.com> wrote:

"I'm somewhat leery about abstractions that seek to improve upon the runtime environment, because it tends to lead to a situation where you need to reimplement at least part of the language's toolchain. "

This is true of CPS as well, things like exceptions and stack traces will be quite unusable in CPS as well, and proper handling of them will need to be re-implemented in a continuation style. 

Yes, but to a lesser degree. Exception propagation is inherently restricted to a single thread, so the standard try-catch block isn't going to work with any form of asynchronicity.

Closures may be opaque, but they're also Turing complete. You can choose not to call the handler, to call it multiple times, to choose between calling different handlers, and so forth. It's not clear to me how you can support the same range of functionality with custom callstack while also maintaining transparency.

However, I'd encourage you to explore your reified callstack idea. It might be possible to write something with equivalent speed and functionality to CPS, yet still maintain a greater degree of transparency.

- James

[James Reeves]

On 31 May 2015 at 04:43, Julien Fantin <julien...@gmail.com> wrote:

Have you considered the fact that this would allow for spec-compliant
HTTP clients in both Clojure and ClojureScript ?

I've in fact been using the exact design you're proposing in a number
of client-side middlewares and it's worked very well for us.

My contention was that all the existing client libraries are very
"complected"; they re-implement the same functionality in ad-hoc,
subtly incompatible ways, and most don't provide clean extension
points like middlewares do. Were they to adopt this spec, most of
their features could be packaged as compatible middlewares, and it
would leave them as simple adapters in Ring parlance.

Certainly the same design could be used in HTTP clients as well, and some functions could be reused, such as those to examine the response, or to asynchronously read the body.

However, most of the middleware would have little overlap, I think. The wrap-params middleware in Ring parses parameters into maps, whereas equivalent middleware for the client would format maps into parameters, literally doing the opposite.

- James

[Julien Fantin]

Reusing parts of Ring-CPS core in the client would indeed help with a
consistent spec implementation.

However in this scenario, for the client to remain compatible with
ClojureScript adapters, the relevant parts of Ring-CPS would have to
use reader-conditionals and provide plaform-specific
implementations. Is this something you'd be willing to consider for
Ring-CPS?

This is also why I was hinting at extracting request/response utils in
a portable library that both Ring-(CPS) core and the client would
depend on.

Regarding middlewares, indeed most of the functionality would be "the
opposite". My thinking was that if the CPS middlewares also shipped
the client part of their functionality, it'd ensure both ends remain
compatible. It'd probably require some changes in the way the projects
are packaged and distributed though.

- Julien

[Dmitri Sotnikov]

Out of curiosity, is this still being considered?

[James Reeves]

In writing Ring-CPS I realised that most of the work would be in writing a library for handling asynchronous I/O. There's no standard way of handling it, so I'd need to create a protocol that works with the lowest common denominator, then write functions for handling piping and so forth.

As asynchronous HTTP isn't currently a priority for me, I decided to wait and see if any Clojure asynchronous I/O libraries crop up that could save me the trouble of writing one specifically for Ring.

- James

[Dmitri Sotnikov]

That makes sense. I think waiting to see if something standard comes up is a good approach here as it's not really a show stopper for most use cases.

[James Reeves]

Thanks for replying, Paul. I actually have a more recent thread in the group that I posted earlier today where I discuss an updated design, so you're not late to the discussion.

I'm aware of Pedestal's interceptors. My understanding is that their functionality is equivalent CPS-style Ring middleware, in that there are functions for entering, leaving and raising an error. 

CPS-style functions are backward compatible with Ring's current design, which is the main draw to me. Interceptors are arguably more transparent, but if I understand right, not significantly so in isolation. An map containing three opaque functions doesn't reveal much more than a single opaque function.

The key advantage of interceptors that I can see is that they are held in a persistent queue, whereas middleware are typically composed. But there's no reason why we can't place middleware in a queue and compose them later.

Am I missing something, or is that an accurate assessment?

- James

On 27 June 2016 at 21:09, Paul deGrandis <paul.de...@gmail.com> wrote:

Hi all,

Sorry for being so late to this thread, someone just linked me to it today.

The system Timothy Baldridge described above is effectively how the interceptor chain works.  On Pedestal Master, the interceptor chain is a separate module (pedestal.interceptor) -- I definitely invite you to explore it, see how it integrates into your existing work, and weigh the trade-offs against a purely CPS approach.  While I don't care for the CPS approach (I don't think the trade-offs shake out as a net positive when compared against alternatives), I do share Zach's view that CPS is the logical async equivalent of Ring's current model.

I've had to tackle the container-specific NIO/AIO challenges in Pedestal.  All response types are programmed against two protocols (one for blocking IO bodies, another for non-blocking bodies).  I'm more than happy to talk through the challenges, approaches we tried and reviewed, and discuss what we finally ended up upon.

At this point, Pedestal really is just a set of separate libraries, all programmed against protocols/interfaces.  If you see something you like, please steal liberally :)  If you have questions, or want to openly work through design decisions and trade-offs, I'm definitely open to participating.

Cheers,
Paul

[Paul deGrandis]

[Paul deGrandis]

I'm aware of Pedestal's interceptors. My understanding is that their functionality is equivalent CPS-style Ring middleware, in that there are functions for entering, leaving and raising an error.

Totally accurate here.  Assuming you introduce (or have introduced) an "error" continuation function argument, the two models share a similar parts.  The difference on the surface is that your "enter" function would be taking the leave and error functions as arguments -- definitely advantages and disadvantages there.  Pedestal took a slightly different path in that all functions take the same, single argument - the context.
 

CPS-style functions are backward compatible with Ring's current design, which is the main draw to me.

I haven't thoroughly gone through all the pieces in the branch, but I'd agree that CPS seems like the best fit for Ring, both in spirit and in compatibility.

 

Interceptors are arguably more transparent, but if I understand right, not significantly so in isolation. An map containing three opaque functions doesn't reveal much more than a single opaque function.

The functions themselves are opaque, but what you get is a first-class shared/established container, on which you can attach other properties (ie: the "interceptor").  You can hold the three functions together as a logical "slice" in the handling/operating pipeline.
 

The key advantage of interceptors that I can see is that they are held in a persistent queue, whereas middleware are typically composed. But there's no reason why we can't place middleware in a queue and compose them later.

Regarding error handling and composition, I think you eventually have to make this jump (or something similar to it), or accept the standard CPS debugging story.  Much of  Pedestal's core capabilities come directly from the interceptor chain's queue -- it's how one can have dynamic and rule-based interceptors, how you can easily trace and inject in the handling of a request, and how Pedestal can treat async-debugging as if it were synchronous (with a chain, you can pattern match on "steps" and errors within those steps).  It's also how we can run execution of steps backwards, which you'd lose in CPS -- this is going to put more strain on developers to correctly compose the independent, but related pieces of the CPS handling within Ring.

Am I missing something, or is that an accurate assessment?

All-in-all an accurate assessment, though I think the fallout of the design decisions might give you some ideas or new angles to consider for Ring.  One example, I can envision a model where Ring's exposed API is in a CPS style, but is then composed and executed on the interceptor chain under the hood.

I'm looking forward to what shakes out!

Cheers,
Paul


 

[James Reeves]

On 27 June 2016 at 22:55, Paul deGrandis <paul.de...@gmail.com> wrote:

Interceptors are arguably more transparent, but if I understand right, not significantly so in isolation. An map containing three opaque functions doesn't reveal much more than a single opaque function.

The functions themselves are opaque, but what you get is a first-class shared/established container, on which you can attach other properties (ie: the "interceptor").  You can hold the three functions together as a logical "slice" in the handling/operating pipeline.

Presumably the functions are passed the interceptor they belong to, so they can read these additional keys?

 

The key advantage of interceptors that I can see is that they are held in a persistent queue, whereas middleware are typically composed. But there's no reason why we can't place middleware in a queue and compose them later.

Regarding error handling and composition, I think you eventually have to make this jump (or something similar to it), or accept the standard CPS debugging story.  Much of  Pedestal's core capabilities come directly from the interceptor chain's queue -- it's how one can have dynamic and rule-based interceptors, how you can easily trace and inject in the handling of a request, and how Pedestal can treat async-debugging as if it were synchronous (with a chain, you can pattern match on "steps" and errors within those steps).  It's also how we can run execution of steps backwards, which you'd lose in CPS -- this is going to put more strain on developers to correctly compose the independent, but related pieces of the CPS handling within Ring.

Interesting. Why would you want to execute the steps backward?

 

All-in-all an accurate assessment, though I think the fallout of the design decisions might give you some ideas or new angles to consider for Ring.  One example, I can envision a model where Ring's exposed API is in a CPS style, but is then composed and executed on the interceptor chain under the hood.

The thought had struck me as well. There are a lot of things I like about the interceptor model, but I also get the impression I don't fully appreciate the advantages of its design.

 

I'm looking forward to what shakes out!

Thanks!

- James

[Paul deGrandis]

Presumably the functions are passed the interceptor they belong to, so they can read these additional keys?

This is the motivating factor behind all functions taking only a single argument -- the context.  The context holds the interceptor queue, which is how other interceptor functions can dynamically change (or instrument) the processing of a request.  Interceptors are just records with :enter, :leave, :error, and :name -- a single interceptor represents that one logical slice.  An interceptor could always look at itself in the queue.
 

Interesting. Why would you want to execute the steps backward?

Once you process the chain in one direction (:enter), you can then just turn around and process it in the other direction (:leave), and you get a similar effect to middleware, but completely decoupled.  Related, you could also just run the chain in a single direction to process streaming data (SSE, WebSockets, etc.), while still leaving all the error-handling properties of the interceptor chain intact.
 

 

All-in-all an accurate assessment, though I think the fallout of the design decisions might give you some ideas or new angles to consider for Ring.  One example, I can envision a model where Ring's exposed API is in a CPS style, but is then composed and executed on the interceptor chain under the hood.

The thought had struck me as well. There are a lot of things I like about the interceptor model, but I also get the impression I don't fully appreciate the advantages of its design.

You're not alone :)
Since taking over Pedestal 3-4 years ago, I've come to hold the interceptor chain in much the same way I hold the core idea of "Lisp."  There are a very small set of primitive operations, from which all sorts of behaviors can be created.  You also see the pattern crop up all over "request processing" systems and for good reason -- it was the primary reason why I wanted the interceptor chain to be a standalone module.

--Paul

[James Reeves]

On 28 June 2016 at 00:43, Paul deGrandis <paul.de...@gmail.com> wrote:

Presumably the functions are passed the interceptor they belong to, so they can read these additional keys?

This is the motivating factor behind all functions taking only a single argument -- the context.  The context holds the interceptor queue, which is how other interceptor functions can dynamically change (or instrument) the processing of a request.  Interceptors are just records with :enter, :leave, :error, and :name -- a single interceptor represents that one logical slice.  An interceptor could always look at itself in the queue.

How often does an interceptor need to change the interceptor queue? It seems like it could make things more confusing if the sequence of interceptors can change on the fly.

 

Interesting. Why would you want to execute the steps backward?

Once you process the chain in one direction (:enter), you can then just turn around and process it in the other direction (:leave), and you get a similar effect to middleware, but completely decoupled.  Related, you could also just run the chain in a single direction to process streaming data (SSE, WebSockets, etc.), while still leaving all the error-handling properties of the interceptor chain intact.

What are the practical advantages of decoupling?

I mean, the author of a middleware function always has the option of decoupling the "request" and "response" parts, so the only time I can think of that running the "enter" or "leave" part of an interceptor in isolation would be to do something the original author didn't intend.

Thanks for answering my questions, by the way.

- James

[Paul deGrandis]

How often does an interceptor need to change the interceptor queue? It seems like it could make things more confusing if the sequence of interceptors can change on the fly.

It occurs more often than one might think, but it rarely is done in an ad-hoc, unpredictable fashion.  Let's consider two (related) use-cases.  Keep in mind, absolutely everything in Pedestal is an interceptor, interceptors are placed on the queue (the interceptor chain), and the chain is executed linearly.

One may want a different set of interceptors for different routes within their application.  They might even want a different set of interceptors for a single route -- this is common when you want that endpoint to perform an upgrade-request (SSE or WebSockets), and you sometimes see it where one, single route offers a different output encoding (common with the rise of demand-driven APIs).
In Pedestal (and really, in the interceptor chain) a route endpoint (interceptor) will just add the additional interceptors it needs for processing that request.

But we need not limit ourselves to just dispatching this kind of behavior on URL alone.  Consider something like Liberator.  You could write a single interceptor that inspects many aspects of a request and queues up the corresponding interceptors to handle that request and correctly format the response.  This is the case anytime you'd write a rules-based interceptor (which is common when attempting to model your API to embody business rules).

Because composition of functionality/interceptors happens via a data structure (the chain/queue), we never need to process and fall-through functionality that is not related to our request.  This improves performance while simplifying execution and debugging.

All of that said, outside of routing/handling, the interceptor chain is almost always stable -- we typically just call that "the default interceptor stack."  Things tend to only go dynamic towards the very end of handling a request.
 

What are the practical advantages of decoupling?

I mean, the author of a middleware function always has the option of decoupling the "request" and "response" parts, so the only time I can think of that running the "enter" or "leave" part of an interceptor in isolation would be to do something the original author didn't intend.

A few things immediately fall out of decoupling the "enter" and "leave".  The first (and most important) is that it allows for finer control over execution -- this is needed if you want to go async.  It also opens up composition -- with the pieces separate, you can choose how to put them together.  Ring Middleware that split "request" and "response" hold those two pieces together only by a namespace, and force composition through wrapping/decorating, to participate in the middleware stack.  Interceptor's "enter" and "leave" are held together by a record, and interceptors are free to be composed in any manner on the interceptor chain.  The creation of that interceptor record is backed by a protocol, which is what drives the data-described capabilities.

When enter and leave are separate, one might elect to only run the chain in a single direction, which allows for a chain to handle streams of data.  This allows an interceptor chain to be used for SSE, WebSockets, and other streaming message workloads.

Thanks for answering my questions, by the way.

Thanks for asking them!


--Paul

[Paul deGrandis]

Totally unrelated, but Pedestal's router is also a separate module now, if you'd like to pull it into Compojure.  It routes in effectively constant-time in the optimized case, and O(log N) in the general case.  It operates on the Ring spec for a request, so it should work for everyone.

Cheers,
Paul

[Dmitri Sotnikov]

It occurs more often than one might think, but it rarely is done in an ad-hoc, unpredictable fashion.  Let's consider two (related) use-cases.  Keep in mind, absolutely everything in Pedestal is an interceptor, interceptors are placed on the queue (the interceptor chain), and the chain is executed linearly.

I typically have different middleware wrap different sets of routes in Ring applications using the wrap-routes Compojure macro that applies the middleware after a particular route has been matched. Consider the following example:

(def app-routes (routes

ws-routes (wrap-routes home-routes wrap-csrf)

 (wrap-routes admin-routes wrap-csrf wrap-auth)

(route/not-found (:body (error-page {:code 404 :title "page not found"})))))

Ring middleware can be wrapped at any point in the request handling process. There's nothing that prevents middleware functions from invoking additional middleware for a specific set of routes based on whatever runtime logic is required.

[James Reeves]

On 28 June 2016 at 14:38, Paul deGrandis <paul.de...@gmail.com> wrote:

One may want a different set of interceptors for different routes within their application.  They might even want a different set of interceptors for a single route -- this is common when you want that endpoint to perform an upgrade-request (SSE or WebSockets), and you sometimes see it where one, single route offers a different output encoding (common with the rise of demand-driven APIs).

So changing the interceptor queue is usually used for the same purpose as route-specific middleware?

 

But we need not limit ourselves to just dispatching this kind of behavior on URL alone.  Consider something like Liberator.  You could write a single interceptor that inspects many aspects of a request and queues up the corresponding interceptors to handle that request and correctly format the response.  This is the case anytime you'd write a rules-based interceptor (which is common when attempting to model your API to embody business rules).

Ah, I see why that would be necessary. Middleware can dynamically wrap the handler in other middleware, whereas the interceptor model achieves the same result through modifying the interceptor queue.

The interceptor model has the advantage that the queue can be inspected, and the "enter" and "leave" parts can be decoupled.

 

When enter and leave are separate, one might elect to only run the chain in a single direction, which allows for a chain to handle streams of data.  This allows an interceptor chain to be used for SSE, WebSockets, and other streaming message workloads.

WebSockets are something I hadn't considered. I can see why discarding the "response" part of the middleware might be a good idea in that case.

The same result could be achieved by making finer-grained middleware, but that does require some additional anticipation on behalf of the original author.

- James

[James Reeves]

On 28 June 2016 at 14:40, Paul deGrandis <paul.de...@gmail.com> wrote:

Totally unrelated, but Pedestal's router is also a separate module now, if you'd like to pull it into Compojure.  It routes in effectively constant-time in the optimized case, and O(log N) in the general case.  It operates on the Ring spec for a request, so it should work for everyone.

I don't think Pedestal's router is compatible with the way Compojure works. Routes in Compojure are functions that when passed a request map, either return a response or nil, indicating the route didn't match. So there's no overall routing table, each route operates in isolation.

My understanding of Pedestal's router is that its efficiency comes from having the entire routing table available to it. If it only knew about one route, my guess is that it would be no more efficient than a regular expression. Would you say this is an accurate assessment?

Incidentally, I'm all for Pedestal becoming more modular. In the past it felt like Pedestal was an isolated ecosystem.

- James

[Paul deGrandis]

I typically have different middleware wrap different sets of routes in Ring applications using the wrap-routes Compojure macro that applies the middleware after a particular route has been matched. Consider the following example:

I'm aware of this capability and understand it's common in Ring applications, just as it's common in Pedestal applications.  The difference is that the kind of composition (wrapping/decorating) is forced on the consumer/user.  This ends up leading to all the issues Timothy and Kovas were talking about in the other thread.  If you split "enter" and "leave", back the "container" creation by a protocol, and compose via a data structure:

 1. All of the composition and introspection problems go away
 2. Data-described capabilities emerge
 3. Async and Sync handling live side-by-side
 4. Fine-grain control over processing each request is possible
 5. Efficient processing of each request falls out naturally from the model
 6. The user-level API can be exposed however you like (CPS, direct interceptors, deferables, etc.)

But the trade-off is that composition is now more than decoration/function-composition, and you need a new "machine" to process the chain/queue.
 

Ring middleware can be wrapped at any point in the request handling process. There's nothing that prevents middleware functions from invoking additional middleware for a specific set of routes based on whatever runtime logic is required.

Definitely understand that -- I was mostly pointing out how "dynamic interceptors" are used, attempting to illustrate how middleware patterns (like you're pointing out) are achieved on the chain.

Thanks for jumping in!

[Paul deGrandis]

I don't think Pedestal's router is compatible with the way Compojure works. Routes in Compojure are functions that when passed a request map, either return a response or nil, indicating the route didn't match. So there's no overall routing table, each route operates in isolation.

Pedestal's linear router uses the `match-fn` approach, similar (albeit not tied to the response value) as what you're describing.  But you are correct, while Pedestal's router would be compatible in response and request format to Compojure, it needs to know about all of the possible routes up front to build the prefix tree.

Given the conversation in this thread --
The router is an interceptor that inspects the request and enqueues the corresponding interceptors to handle that request.  The core matching capabilities of the router are backed by a protocol (and that's how you'd consume it in Ring applications).
 

My understanding of Pedestal's router is that its efficiency comes from having the entire routing table available to it. If it only knew about one route, my guess is that it would be no more efficient than a regular expression. Would you say this is an accurate assessment?

I was under the impression that Compojure now has a "compile routes" step.  Is that just to turn routes into Records for efficient lookup?  Presumably you have a collection of all the routes in some sequential container, could you not pass that sequence to the Pedestal router and package the result in a middleware?

--Paul
 

[James Reeves]

On 28 June 2016 at 15:35, Paul deGrandis <paul.de...@gmail.com> wrote:

My understanding of Pedestal's router is that its efficiency comes from having the entire routing table available to it. If it only knew about one route, my guess is that it would be no more efficient than a regular expression. Would you say this is an accurate assessment?

I was under the impression that Compojure now has a "compile routes" step.  Is that just to turn routes into Records for efficient lookup?  Presumably you have a collection of all the routes in some sequential container, could you not pass that sequence to the Pedestal router and package the result in a middleware?

No, Compojure compiles routes into functions individually. It has no knowledge of the overall routing table.

- James

[Paul deGrandis]

I was under the impression that Compojure now has a "compile routes" step.  Is that just to turn routes into Records for efficient lookup?  Presumably you have a collection of all the routes in some sequential container, could you not pass that sequence to the Pedestal router and package the result in a middleware?

No, Compojure compiles routes into functions individually. It has no knowledge of the overall routing table.

Ah, darn!  Well, more food for thought I suppose.  And the router is still an option for anyone working directly at the Ring layer.  Thanks for helping set my understanding straight.

Digression over.  Let's all jump back into the async conversation :)

--Paul

[Paul deGrandis]

So changing the interceptor queue is usually used for the same purpose as route-specific middleware?

Correct.  It's most common that an interceptor appends/enqueues additional interceptors on the queue, rather than it changing/modifying something already on the queue.  While the common case is route-specific middleware/interceptors, in general I tend to say, as you get closer to handling the request, the chain becomes increasingly more dynamic.  In the simplest case, the "default" interceptor stack could be *just* a router, in which case, the interceptor chain/queue is only determined by the route-specific middleware/interceptors.
 

 

But we need not limit ourselves to just dispatching this kind of behavior on URL alone.  Consider something like Liberator.  You could write a single interceptor that inspects many aspects of a request and queues up the corresponding interceptors to handle that request and correctly format the response.  This is the case anytime you'd write a rules-based interceptor (which is common when attempting to model your API to embody business rules).

Ah, I see why that would be necessary. Middleware can dynamically wrap the handler in other middleware, whereas the interceptor model achieves the same result through modifying the interceptor queue.

Correct.  And composition is decided by whatever consumes that queue data structure.  By default, that's the `execute` function within Pedestal's interceptor chain namespace, but anyone is free to write their own function, consume the queue, and compose however they like.
 

The interceptor model has the advantage that the queue can be inspected, and the "enter" and "leave" parts can be decoupled.

 

When enter and leave are separate, one might elect to only run the chain in a single direction, which allows for a chain to handle streams of data.  This allows an interceptor chain to be used for SSE, WebSockets, and other streaming message workloads.

WebSockets are something I hadn't considered. I can see why discarding the "response" part of the middleware might be a good idea in that case.

This use-case/pattern pops up again in microservices.  Once composition rises to the macro level (a system-of-systems), sometimes you want to retain "enter" functionality, but just change "leave" functionality to a simple ACK.
In short, splitting enter/leave is about finer control over execution. more flexibility in composition/reuse, and powerful introspection.
 

The same result could be achieved by making finer-grained middleware, but that does require some additional anticipation on behalf of the original author.

Agreed.  When the model of composition is wrapping/decorating, it doesn't seem natural (on the surface) to make the split.  The moment you want to control the composition or the execution, the split becomes essential.

Looping this back into the error-handling case-
You can also understand the motivating factor for pulling that out as a separate piece.  If we need flexibility for execution, composition, and introspection of enter/leave, we need to force the same constraints and requirements on error handling.  The immediate logical choice is to decouple it from enter/leave entirely.  This design decision is common across other implementations of the interceptor pattern.

--Paul