Ring 1.6.0-beta2 released

[James Reeves]

Ring 1.6.0-beta2 is now up on Clojars. It's the first release with support for asynchronous handlers.

This is a beta release, and the current design is not finalized. Many things might change before full release, including a full rollback of this functionality. However, I'm reasonably confident in the current design.

Please don't think of asynchronous handlers as the new norm. I intend them to be a specialized tool, and suitable for a narrow set of use cases. The synchronous handlers you're used to are still the recommended way of developing in Ring in most cases.

In order to use asynchronous handlers, set the :async? option on the Jetty adapter to true:

  (defn hello-world [request respond raise]

    (respond {:status 200, :headers {}, :body "Hello World"}))

  (ring.adapter.jetty/run-jetty hello-world {:port 3000, :async? true})

- James

[Timothy Baldridge]

>>Please don't think of asynchronous handlers as the new norm. I intend them to be a specialized tool, and suitable for a narrow set of use cases. 

What is the rationale behind this statement?

--
You received this message because you are subscribed to the Google Groups "Ring" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ring-clojure...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

“One of the main causes of the fall of the Roman Empire was that–lacking zero–they had no way to indicate successful termination of their C programs.”
(Robert Firth)

[James Reeves]

On 7 July 2016 at 18:49, Timothy Baldridge <tbald...@gmail.com> wrote:

>>Please don't think of asynchronous handlers as the new norm. I intend them to be a specialized tool, and suitable for a narrow set of use cases. 

What is the rationale behind this statement?

I'm working on a post that discusses this in a little more detail, but it might be useful to go into more depth here.

Asynchronous HTTP makes sense under three circumstances:

1. Your HTTP servers are your performance bottleneck

2. Your HTTP servers are primarily waiting around for I/O

3. You're running a large enough number of HTTP servers that increasing capacity is not a trivial cost

This discounts a fairly large set of applications.

For example, in 2008 Wikipedia was handling 50,000 requests per second. Assume you had this level of traffic, assume it took 100ms to return a page, and most of your web servers were just waiting around for I/O. Even in that case, you'd only need a set of servers capable of maintaining a pool of around 5000 threads. It's not a high cost.

If you're deliberately keeping connections open, for instance to support long polling or streaming, then asynchronous I/O begins to make more sense. However even that use-case is rapidly being supplanted by WebSockets.

If asynchronous support were that important to most Ring users, then I'd hope I'd have heard more about it, or we'd see Pedestal being the predominant platform. That's why I see asynchronous HTTP as a tool with a narrow use-case.

- James

[Zach Tellman]

To address the Wikipedia example, the thread-per-request model shouldn't be judged by its behavior under ideal (or even normal) conditions, but rather its behavior under adverse conditions.  What if the caching layer has to be restarted, and the mean response time climbs to 1000ms?  Assuming we built in some headroom, and have 10k available threads, we're still rejecting 80% of traffic.  

Of course, even if we can handle the additional in-flight requests, we may still want to reject some of them to protect our backend services.  But in thread-per-request services, it's usually the max thread count which ends up being the limiting factor, rather than any properties in the rest of our system.  In cases where we have backend services (which we usually do), and the latency of those services can spike under pressure (which it usually does), and our maximum capacity for in-flight requests isn't two orders of magnitude higher than the normal level, the async model has real benefits.

I don't know how typical that last property is; probably most services have a ridiculous amount of headroom, and this is a non-issue.  However, resiliency to adverse conditions, and control over the behavior in those conditions, is a real benefit which you seem to be ignoring.

Zach

--

[James Reeves]

On 7 July 2016 at 21:11, Zach Tellman <ztel...@gmail.com> wrote:

To address the Wikipedia example, the thread-per-request model shouldn't be judged by its behavior under ideal (or even normal) conditions, but rather its behavior under adverse conditions.  What if the caching layer has to be restarted, and the mean response time climbs to 1000ms?  Assuming we built in some headroom, and have 10k available threads, we're still rejecting 80% of traffic.

Even so, we're talking about traffic levels that are far in excess of what most web applications have to handle. Twitter in 2009 had around 350,000 users, and the number of concurrent connections went from an average of 200-300, spiking at 800. That's not a particularly large number of threads!

I don't know how typical that last property is; probably most services have a ridiculous amount of headroom, and this is a non-issue.  However, resiliency to adverse conditions, and control over the behavior in those conditions, is a real benefit which you seem to be ignoring.

That's true, and it was something that hadn't occurred to me. But as you point out, the capacity of most web servers is far more than most applications need. Perhaps an application that makes heavy use of SSEs or large I/O transfers would begin to benefit earlier from an asynchronous approach, but that doesn't seem like a common use-case.

- James

[Zach Tellman]

If you're writing a consumer web application (like Wikipedia or Twitter), then I agree, the threading model isn't going to be a scaling bottleneck for quite a while.  Speaking personally, I've never worked on a consumer web application, I've worked on APIs which speak HTTP.  A combination of fan-in from many applications and people running batch jobs against the API make it easy to achieve higher numbers than what we're discussing here.  Unfortunately, the economics don't work in your favor: the value of a human making a request is much higher than another computer making a request.  All of this is to say that your criteria for when async "makes sense" is easy to achieve in certain applications.

My professional experience is clearly biased, and maybe you're correct in treating it as a niche use of Ring.  In either case, we can have supersets of the Ring spec, such as Aleph, that relieve some of the pressure.  But it might be worthwhile to have some examples (real or imagined) of applications which the core Ring spec tries to support, and those it doesn't, to make the assumptions beneath the design decisions a bit clearer to everyone.

--

[James Reeves]

On 7 July 2016 at 22:09, Zach Tellman <ztel...@gmail.com> wrote:

If you're writing a consumer web application (like Wikipedia or Twitter), then I agree, the threading model isn't going to be a scaling bottleneck for quite a while.  Speaking personally, I've never worked on a consumer web application, I've worked on APIs which speak HTTP. 

APIs are an interesting use-case, but wouldn't you still find yourself more constrained by the database than the web servers? Even if you had a distributed database, I'd imagine you'd need more database nodes that web servers, comparatively.

My professional experience is clearly biased, and maybe you're correct in treating it as a niche use of Ring.  In either case, we can have supersets of the Ring spec, such as Aleph, that relieve some of the pressure.  But it might be worthwhile to have some examples (real or imagined) of applications which the core Ring spec tries to support, and those it doesn't, to make the assumptions beneath the design decisions a bit clearer to everyone.

I should mention that don't intend to use "niche" to mean "bad". Perhaps "specialised" is a more neutral term. I see asynchronous support in the same vein as, say, data.int-map. Ideally you'd start with a map, then use a more specialized structure if your performance requirements demanded it. In the same way, it seems reasonable to start with synchronous handlers, and move to asynchronous handlers when required.

This beta release is intended to allow people to kick the proverbial tyres of the current asynchronous proposal. It's sat around for a year without additional feedback, so producing an implementation that people can try out seems the next logical step. If it turns out there are problems, the proposal can be altered or shelved.

What sort of examples did you have in mind?

- James

[Zach Tellman]

> APIs are an interesting use-case, but wouldn't you still find yourself more constrained by the database than the web servers? Even if you had a distributed database, I'd imagine you'd need more database nodes that web servers, comparatively.

Only if the database spends more computational effort on each request than the web server does.  It's very common to have a single Redis instance handling the combined request volume from many servers, and in general in-memory caching tiers are smaller than the tier they support.  Given that RAID arrays of modern SSDs can yield over a million IOPS, it's not impossible to imagine a disk-backed database which does the same.

This is also false where network RTT is higher than the time to process a request on the backend.  If a request takes < 100us from the backend's perspective, but network overhead adds 10ms from the web server's perspective, the number of in-flight requests is 100x higher for the web server than for the backend.  In this situation, a database server needs to only handle 10 requests concurrently to exhaust a web server with 1000 threads for handling in-flight requests.

> I should mention that don't intend to use "niche" to mean "bad"

I didn't take it this way.  But if data.int-map weren't a drop-in replacement for a normal hash-map with integer keys, it wouldn't really be worth writing or maintaining.  If you're right about how niche the async model is, then I question the need for any sort of update at all; just leave it to the supersets exposed by Aleph, Pedestal, et al.

As far as I can tell, the only value of having a standard API for async is that the core Ring middleware can be async-compatible.  Since only a few of these modify the response (which is the only case where special async middleware is required), maybe we can just make them pluggable?  That certainly doesn't solve the general form of the problem, but speaking from experience, that's a very hard problem to solve in a way that doesn't bring a lot of API baggage with it.

As far as examples, I just meant that when you're building a product, it's common to have prototypical customers that you can reference when evaluating a new feature or design.  We could define half a dozen or so use cases for Ring which we think span the design space, and try to understand how our approaches succeed or fail for each.

At the end of the day, this design is easy for me to support in Aleph, and I don't feel the need to strongly argue against it.  Certainly I understand the desire to use something simple and universal like callbacks, instead of a more complex promise-like data structure.  But I can't reconcile the fact that you think async is so niche and that you think this was a worthwhile change.  I'd be interested to hear your thoughts.

--

[James Reeves]

On 7 July 2016 at 23:58, Zach Tellman <ztel...@gmail.com> wrote:

If you're right about how niche the async model is, then I question the need for any sort of update at all; just leave it to the supersets exposed by Aleph, Pedestal, et al.

I consider asynchronous handlers to be a narrow use-case, in that I think synchronous handlers are the better solution for a majority of applications. However, that doesn't necessarily mean the number of people needing asynchronous support is insignificant, merely that they exist in the minority.

With the current proposal, the existing functionality of Ring would be unaffected. Supporting something that would benefit a minority, while not affecting the majority, seems like something that has few disadvantages. Clojure has plenty of tools, even in the core language, that are not commonly used, but are extremely useful the small number of times you actually need them.

I could leave it to Aleph, Pedestal, and other libraries, and indeed that's what Ring has done so far. However, my concern is that by doing that we encourage continuing incompatibility. With Aleph, for instance, one needs to rewrite Ring middleware to be compatible with handlers that return a deferred. With asynchronous middleware, we can convert between Ring and Aleph automatically.

Pedestal's interceptors are trickier, because an interceptor can be converted into a Ring handler, but not vice versa. However, because the synchronous and asynchronous parts of middleware will share the same code, we end up naturally separating middleware into "request" and "response" parts, which makes it easier to convert middleware to interceptors.

- James