Re: [Rails-core] Should chunked encoding (by Rails) be optional for HTTP streaming?

[Aaron Patterson]

Hi Tim!

On Fri, Aug 24, 2012 at 06:44:02AM -0700, Tim Olsen wrote:
> Hello,
>
> I have been using the HTTP streaming feature for my company's website for a
> few months now. It has significantly decreased the page load times for our
> asset-heavy pages.
>
> I have encountered a few problems with HTTP streaming which can be traced
> to Rails always doing the chunked encoding itself:
>
> - Incompatible middleware such as newrelic (granted it's easy to solve by
> putting the newrelic header and footer in manually but there are probably
> other incompatible middleware).

I'm not familiar with this, but it seems like something newrelic should
provide?

> - Problems with running assertions against a response body in integration
> tests. The response has chunk markers in it making it difficult to parse
> the body.

Are these *rails* integration tests? If so, we should consider this a
bug. I would expect chunks to be assembled in to normal bodies in the
integration tests.

> - Incompatible servers that chunk encode again resulting in a double
> chunked encoding. I had to monkey patch Rails to not chunk encode in order
> to get streaming working properly with JRuby / Tomcat. See
> https://github.com/jruby/jruby-rack/issues/117

Not fun. :(

> I understand the chunked encoding is required in the end in order to
> stream. But I wonder if instead of having Rails doing it at the core, it
> can instead be moved to a Rack middleware that is placed at the end of the
> middleware stack when needed and can be left out when the server is capable
> of doing the chunked encoding. This could be controlled by a configuration
> option.
>
> As an intermediate step towards moving it to a Rack middleware, chunked
> encoding could first be made optional. This would at least solve the JRuby
> / Tomcat issue.

It seems entirely possible to do this. Rack has a chunking middleware
that can be inserted in the stack. I think the main problem is that by
the time the framework learns that you want a streaming response (the
point when you call render with stream), the middleware stack is already
assembled. In order to make the streaming Just Work, you'd have to add
a chunked body right at that point.

We could eliminate the chunked body [from here][1], but it means two
things:

1. You'd have to manually add the chunking middleware for *that*
controller.

2. It would break backwards compatibility

We could add the chunking middleware by default, but I'm not sure
everyone would want that.

I have another idea to fix this, but it's a bit more effort. AC::Live
doesn't chunk responses. I think it's possible to get the ERB to write
to a buffer that AC::Live uses. I've been looking in to this, but it
hasn't been my highest priority lately. :(

[1]: https://github.com/rails/rails/blob/master/actionpack/lib/action_controller/metal/streaming.rb#L219

--
Aaron Patterson
http://tenderlovemaking.com/

[Tim Olsen]

On Friday, August 24, 2012 6:58:08 PM UTC-4, Aaron Patterson wrote:

On Fri, Aug 24, 2012 at 06:44:02AM -0700, Tim Olsen wrote:
> I have encountered a few problems with HTTP streaming which can be traced
> to Rails always doing the chunked encoding itself:
>
>  - Incompatible middleware such as newrelic (granted it's easy to solve by
> putting the newrelic header and footer in manually but there are probably
> other incompatible middleware).

I'm not familiar with this, but it seems like something newrelic should
provide?

Maybe, but it seems a little redundant for everything along the middleware stack to have to dechunk if it wants to read the body and rechunk if it wants to modify the body.

 

>  - Problems with running assertions against a response body in integration
> tests.  The response has chunk markers in it making it difficult to parse
> the body.

Are these *rails* integration tests?  If so, we should consider this a
bug.  I would expect chunks to be assembled in to normal bodies in the
integration tests.

It's happening for me in rails request specs, which I assume means it is happening in rails integration tests.  The problem went away when I monkey patched Rails to not chunk.

> I understand the chunked encoding is required in the end in order to
> stream.  But I wonder if instead of having Rails doing it at the core, it
> can instead be moved to a Rack middleware that is placed at the end of the
> middleware stack when needed and can be left out when the server is capable
> of doing the chunked encoding.  This could be controlled by a configuration
> option.  
>
> As an intermediate step towards moving it to a Rack middleware, chunked
> encoding could first be made optional.  This would at least solve the JRuby
> / Tomcat issue.

It seems entirely possible to do this.  Rack has a chunking middleware
that can be inserted in the stack.  I think the main problem is that by
the time the framework learns that you want a streaming response (the
point when you call render with stream), the middleware stack is already
assembled.  In order to make the streaming Just Work, you'd have to add
a chunked body right at that point.

How about a chunking middleware that only chunk encodes if the Content-Length header is not set?  Otherwise it allows the response to pass through as is.

 

We could eliminate the chunked body [from here][1], but it means two
things:

1. You'd have to manually add the chunking middleware for *that*
   controller.

2. It would break backwards compatibility

The only thing I can think of breaking is middleware that expects a chunked encoding when Content-Length is not set.  I would hope they would detect chunked encoding by the Transfer-Encoding header, but who knows.  Is there anything else you think can break?

 

We could add the chunking middleware by default, but I'm not sure
everyone would want that.

If chunking middleware proves distasteful for too many, maybe just having a configuration option to turn off the chunking (for servers such as Tomcat) is the way to go.

 

I have another idea to fix this, but it's a bit more effort.  AC::Live
doesn't chunk responses.  I think it's possible to get the ERB to write
to a buffer that AC::Live uses.  I've been looking in to this, but it
hasn't been my highest priority lately. :( 

Interesting.  Does AC::Live just rely on the webserver to do the chunking?

Tim

[Michael Schuerig]

Is there a problem with chunking and Tomcat? jruby-rack doesn't support
it directly, but it doesn't prohibit it either. It appears to work if
you set a response_body whose #each yields properly formatted chunks and
a terminating empty chunk. I've just started to use this to pass through
(already chunked) responses from another services.

Michael

--
Michael Schuerig
mailto:mic...@schuerig.de
http://www.schuerig.de/michael/

[Tim Olsen]

On Sat, Aug 25, 2012 at 2:55 AM, Michael Schuerig <michae...@schuerig.de> wrote:

Is there a problem with chunking and Tomcat? jruby-rack doesn't support
it directly, but it doesn't prohibit it either. It appears to work if
you set a response_body whose #each yields properly formatted chunks and
a terminating empty chunk. I've just started to use this to pass through
(already chunked) responses from another services.

I and others have seen Tomcat double chunk streamed responses coming from Rails.  It may be the case, however, that this only happens for responses over a certain size.  I've filed an issue with more details at https://github.com/jruby/jruby-rack/issues/117 .

How large are your responses?  I've only tried so far to get one of our streamed pages to work in a jruby setup.  It's response is 68,019 bytes.

 

Tim

[Michael Schuerig]

On Saturday 25 August 2012, Tim Olsen wrote:

> On Sat, Aug 25, 2012 at 2:55 AM, Michael Schuerig
> <michae...@schuerig.de
>
> > wrote:
> >
> > Is there a problem with chunking and Tomcat? jruby-rack doesn't
> > support it directly, but it doesn't prohibit it either. It appears
> > to work if you set a response_body whose #each yields properly
> > formatted chunks and a terminating empty chunk. I've just started
> > to use this to pass through (already chunked) responses from
> > another services.
>
> I and others have seen Tomcat double chunk streamed responses coming
> from Rails. It may be the case, however, that this only happens for
> responses over a certain size. I've filed an issue with more
> details at https://github.com/jruby/jruby-rack/issues/117 .

I think our cases are different. I'm not using the Rails streaming
support at all. Instead, I'm setting the necessary headers and the
response_body myself. In particular, I'm setting response_body to an
object implementing #each in such a way that it yields properly
formatted chunks. jruby-rack writes these chunks individually to the
servlet's output stream. I haven't yet tested this exhaustively, but as
far as I can tell, Tomcat (or rather Winstone in my tests) simply passes
the chunks through to its browser client in turn. That is exactly the
behavior I want, as my Rails app in this case only proxies attachments
stored by another, internal webservice.

> How large are your responses? I've only tried so far to get one of
> our streamed pages to work in a jruby setup. It's response is
> 68,019 bytes.

I don't think our cases are comparable. I've tried my mechanism on cases
where the document body is at most 1MB; but I haven't seen any
indication that it would work differently for much larger or smaller
responses.

[Tim Olsen]

On Sat, Aug 25, 2012 at 10:14 AM, Michael Schuerig <michae...@schuerig.de> wrote:

I think our cases are different. I'm not using the Rails streaming
support at all. Instead, I'm setting the necessary headers and the
response_body myself. In particular, I'm setting response_body to an
object implementing #each in such a way that it yields properly
formatted chunks. jruby-rack writes these chunks individually to the
servlet's output stream. I haven't yet tested this exhaustively, but as
far as I can tell, Tomcat (or rather Winstone in my tests) simply passes
the chunks through to its browser client in turn. That is exactly the
behavior I want, as my Rails app in this case only proxies attachments
stored by another, internal webservice.

By chunks do you mean HTTP/1.1 chunks [1]?  In other words, do you set the Transfer-Encoding to "chunked" and are your chunks prefixed by the size of the chunk in hex and a CRLF?  If not, then that is not HTTP chunked encoding.  It's by removing the chunked encoding (the header, the size in hex and CRLF) that I was able to get things working properly with Tomcat.  

As for Winstone, I have tried that server and it works with the chunked encoding.

 

I don't think our cases are comparable. I've tried my mechanism on cases
where the document body is at most 1MB; but I haven't seen any
indication that it would work differently for much larger or smaller
responses.

With that size, I would expect you to see double chunked encoding.  Double chunked encoding results in the hex showing through, that is if you're not using a browser such as Chrome which refuses to display a page with two Transfer-Encoding headers.

Tim

[1] http://greenbytes.de/tech/webdav/rfc2616.html#chunked.transfer.encoding

 

[Michael Schuerig]

On Sunday 26 August 2012, Tim Olsen wrote:
> On Sat, Aug 25, 2012 at 10:14 AM, Michael Schuerig <
>
> michae...@schuerig.de> wrote:
> > I think our cases are different. I'm not using the Rails streaming
> > support at all. Instead, I'm setting the necessary headers and the
> > response_body myself. In particular, I'm setting response_body to
> > an object implementing #each in such a way that it yields properly
> > formatted chunks. jruby-rack writes these chunks individually to
> > the servlet's output stream. I haven't yet tested this
> > exhaustively, but as far as I can tell, Tomcat (or rather Winstone
> > in my tests) simply passes the chunks through to its browser
> > client in turn. That is exactly the behavior I want, as my Rails
> > app in this case only proxies attachments stored by another,
> > internal webservice.
>
> By chunks do you mean HTTP/1.1 chunks [1]? In other words, do you
> set the Transfer-Encoding to "chunked" and are your chunks prefixed
> by the size of the chunk in hex and a CRLF?

Yes. I do this myself in the #each method of the object to which I set
response_body.

> If not, then that is
> not HTTP chunked encoding. It's by removing the chunked encoding
> (the header, the size in hex and CRLF) that I was able to get things
> working properly with Tomcat.
>
> As for Winstone, I have tried that server and it works with the
> chunked encoding.

Ouch. I mistakenly thought that Winstone was an embedded variant of
Tomcat. I was probably confusing it with Trinidad. Anyway, for
Tomcat/Trinidad, now do not set the Transfer-Encoding header and I just
yield the data then Tomcat does the right thing. It does not use the
exact blocks of data for chunks that I yield, but it generates properly
formatted chunks and a trailing empty chunk.

[Tim Olsen]

If I am understanding you correctly, you are not sending chunked data to Tomcat and Tomcat does the chunking.  The problem I'm having is that Tomcat chunks again the already chunked data coming from Rails streaming.  So my solution is similar to yours (if I'm understanding you correctly): I monkey patched Rails to not send chunked data to Tomcat and Tomcat takes care of the chunking.

Tim