Enumerable Templates
Joshua Peek
Heres the basic idea:
template = Tilt::ERBTemplate.new { 'Hey <%= name %>!' }
enum = template.enum_for(nil, :name => "Joshua")
enum.each { |part| puts part } # "Hey ", "Josh", "!"
The obvious use case is to use tilt templates as rack bodies so we
could stream the template out as its rendering.
I sketched out a rough implementation for ERB and Erubis.
http://github.com/josh/tilt/commit/6335219ddcae3d2ed2c6c369aff0c96ef83d327c
I'm not really sure about any of the method names, all that could
radically change.
But do you like the idea?
Implementation concerns
1) We'd need a default implementation for "unsupported" templates. I
think they should just yield the result as a single chunk.
2) We probably want to expose a public api for initializing the
template buffer to something other than "". I can think of some
alternate use cases here. In Rails, they use a "HtmlSafeBuffer" class.
And we can probably do better than a "_tilt_buf" local hack.
3) Though tilt isn't concerned with partials, we should think about
how Sinatra (and others) would want to pass its buffer along to
subtemplates.
Magnus Holm
The implementation itself looks fine enough, but the question is if we
want to support it in the core.
First of all, I believe that enumerable templates are *not* a good
thing when it comes to HTTP. Here's something I wrote some time ago:
---
I’ve been doing some research for this earlier, and my conclusion was:
This is very hard, if not impossible, to implement automatically. The
main problem is that it’s impossible to handle exceptions correctly
without making the whole stack aware of it.
Currently, when an exception occurs, the system can simply change the
response (since the response hasn’t been sent to the client yet, but
is only buffered inside the system). With this approach, a response
can be in x different states: before flushing, after the 1st flushing,
… and after the xth flushing. And after the 1st flushing, the status,
headers and some content has been sent to the client.
Imagine that something raises an exception after the 1st flushing.
Then a 200 status has already been sent, togeher with some headers and
some content. First of all, the system has to make sure the HTML is
valid and at least give the user some feedback. It’s not impossible,
but still a quite hard problem (because ERB doesn’t give us any hint
of where tags are open/ closed). The system also need to take care of
all the x different state and return correct HTML in all of them.
Another issue is that we’re actually sending an error page with a 200
status. This means that the response is cacheable with whatever
caching rules you decied earlier in the controller (before you knew
that an error will occur). Suddenly you have your 500.html cached all
over the placed, at the client-side, in your reverse proxy and
everywhere.
Let’s not forget that exceptions don’t always render the error page,
but do other things as well. For instance, sometimes an exception is
raised to tell the system that the user needs to be authenticated or
doesn’t have permission to do something. These are often implemented
as Rack middlewares, but with automatic flushing they also need to
take care of each x states. And if it for instance needs to redirect
the user, it can’t change the status/headers to a 302/ Location if
it’s already in the 1st state, and therefore needs to inject a
<script>window.location=’foo’</ script> in a cacheable 200 response.
Of course, the views shouldn’t really raise any exceptions because it
should be dumb. However, in Rails it’s very usual in Rails to defer
the expensive method calls to the view. The controllers sets
everything up, but it’s not until it needs to be rendered that it’s
actually called. This increases the possibilty that an exception is
raised in the rendering phrase.
Maybe I’m just not smart enough, but I just can’t come up with a way
to tackle all of these problems (completely automated) without
requiring any changes in the app.
---
I feel that this change will make Tilt more complex without any
immediate gain. In order to use enumerated template you'll have to
make sure the whole stack supports it, and currently no stack supports
it. Those who wish to add support can easily write a special
Template-class for each engine. If it turns out to be useful, *then*
we can merge it into Tilt.
--
// Magnus Holm
Joshua Peek
layout immediately, the client would be able to start parsing and
downloading styles and scripts while the rest of the page was being
rendered. This would be an opt in thing in frameworks. Rails is going
to be doing this soon for its layouts but they need some erubis hacks.
I'd like to see Rails using tilt in the future and having this baked
in would be a good sell.
I agree this needs to evolve some more and shouldn't be merged into
master as is (not a pull request). But I do think the buffer
initialization should be considered regardless of streaming bodies
being the worst idea in the world.
Still not sure about the API but:
template = Tilt::ERBTemplate.new { 'Hey <%= name %>!' }
tempalte.render(nil, :name => "Josh", :outbuf => buf)
puts buf.to_s # => "Hey Josh!"
Supporting this allows the enumerable stuff to be laid on top of the
template api without any modifications.
On Oct 15, 2:03 am, Magnus Holm <judo...@gmail.com> wrote:
> Hey Joshua,
>
> The implementation itself looks fine enough, but the question is if we
> want to support it in the core.
>
> First of all, I believe that enumerable templates are *not* a good
> thing when it comes to HTTP. Here's something I wrote some time ago:
>
> immediate gain. In order to use enumerated template you'll have to
> make sure the whole stack supports it, and currently no stack supports
> it. Those who wish to add support can easily write a special
> Template-class for each engine. If it turns out to be useful, *then*
> we can merge it into Tilt.
>
Magnus Holm
// Magnus Holm
Konstantin Haase
On Oct 15, 6:37 am, Joshua Peek <j...@37signals.com> wrote:
> I'd like to propose adding an enumerable render API to Tilt.
>
> Heres the basic idea:
>
> template = Tilt::ERBTemplate.new { 'Hey <%= name %>!' }
> enum = template.enum_for(nil, :name => "Joshua")
> enum.each { |part| puts part } # "Hey ", "Josh", "!"
>
> The obvious use case is to use tilt templates as rack bodies so we
> could stream the template out as its rendering.
>
> I sketched out a rough implementation for ERB and Erubis.
>
> I'm not really sure about any of the method names, all that could
> radically change.
>
> But do you like the idea?
3.1's streaming templates to Sinatra. +1 from me. However, enum_for is
a rather bad name, as it is overloading Object#enum_for (introduced in
Ruby 1.8.7) and could therefore lead to unexpected behavior. Also, a
Haml implementation would probably be somewhat similar.
> Implementation concerns
>
> 1) We'd need a default implementation for "unsupported" templates. I
> think they should just yield the result as a single chunk.
left to the Rack handler.
> 2) We probably want to expose a public api for initializing the
> template buffer to something other than "". I can think of some
> alternate use cases here. In Rails, they use a "HtmlSafeBuffer" class.
> And we can probably do better than a "_tilt_buf" local hack.
so this is already possible, but not for all Engines plus somewhat
hackish.
> 3) Though tilt isn't concerned with partials, we should think about
> how Sinatra (and others) would want to pass its buffer along to
> subtemplates.
Sinatra, since Sinatra uses both `yield` (for layouts) and just
calling render methods for partials. I think there are some edge cases
to be aware of when handling it (concurrency and such). Plus it would
essentially change the order templates are rendered in Sinatra (layout
first, currently it's layout last), which would break all current
`content_for` extensions out there. This would however not be an
issue, if content_for would ship with Sinatra or Tilt, which would
allow handling streaming correctly.
Konstantin
Ryan Tomayko
> I'd like to propose adding an enumerable render API to Tilt.
>
> Heres the basic idea:
>
> template = Tilt::ERBTemplate.new { 'Hey <%= name %>!' }
> enum = template.enum_for(nil, :name => "Joshua")
> enum.each { |part| puts part } # "Hey ", "Josh", "!"
>
> The obvious use case is to use tilt templates as rack bodies so we
> could stream the template out as its rendering.
>
> I sketched out a rough implementation for ERB and Erubis.
>
> http://github.com/josh/tilt/commit/6335219ddcae3d2ed2c6c369aff0c96ef83d327c
>
> I'm not really sure about any of the method names, all that could
> radically change.
>
> But do you like the idea?
I do. It's definitely an interesting concept. Actually, I've been kind
of fascinated with the basic idea for a while now. It's just cool. I
saw Yehuda's Rails 3.1 talk recently and was impressed with all the
stuff that's planned around this.
That said, as much as I find the concept interesting from an Ruby /
API design standpoint, I'm pretty skeptical about the value it
provides. Magnus laid out some good arguments around feasibility
within web frameworks but, even assuming things like exception
handling are figured out, I'm still not sure under what circumstances
this would actually be beneficial. I mean, I can imagine places where
it could be useful, but they're not typically problems associated with
template languages:
* Very large responses. Not having to store the entire response in
memory before sending to the client could theoretically be useful. But
how big does "large" have to be here to get real value? Bodies under a
few MB don't bother me.
* Long running, slowly generated responses. I can't even think of a
good example here. Maybe some kind of long running API call that wrote
the current state of some thing every 1s or something?
* Slow clients. In a multi-threaded or event based system, it could be
interesting to have a bunch of templates going at the same time, each
generating only as much data as their client is capable of immediately
receiving. But, again, this doesn't seem super beneficial unless
you're generating many large responses.
Also consider that doing many small writes on a socket can be more
expensive than buffering and doing fewer large writes. If my web
framework did support generating template output iteratively, I'd
probably disable it by having some kind
buffer-everything-as-fast-as-you-can middleware in front of it.
So, I don't know. I'm skeptical but I'd love for someone to make a
strong case for doing this because it'd be fun to hack on :)
> Implementation concerns
>
> 1) We'd need a default implementation for "unsupported" templates. I
> think they should just yield the result as a single chunk.
>
> 2) We probably want to expose a public api for initializing the
> template buffer to something other than "". I can think of some
> alternate use cases here. In Rails, they use a "HtmlSafeBuffer" class.
> And we can probably do better than a "_tilt_buf" local hack.
>
> 3) Though tilt isn't concerned with partials, we should think about
> how Sinatra (and others) would want to pass its buffer along to
> subtemplates.
The implementation looks good to me and agreed on the points listed above.
Ryan