Precompiled (Haml) template cache

[Adam Strzelecki]

Here's some mod to Sinatra 0.9 that caches precompiled templates for
Haml. I got ~3x speed increase on this on production since Haml does
not need to re-parse the template on every run, but just executes
precompiled code.

Note it does NOT cache the output (like Rails may do) but just result
of Haml.new(sourcecode) (instance) so we don't need to call it all
over again.
The problem with sinatra is it calls Haml.new
(template_sourcecode).render(locals) alltogether on every
haml :template
While locals do change on every requests, template_sourcecode is still
the same (unless we change the file), so Haml.new(template_sourcecode)
is still the same, and we waste CPU cycles calling it all over again.

This is just optional mod now, but Sinatra authors may consider adding
such caching functionality directly into Sinatra and also for ERB &
SASS (I don't use actually)

Here goes my mod:

# Sinatra 0.9.x template cache module
#
# Caches Haml templates in @@template_cache class variable, avoiding
recompilation of the
# template on every page reload.
#
# http://pastie.org/445282

class Sinatra::Base
# Overriden function responsible for calling internal Haml rendering
routines
def haml(template, options={})
return super(template, options) unless (defined? @@template_cache)
&& @@template_cache.has_key?(template)
output = @@template_cache[template].render(self, options[:locals]
|| {})
if @@template_cache.has_key?(:layout) && options[:layout] != false
@@template_cache[:layout].render(self, options[:locals] || {})
{ output }
else
output
end
end
# Overriden internal function responsible for template
precompilation and rendering
def render_haml(template, data, options, &block)
@@template_cache ||= {}
@@template_cache[template] ||= ::Haml::Engine.new(data, options
[:options] || {})
@@template_cache[template].render(self, options[:locals] || {},
&block)
end
# Clears the cache
def empty_cache!
@@template_cache = {}
end
end

[Widi Harsojo]

What about coding in HAML like syntax when in "development" and there
is a services in sinatra watching the date change of that file and
compiled it.

I've done it .
http://rubyforge.org/frs/shownotes.php?group_id=6464&release_id=33258

/wh

[candlerb]

On Apr 13, 8:33 pm, Adam Strzelecki <o...@java.pl> wrote:
> Here's some mod to Sinatra 0.9 that caches precompiled templates for
> Haml.

I think this is a good idea. However I would suggest:

- use class instance variable (@template_cache) instead of a class
variable (@@template_cache). This is because many different
applications may all be subclasses of Sinatra::Base, and they should
not share caches.

- hook deeper into the render logic so that you can do a File.stat()
on the template file, and check for changes. This check is cheap (the
inode will remain in RAM for a frequently-accessed template) but
avoids the problem of stale templates in both development and
production mode.

Perhaps render_foo should become compile_foo, which returns a closure
that performs the actual work.

Some care is needed with Proc templates. That is, it's possible to
write

template :foo, Proc { "..." }

I don't know who uses this or why, but it needs to be considered
whether it's OK to execute this only once and cache the compiled
template.

[Adam Strzelecki]

> I think this is a good idea. However I would suggest:
>
> - use class instance variable (@template_cache) instead of a class
> variable (@@template_cache). This is because many different
> applications may all be subclasses of Sinatra::Base, and they should
> not share caches.

Tried. Doesn't really work, since it seems Sinatra::Base instance, so
@template_cache, is re-created upon each request (at least at my
machine Rack+Thin).

> - hook deeper into the render logic so that you can do a File.stat()
> on the template file, and check for changes. This check is cheap (the
> inode will remain in RAM for a frequently-accessed template) but
> avoids the problem of stale templates in both development and
> production mode.

This make sense, however Sinatra doesn't reload its files when
modified on production, so I don't see why it should reload the
templates.
While on development it does reload all classes on each request.

> Perhaps render_foo should become compile_foo, which returns a closure
> that performs the actual work.

I think the performance problem also related to fast that haml =
Haml.new(template_src) after it parses Haml template does produce
internally haml.@precompiled which is not-yet evaluated Ruby src code.
And then on every haml.render(bindings) it does call
eval(@precompiled) which does Ruby compilation all over again.
It would be better if Haml.new(template_src) produced Proc instance,
but I don't have a clue how to run Proc with specified bindings, same
as it is done with eval.

Regards,
--
Adam

[candlerb]

> > - use class instance variable (@template_cache) instead of a class
> > variable (@@template_cache). This is because many different
> > applications may all be subclasses of Sinatra::Base, and they should
> > not share caches.
>
> Tried. Doesn't really work, since it seems Sinatra::Base instance, so  
> @template_cache, is re-created upon each request (at least at my  
> machine Rack+Thin).

I mean an instance variable in the class object itself. Have a look at
how 'templates' is implemented.

def lookup_template(engine, template, views_dir)
case template
when Symbol
if cached = self.class.templates[template]
...
@templates = {}
...
class << self
attr_accessor :routes, :filters, :conditions, :templates,
...
# Define a named template. The block must return the template
source.
def template(name, &block)
templates[name] = block
end

> This make sense, however Sinatra doesn't reload its files when  
> modified on production, so I don't see why it should reload the  
> templates.

I believe Rails is smart enough to do this even in production mode.
Personally I find it's useful to update templates in a running
instance.

> It would be better if Haml.new(template_src) produced Proc instance,  
> but I don't have a clue how to run Proc with specified bindings, same  
> as it is done with eval.

I don't think you can. However I'm sure I saw a solution along the
lines of the following:

x = eval <<EOS
Proc.new do |locals|
foo = locals[:foo]
bar = locals[:bar]
#.. stuff which uses foo and bar ..
end
EOS

...

x.call(:foo => 123, :bar => 456)

Ah yes, this is in Rails, in actionpack/lib/action_view/template/
renderable.rb

private
def compile!(render_symbol, local_assigns)
locals_code = local_assigns.keys.map { |key| "#{key} =
local_assigns[:#{key}];" }.join

source = <<-end_src
def #{render_symbol}(local_assigns)
old_output_buffer = output_buffer;#{locals_code};#
{compiled_source}
ensure
self.output_buffer = old_output_buffer
end
end_src

[Adam Strzelecki]

> I mean an instance variable in the class object itself. Have a look at
> how 'templates' is implemented.

Okay, I got you point, here's fixed source:

# Sinatra 0.9.x Haml template cache module
#
# Caches Haml templates in self.class.template_cache class variable,

avoiding recompilation of the
# template on every page reload.

class Sinatra::Base
class <<self
attr_accessor :template_cache
end

# Overriden function responsible for calling internal Haml
rendering routines
def haml(template, options={})

self.class.template_cache ||= {}
return super(template, options) unless
self.class.template_cache.has_key?(template)
output = self.class.template_cache[template].render(self,
options[:locals] || {})
if self.class.template_cache.has_key?(:layout) &&
options[:layout] != false
self.class.template_cache[:layout].render(self,

options[:locals] || {}) { output }
else
output
end
end
# Overriden internal function responsible for template
precompilation and rendering
def render_haml(template, data, options, &block)

self.class.template_cache[template] ||= ::Haml::Engine.new(data,
options[:options] || {})
self.class.template_cache[template].render(self, options[:locals]

|| {}, &block)
end
# Clears the cache
def empty_cache!

self.class.template_cache = {}
end
end

Regards,
--
Adam

[Adam Strzelecki]

Another improved version that caches also Ruby compiled template
trough running instance (one per request) inside
"render_haml_#{template}" local method.
This improves performance when using nested template calls inside
Haml, i.e.:
- links.each do |link|
= haml :link_body, :layout => false, :locals => {:link => link}

You may read about my benchmarks with this caching at: http://groups.google.com/group/haml/msg/92af7904b69042cd
(Actually I've replaced render_proc into def_method below, since it is
even slightly faster)

# Sinatra 0.9.x Haml template cache module
#
# Caches Haml templates in self.class.template_cache class variable,
avoiding reparsing of the

# template on every page reload.

# Caches Haml templates in "render_haml_#{template}" for running
instance, avoiding Ruby's code
# recompilation in case of loops, and nested "= haml :template" calls.

class Sinatra::Base
class <<self
attr_accessor :template_cache
end

# Overriden function responsible for calling internal Haml
rendering routines
def haml(template, options={})

self.class.template_cache ||= {}
return super(template, options) unless
self.class.template_cache.has_key?(template)

# define local method that renders the template, this improves
performance since the code is compiled once
method = "render_haml_#{template}".to_sym
self.class.template_cache[template].def_method(self, method,
*((options[:locals] || {}).keys)) unless self.respond_to?(method)
output = self.send(method, options[:locals] || {})
if self.class.template_cache.has_key?(:layout) &&
options[:layout] != false
# :layout is usually called once per instance so we won't have
much performnace improvement
# caching method here
self.class.template_cache[:layout].render(self,

options[:locals] || {}) { output }
else
output
end
end
# Overriden internal function responsible for template
precompilation and rendering
def render_haml(template, data, options, &block)

self.class.template_cache[template] ||= ::Haml::Engine.new(data,
options[:options] || {})
# define local method that renders the template, this improves
performance since the code is compiled once
method = "render_haml_#{template}".to_sym
self.class.template_cache[template].def_method(self, method,
*((options[:locals] || {}).keys))
self.send(method, options[:locals] || {}, &block)

end
# Clears the cache
def empty_cache!

[Adam Strzelecki]

Hello,

Since we got brand new 0.9.2 I decided to rework my Haml cache to be
even faster & simpler.
Actually I found out we can use Haml's def_method for our Application
class rather than just the current instance, so parsed and compiled
methods via `Haml::Engine::def_method` stay for Application's life-time.
This brings even more speed boost comparing to the code I've posted
last month, as each template is now touched by Haml engine just once,
and adds `render_haml_templatename` method that is Ruby compiled code,
that's called later on upon each template call.

Here's the code:

-------------------- CUT --------------------
# Sinatra >= 0.9.2 Haml template cache module

# Caches Haml templates in class instance "render_haml_#{template}"
methods,
# avoiding reparsing and recompiling of the template on every page
reload.

class Sinatra::Base
# Override function responsible for calling internal Haml rendering
routines
def haml(template, options={}, locals={})

method = "render_haml_#{template}".to_sym

return super(template, options, locals) unless self.respond_to?
method
locals = options.delete(:locals) || locals || {}
if options[:layout] != false
__send__(:render_haml_layout, locals) { __send__(method,
locals) }
else
__send__(method, locals)
end
end
def render_haml(template, data, options, locals, &block)

method = "render_haml_#{template}".to_sym

::Haml::Engine.new(data, options).def_method(self.class, method,
*(locals.keys))
__send__(method, locals, &block)
end
def self.clear_cache!
instance_methods.grep(/^render_haml_/).each{|m| remove_method m}
end
def clear_cache!; self.class.clear_cache! end
end
-------------------- CUT --------------------

You may add cache clear or reload based on Rack::Reloader.
I will try to post my code regarding this reloaded as soon its finished.

Have fun with my cache in your production environment :)
Waiting for your comments and suggestions...

Cheers,
--
Adam