Unit Test Mako templates with coverage

[Florian Rüchel]

I like testing. And I also like coverage. I also think templates should be tested, but they should be tested differently: You should just let each template run once with normal input. Since a template should not contain much logic beyond basic branches and statements anyway that should be enough. However, after some googling I did not find a single resource on how to test your templates.

So my first point here would be: Does noone test their templates? Am I on the wrong track? I know I can test them with integration tests but that's not what I'm going for here. To solve this problem, I used a very simple approach: I just render the template and check its output, much like with an integration test, so nothing special here.

But then I came to my second point: Since I think having just executed a template is fine, I also want to make sure I execute everything in this template, i.e. every branch and every loop should have been taken at least once. Then I can be sure there are no basic flaws in my template. The obvious solution here for a normal ".py" file would be coverage, so I tried that. Unfortunately, you have no module to refer to - but luckily Mako compiles to py files so I added a module_directory setting and now I have those files. I assumed the process would go like this:

1. Start coverage
2. Run a template test
2.1 The template gets compiled to python
2.2 The python module gets imported
3. Coverage sees the import and gives coverage information (note that, of course, I added the module_directory to coverage's source setting)

However, coverage says that the template modules were never imported. Looking at the Mako source, I see that no normal import is made, but rather an import via imp.load_source. Could this be the reason?

So my second issue is: Does Mako not import in a coverage compatible way? If not, is there even a way to measure coverage or should I just abandon my attempt on this? Or isn't this a problem with Mako and rather with coverage? I have a hard time figuring out where to ask this question.

Note that mostly I use this approach to test defs to make sure their output is valid when used under different circumstances.

Thanks in advance and regards,
Florian

[Michael Bayer]

coverage should be using the sys.settrace() function to get at code to be covered. I don't know that it has a specific dependency on using __import__ hooks, which Mako bypasses so that the compiled templates don't show up in sys.modules.

It works for me:

test script:

from mako.template import Template

with open("some_template.mako", "w") as f:
f.write("hello world")

t = Template(filename="some_template.mako", module_directory='data')

print(t.render())

usage example:

coverage run test.py
hello world
classics-MacBook-Pro:mako classic$ coverage report
Name Stmts Miss Cover
---------------------------------------------
data/some_template.mako 19 0 100%
mako/__init__ 1 0 100%
mako/_ast_util 538 437 19%
mako/ast 80 65 19%
mako/cache 63 36 43%
mako/codegen 573 370 35%
mako/compat 118 57 52%
mako/exceptions 155 117 25%
mako/ext/__init__ 0 0 100%
mako/ext/pygmentplugin 41 11 73%
mako/filters 85 47 45%
mako/lexer 258 128 50%
mako/parsetree 260 145 44%
mako/pygen 126 58 54%
mako/pyparser 364 328 10%
mako/runtime 418 263 37%
mako/template 230 103 55%
mako/util 222 144 35%
test 5 0 100%
---------------------------------------------
TOTAL 3556 2309 35%


then:

coverage annotate data/some_template.mako.py

output file looks like:

# -*- coding:ascii -*-
> from mako import runtime, filters, cache
> UNDEFINED = runtime.UNDEFINED
> __M_dict_builtin = dict
> __M_locals_builtin = locals
> _magic_number = 9
> _modified_time = 1380833728.719491
> _enable_loop = True
> _template_filename = 'some_template.mako'
> _template_uri = 'some_template.mako'
> _source_encoding = 'ascii'
> _exports = []


> def render_body(context,**pageargs):
> __M_caller = context.caller_stack._push_frame()
> try:
> __M_locals = __M_dict_builtin(pageargs=pageargs)
> __M_writer = context.writer()
# SOURCE LINE 1
> __M_writer(u'hello world')
> return ''
> finally:
> context.caller_stack._pop_frame()


So I'd say make sure coverage is running before you do anything with templates and also that it knows where to look for source code.