[ANN] Benchmark of template engines

1,155 views
Skip to first unread message

slinso

unread,
Feb 8, 2016, 12:04:49 PM2/8/16
to golang-nuts
Hi,

I've created a benchmark for the template engines mentioned on the awesome-go list. The benchmark consists of two tests:
1. simple "hello world"
2. classic index page with header, footer and navigation


My conclusion is that I will pre generate templates that can not be cached and the performance is needed.
IMHO for most use-cases builtin Golang templating is sufficient.

Any feedback is welcome!

Santiago Corredoira

unread,
Feb 8, 2016, 12:33:45 PM2/8/16
to golang-nuts
Hi, nice comparison. 

I don't get why your fork of ego is so much faster that the original. What are you doing different?

slinso

unread,
Feb 9, 2016, 3:38:05 AM2/9/16
to golang-nuts

I don't get why your fork of ego is so much faster that the original. What are you doing different?

Currently I have only implemented string and integer optimisation. you can look at the code generation in (https://github.com/SlinSo/egon/blob/master/print_block.go).
The benchmarks favors string optimisations, that's why I could reduce allocs to zero in the simple benchmark.

General idea is to provide type information in the template code and optimisations are made for specific types. If no optimisations is implemented or no type provided use Sprintf with provided type or "%v" to render the content.

Egon

unread,
Feb 9, 2016, 5:33:25 AM2/9/16
to golang-nuts
Could you include how each template engine handles security. Performance isn't (usually) the most important thing. I wouldn't use any template language that hasn't mentioned how it handles security in the documentation.

+ Egon

Taru Karttunen

unread,
Feb 9, 2016, 7:17:59 AM2/9/16
to slinso, golang-nuts
On 08.02 08:52, slinso wrote:
> I've created a benchmark for the template engines mentioned on the
> awesome-go list. The benchmark consists of two tests:
> 1. simple "hello world"
> 2. classic index page with header, footer and navigation

Does this include escaping the values to avoid injections?

- Taru Karttunen

slinso

unread,
Feb 9, 2016, 8:01:12 AM2/9/16
to golang-nuts, sven...@googlemail.com
I will take another look at the engines and create an new test, that will consist of different possible XSS vulnerabilities. For example, I know that my current egon implementations only handles HTML-escaping and you would have to include go code to do url escaping.

slinso

unread,
Feb 9, 2016, 10:49:47 AM2/9/16
to golang-nuts
Could you include how each template engine handles security. Performance isn't (usually) the most important thing. I wouldn't use any template language that hasn't mentioned how it handles security in the documentation.

I thought of implementing security concerns based on OWASP XSS threats. Or did you mean other security concerns?


RULE #0 - Never Insert Untrusted Data Except in Allowed Locations
<script>{{.BaseScript}}</script>
<!-- {{.BaseHtmlComment}} -->
<div {{.BaseHtmlAttribute}}=test></div>
<{{.BaseHtmlStartElement}} href="/test" />
<style>{{.BaseStyle}}</style>

RULE #1 - HTML Escape Before Inserting Untrusted Data into HTML Element Content
<body>{{.HtmlElement}}</body>
<div>{{.HtmlElement}}</div>

RULE #2 - Attribute Escape Before Inserting Untrusted Data into HTML Common Attributes
<div attr={{.AttributeNothing}}>content</div>
<div attr='{{.AttributeSingle}}'>content</div>
<div attr="{{.AttributeDouble}}">content</div>

RULE #3 - JavaScript Escape Before Inserting Untrusted Data into JavaScript Data Values
<script>alert('{{.JsFnQuotedSingle}}')</script>
<script>x='{{.JsVar}}'</script>
<a onx='f("{{.JsFnQuotedDouble}}")' />
<a onx='f({{.JsFn}})' />
<a onx='pattern = /{{.JsPattern}}/;' />

RULE #3.1 - HTML escape JSON values in an HTML context and read the data with JSON.parse
<script>var pair = {{.JsonValue}};</script>
<script id="init_data" type="application/json">{{.JsonValue}}</script>

RULE #4 - CSS Escape And Strictly Validate Before Inserting Untrusted Data into HTML Style Property Values
<a style="border-{{.CssAttribute}}: 4px" />
<a style="align: {{.CssValue}}" />
<a style="background: '{{.CssValueSingle}}' />
<a style="background: url({{.CssValueUrl}}) />
<style>
div { background: "{{.CssValueDouble}}";}
div { background-url : {{.CssValueUrlJs}}; }
div { text-size: "expression({{.CssValueExpression}})"; }
</style>

RULE #5 - URL Escape Before Inserting Untrusted Data into HTML URL Parameter Values
<a href='{{.Url}}' />
<a href='/{{.UrlPath}}' />
<a href='?dir={{.UrlParam}}' />

RULE #6 - Sanitize HTML Markup with a Library Designed for the Job

RULE #7 - DOM XSS
<script>document.write({{.DomXss}} + document.location.hash);</script>

Egon

unread,
Feb 9, 2016, 10:58:29 AM2/9/16
to golang-nuts


On Tuesday, 9 February 2016 17:49:47 UTC+2, slinso wrote:
Could you include how each template engine handles security. Performance isn't (usually) the most important thing. I wouldn't use any template language that hasn't mentioned how it handles security in the documentation.

I thought of implementing security concerns based on OWASP XSS threats. Or did you mean other security concerns?

It simply means when it doesn't contain any information about security -- I won't use it. If the documentation already contains "You need to use manual sanitization and here are the things that can happen if you forget it.", I might consider using it. To me it shows that the person at least has done "his homework" with regards to writing a template package, knows the hard-to-get-right-stuff.

html/template approach is pretty good, i.e. it is based on http://js-quasis-libraries-and-repl.googlecode.com/svn/trunk/safetemplate.html#problem_definition approach. It contains different trade-offs/problems with using something.

Daniel Skinner

unread,
Feb 9, 2016, 6:51:35 PM2/9/16
to Egon, golang-nuts
I'm the author of damsel and the documentation doesn't discuss security.

The original implementation was in python and had details about security since it supported some rather arbitrary things, like embedded python in a sandbox. Porting to go, I removed a lot of functionality and it literally only functions as a transpiler to html that get's passed to html/template or any template package of your choice, so the discussion of such wasn't evident, or relevant.

Regardless, I don't advocate the usage of damsel or similar markups. While interesting for describing document structure, such things become an eye-sore once you need to add static content.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

slinso

unread,
Feb 12, 2016, 4:26:09 AM2/12/16
to golang-nuts
Hi Egon,

I've added a paragraph "security" to the readme. Only html/template supports contextual escaping at the moment. Thats why I will not try to implement escaping with every engine, because its not possible. I hoped, that I could use the contextual escaper from the stdlib directly, but they are not exported. Only HTML, JS and URLQuery escaper are exported.

For me it is still ok to use ego from a performance perspective, as long as you are aware of the XSS threats. It depends on your template structure if it is safe enough. Otherwise, as I said before, html/template work really well and is my first choice if performance is good enough. And that's almost always the case.

slinso

unread,
Feb 20, 2016, 4:26:04 AM2/20/16
to golang-nuts, egon...@gmail.com
Hi Daniel,

I've rerun the benchmark with go1.6 and there is some kind of regression with Damsel. I haven't investigated any further, just wanted to let you know.
Here are the results:

benchmark                          old ns/op     new ns/op     delta
BenchmarkDamsel-2                  108008        328087        +203.76%

benchmark                          old allocs     new allocs     delta
BenchmarkDamsel-2                  50             165            +230.00%


Daniel Skinner

unread,
Feb 23, 2016, 9:20:14 AM2/23/16
to slinso, golang-nuts, egon...@gmail.com
that's quite a difference. I'd be curious what the difference was for the Go template package as well since damsel essentially only produces text to be consumed by html/template. Any slow down in the std lib package would manifest itself in damsel since it really is nothing more than overhead to html/template

Rob Pike

unread,
Feb 23, 2016, 4:13:50 PM2/23/16
to Daniel Skinner, slinso, golang-nuts, Egon Elbre
If you can turn this into a small example that shows the regression, please file an issue.

-rob

slinso

unread,
Feb 24, 2016, 5:19:45 AM2/24/16
to golang-nuts, dan...@dasa.cc, sven...@googlemail.com
Sorry, my fault. This is not a regression!
I didn't implement correct error handling for the benchmark functions. Damsel template engine always executes template.Parse and before 1.6 that resulted in a template redefinition error and i did not catch that. With the introduction of the "block" statement it is now possible to redefine templates.

So allocs are now correct. But the benchmark it is not optimal for Damsel, because parsing is left out of the benchmark for the other engines.

Daniel Skinner

unread,
Feb 24, 2016, 11:05:22 PM2/24/16
to slinso, golang-nuts
looking at your test, it's understandable due to a poorly named export which is documented as "HtmlTemplate is an inefficient example of external template integration that is also used with tests using html/template". What you want is 

t, _ := damsel.ParseFile(string)
r := t.Result()

The result should be cached on disk or in memory and passed on to html/template, but at that point the benchmark doesn't make sense as you're just testing html/template again, also again why "Security: NO" doesn't make sense here as damsel is a precompilation step that takes no variable input.

The only subheadings it makes sense to add results is for "Template Engines with manual precompilation" in which case you could benchmark t.Result(), and maybe "more complex test with template inheritance (if possible)" as it supports that as documented here: https://github.com/dskinner/damsel#reusable-templates

Template inheritance may or may not be faster than html/template inheritance depending on how you construct the benchmark but the result would be misleading given the nature of damsel as a precompilation step.

slinso

unread,
Feb 25, 2016, 3:27:44 AM2/25/16
to golang-nuts, sven...@googlemail.com
i haven't read your comment "HtmlTemplate is an inefficient example of external template integration that is also used with tests using html/template", but that changes everything. now it is clear to me, that damsel is just about transpiling and using html/template afterwards. it is not suggested to be used the way i wrote the benchmark.

In my tests i used HtmlTemplate implementation and because of the damsel input that is executed with html/template the contextual escapers were not working. but if transpiling happens before that everything is fine and security is handled by html/template. i will change that part in the readme.

The only subheadings it makes sense to add results is for "Template Engines with manual precompilation" in which case you could benchmark t.Result(), and maybe "more complex test with template inheritance (if possible)" as it supports that as documented here: https://github.com/dskinner/damsel#reusable-templates
i think i it does not fit there. imho it doesnt make sense to benchmark transpilation because you would do that once on startup an cache the transpiled output as you said. if i find more transpiling template engines i could make an extra section, but at the moment i think it is better to delete damsel from the benchmark and just document that.

Template inheritance may or may not be faster than html/template inheritance depending on how you construct the benchmark but the result would be misleading given the nature of damsel as a precompilation step.
i haven't looked at the damsel inheritance implementation, but i think it should be a little faster because it happens at transpilation and html/template doensn't have to do that at runtime. currently i don't have plans to write template inheritance test cases for every engine, because the performance difference can already be seen in the simple benchmark and imho for complex templates security is even more a concern.

Daniel Skinner

unread,
Feb 25, 2016, 10:35:53 AM2/25/16
to slinso, golang-nuts
> i think it is better to delete damsel from the benchmark and just document that

sounds good

--
Reply all
Reply to author
Forward
0 new messages