mustache templates

[Bas van der Vlies]

I am playing with mustache and have a question about how to access the data variable in mustache. There is a code about keys and values, see below.
What i want is to access the value for a key, eg:
* vars.sara_mustache_expand_file.ldata[acl_internal_clients_management]

How can I accomplish this in the mustache language?


The following pseudo code how list key/value pairs:

bundle agent sara_mustache_expand_file(bundle_name, template_data, source, target)
{

vars:
"template_dir" string => "$(g.node_status)/templates/$(bundle_name)";
"ldata" data => @(template_data);

files:
"$(target)"
comment => "Expand the template to a temporary location to prevent zero files",
create => "true",
edit_template => "$(template_dir)/$(source)",
template_method => "mustache",
classes => sara_if_repaired( "template_expanded_$(source)", "template_$(source)" );
}

Now I can use this code for template generation:

==== template.mustache
<h1>{{vars.sara_mustache_expand_file.template_dir}}</h1

// ACL - internal clients management network allowed to query Bas
acl "internal_clients_management" {
{{#vars.sara_mustache_expand_file.ldata}}
{{@}} : {{%.}};
{{/vars.sara_mustache_expand_file.ldata}}
};
=====

This is the output of the generated file:

<h1>/etc/node_status/templates/bind</h1

// ACL - internal clients management network allowed to query Bas
acl "internal_clients_management" {
acl_internal_clients_management : [
{
"ip": "145.100.12.242"
}
];
acl_internal_masters : [
{
"ip": "145.100.29.50"
}
];


---
Bas van der Vlies
| Operations, Support & Development | SURFsara | Science Park 140 | 1098 XG Amsterdam
| T +31 (0) 20 800 1300 | bas.van...@surfsara.nl | www.surfsara.nl |

[Neil Watson]

See if this helps:
http://evolvethinking.com/mustache-template-in-cfengine/

--
Neil H Watson
Sr. Partner, Architecture and Infrastructure
CFEngine reporting: https://github.com/evolvethinking/delta_reporting
CFEngine policy: https://github.com/evolvethinking/evolve_cfengine_freelib
CFEngine and vim: https://github.com/neilhwatson/vim_cf3
CFEngine support: http://evolvethinking.com

[Bas van der Vlies]

thanks nick for the pointer. I found it out. I will make a example or blog. First explain why I want this. I wanted to use the classes/vars that cfengine will record and can be extracted with
the ‘datastate’ function. But this function is very expensive one, execution time will be raised significantly. Mustache does not need the ‘template_data’ to be filled in. It used the internal cfengine data structure. This
much faster then merging of json data with datastate no the solution. To access:
{
"acl_internal_clients_management": [
{"ip": "145.100.12.242"}
{"ip": "145.100.12.243"}
],
};

acl "internal_clients_management" Bas {
{{#vars.sara_mustache_expand_file.ldata.acl_internal_clients_management}}
{{ip}}
{{/vars.sara_mustache_expand_file.ldata.acl_internal_clients_management}}
};

Noticed the dot notation!. This works for me ;-)

> --
> You received this message because you are subscribed to the Google Groups "help-cfengine" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to help-cfengin...@googlegroups.com.
> To post to this group, send email to help-c...@googlegroups.com.
> Visit this group at http://groups.google.com/group/help-cfengine.
> For more options, visit https://groups.google.com/d/optout.

[Ted Zlatanov]

On Fri, 25 Sep 2015 11:28:14 +0200 Bas van der Vlies <bas.van...@surfsara.nl> wrote:

BvdV> I wanted to use the classes/vars that cfengine will record and can
BvdV> be extracted with the ‘datastate’ function.

Great. This function has other uses too, not just Mustache.

BvdV> But this function is very expensive one, execution time will be
BvdV> raised significantly.

You'd call it just once, of course.

BvdV> Mustache does not need the ‘template_data’ to be filled in. It
BvdV> used the internal cfengine data structure. This much faster then
BvdV> merging of json data with datastate no the solution.

The Mustache code calls the same code as datastate() except it uses a
temporary variable. The speed when merging with more JSON data
shouldn't be too bad, you're only creating one persistent variable:

vars:
"my_data" data => datastate();
"my_overlay" data => ... whatever you need ...;

...

files:
"myfile" ... template_data => mergedata(my_data, my_overlay);

Also look at the bundlestate() function which is just like datastate()
except it only collects the variables of a single bundle, so it's a bit
faster.

Ted

[Bas van der Vlies]

> On 25 sep. 2015, at 12:04, Ted Zlatanov <t...@lifelogs.com> wrote:
>
> On Fri, 25 Sep 2015 11:28:14 +0200 Bas van der Vlies <bas.van...@surfsara.nl> wrote:
>
> BvdV> I wanted to use the classes/vars that cfengine will record and can
> BvdV> be extracted with the ‘datastate’ function.
>
> Great. This function has other uses too, not just Mustache.
>
> BvdV> But this function is very expensive one, execution time will be
> BvdV> raised significantly.
>
> You'd call it just once, of course.
>

> The Mustache code calls the same code as datastate() except it uses a
> temporary variable. The speed when merging with more JSON data
> shouldn't be too bad, you're only creating one persistent variable:
>
> vars:
> "my_data" data => datastate();
> "my_overlay" data => ... whatever you need ...;
>
> ...
>
> files:
> "myfile" ... template_data => mergedata(my_data, my_overlay);
>

That is the approach I’m following:

I call it once and my execution time went from 5 seconds to 20 seconds. That was only a test for a single
template rendering bundle. I want all my variable data in json files and that I can override it, eg:
* default.json, cluster.json, host.json

I will merge this json files with cfengine and then the template file will be rendered. I merged the cfengine_data
and the json parsed data and pass this to the mustache template:
* template_data => mergedata(cfengine_data, json_parsed_data);

But this is expensive when I just use the “internal’ datastate it is much faster. I have to learn the mustache language
and maybe I am doing it wrong. I gonna do some more test with multiple mustache file template rendering.

> Also look at the bundlestate() function which is just like datastate()
> except it only collects the variables of a single bundle, so it's a bit
> faster.
>

Thanks for the hint


> Ted

[Ted Zlatanov]

On Fri, 25 Sep 2015 12:51:30 +0200 Bas van der Vlies <bas.van...@surfsara.nl> wrote:

BvdV> I call it once and my execution time went from 5 seconds to 20 seconds. That was only a test for a single
BvdV> template rendering bundle.
...
BvdV> But this is expensive when I just use the “internal’ datastate it is much faster. I have to learn the mustache language
BvdV> and maybe I am doing it wrong. I gonna do some more test with multiple mustache file template rendering.

That sounds like it needs optimization. Maybe function calls are copied
instead of reusing the variable. Maybe you have a lot of data in the
generated data container. It's hard to tell. Can you dump the state to a
file? In 3.7 you have a Mustache template extension {{%variable}} to
dump as JSON, so you'd have just {{%.}} in your template. The state for
a small self-contained test should be very small but it sounds like
you're mixing it with existing policy.

Maybe you can even profile the execution but I know that's a lot more work.

Ted

[Bas van der Vlies]

Ted thanks. I dumped everything in a file, very helpful. De file is 700K in size. We have a lot of bundles/libs that sets a lot of vars/classes. The json data for this template is very small.

[Ted Zlatanov]

On Fri, 25 Sep 2015 15:24:24 +0200 (CEST) Bas van der Vlies <bas.van...@surfsara.nl> wrote:

BvdV> Ted thanks. I dumped everything in a file, very helpful. De file is
BvdV> 700K in size.

Yeah, that's pretty extreme :) You'll definitely want bundlestate() then.

BvdV> We have a lot of bundles/libs that sets a lot of vars/classes. The
BvdV> json data for this template is very small.

Yeah, since CFEngine variables survive bundle execution, a lot of junk
can accumulate. Maybe you want to move your configuration bundle to the
beginning of your bundlesequence and call datastate() early as well, so
it doesn't collect so much data.

Also note that if you capture the datastate() in variable A and then
capture it later in variable B, variable B will contain variable A! That
will slow things down for sure.

Ted