Mustache and JSON examples

[Erik S]

Quick overview

CFEngine's (relatively) recent support for Mustache templates, coupled with JSON input data, is a big deal - and a game changer, in my opinion. It makes it trivially simple to write dynamic promises - i.e. promises where input data can be tweaked to affect the system-state outcome rather than modifying a promise file, syntax-checking it, committing it to git, etc. In other words, a mature promise ruleset can remain static and input data can be modified to meet changing system needs.

To the end of making this valuable capability accessible to a wider audience (e.g. dummies like me), I want to share some brief example code here on the group / list.



Promise example

bundle agent tester_test
{

vars:

    "s_conf" string => "/usr/local/etc/tester.mustache";
    "s_json" string => "/usr/local/etc/tester.json";
    
    # Read in JSON file, then assign array of values to
    # a promise slist
 
    "s_data" data => readjson("$(s_json)", 50000);
    "s_names" slist => getvalues("s_data[servernames]");

files:

    # Read in JSON file for use in Mustache template

    "/tmp/tester.rendered"
    handle          => "xx_render_mustache",
    create          => "true",
    perms           => mog("644","root","root"),
    template_method => "mustache",
    edit_template   => "$(s_conf)",
    template_data   => readjson("$(s_json)", 50000),
    classes         => if_repaired("xx_fixed_tester");

commands:

    # Using a class + depends_on ensures that this command
    # only runs after the Mustache template is successfully
    # rendered

    xx_fixed_tester::
    "/bin/touch /tmp/tester.ran-command"
    depends_on      => { "xx_render_mustache" };

reports:

    # Report the slist read in from JSON

    "$(s_names)";

}

JSON example (tester.json)

{
    "nodejsapps": [
        {
            "name": "sales",
            "port": 3110
        },
        {
            "name": "shipping",
            "port": 3116
        },
        {
            "name": "inventory",
            "port": 3115
        }
    ],
    "servernames": [
        "trivia",
        "lofn",
        "victoria",
        "nesoi"
    ],
    "trivia": [
        {
            "pythonapps": [
                {
                    "name": "sales",
                    "port": 8110
                },
                {
                    "name": "shipping",
                    "port": 8116
                },
                {
                    "name": "inventory",
                    "port": 8115
                }
            ]
        }
    ]
}

Mustache example (tester.mustache)

This template demos the use of JSON values / arrays.

For more on Mustache, read the mustache(5) manpages
and the CFEngine documentation on template_method  =>  "mustache"

-- array multi-value expansion (start) --
{{#nodejsapps}}
{{name}}:{{port}}
{{/nodejsapps}}
-- array multi-value expansion (end) --

-- nested array expansion (start) --
{{#trivia}}
{{#pythonapps}}
{{name}}
{{/pythonapps}}
{{/trivia}}
-- nested array expansion (end) --

And that is that.

(Made several edits after initially posting - to improve clarity..)

[Neil Watson]

On Thu, Feb 04, 2016 at 03:46:42PM -0800, Erik S wrote:
> where input data can be tweaked to affect the system-state outcome
> rather than modifying a promise file, syntax-checking it, committing it
> to git, etc. In other words, a mature promise ruleset can remain static
> and input data can be modified to meet changing system needs.

Have you seen EFL? :)
https://github.com/evolvethinking/evolve_cfengine_freelib


--
Neil H Watson
Sr. Partner, Architecture and Infrastructure
CFEngine reporting: https://github.com/evolvethinking/delta_reporting
CFEngine policy: https://github.com/evolvethinking/evolve_cfengine_freelib
CFEngine and vim: https://github.com/neilhwatson/vim_cf3
CFEngine support: http://evolvethinking.com

[Ted Zlatanov]

On Thu, 4 Feb 2016 15:46:42 -0800 (PST) Erik S <schwart...@gmail.com> wrote:

Hi Erik. It's great to see your nice example. I just had one comment:

ES> "s_data" data => readjson("$(s_json)", 50000);
...
ES> "/tmp/tester.rendered"
ES> handle => "xx_render_mustache",
ES> create => "true",
ES> perms => mog("644","root","root"),
ES> template_method => "mustache",
ES> edit_template => "$(s_conf)",
ES> template_data => readjson("$(s_json)", 50000),
ES> classes => if_repaired("xx_fixed_tester");

You already have the data in `s_data` so just use

# 3.6
template_data => mergedata(s_data)

# 3.7 and higher
template_data => @(s_data)

to save a few I/O cycles. It's not a big deal though :)

Thanks
Ted

[Erik S]

On Thursday, February 4, 2016 at 7:42:32 PM UTC-6, Neil Watson wrote:

On Thu, Feb 04, 2016 at 03:46:42PM -0800, Erik S wrote:
>   where input data can be tweaked to affect the system-state outcome
>   rather than modifying a promise file, syntax-checking it, committing it
>   to git, etc. In other words, a mature promise ruleset can remain static
>   and input data can be modified to meet changing system needs.

Have you seen EFL? :)
https://github.com/evolvethinking/evolve_cfengine_freelib

Neil, definitely - I specifically thought of several past threads where you've mentioned it. I (of course) agree with your design philosophy after going through several cycles in my department like this:

1. Promise is designed to manage a webapp (i.e. its service account, its config files, and its running / not-running state)
2. Customer requirements change
3. Promise needs to be modified, then committed / pushed / fetched / merged via our git repo
   

Step #3 was proving to be an obnoxious pattern. By moving the regularly changing values to a "system local" (which is appropriate for our use case) JSON file, we've eliminated step #3.

This may seem like a no brainer to some of y'all, but it became a lot easier to justify once we felt the pain.

[Erik S]

Thanks Ted - that's a good idea. This is where language mastery (which I don't have) comes into play. I hadn't used mergedata() before; it's very useful.

And I totally agree that it's better to avoid reading from disk a second time when the values are already in RAM.

[Erik S]

On Friday, February 5, 2016 at 8:22:03 AM UTC-6, Ted Zlatanov wrote:

And now that I'm eyeballing this again, I also could have re-used my data variable in the files promise.
 

    template_data   => "$(s_data)",

Didn't think of it until you mentioned this. Thanks again.