Template validation

[Markus Rexhepi-Lindberg]

Hi,

Is there something equivalent to the validate function that is used for the template module in Ansible [1] for CFEngine's template engine? I have built some logic to validate the `/etc/ssh/sshd_config` config file for the sshd application before writing to it in one of my CFEngine policies but it has some drawbacks and it is not really an elegant solution.

[1] https://docs.ansible.com/ansible/latest/collections/ansible/builtin/template_module.html#parameter-validate

[Lars Erik Wik]

Hi Markus,

We don't have an attribute like `validate` in CFEngine as far as I know. However, the same functionality can be achieved by splitting up the promise. One promise for rendering a temporary file, another for copying that file into the final destination. Here is an example:

```cf3

body copy_from cp(source) {
  source => "$(source)";
}

bundle agent main {
  vars:
    "temp_file"
      string => "/tmp/TEMP.cfengine";

    "dest_file_1"
      string => "/tmp/file-1.txt";

    "dest_file_2"
      string => "/tmp/file-2.txt";

  files:
    "$(temp_file)"
      content => "Hello CFEngine";

    "$(dest_file_1)" # This file will pass validation
      copy_from => cp("$(temp_file)"),
      if => fileexists($(temp_file)),
      unless => not(returnszero("/usr/bin/grep --quiet 'Hello CFEngine' $(temp_file)", noshell));

    "$(dest_file_2)" # This file will not pass validation
      copy_from => cp("$(temp_file)"),
      if => fileexists($(temp_file)),
      unless => not(returnszero("/usr/bin/grep --quiet 'Hello World' $(temp_file)", noshell));
}
```

I used the `if` attribute to make sure the file is created before I running the command in the `unless` attribute. I use them in this order, because `if` is executed before `unless`.

By running the example, we can see that only '/tmp/file-1.txt' was created (with the exception of '/tmp/TEMP.cfengine' ofc.)

```

# cf-agent -KIf ~/example.cf
    info: Created file '/tmp/TEMP.cfengine', mode 0600
    info: Updated file '/tmp/TEMP.cfengine' with content 'Hello CFEngine'
    info: Copied file '/tmp/TEMP.cfengine' to '/tmp/file-1.txt.cfnew' (mode '600')
    info: Moved '/tmp/file-1.txt.cfnew' to '/tmp/file-1.txt'
    info: Updated file '/tmp/file-1.txt' from 'localhost:/tmp/TEMP.cfengine'
```

However, this is quite a bit of code which compared to Ansibles one liner. The validate attribute sounds like a very nice feature, thus I created a ticket (see https://northerntech.atlassian.net/browse/CFE-4356) to add it to CFEngine as well. If you plan on using this multiple places, you can consider putting it into a bundle for reusability. 

[Markus Rexhepi-Lindberg]

Hi Lars,

Thanks for the example! Much more elegant than my implementation :-)

If the content of the `$(temp_file)` would change would it then trigger the other file(s) to be updated as well (if the unless is still false)? If that's the case then I assume the `$(temp_file)` can be templated using something like the `edit_template` attribute.

[Lars Erik Wik]

Hi again Markus,

If I understand the question correctly, then the answer is yes. If we modify the example a bit, we can see this in action:

```cf3
body copy_from cp(source) {
  source => "$(source)";
}

bundle agent main {
  vars:
    "temp_file"
      string => "/tmp/TEMP.cfengine";

    "dest_file_1"
      string => "/tmp/file-1.txt";

    "dest_file_2"
      string => "/tmp/file-2.txt";

  files:
    "$(temp_file)"

      create => "true",
      edit_template => "/tmp/my_template.tmpl",
      template_method => "cfengine";

    "$(dest_file_1)"

      copy_from => cp("$(temp_file)"),
      if => fileexists("$(temp_file)"),
      unless => not(returnszero("/usr/bin/grep --quiet 'Hello CFEngine' $(temp_file)", noshell));

    "$(dest_file_2)"
      copy_from => cp("$(temp_file)"),
      if => fileexists("$(temp_file)"),

      unless => not(returnszero("/usr/bin/grep --quiet 'Hello World' $(temp_file)", noshell));
}

```

Let's create our template file:

```
# echo "Hello CFEngine" > /tmp/my_template.tmpl
```

Now we run the agent and see that `/tmp/file-1.txt` is created:

```
# cf-agent -KIf ~/example.cf
    info: Created file '/tmp/TEMP.cfengine', mode 0600

    info: Inserted the promised line 'Hello CFEngine' into '/tmp/TEMP.cfengine' after locator
    info: insert_lines promise 'Hello CFEngine' repaired
    info: Edited file '/tmp/TEMP.cfengine'

    info: Copied file '/tmp/TEMP.cfengine' to '/tmp/file-1.txt.cfnew' (mode '600')
    info: Moved '/tmp/file-1.txt.cfnew' to '/tmp/file-1.txt'
    info: Updated file '/tmp/file-1.txt' from 'localhost:/tmp/TEMP.cfengine'
```

Let's modify our template and run it again:

```

# echo "Hello World" > /tmp/my_template.tmpl
# cf-agent -KIf ~/example.cf
    info: Inserted the promised line 'Hello World' into '/tmp/TEMP.cfengine' after locator
    info: insert_lines promise 'Hello World' repaired
    info: Edited file '/tmp/TEMP.cfengine'

    info: Copied file '/tmp/TEMP.cfengine' to '/tmp/file-1.txt.cfnew' (mode '600')

    info: Backed up '/tmp/file-1.txt' as '/tmp/file-1.txt.cfsaved'

    info: Moved '/tmp/file-1.txt.cfnew' to '/tmp/file-1.txt'

    info: Updated '/tmp/file-1.txt' from source '/tmp/TEMP.cfengine' on 'localhost'
```

Here something unexpected happens... Due to pre-evaluation (see https://docs.cfengine.com/docs/3.21/reference-language-concepts-normal-ordering.html#top), the functions in the `if` and `unless` attributes are evaluated and cached before the promises are executed. Thus, the conditions are initially correct for `/tmp/file-1.txt`, but not for `/tmp/file-2.txt`. Then, `/tmp/TEMP.cfengine` is modified, and the wrong file gets updated. If we run the agent again, we can see that the correct file gets updated, but now the other file still contains the wrong content.

```
# cf-agent -KIf ~/example.cf
    info: Copied file '/tmp/TEMP.cfengine' to '/tmp/file-2.txt.cfnew' (mode '600')
    info: Moved '/tmp/file-2.txt.cfnew' to '/tmp/file-2.txt'
    info: Updated file '/tmp/file-2.txt' from 'localhost:/tmp/TEMP.cfengine'
```

We can fix this by making sure the two other `files` promises are out of context until after the rendering the temporary file `/tmp/TEMP.cfengine`. We can do this by using the classes attribute as shown below:

```

body copy_from cp(source) {
  source => "$(source)";
}

body classes check_if_changed
{
  promise_repaired => { "it_was_changed" };

}

bundle agent main {
  vars:
    "temp_file"
      string => "/tmp/TEMP.cfengine";

    "dest_file_1"
      string => "/tmp/file-1.txt";

    "dest_file_2"
      string => "/tmp/file-2.txt";

  files:
    "$(temp_file)"

      create => "true",
      edit_template => "/tmp/my_template.tmpl",
      template_method => "cfengine",
      classes => check_if_changed;

    it_was_changed::
      "$(dest_file_1)"

        copy_from => cp("$(temp_file)"),
        if => fileexists("$(temp_file)"),
        unless => not(returnszero("/usr/bin/grep --quiet 'Hello CFEngine' $(temp_file)", noshell));

      "$(dest_file_2)"
        copy_from => cp("$(temp_file)"),
        if => fileexists("$(temp_file)"),

        unless => not(returnszero("/usr/bin/grep --quiet 'Hello World' $(temp_file)", noshell));
}
```

Now the promises for `/tmp/file-1.txt` and `/tmp/file-2.txt` are executed if and only if the template `/tmp/TEMP.cfengine` changed.

```
# echo "Hello CFEngine" > /tmp/my_template.tmpl
# cf-agent -KIf ~/example.cf
    info: Inserted the promised line 'Hello CFEngine' into '/tmp/TEMP.cfengine' after locator
    info: insert_lines promise 'Hello CFEngine' repaired
    info: Edited file '/tmp/TEMP.cfengine'

    info: Copied file '/tmp/TEMP.cfengine' to '/tmp/file-1.txt.cfnew' (mode '600')
    info: Moved '/tmp/file-1.txt.cfnew' to '/tmp/file-1.txt'
    info: Updated file '/tmp/file-1.txt' from 'localhost:/tmp/TEMP.cfengine'

# echo "Hello World" > /tmp/my_template.tmpl
# cf-agent -KIf ~/example.cf
    info: Inserted the promised line 'Hello World' into '/tmp/TEMP.cfengine' after locator
    info: insert_lines promise 'Hello World' repaired
    info: Edited file '/tmp/TEMP.cfengine'
    info: Copied file '/tmp/TEMP.cfengine' to '/tmp/file-2.txt.cfnew' (mode '600')
    info: Moved '/tmp/file-2.txt.cfnew' to '/tmp/file-2.txt'
    info: Updated file '/tmp/file-2.txt' from 'localhost:/tmp/TEMP.cfengine'
```

I hope this helps!