CFEngine evaluation sequence: case studies, user surprise, and expectations

[Mike Weilgart]

I started to write this a few days ago in the thread, "In search of a CFEngine language specification (for 4.x)," but I've decided it deserves its own thread as it potentially applies just as much to future CFEngine 3 releases.

P. Christeas wrote the following:

PC> But, as you said, problem is that many of the existing promise types 

PC> (implementations, that is) have arbitrary quirks in the way they are evaluated 

PC> and executed.

Indeed.

The theory is that one shouldn't have to worry about sequence when writing CFEngine 3 policy.  The reality is that one has to worry about sequence MUCH, MUCH MORE than when writing in any language wherein commands are simply executed one after another.

Here is a real case study in CFEngine evaluation sequence.

Consider the following three promises:

    [root@hub ~]# cat nameservers.cf

    bundle agent main {

        commands:

          any::

            "/bin/sed -f $(this.promise_dirname)/nameservers.sed /etc/resolv.conf"

              classes => kept_successful_command,

              module => "true";

        vars:

          DEBUG::

            "report_indices"

              slist => getindices("inventory_module.nameserver");

        reports:

          DEBUG::

            "Inventorying nameserver: '$(inventory_module.nameserver[$(report_indices)])'";

    }

    body classes kept_successful_command {

      # from standard library

      kept_returncodes => { "0" };

    }

Below is the Sed script referenced.  It converts the nameservers from the resolv.conf file into module protocol for CFEngine inventory.  (As a side note, this sort of text parsing is extraordinarily difficult/overcomplicated with pure CFEngine, but as I'm quite decent with POSIX text processing tools, this doesn't bother me too much.)

    [root@hub ~]# cat nameservers.sed 

    #!/bin/sed -f

    1i\

    ^context=inventory_module

    1i\

    ^meta=inventory,attribute_name=Nameservers

    /^nameserver[[:space:]]*/!d

    s///

    s/[[:space:]]*$//

    s/.*/=nameserver[&]=&/

Note the vars promise and reports promise in the bundle up above, written so we can see what CFEngine is adding to inventory.

Now, before going any further—do you see the error in the policy up above?  (If you do see it, it's possible you've been spending way too much debugging CFEngine's evaluation sequence.)  ;)

What do we get if we run this policy with the DEBUG class set?

    [root@hub ~]# cf-agent -KIC -DDEBUG -f ./nameservers.cf 

        info: Executing 'no timeout' ... '/bin/sed -f /root/./nameservers.sed /etc/resolv.conf'

        info: Command related to promiser '/bin/sed -f /root/./nameservers.sed /etc/resolv.conf' returned code defined as promise kept 0

        info: Completed execution of '/bin/sed -f /root/./nameservers.sed /etc/resolv.conf'

    R: Inventorying nameserver: '$(cf_null)'

    R: Inventorying nameserver: '10.0.2.3'

    [root@hub ~]# 

Why do we see cf_null?  We don't want cluttering up our inventory, do we?

Well actually, it doesn't show up in inventory.  Just in the debug reporting.  Try and see if you can work out why.

Then, a pop quiz (actually a trick question): Where will you add the depends_on attribute to avoid this noise in the debug reporting?

Now please notice that these extra two promises, added for the sole purpose of assisting with debugging the commands promise, themselves need quite a bit of debugging before they work as expected.

(Can anyone guess what's going on in the above code and what would have to be done to fix it?)  :)

--------

I posit that it would be less surprising and easier to learn (and to debug!) if CFEngine simply evaluated promises in the sequence they are specified in a bundle, unless overridden with "depends_on" attributes.

And actually, considering the great number of "isvariable()" calls I have had to add in a single bundle to ensure that production data didn't get polluted with unresolved variables, I think that passing unresolved variables as-is is a serious misfeature.  This is about the only "magic ordering" I would personally support: skip any promise that makes reference to an unresolved variable (just like most CFEngine function calls already do.)

Interested in the thoughts of the developers on these points.

Best,

--Mike Weilgart

Vertical Sysadmin, Inc.

[Ted Zlatanov]

On Thu, 1 Dec 2016 22:47:23 -0800 (PST) Mike Weilgart <mike.w...@verticalsysadmin.com> wrote:

MW> I posit that it would be *less* surprising and easier to learn (and to
MW> debug!) if CFEngine simply evaluated promises *in the sequence they are
MW> specified* in a bundle, unless overridden with "depends_on" attributes.

Regarding the convergence model, I proposed a simple way to provide
ordered evaluation with today's core:

https://tracker.mender.io/browse/CFE-2399?focusedCommentId=73474&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-73474

The nice part in that proposal is that if we provide a way to "rank"
promises, then evaluation in the order as written would simply use
`rank=$(current_linenumber"` for every promise.

MW> And actually, considering the great number of "isvariable()" calls I have
MW> had to add in a single bundle to ensure that production data didn't get
MW> polluted with unresolved variables, I think that passing unresolved
MW> variables as-is is a *serious* misfeature. This is about the only "magic
MW> ordering" I would personally support: skip any promise that makes reference
MW> to an unresolved variable (just like most CFEngine function calls already
MW> do.)

If you look back in the cfengine-devel archives, William Orr did work
last year on a patch along those lines. I don't know what happened with
it.

I personally would prefer all evaluation to be tunable per bundle
definition *and* per bundle call. Different contexts require different
techniques.

For example, you don't want independent security measures to stop
working because of a shell error somewhere in one of them. But you do
want application deployment to abort if the port number is not defined.

Ted

[William Orr]

Still listening :-)

I have the early progress on it, though it's still very rough. At this
point, since I've switched jobs at LinkedIn, I no longer have access
to the CFE infrastructure or the time to work on it on hours.

I don't realistically have time to get a substantial amount of time
done on it near term, but I can show what I had before I had to put it
down.