control_common_bundlesequence_end .. a bug??

[H. Harun]

Hello,
I am using cfengine 3.24.2
I have a bundle called end_always that I would like to run after my autorun bundles.

This is how I declare it in def.json
{
  "inputs": [ "services/end_always.cf" ],
  "variables":
  {
    "default:def.control_common_bundlesequence_end": {
      "value": [ "end_always" ]
    }
  },
  "classes": {
    "services_autorun": [ "any::" ]
  }
}

However, bundle end_always seems to run 2 times (before and after autorun).
cfagent -v -K > /tmp/cf.txt

 verbose: Using bundlesequence =>  {"inventory_control","inventory_control","inventory_any","inventory_autorun","inventory_linux","inventory_lsb","inventory_debian","inventory_os","def","cfengine_enterprise_hub_ha","end_always","services_autorun","autorun","cfe_internal_management","mpf_main","cfengine_enterprise_hub_ha","end_always"}


The only way to run end_always once at the end is to declare an empty bundle in control_common_bundlesequence_classification.


in inputs/controls/def.cf
      "tbse" data => mergedata( "def.control_common_bundlesequence_end" );
       "tbse" data => mergedata( "def.control_common_bundlesequence_classification" );



A bug???

[Nick Anderson]

Hi, thanks for bringing it up.

Indeed, I was able to reproduce the issue. And, I think you also correctly spotted what is related to it.

https://github.com/cfengine/masterfiles/blob/843a917db0e541d9a0c21a11e7886ddc30a2bec4/controls/def.cf#L285-L290

"tbse" data => mergedata( "def.control_common_bundlesequence_end" );

"bundlesequence_end" slist => getvalues( tbse );

"tbse" data => mergedata( "def.control_common_bundlesequence_classification" );

"bundlesequence_classification" slist => getvalues( tbse );

Here a temporary variable tbse is set twice, probably with the thought that it would be fully re-defined. But when def.control_common_bundlesequence_classification is not defined, then mergedata() returns nothing.

For example here we see j2m doesn't get defined:

bundle agent main
{
      vars:
        "j1" data => '["end_always"]';

        "j1m" data => mergedata( "j1" );

        "j2m" data => mergedata( "j2" );


      reports:
        "j1m: $(with)" with => storejson( "j1m" );
        "j2m: $(with)" with => storejson( "j2m" );

}

# cf-agent --no-lock --log-level info --show-evaluated-vars=main\\. --file ./example.cf
R: j1m: [
  "end_always"
]
R: j2m: $(with)
Variable name                            Variable value                                               Meta tags                                Comment
default:main.j1                          ["end_always"]                                               source=promise
default:main.j1m                         ["end_always"]                                               source=promise

To count the number of end_always in bundlesequence:

cf-agent -KIf update.cf; cf-agent -Kv > out.log && grep "Using bundlesequence" out.log | awk '{print gsub("end_always", "&")}'

Potential ways to address:

We could make the temporary variables uniquely named:

"tbse1" data => mergedata( "def.control_common_bundlesequence_end" );
"bundlesequence_end" slist => getvalues( tbse1 );
"tbse2" data => mergedata( "def.control_common_bundlesequence_classification" );
"bundlesequence_classification" slist => getvalues( tbse2 );

Define empty lists if control var not defined (as is the case when defining from Augments), override with values of data from Augments.

We need to define an empty list or policy will error if it's not defined.

"bundlesequence_end"
  slist => { },
  if => not( isvariable( "def.control_common_bundlesequence_end" ) );
"bundlesequence_end" slist => getvalues( mergedata( "def.control_common_bundlesequence_end" ) );

"bundlesequence_classification"
  slist => { },
  if => not( isvariable( "def.control_common_bundlesequence_classification" ) );
"bundlesequence_classification" slist => getvalues( mergedata( "def.control_common_bundlesequence_classification" ) );

Nested functions doesn't work

"bundlesequence_end" slist => getvalues( mergedata( "def.control_common_bundlesequence_end" ) );
"bundlesequence_classification" slist => getvalues( mergedata( "def.control_common_bundlesequence_classification" ) );

Because getvalues() on a variable that doesnt exist returns an empty slist but getvalues() on mergedata() that fails does not return an empty list.

bundle agent main
{
      vars:
        "l" slist => getvalues( "does_not_exist"  );
        "j" slist => getvalues( mergedata( "does_not_exist" )  );
}

# cf-agent --no-lock --log-level info --show-evaluated-vars=main\\. --file ./example2.cf
Variable name                            Variable value                                               Meta tags                                Comment
default:main.l                                                                                        source=promise

I filed https://northerntech.atlassian.net/browse/CFE-4588

[Nick Anderson]

PR here: https://github.com/cfengine/masterfiles/pull/3044