surprsiing promises outcomes

[Jeroen van den Haspel]

Dear all,

When changing a file including permissions I get surprising results back when setting classes based on if_repaired and if_ok.

I made an overview in the attached file.

Usually I would check for if_repaired. If that's the case then the file has changed and a follow up action can be done for example.

If not set then the file would be unchanged and therefore be ok like with promise_kept.

But, in case the file cannot be changed due to read only attribute (immutable flag) or read only file system, it happens that if_repaired flag class is set while the file is still unchanged....

I even had the situation that promise_kept, promise_repaired, repair_failed ánd repair_denied all became true at the same time....

I am actually looking for a simple check with three possible autcomes like ok, changed and error.

So I made a custom body class for it.

body classes ok_changed_error(x)

{

      promise_kept     => { "$(x)_ok" };

      promise_repaired => { "$(x)_changed" };

      repair_failed    => { "$(x)_error" };

      repair_denied    => { "$(x)_error" };

      repair_timeout   => { "$(x)_error" };

}

And to check the different errors:

body classes ok_changed_error(x)

{

      promise_kept     => { "$(x)_ok" };

      promise_repaired => { "$(x)_changed" };

      repair_failed    => { "$(x)_failed" };

      repair_denied    => { "$(x)_denied" };

      repair_timeout   => { "$(x)_timeout" };

}

Any ideas?

Regards,

Jeroen.

[Nick Anderson]

Jeroen van den Haspel <jvdh...@gmail.com> writes:

> When changing a file including permissions I get surprising results back

> when setting classes based on if_repaired and if_ok.

> I made an overview in the attached file.

Hi Jeroen,

When posting policy questions to the list, it's best if you can include a self
contained runnable example.

It's hard to tell exactly what you are trying to do but I think you might be
confused by multiple outcomes from compound promises, or multiple outcomes as
the result of subsequent passes.

First, promises can have multiple outcomes. The devil is in the details.

Let's start by creating an immutable file.

#+Caption Create an immutable file

touch /tmp/immutable_file
chatter +i /tmp/immutable_file
lsattr /tmp/immutable_file

Illustration of multiple outcomes.

bundle agent main
{
  vars:
      "immutable" string => "/tmp/immutable_file";
  files:

      # Here we make a promise that the immutable file exists. Note we do not
      # make any statement about its extended attributes, just that the file
      # should exist. We attach the results classes body to make it easy to
      # visualize the classes that are defined by the promise. Because this file
      # exists before the agent run we expect this promise to be kept.

      "$(immutable)"
        handle => "immutable_present",
        create => "true",
        classes => results("bundle", "immutable_present");

      # Here we we make no promise not that the file exists, but we do promise
      # that if the file does exist, it should contain the current value of the
      # $(sys.date) variable which is initalized at the beginning of the agent
      # run. Because this file is immutable and does not contain the desired
      # string, we expect that the agent will attempt to correct the content and
      # fail, resulting in a promise not kept.

      "$(immutable)"
        handle => "immutable_content_date",
        edit_line => append_if_no_line( $(sys.date) ),
        classes => results("bundle", "immutable_content_date");


      # Here we promise not only that the file exists, but also that it contains
      # a line with the current value of the $(sys.date) variable which is
      # initalized at the beginning of the agent run. This promise compounds
      # multiple promises into a single statement. We expect that the promise
      # will be kept because the file does indeed exist, however we also expect
      # that the agent will fail to correct the content of the file since it is
      # immutable.

      "$(immutable)"
        handle => "immutable_present_content_date",
        create => "true",
        edit_line => append_if_no_line( $(sys.date) ),
        classes => results("bundle", "immutable_present_content_date");


  vars:
      "immutable_present" slist => classesmatching("immutable_present_.*");
      "immutable_content_date" slist => classesmatching("immutable_content_date_.*");
      "immutable_present_content_date" slist => classesmatching("immutable_present_content_date_.*");

  reports:
      "CFEngine Version: $(sys.cf_version)";
      "$(immutable_present)";
      "$(immutable_content_date)";
      "$(immutable_present_content_date)";
}

It's also possible to have a condition where on the first pass (remember when
evaluating policy cf-agent will give each explicitly actuated bundle three
passes of evaluation) a promise is not_kept, but later in that pass something
changes wich allows the promise to succeed resulting in the promise now having a
kept outcome.

Also of potential interest is the depends_on attribute. It works on promise
outcomes, not classes. You can have one promise depend on one or more promise
handles. The promise that depends on the other will only actuate if all of the
other listed promises have had an outcome of kept or repaired (this is
independant of having any classes defined as a result of the other promsies).

I hope this helps.

[Nick Anderson]

On 10/03/2016 02:32 PM, Nick Anderson wrote:
> Jeroen van den Haspel <jvdh...@gmail.com> writes:
>
>> When changing a file including permissions I get surprising results back

>> when setting classes based on if_repaired and if_ok .

>> I made an overview in the attached file.
>
> Hi Jeroen,
>
> When posting policy questions to the list, it's best if you can include
> a self
> contained runnable example.
>
> It's hard to tell exactly what you are trying to do but I think you might be
> confused by multiple outcomes from compound promises, or multiple
> outcomes as
> the result of subsequent passes.
>

> First, *promises can have multiple outcomes*. The devil is in the details.

This output was supposed to follow the policy. (not shown in the example
I included the stdlib so that I could use the results classes body.


error: Can't rename '/tmp/immutable_file.cf-after-edit' to
'/tmp/immutable_file' - so promised edits could not be moved into place.
(rename: Operation not permitted)
error: Unable to save file '/tmp/immutable_file' after editing
R: CFEngine Version: 3.9.1
R: immutable_present_content_date_kept
R: immutable_present_content_date_reached
R: immutable_present_kept
R: immutable_present_reached
R: immutable_content_date_failed
R: immutable_content_date_not_kept
R: immutable_content_date_error
R: immutable_content_date_reached

[Jeroen van den Haspel]

Thank you Nick!

You wrote: "First, *promises can have multiple outcomes*. The devil is in the details."

This seems to happen indeed! I am working very shortly with CFEngine and my colleague showed and proved the multiple outcomes.

And indeed it would have been useful to post my code as well. Find below my code, it's part of a larger set so it contains some variables.

********************

bundle agent check_user_and_copy_private_key(username) {

 vars:

  "privatekeydir"     string => "/etc/.ssh/private/service";

 classes:

  "user_exists"   expression => userexists("$(username)");

 files:

  user_exists::

   "$(privatekeydir)/$(username).priv"

     comment                     => "Copy private rsa key",

     create                      => "true",

     perms                       => mo("0600","$(username)"),

     action                      => immediate_dlc,

     copy_from                   => cp_dlc("$(file_location.systemfiles_common)/rsa_keys/private/service/$(username).priv"),

     classes                     => ok_changed_error("private_key_$(username)");

 reports:

  !user_exists::

   "WARNING : ${this.bundle} : User $(username) does not exist. Private key NOT copied!";

  "private_key_$(username)_ok"::

   "OK      : ${this.bundle} : User private key $(username) already uptodate";

  "private_key_$(username)_changed"::

   "CHANGED : ${this.bundle} : User private key $(username) copied";

  "private_key_$(username)_failed"::

   "FAILED  : ${this.bundle} : Cannot copy private key) $(username)";

  "private_key_$(username)_denied"::

   "DENIED  : ${this.bundle} : Cannot copy private key) $(username)";

  "private_key_$(username)_timeout"::

   "TIMEOUT : ${this.bundle} : Cannot copy private key) $(username)";

}

body classes ok_changed_error(x)

{

      promise_kept     => { "$(x)_ok" };

      promise_repaired => { "$(x)_changed" };

      repair_failed    => { "$(x)_failed" };

      repair_denied    => { "$(x)_denied" };

      repair_timeout   => { "$(x)_timeout" };

}

***********************

Now a snippet from the verbose logging when tmpadmin_jeroen is processes. The file exists but content, ownership and permissions are wrong and would be set by CFEngine if file wasn't read only.

 verbose: File '/etc/.ssh/private/service/tmpadmin_jeroen.priv' exists as promised

 verbose:

 verbose: C:    + promise outcome class 'private_key_tmpadmin_jeroen_ok'

 verbose:

 verbose: Handling file existence constraints on '/etc/.ssh/private/service/tmpadmin_jeroen.priv'

    info: Owner of '/etc/.ssh/private/service/tmpadmin_jeroen.priv' was 0, setting to 211224

 verbose:

 verbose: C:    + promise outcome class 'private_key_tmpadmin_jeroen_changed'

 verbose:

    info: Cannot set ownership on file '/etc/.ssh/private/service/tmpadmin_jeroen.priv'. (chown: Operation not permitted)

 verbose:

 verbose: C:    + promise outcome class 'private_key_tmpadmin_jeroen_denied'

 verbose:

   error: chmod failed on '/etc/.ssh/private/service/tmpadmin_jeroen.priv'. (chmod: Operation not permitted)

 verbose: File '/etc/.ssh/private/service/tmpadmin_jeroen.priv' copy_from '/var/cfengine/dc_data/systemfiles/common/rsa_keys/private/service/tmpadmin_jeroen.priv'

 verbose: Destination file '/etc/.ssh/private/service/tmpadmin_jeroen.priv' already exists

 verbose: Image file '/etc/.ssh/private/service/tmpadmin_jeroen.priv' has a wrong digest/checksum, should be copy of '/var/cfengine/dc_data/systemfiles/common/rsa_keys/private/service/tmpadmin_jeroen.priv'

 verbose: Copy of regular file succeeded '/var/cfengine/dc_data/systemfiles/common/rsa_keys/private/service/tmpadmin_jeroen.priv' to '/etc/.ssh/private/service/tmpadmin_jeroen.priv.cfnew'

   error: Could not install copy file as '/etc/.ssh/private/service/tmpadmin_jeroen.priv', directory in the way?. (rename: Operation not permitted)

 verbose:

 verbose: C:    + promise outcome class 'private_key_tmpadmin_jeroen_failed'

 verbose:

   error: Was not able to copy '/var/cfengine/dc_data/systemfiles/common/rsa_keys/private/service/tmpadmin_jeroen.priv' to '/etc/.ssh/private/service/tmpadmin_jeroen.priv'

 verbose:

 verbose: C:    + promise outcome class 'private_key_tmpadmin_jeroen_failed'

 verbose:

 verbose: Handling file existence constraints on '/etc/.ssh/private/service/tmpadmin_jeroen.priv'

    info: Owner of '/etc/.ssh/private/service/tmpadmin_jeroen.priv' was 0, setting to 211224

 verbose:

 verbose: C:    + promise outcome class 'private_key_tmpadmin_jeroen_changed'

 verbose:

    info: Cannot set ownership on file '/etc/.ssh/private/service/tmpadmin_jeroen.priv'. (chown: Operation not permitted)

 verbose:

 verbose: C:    + promise outcome class 'private_key_tmpadmin_jeroen_denied'

 verbose:

   error: chmod failed on '/etc/.ssh/private/service/tmpadmin_jeroen.priv'. (chmod: Operation not permitted)

 verbose: A: Promise NOT KEPT - denied

***********

Now I see that there are multiple outcomes! Thanks for pointing me in the right direction!

So the file block for "$(privatekeydir)/$(username).priv" cannot be seen as "one" action.

Is there a way to threat the combination of create-perms-copy as one action with one outcome? 

Regards,

Jeroen.

Op maandag 3 oktober 2016 21:35:46 UTC+2 schreef Nick Anderson:

[Aleksey Tsalolikhin]

Dear Jeroen,

Here are a few more resources you might find useful when starting to work with CFEngine:

https://digitalelf.net/cf-primer/ (short primer)

http://www.cfenginetutorial.org/ (longer tutorial)

https://docs.cfengine.com/lts/guide-language-concepts.html (documenation on CFEngine language basics from the vendor)

http://shop.oreilly.com/product/0636920022022.do (Diego's excellent book)

Sincerely,

Aleksey

-- 

Need training on CFEngine, Git or Time Management?  Email trai...@verticalsysadmin.com.

--
You received this message because you are subscribed to the Google Groups "help-cfengine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to help-cfengine+unsubscribe@googlegroups.com.
To post to this group, send email to help-c...@googlegroups.com.
Visit this group at https://groups.google.com/group/help-cfengine.
For more options, visit https://groups.google.com/d/optout.

ᐧ

[Nick Anderson]

Jeroen van den Haspel <jvdh...@gmail.com> writes:

> Thank you Nick!

You're most welcome.

files:

 user_exists::
  "$(privatekeydir)/$(username).priv"
    comment                     => "Copy private rsa key",
    create                      => "true",
    perms                       => mo("0600","$(username)"),
    action                      => immediate_dlc,
    copy_from                   => cp_dlc("$(file_location.systemfiles_common)/rsa_keys/private/service/$(username).priv"),
    classes                     => ok_changed_error("private_key_$(username)");

> So the file block for "(username).priv" cannot be seen as

> "one" action.
> Is there a way to threat the combination of create-perms-copy as one action
> with one outcome?

Not exactly. It's considered "best practice" to make the smallest possible
promises. One thing you could do is to break that promise up in to it's smallest
parts and then place them into a bundle together. Bundles when actuated as
methods also have outcomes. They roll up the worst result. So if any promise in
a bundle is not kept the whole bundle is considered not kept.

bundle agent main
{
  methods:
      "kept"
        classes => results("bundle", "test_first_bundle");

      "not_kept"
        classes => results("bundle", "test_second_bundle");

      "repaired"
        classes => results("bundle", "test_third_bundle");
  vars:
     "c" slist => classesmatching("test_.*");

  reports:
      "$(c)";
}

bundle agent kept
{
  reports:
      "Reports are always kept, so this bundle should be kept";
}

bundle agent not_kept
{
  commands:
      "/bin/false"; # not_kept
      "/bin/true"; # repaired

  reports:
      "Any promise resulting in  not kept results in the bundle being not kept";
}

bundle agent repaired
{
  commands:
    "/bin/true";

  reports:
    "One promise kept, one promise repaired results in a repaired bundle because repaired is worse than kept";
}

R: Reports are always kept, so this bundle should be kept
   error: Finished command related to promiser '/bin/false' -- an error occurred, returned 1
R: Any promise resulting in  not kept results in the bundle being not kept
   error: Method 'not_kept' failed in some repairs
R: One promise kept, one promise repaired results in a repaired bundle because repaired is worse than kept
R: test_third_bundle_repaired
R: test_third_bundle_reached
R: test_second_bundle_failed
R: test_second_bundle_not_kept
R: test_second_bundle_error
R: test_second_bundle_reached
R: test_first_bundle_kept
R: test_first_bundle_reached

Note: Promises that have potential for multiple outcomes because they are
compound are not rolled into a single atomic transaction. Each part of that
compound promise attempts to converge independently. Being able to compound them
into a single statement simply allows you to express multiple promises in a more
concise way.

Does that help?

[Jeroen van den Haspel]

Thanks! Great example.

This absolutely helps us in the right direction.
I like the methods method you use. I only knew the usebundle way but now it can set a class as result of a bundle.
So cutting it in smaller portions (bundles) and separating perms and copy is the way to go.

Regards,
Jeroen.

Verzonden vanaf mijn mobiel.

Op 4 okt. 2016 15:46 schreef "Nick Anderson" <nick.a...@cfengine.com>:

[Jeroen van den Haspel]

Great. Thanks!

Regards,
Jeroen.

Verzonden vanaf mijn mobiel.

Op 4 okt. 2016 13:12 schreef "Aleksey Tsalolikhin" <ale...@verticalsysadmin.com>:

[Nick Anderson]

Jeroen van den Haspel writes:

> Thanks! Great example.
>
> This absolutely helps us in the right direction.
> I like the methods method you use. I only knew the usebundle way but now it
> can set a class as result of a bundle.
> So cutting it in smaller portions (bundles) and separating perms and copy
> is the way to go.

I'm glad it's helpful.

One more thing to keep in mind. Bundles are not functions. They are
collections of promises, and they maintain state between actuation during the
same agent run.

bundle agent main
{
  methods:
    "one" usebundle => mybundle("first_actuation");
    "two" usebundle => mybundle("second_actuation");

    "report";
}
bundle agent mybundle(param)
{
  vars:
    # Create a var named for the param containing its own name.
    "$(param)" string => "$(param)";
}

bundle agent report
{
  vars:
   "in_mybundle" slist => variablesmatching("default\:mybundle\..*");

  reports:
   "Vars defined in mybundle $(in_mybundle) == $($(in_mybundle))";
}

R: Vars defined in mybundle default:mybundle.first_actuation == first_actuation
R: Vars defined in mybundle default:mybundle.second_actuation == second_actuation

Similarly a promise that has already been kept or repaired won't run again.

bundle agent main
{
  methods:
    "one" usebundle => mybundle("first_actuation");
    "two" usebundle => mybundle("second_actuation");
}

bundle agent mybundle(param)
{
  commands:
    # This won't be actuated when mybundle is called a second time.

    "/bin/echo"
      args => "Hello1";

    # This will because the promise is different from the previous actuation (differnt param value).
    "/bin/echo"
      args => "$(param)";

    # THis will because the promise is different from the previous actuation (unique handle);
    "/bin/echo"
      args => "Hello2",
      handle => "$(param)";

  vars:
    # Create a var named for the param containing its own name.
    "$(param)" string => "$(param)";
}

bundle agent report
{
  vars:
   "in_mybundle" slist => variablesmatching("default\:mybundle\..*");

  reports:
   "Vars defined in mybundle $(in_mybundle) == $($(in_mybundle))";
}

notice: Q: ".../bin/echo Hello": Hello1
notice: Q: ".../bin/echo first": first_actuation
notice: Q: ".../bin/echo Hello": Hello2
notice: Q: ".../bin/echo secon": second_actuation
notice: Q: ".../bin/echo Hello": Hello2

–
Nick Anderson
Senior Solutions Engineer, https://cfengine.com/
mobile: +1-785-550-1767

[Jeroen van den Haspel]

Thanks again Nick.

At the bottom of this page:

https://docs.cfengine.com/docs/3.9/reference-promise-types-methods.html

there is an example of sending return values.

That's quite close to function behaviour.

Regards,

Jeroen.

[Nick Anderson]

Jeroen van den Haspel writes:
> Thanks again Nick.
>
> At the bottom of this page:
> https://docs.cfengine.com/docs/3.9/reference-promise-types-methods.html
> there is an example of sending return values.
>
> That's quite close to function behaviour.

Yes, it's similar, and you can successfully use bundles like functions as long
as you remember the details especially about how promises wont be actuated again
once they have been kept or repaired (you may need to make them unique).

bundle agent main
{
  methods:
     "test"
       useresult => "test_result";

     "test2a"
       usebundle => test2("test2a"),
       useresult => "test2_result";

     "test2b"
       usebundle => test2("test2b"),
       useresult => "test2_result";

       # Note I put the result into the same array, this may or may not be
       # useful depending on your sitatuation.

  reports:
    "$(test_result[1])";
    "$(test2_result[test2a])";
    "$(test2_result[test2b])";
}

bundle agent test
{
  reports:
    "Returned Value"
      bundle_return_value_index => "1";
}

bundle agent test2(return_value_index)
{
  reports:
    "Passed $(return_value_index)"
      bundle_return_value_index => "$(return_value_index)";
}

R: Returned Value
R: Passed test2a
R: Passed test2b

[Jeroen van den Haspel]

Brilliant!

Returning in an array is indeed very usefull as I can return more than one value like a custom status code.

Based on that statuscode I am able to set classes and with classes I can decide wether or not to do something else.

The classes have to be unique as well offcourse which I did by adding the variable to the classes name. Works!

Regards,

Jeroen.

Op dinsdag 11 oktober 2016 16:16:05 UTC+2 schreef Nick Anderson: