question on augments_file

[robu...@gmail.com]

Hi all,

I've got some question about the usage of augemnts_file.

I'm currently testing the upgrade from our 3.7.6 installation to 3.10.2.

we have all our custom policies in 

masterfiles/services/autorun

controlling the bundle-sequence through 

masterfiles/services/main.cf

and have some additional lib<xyzz>.cf files in

masterfiles/services/autorun/libs/

Up to now we haven't made usage of the augemnts_file but instead had

promises.cf extended to create an slist of all additional inputs in

masterfiles/services/autorun/libs/

I would like to move away from this "home cooked" solutions and get closer 

to what seems to be a more "standard" approach.

with the augments_file inputs variable I seem to can't include our whole lib-dir?

Is that right? 

do I have to statically list each and any file that I want to include?

OR, 

as I have to set 

"classes:" { "services_autorun": [ "any" ] }

to make services/autorun run in the way we had it so far,

are ALL subdirs of  services/autorun automatically parsed?

And my next question, 

is setting the above mentioned class in the augemnts_file sufficient to parse all files in the service autorun dir

(and get thm executed according to services/main.cf) or do I have to manipulate

controls/def.cf to set the classes there too? (I fear so, but that would draw a bunch of question 

marks on the whole point of the augments_file or not?)

Is it right that in the augments_file I can only set variables and classes for the class any?

(no same variable with different values for different classes?)
Up to now we have a quite extensive "bundle common def1" that is evaluated first and defines 

vars and classes, I wonder if and how much I could/should port from there to the augments_file?

TIA and regards

[Nick Anderson]

robu...@gmail.com writes:

with the augments_file inputs variable I seem to can't include our whole
lib-dir?
Is that right?

Yes, that is correct. The inputs key gets mapped to def.augments_inputs, and
that list is included in the stock promises.cf.

do I have to statically list each and any file that I want to include?

Yes. However you could include a file that includes other files.

as I have to set

"classes:" { "services_autorun": [ "any" ] }

to make services/autorun run in the way we had it so far,
are ALL subdirs of services/autorun automatically parsed?

No, only the policy files in the top level of the services/autorun directory
are automatically added to inputs when services_autorun is defined.

is setting the above mentioned class in the augemnts_file sufficient to

parse all files in the service autorun dir
(and get thm executed according to services/main.cf) or do I have to
manipulate
controls/def.cf to set the classes there too? (I fear so, but that would
draw a bunch of question

marks on the whole point of the augments_file or not?)

No, there is no need to modify def.cf to define the services_autorun class.
Defining it from augments is sufficient to enable the addition of
services/autorun/*.cf to inputs and run the bundles tagged with autorun in
lexical order.

Is it right that in the augments_file I can only set variables and classes 
for the class any?

No, you should be able to use any of the hard classes except for those related
to the hostname and am_policy_hub|policy_server. So you should able to use
things like redhat_5 and 1cpu and 64bit.

Variables are not set conditionally from augments. They are for the any class
all the time.

Augments is not trying to allow the level of decision making you can achieve
inside the policy language. It's intended to be pure data that applies to your
infrastructure from a high level.

… In master an augments key has been introduced where you can merge more
specific augments on top of def.json using the same strategy as mergedata()
variables conditionally using multiple augments. So you could have different
augments for different platforms.

(no same variable with different values for different classes?)
Up to now we have a quite extensive "bundle common def1" that is evaluated
first and defines
vars and classes, I wonder if and how much I could/should port from there

to the augments_file?

Augments is useful for defining classes and variables very early during agent
initialization (earlier than def.cf). Probably your common bundle could be
included via the inputs key in the augments file. If it's setting classes
conditionally based on other things it will likely still be useful to you. If
it's just turning on some classes and setting some variables the same for the
entire infrastructure they might do just as well in augments.

–
Nick Anderson
Doer of things, CFEngine

[robu...@gmail.com]

Thanks a lot for the clarification,

I appreciate it!

[robu...@gmail.com]

... sorry, still struggling with the augemnts_file:

I want to set the the mail variables in def.json as suggested in

https://github.com/cfengine/masterfiles/blob/3.7.0/example_def.json

so I did:

{

 "vars": {

             "domain": "<domain.name>",

             "mailto": "me@<domain.name>",

             "mailfrom": "cfengine@$(fqhost)",

             "smtpserver": "localhost";

 }

}

controls/def.cf is unmodified and hence has

      "mailto"

        string => "root@$(def.domain)",

        ifvarclass => not(isvariable("mailto"));

      "mailfrom"

        string => "root@$(sys.uqhost).$(def.domain)",

        ifvarclass => not(isvariable("mailfrom"));

      "smtpserver"

        string => "localhost",

        ifvarclass => not(isvariable("smtpserver"));

mail.log and our mailserver tell me that the mail is trying to be send to root@$(def.domain)

... which is wrong.

What am I missing?

TIA and regards

[Nick Anderson]

Did you restart cf-serverd after deploying that change?

[robu...@gmail.com]

cf-serverd?

Isn't cf-execd sending the mails?

in any case I did a

systemctl restart cfengine3.service and verified that all services running with the new pid.

BUT

it is still cfengine 3.7.6 running on that system, just trying to upgrade the masterfiles ...

though 

/var/cfengine/inputs/controls/3.7/def.cf

has the same settings as def.json

okay, so you assume it just should work

meaning $(sys.uqhost).$(def.domain)

should work in def.json (at least once domain is defined in def.json)?

let me try to run it with cfengine version 3.10.2

...

thnx for having a look

 I will come back on this later on

[nick.a...@northern.tech]

Oh yes your correct, cf-execd, my mistake.

Yes please try with 3.10.2, but 3.7.6 with the 3.10.2 policy should work as well.



Sent from my android device.

--
You received this message because you are subscribed to a topic in the Google Groups "help-cfengine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/help-cfengine/1yirEuDGWwo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to help-cfengin...@googlegroups.com.
To post to this group, send email to help-c...@googlegroups.com.
Visit this group at https://groups.google.com/group/help-cfengine.
For more options, visit https://groups.google.com/d/optout.

[Nick Anderson]

In the JSON you should be able to use sys scope vars:

{

 "vars": {

             "domain": "$(sys.domain)",

             "mailto": "m...@mydomain.com",

             "mailfrom": "cfengine@$(sys.fqhost)",

             "smtpserver": "localhost";

 }

}

To unsubscribe from this group and all its topics, send an email to help-cfengine+unsubscribe@googlegroups.com.

[robu...@gmail.com]

well I'm out of ideas on this one.

I have another test variable set in the augments file

and in controls/def.cf with the same 

class constrain like

ifvarclass => not(isvariable("mailto"));

and it works as expected.

NOT the mailto/mailfrom variables though.

whatever I try, 

mailto/mailfrom seems to be overwritten by controls/def.cf

I know it seems impossible (to me at this point at least)

especially since I have an extended report now

showing me the values for mailto and mailfrom like so:

    vars:

        any::

            "cf_execd_pid"

                string              =>  execresult("/bin/cat /var/cfengine/cf-execd.pid", "noshell");

            "cf_monitord_pid"

                string              =>  execresult("/bin/cat /var/cfengine/cf-monitord.pid", "noshell");

            "cf_serverd_pid"

                string              =>  execresult("/bin/cat /var/cfengine/cf-serverd.pid", "noshell");

            "m_time"

                string              =>  execresult("/bin/date", "noshell");

    reports:

        cfengine::

            "augemts_test   is ${def.augments_test}

            cf_execd_pid    is ${cf_execd_pid}

            cf_monitord_pid is ${cf_monitord_pid}

            cf_serverd_pid  is ${cf_serverd_pid}

            mailto          is ${def.mailto}

            mailfrom        is ${def.mailfrom}

            time            is ${m_time}

            sys.last_policy_update  is ${sys.last_policy_update}"

                comment         =>  "test";

mailto/mailfrom in this report shows ALWAYS the correct values from def.json

(and def.augments_test can be switched by either removing the var from def.json or removing the

class constrain for that var in controls/def.cf - so it seems def.josn GETS parsed correctly)

But still cf-execd uses the wrong (unaltered) settings from controls/def.cf to send the mail.

I restarted cfengine (and the whole instance), quadruple checked the spelling in def.json

(also with od -c) and the file seems all right.

I'm truly lost, I can set the mailto/mailfrom in controls/def.cf obviously and it works,

but HOW can it be possible that my test var works but not mailto/mailfrom??

And how can the mailto/mailfrom variables in the report above can be right but not being used by cf-execd?

AND to make the whole thing even more bizarre is that ... after a couple of cfengine restarts it works correctly - for a while.

than it fails again until I restart cfengine a couple of times, than it will work again for a while.

I browsed and greped what I could but this truly leaves me clue- and speech-less.

Any idea where to look or how to debug this further? Is there anything about mail hardcoded in cf-execd?

btw. I started this test with cfengine 3.7.6 on the server, than upgraded the server to cfengine 3.10.2 

I was cfengine 3.7.6 on the test client than I manually upgraded the client to 3.10.2 

all three cases behaved the same

best regards 

and thnx for any comment

[robu...@gmail.com]

well,

must have been caught by some ... dunno what ... 

something in the upgrade process, where I didn't followed the recommend procedure :-/

I upgraded the masterfiles and than installed the 3.10.2 community .deb on top of the running 3.7.6

...

meanwhile it works as expected.

[Nick Anderson]

So it's working? Great.

LMK if you run into more issues.

--