Standard Library with community 3.10

[Bob]

I am moving from community 3.6.2 to 3.10.2. When I was using 3.6 I had a cfengine_stdlib.cf file and am now getting errors related to bundles there. Example:

./main.cf:192:0: error: Undefined body tidy with type delete
./main.cf:193:0: error: Undefined body days_old with type file_select
./main.cf:194:0: error: Undefined body recurse with type depth_search
./site.cf:347:0: error: Undefined body std_defs with type edit_defaults
./site.cf:348:0: error: Undefined body mog with type perms
./site.cf:353:0: error: Undefined body std_defs with type edit_defaults

I see some of these bundles defined in 3.10 under inputs/lib/files.cf but I am not sure how to use them. When I search the 3.10 documentation (from the website) I don't see any hints.

Could someone provide me with an example? Thanks.

[Nick Anderson]

The stdlib was broken up into multiple files and split into version speicifc directories for 3.6.0. The version split was necessary because some parts of the stdlib include syntax for newer versions, and it was not until 3.7 that we gained the minimum_version macro.

In 3.9 the stdlib was merged back into a single directory.

To include the stdlib you should be able to add lib/stdlib.cf to your intputs. That policy file includes the other files that are included by default.

https://docs.cfengine.com/docs/3.10/reference-masterfiles-policy-framework-lib-stdlib.html


Does that make sense?

[David Lee]

Going from 3.6.x to 3.10.x is a big overall jump.

We did this during summer last year, and followed the suggestions published by CFEngine, on two pages:

https://docs.cfengine.com/docs/3.10/guide-installation-and-configuration-upgrading.html

https://docs.cfengine.com/docs/3.10/examples-tutorials-masterfiles_policy_framework_upgrade.html

I would particularly draw you attention to a short paragraph near the top of the first document:

# NOTE: Upgrading between major LTS releases is safest to be done step-by-step
# from one LTS version to the next, for example from 3.6.x to 3.7.x to 3.10.x.
# Try not to do multi-version upgrade at once.

Two steps, not a giant leap.

The process wasn't "do it in you your sleep" easy. But it was OK. And the two-step approach was a major help.

Another thing in our favour is that, prior to this, I had always tried to keep the main "masterfiles" as close as reasonably possible to what CFEngine had been naturally distributing. Of course, we had our own major policy... after all, that is the very reason why one runs CFEngine in the first place! But I had been keeping as much separation as possible (as little overspill as possible) between the two, and keeping the points of contact between them minimal, simple, and well-defined.

So taking the high-level overview:

1. On your existing (3.6.x) set-up, try to get things in order:
(a) clean separation of CFEngine policy from your own;
(b) closeness of your copy of CFEngine masterfiles to what they had distributed;
2. Consider the two-step approach (3.6 -> 3.7; 3.7 -> 3.10) rather than a giant leap;
3. Use the two documents listed above to guide you on the work itself.

I hope that helps.

-- David Lee, ECMWF

> --
> You received this message because you are subscribed to the Google Groups
> "help-cfengine" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to help-cfengin...@googlegroups.com.
> To post to this group, send email to help-c...@googlegroups.com.
> Visit this group at https://groups.google.com/group/help-cfengine.
> For more options, visit https://groups.google.com/d/optout.

[Bob]

So taking the high-level overview:

1. On your existing (3.6.x) set-up, try to get things in order:
   (a) clean separation of CFEngine policy from your own;
   (b) closeness of your copy of CFEngine masterfiles to what they had distributed;
2. Consider the two-step approach (3.6 -> 3.7; 3.7 -> 3.10) rather than a giant leap;
3. Use the two documents listed above to guide you on the work itself.

I hope that helps.

-- David Lee, ECMWF

Thanks, David. I am accepting your advice and going from 3.6 to 3.7 first. I take your point about leaving "masterfiles" untainted as possible -- which I will try to do.

Where do people store the files that will be copied to the target system (/etc/issue, /etc/motd, /etc/yp.conf)?

I assume my policy files reside in masterfiles/services ?

[Neil H. Watson]

On Tue, Jan 09, 2018 at 03:10:33PM -0800, Bob wrote:
> Where do people store the files that will be copied to the target system
> (/etc/issue, /etc/motd, /etc/yp.conf)?
> I assume my policy files reside in masterfiles/services ?

/var/cfengine/sitefiles

[Nick Anderson]

Hi Bob,

Where do people store the files that will be copied to the target system
(/etc/issue, /etc/motd, /etc/yp.conf)?

There is no standard that I am aware of.

It can be nice to track versions of hand crafted files separately from the
policy itself.

Neil mentions /var/cfengine/sitefiles. If you select a place outside of
masterfiles, then you just need to be sure to have an appropriate access
promise granting hosts access to the files stored under that path.

bundle server bobs_access_rules
{

  access:

    policy_server|am_policy_hub::

      "/var/cfengine/sitefiles"
        admit_ips => { @(def.acl) },
        comment => "Hand crafted configs are stored here. All hosts need to be
                    able to copy assets from this directory on the hub.";

}

I assume my policy files reside in masterfiles/services ?

The services directory is a place you can store your custom policy. The
masterfiles policy framework does ship services/main.cf which contains an
empty bundle agent main. You can use this as a branching off point.

I commonly see directories created under the top level as well. For example
bob or mycompany. The main guidance is that you should try to avoid
modifying the files that are shipped as part of masterfiles.

The masterfiles policy framework is rendered in the documentation here:
https://docs.cfengine.com/docs/3.10/reference-masterfiles-policy-framework.html

–
Nick Anderson

[Stefan Skoglund]

ons 2018-01-10 klockan 11:18 -0600 skrev Nick Anderson:
> Neil H. Watson <cfen...@watson-wilson.ca> writes:
>
> > On Tue, Jan 09, 2018 at 03:10:33PM -0800, Bob wrote:
> > >   Where do people store the files that will be copied to the 
> > >   target system
> > >   (/etc/issue, /etc/motd, /etc/yp.conf)?
> > >   I assume my policy files reside in masterfiles/services ?
> >
> > /var/cfengine/sitefiles
>
> Hi Bob,
>

> #+BEGIN_QUOTE

>   Where do people store the files that will be copied to the 
>   target system
>   (/etc/issue, /etc/motd, /etc/yp.conf)?

> #+END_QUOTE

>
> There is no standard that I am aware of.
>

I have a dual policy regarding that:

more complicated files (and some not so complicated) is handled in git
(a sister repo to the masterfile's one) and the hub pulls updates from
that git repo.

Clients fetches the files from the hub (/var/cfengine/sitefiles) and
places them in /var/cfengine/local-sitefiles.


Some other files is handled with a normal bundle Insert_text (for
example a /etc/motd template using mustache)

[Bas van der Vlies]

Mostly I have a bundle for each service and each service has its own data or template directory, where template is a shortcut “templates” in masterfiles framework, eg:

   * resolv.cf —> generated resolv.conf from mustache template -> mustache/json files  in “templates/resolv”, see https://github.com/basvandervlies/cf_surfsara_lib

   * we have also data that is static we put this in “data/<bundle_name>” and all the logic is handle in <bundle_name>.cf

--

Bas van der Vlies
| Operations, Support & Development | SURFsara | Science Park 140 | 1098 XG  Amsterdam
| T +31 (0) 20 800 1300  | bas.van...@surfsara.nl | www.surfsara.nl |