Question about example policies from the book "Learning CFEngine 3"

[Alex Voiz]

Hello everyone

I have a noobs question :)

Im trying to use example on page 76 (Editing /etc/sysctl.conf) it works if I run it manually with command cf-agent -Kf /root/sysctl-edit.cf . Config looks like:

####comm

body common control

{

                 inputs => { "/var/cfengine/inputs/libraries/cfengine_stdlib.cf" };

                 bundlesequence => { "configfiles" };

}

bundle agent configfiles

{

        vars:

                # Files to edit

                "files[sysctl]" string => "/root/sysctl.conf";

                # Sysctl variables to set

                "sysctl[net.ipv4.tcp_syncookies]"                       string => "1";

                "sysctl[net.ipv4.conf.all.accept_source_route]"         string => "0";

                "sysctl[net.ipv4.conf.all.accept_redirects]"            string => "0";

                "sysctl[net.ipv4.conf.all.rp_filter]"                   string => "1";

                "sysctl[net.ipv4.conf.all.log_martians]"                string => "1";

        methods:

                "sysctl" usebundle => edit_sysctl,

                comment => "Configure $(files[sysctl])";

}

bundle agent edit_sysctl

{

        files:

                "$(configfiles.files[sysctl])"

                handle => "edit_sysctl",

                comment => "Make sure sysctl.conf contains desired configuration",

                create => "true",

                edit_line => set_variable_values("configfiles.sysctl"),

                classes => if_repaired("sysctl_modified");

        commands:

          sysctl_modified.!no_restarts::

                "/sbin/ifconfig -a"

                handle => "reload_sysctl",

                comment => "Make sure new sysctl settings are loaded";

        reports:

          cfengine_3::

            "bundle agent host_groups_new D O N E";

}

But I need so that it runs every 5 minutes on a specific host for this I did the following:

added to promises.cf (full file promises.cf is attached to the theme) next bundle agent:

bundle agent host_groups_new

{

classes:

        "11.cc.lan" or => {"ipv4_192_168_16_11"};

methods:

        11.cc.lan::

                "configs_policies/sysctl-edit.cf" usebundle => "configfiles";

}

Same sysctl-edit.cf but without way to libs without this:

body common control

{

inputs => { "/var/cfengine/inputs/libraries/cfengine_stdlib.cf" };

bundlesequence => { "configfiles" };

}

If I did't delete "body common control" section in sysctl-edit.cf I get an error when check syntax :

cf-promises -f /root/promises_cfengine/promises.cf 

/root/promises_cfengine/promises.cf:9:0: error: Duplicate definition of body control with type common

/root/promises_cfengine/configs_policies/sysctl-edit.cf:3:0: error: Duplicate definition of body control with type common

Thereafter this policy don't work, no errors in syslog, after that I cheked sysctl-edit.cf syntax

cf-promises -f /root/promises_cfengine/configs_policies/sysctl-edit.cf 

2014-02-10T17:23:14+0200    error: No such FnCall "if_repaired()" in promise @ /root/promises_cfengine/configs_policies/sysctl-edit.cf near line 21

2014-02-10T17:23:14+0200    error: No such FnCall "if_repaired()" in promise @ /root/promises_cfengine/configs_policies/sysctl-edit.cf near line 21

I think cfengine can't find libs with "if_repaired" I can't understand how I can show this libs for sysctl-edit.cf ? Just manually we add these libs in to the policy and it's work. 

[Brian Bennett]

You need to add the name of the bundle to bundlesequence and the file to inputs, both of which are under body common control in promises.cf.

-- 

Brian

--
You received this message because you are subscribed to the Google Groups "help-cfengine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to help-cfengin...@googlegroups.com.
To post to this group, send email to help-c...@googlegroups.com.
Visit this group at http://groups.google.com/group/help-cfengine.
For more options, visit https://groups.google.com/groups/opt_out.

<promises.cf>

[Diego Zamboni]

Hi Alex,

Once you remove body common control from sysctl-edit.cf, you will of course get an error if you try cf-promises on it, because it's no longer loading the standard library. Since sysctl-edit.cf is now being loaded from promises.cf, you need to run cf-promises against that file to verify it.

In addition, your host_groups_new bundle is broken. Classes cannot have periods in their names, so the class name "11.cc.lan" is invalid. In the following condition:

11.cc.lan::

Periods are interpreted as "AND", so the following line will only be evaluated if class "11" and class "cc" and class "lan" are all set (see http://cfengine.com/docs/3.5/manuals-language-concepts-classes.html#operators-and-precedence).

You should replace the periods with underscores to make a valid class name (also, I'm not completely sure class names starting with numbers are allowed, but haven't tested).

Hope this helps,

--Diego

[Neil Watson]

> {

> classes:
> "11.cc.lan" or => {"ipv4_192_168_16_11"};
> methods:
> 11.cc.lan::
> "configs_policies/sysctl-edit.cf" usebundle =>

This is not a class. Classes are [a-zA_Z0-9_]. The class promiser
"11.cc.lan" will be automatically canonized to 11_cc_lan. Use that in
in the methods call.

--
Neil H Watson
Reporting with Cfengine Community http://evolvethinking.com/products/delta-reporting/
http://evolvethinking.com/evolve-thinkings-free-cfengine-library/
VIM and Cfengine https://github.com/neilhwatson/vim_cf3

[Alex Voiz]

Thank very much to all

[Joan Gonzalez]

hey hello to all people in this group, excuse me i'm new using cfengine but i have a problem, maybe someone can help me: this my promise by default :

body common control

{

      bundlesequence => {
                        # Common bundle first (Best Practice)
                          "def",
                          "cfe_internal_hub_vars",

                          # Design Center
                          "cfsketch_run",

                          # autorun system
                          "services_autorun",

                         # Agent bundle
                          "cfe_internal_management",   # See cfe_internal/CFE_cfengine.cf
                          "def",
                          "package_apt",
                          "replace_apt",

      };

      inputs => {
                 # File definition for global variables and classes
                  "def.cf",
                  "package_apt.cf",
                  "replace_apt.cf",

                 # Design Center
                  "sketches/meta/api-runfile.cf",

                 # CFEngine internal processes for Mission Portal mostly
                  "cfe_internal/CFE_cfengine.cf",
                  "cfe_internal/CFE_hub_specific.cf",
                  "cfe_internal/CFE_knowledge.cf",

                 # Control body for all CFEngine robot agents
                  "controls/cf_agent.cf",
                  "controls/cf_execd.cf",
                  "controls/cf_monitord.cf",
                  "controls/cf_runagent.cf",
                  "controls/cf_serverd.cf",

                 # COPBL/Custom libraries.  Eventually this should use wildcards.
                  @(cfengine_stdlib.inputs),

                  # autorun system
                  @(services_autorun.inputs),

                  # Add update files to build Knowledge Map relationship
                  "update/update_bins.cf",
                  "update/cfe_internal_dc_workflow.cf",
                  "update/cfe_internal_local_git_remote.cf",
                  "update/cfe_internal_update_from_repository.cf",
                  "update/update_policy.cf",
                  "update/update_processes.cf",

                # List of services here
                  "services/file_change.cf",
                 # "packages_apt",

      };

      version => "CFEngine Promises.cf 3.6.7";

      # Uncomment to connect to the hub using latest protocol.
      #protocol_version => "latest";

      #   goal_categories => { "goals", "targets", "milestones" };
      #   goal_patterns   => { "goal_.*", "target.*","milestone.*" };
}

      #

bundle common inventory
# @brief Set up inventory inputs
#
# This bundle creates the inputs for inventory bundles.
#
# Inventory bundles are simply common bundles loaded before anything
# else in promises.cf
#
# Tested to work properly against 3.5.x
{
  classes:
      "other_unix_os" expression => "!windows.!macos.!linux";
      "specific_linux_os" expression => "redhat|debian|suse";

  vars:
      # This list is intended to grow as needed
    !(cfengine_3_4|cfengine_3_5).debian::
      "inputs" slist => { "inventory/any.cf", "inventory/linux.cf", "inventory/lsb.cf", "inventory/debian.cf", "inventory/os.cf" };
      "bundles" slist => { "inventory_control", "inventory_any", "inventory_autorun", "inventory_linux", "inventory_lsb", "inventory_debian", "inventory_os" };
    !(cfengine_3_4|cfengine_3_5).redhat::
      "inputs" slist => { "inventory/any.cf", "inventory/linux.cf", "inventory/lsb.cf", "inventory/redhat.cf", "inventory/os.cf" };
      "bundles" slist => { "inventory_control", "inventory_any", "inventory_autorun", "inventory_linux", "inventory_lsb", "inventory_redhat", "inventory_os" };
    !(cfengine_3_4|cfengine_3_5).suse::
      "inputs" slist => { "inventory/any.cf", "inventory/linux.cf", "inventory/lsb.cf", "inventory/suse.cf", "inventory/os.cf" };
      "bundles" slist => { "inventory_control", "inventory_any", "inventory_autorun", "inventory_linux", "inventory_lsb", "inventory_suse", "inventory_os" };
    !(cfengine_3_4|cfengine_3_5).windows::
      "inputs" slist => { "inventory/any.cf", "inventory/windows.cf", "inventory/os.cf" };
      "bundles" slist => { "inventory_control", "inventory_any", "inventory_autorun", "inventory_windows", "inventory_os" };
    !(cfengine_3_4|cfengine_3_5).macos::
      "inputs" slist => { "inventory/any.cf", "inventory/macos.cf", "inventory/os.cf" };
      "bundles" slist => { "inventory_control", "inventory_any", "inventory_autorun", "inventory_macos", "inventory_os" };
    !(cfengine_3_4|cfengine_3_5).linux.!specific_linux_os::
      "inputs" slist => { "inventory/any.cf", "inventory/linux.cf", "inventory/lsb.cf", "inventory/os.cf" };
      "bundles" slist => { "inventory_control", "inventory_any", "inventory_autorun", "inventory_linux", "inventory_lsb", "inventory_os" };
    !cfengine_3_5.other_unix_os::
      "inputs" slist => { "inventory/any.cf", "inventory/generic.cf", "inventory/os.cf" };
      "bundles" slist => { "inventory_control", "inventory_any", "inventory_autorun", "inventory_generic", "inventory_os" };

    (cfengine_3_4|cfengine_3_5)::
      "inputs" slist => { cf_null };
      "bundles" slist => { cf_null };

  reports:
    verbose_mode::
      "$(this.bundle): loading inventory module '$(inputs)'";
}

      #

bundle common cfengine_stdlib
{
  vars:
    cfengine_3_4::
      # This is the standard library for CFEngine 3.4 and earlier
      # (only 3.4 is explicitly supported)
      "inputs" slist => { "libraries/cfengine_stdlib.cf" };
    cfengine_3_5::
      # 3.5 doesn't have "body file control" so all the includes are listed here
      "inputs" slist => {
                          "lib/3.5/paths.cf",
                          "lib/3.5/common.cf",
                          "lib/3.5/commands.cf",
                          "lib/3.5/packages.cf",
                          "lib/3.5/files.cf",
                          "lib/3.5/services.cf",
                          "lib/3.5/processes.cf",
                          "lib/3.5/storage.cf",
                          "lib/3.5/databases.cf",
                          "lib/3.5/monitor.cf",
                          "lib/3.5/guest_environments.cf",
                          "lib/3.5/bundles.cf",
                          "lib/3.5/reports.cf",
                          "lib/3.5/cfe_internal.cf",
      };

    !(cfengine_3_4|cfengine_3_5)::
      # CFEngine 3.6 and higher can include through a secondary file
      "inputs" slist => { "$(sys.local_libdir)/stdlib.cf" };

  reports:
    verbose_mode::
      "$(this.bundle): defining inputs='$(inputs)'";
}

      #

bundle common cfengine_controls
{
  vars:
    cfengine_3_4::
      # This is control body for CFEngine 3.4 and earlier
 # server control seems to be a bit problem here
      # (only 3.4 is explicitly supported)
      "inputs" slist => {
                          "controls/cf_agent.cf",
                          "controls/cf_execd.cf",
                          "controls/cf_hub.cf",
                          "controls/cf_monitord.cf",
                          "controls/cf_runagent.cf",
                          "controls/3.4/cf_serverd.cf",
      };

    !cfengine_3_4::
      # 3.5 and higher are fine with a standard version
      "inputs" slist => {
                          "controls/cf_agent.cf",
                          "controls/cf_execd.cf",
                          "controls/cf_hub.cf",
                          "controls/cf_monitord.cf",
                          "controls/cf_runagent.cf",
                          "controls/cf_serverd.cf",
      };

  reports:
    verbose_mode::
      "$(this.bundle): defining inputs='$(inputs)'";
}

bundle common cfengine_reports
{
  vars:
    (cfengine_3_4|cfengine_3_5)::
      "inputs" slist => { cf_null };

    !(cfengine_3_4|cfengine_3_5)::
      "inputs" slist => {
                          "cfe_internal/host_info_report.cf",
      };

  reports:
    verbose_mode::
      "$(this.bundle): defining inputs='$(inputs)'";
}

bundle common services_autorun
{
  vars:
    cfengine_3_4||cfengine_3_5||!services_autorun::
      "inputs" slist => { };
      "found_inputs" slist => {};
      "bundles" slist => { "services_autorun" }; # run self

    !(cfengine_3_4||cfengine_3_5)::
      "inputs" slist => { "services/autorun.cf" };
      "found_inputs" slist => lsdir("$(this.promise_dirname)/services/autorun", ".*\.cf", "true");
      "bundles" slist => { "autorun" }; # run loaded bundles

  reports:
    verbose_mode::
      "$(this.bundle): defining inputs='$(inputs)'";
}

###############################################################################
#
# bundle agent service_catalogue
#  - general purposes of usage
#
###############################################################################

bundle agent service_catalogue

{

  methods:

    any::

      "security" -> { goal_infosec, goal_compliance }
      comment => "Basic change management",
      handle => "service_catalogue_change_management",
      usebundle => change_management;

}

# special body for update/*.cf compatibility
body classes u_kept_successful_command
# @brief Set command to "kept" instead of "repaired" if it returns 0
{
      kept_returncodes => { "0" };
}
 
and thast it's my first policy based on example of documentation of cfengine like replace line in file:

body common control
{
        any::
        bundlesequence => { "insert" };
}

bundle agent replace_apt
{
vars:
        "lines" string =>
        "deb http://ft.us.debian.org/debian jessie main contrib non-free
        deb-src http://ftp.us.debian.org/debian jessie main contrib non-free";

files:
        "/etc/apt/sources.list"
        create => "true",
        edit_line => append_if_no_line("#$(insert.lines)");
}
~ 

my problem are the system says: Starting cf-execd: Starting /var/cfengine/bin/cf-execd...
/var/cfengine/inputs/promises.cf:7:0: error: Duplicate definition of body control with type common
/var/cfengine/inputs/replace_apt.cf:1:0: error: Duplicate definition of body control with type common
root@cfengine3-6:/var/cfengine/masterfiles#

do someone can help me whit this?





El martes, 11 de febrero de 2014, 3:53:35 (UTC-4:30), Alex Voiz escribió:

Thank very much to all

[Nick Anderson]

On 07/19/2016 09:32 AM, Joan Gonzalez wrote:
> my problem are the system says: Starting cf-execd: Starting
> /var/cfengine/bin/cf-execd...
> /var/cfengine/inputs/promises.cf:7:0: error: Duplicate definition of
> body control with type common
> /var/cfengine/inputs/replace_apt.cf:1:0: error: Duplicate definition of
> body control with type common
> root@cfengine3-6:/var/cfengine/masterfiles#
>
> do someone can help me whit this?

Bodies and bundles must be uniquely named. There is a body common
control in promises.cf that defines the inputs and high level
bundlesequence. You should just need to remove the body common control
from your replace_apt.cf policy.

[Aleksey Tsalolikhin]

Welcome to CFEngine!  Take a look also at:

- CFEngine Primer  https://digitalelf.net/cf-primer  (short, just the basics)

- http://www.cfenginetutorial.org/  (longer, lots of examples)

They complement the "Learning CFEngine" book and the online documentation.  :-)

ᐧ

--

You received this message because you are subscribed to the Google Groups "help-cfengine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to help-cfengin...@googlegroups.com.
To post to this group, send email to help-c...@googlegroups.com.

Visit this group at https://groups.google.com/group/help-cfengine.
For more options, visit https://groups.google.com/d/optout.

--

Aleksey Tsalolikhin

Founder and Chief Trainer

Vertical Sysadmin, Inc.

+1-323-393-0779

[Christian Linden]

- CFEngine Primer  https://digitalelf.net/cf-primer 

That's a very good one!

.. but on slide 44: is it correct that on the hub first masterfiles is copied to inputs and the clients pick it up from there?

I watched as soon as I save a promise in masterfiles and do a cf-agent -Dvalidated_updates_ready on the client it picks it up before the hub had any chance to copy that to inputs first;

hence I think they pull it from masterfiles directly.

Wrong?

Chris

[Aleksey Tsalolikhin]

Good attention to detail, Chris.  :)

It's meant to show that the workflow is:

- from masterfiles on the hub to "inputs" on the hub and to "inputs" on the clients"

- definitely not from "inputs" on the hub to "inputs" on the clients".

Brian, cold you clarify that, please?  :)

--

You received this message because you are subscribed to the Google Groups "help-cfengine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to help-cfengin...@googlegroups.com.
To post to this group, send email to help-c...@googlegroups.com.
Visit this group at https://groups.google.com/group/help-cfengine.
For more options, visit https://groups.google.com/d/optout.

--

Aleksey Tsalolikhin

Founder and Chief Trainer

Vertical Sysadmin, Inc.

+1-323-393-0779

ᐧ

[Aleksey Tsalolikhin]

On Tue, Jul 19, 2016 at 11:14 AM, Joan Gonzalez <joanlgo...@gmail.com> wrote:

thak's my problem was solvent!!! excelent thanks for your time and answer... (y) i'm so grateful!!!!

Great!  You are most welcome.  :-) 

ᐧ

[Nick Anderson]

On Thursday, July 21, 2016 at 7:32:51 AM UTC-5, Aleksey Tsalolikhin wrote:

Good attention to detail, Chris.  :)

It's meant to show that the workflow is:

- from masterfiles on the hub to "inputs" on the hub and to "inputs" on the clients"

- definitely not from "inputs" on the hub to "inputs" on the clients".

Brian, cold you clarify that, please?  :)

When you define validated_inputs_ready you are explicitly skipping the simple gating check and all of masterfiles is scanned for changes. Normal runs only define validated_updates_ready if the cf_promises_validated file was different (and repaired by the client updating its copy) in the hubs masterfiles vs the executing agents inputs.

If you don't run with validated_updates_ready then the agent will wait until the hub has validated policy and updated the cf_promises_validated flag file.

[Aleksey Tsalolikhin]

So just to sum, the policy won't trickle out of the hub to the hosts until the hub has validated it and installed it in it's own inputs, right? The way you ship CFEngine?

--

[Nick Anderson]

Correct.

I think this has a decent explanation.

http://www.cmdln.org/2012/10/24/cfengine-3-policy-update-or-how-cf_promises_validated-works/

Sent from my mobile device.

[Aleksey Tsalolikhin]

Thanks, Brian!

Here is the policy for the hub, or the hosts that have discovered there is a validated policy update ready for them to pickup  -- the files flow from from /var/cfengine/masterfiles on the hub to /var/cfengine/inputs on the hub or hosts.:

files:

am_policy_hub|validated_updates_ready:: # policy hub should always put masterfiles in inputs in order to check new policy

"$(inputs_dir)"

comment => "Copy policy updates from master source on policy server if a new validation was acquired",

handle => "cfe_internal_update_policy_files_inputs_dir",

copy_from => u_rcp("$(master_location)", @(update_def.policy_servers)),

depth_search => u_recurse("inf"),

file_select => u_input_files,

action => u_immediate;

$(master_location) is defined as follows: vars: !windows::
	"master_location" string => "$(sys.masterdir)",
	comment => "The master CFEngine policy directory on the policy host",
	handle => "cfe_internal_update_policy_vars_master_location";

On Thu, Jul 21, 2016 at 8:59 AM, Brian Bennett <bah...@digitalelf.net> wrote:

Circling back on this,

Here's the update policy for !am_policy_hub, which updates from $(input_dir), which has been defined as $(sys.inputdir).

https://github.com/cfengine/masterfiles/blob/master/cfe_internal/update/update_policy.cf#L140

Whereas am_policy_hub updates from $(master_location)/., which for non-windows systems is defined as $(sys.masterdir).

https://github.com/cfengine/masterfiles/blob/master/cfe_internal/update/update_policy.cf#L158

-- 

Brian

--

Aleksey Tsalolikhin

Founder and Chief Trainer

Vertical Sysadmin, Inc.

+1-323-393-0779

ᐧ

[Christian Linden]

Thanks for clarifying, Nick.
W/o validated_updates_ready development with CFEngine were taking hundredfold of time and patience..
At least for me or do you have another way if you work and want to test your new policies on a host?

c

[Christian Linden]

This slide is misleading as one see the arrows from inputs to the clients =)

Move client1 and 2 to the left and right and draw all arrows from masterfiles down ;-)

.. but it’s the only point. It’s the best summarized introduction I read.

[Nick Anderson]

A flexible pattern that I like is to serve multiple versions of
masterfiles. Then your hosts can subscribe to the version desired.

So for example you could keep all your policy in your git repository.
Then on the hub instead of only serving /var/cfengine/masterfiles you
could serve /var/cfengine/policy-channels/<BRANCH|TAG|COMMIT>. This way
you can have short lived branches and you can subscribe the hosts you
want to that branch without affecting any other hosts. Doing something
like that requires some additional tooling and modification of the
Masterfiles Policy Framework but I have seen some good results from that
and similar patterns.

A very simple alternative could be to have some policy file named for
each host. The host can download its host specific policy file and use that.

Something like:

/var/cfengine/policy/uqhost/host0001.cf
bundle agent this_host
{
reports:
"This is a host specific policy file for host0001";
}

/var/cfengine/policy/uqhost/host0002.cf
bundle agent this_host
{
reports:
"This is a host specific policy file for host0002";
}


And then in yoru update policy do something like:
bundle agent custom_update_bundle
{
vars:
"uqhost_policy"
string => "/var/cfengine/policy/uqhost/$(sys.uqhost)";

files:
"/var/cfengine/policy/this_host.cf"
copy_from => remote_dcp( $(uqhost_policy), "$(sys.policy_hub);
}

And that this_host.cf policy could be included in your inputs and
`this_host` bundle (note its statically named) could be called from your
normal policy. Then you can slip any test policy you want into a host
specific policy file and that policy file would only be used on the host
the file was named after.

It's not my favorite pattern, but its relatively simple, and I have seen
it used with success.

[Nick Anderson]

Oh one more thing I will common do when testing policy changes:

I will grab a test host, kill cf-execd and cf-serverd then modify the policy in inputs. I am usually careful to keep any modified files in open buffers just in case something comes around to try and repair my modified files.

When I am happy I copy my changed files back to the main repo and then restart the services from my test host.

I have seen others simply copy from inputs to masterfiles on the local host and then bootstrap to self.

[mike.w...@verticalsysadmin.com]

Christian,

I extended the git integration of the "masterfiles-stage.sh" script and the corresponding portion of "update.cf" to allow any arbitrary number of policy channels, each associated with a particular git tag or branch or commit, and to which could be assigned any arbitrary number of hosts.  It enables testing workflows as well as policy rollout workflows and flexibility that would not otherwise be possible, and is indispensable for me in all my policy writing and testing.

(Nick Anderson and Ted Zlatonov helped a lot with their feedback and suggestions; you can read the discussion on the "masterfiles-stage.sh" pull request here: https://github.com/cfengine/core/pull/2465)

I've been meaning to write a blog post on the subject and publish all the relevant scripts and policies, just haven't gotten around to it yet.  :)

--Mike Weilgart

[Christian Linden]

Mike,

thanks for your info. But I didn't have masterfiles on git yet; so that's some steps ahead of my use level/case.

Chris