Community 3.6.5 complains about standby interface in Solaris 10 LDOM,

[berntjernberg]

Hi all,

I get a lot of these messages about a standby interface in an IPMP group in a Solaris 10 LDOM:

2015-04-09T16:45:45+0200    error: Could not get interface nxge0 addresses (ioctl(SIOCGARP): 'No such device or address')
2015-04-09T16:45:45+0200    error: Could not get interface nxge0 addresses (ioctl(SIOCGARP): 'No such device or address')
2015-04-09T16:45:47+0200    error: Could not get interface nxge0 addresses (ioctl(SIOCGARP): 'No such device or address')

Bernt

[Nick Anderson]

Hi Bernt, you may want to try excluding that interface using the
ignore_interfaces.rx file.

Normation wrote up a nice blog post about it a couple years ago:
https://blog.normation.com/2013/08/26/about_cfengine_ignore_interfaces_mechanism/

Its mentioned in the CFEngine documentation under hard classes:
https://docs.cfengine.com/latest/reference-classes.html#hard-classes

[berntjernberg]

Hi,

Creating /var/cfengine/inputs/ignore_interfaces.rx seem to work.
I created a small script to generate it. Comments are welcome.

Thanks
Bernt

[Tim]

Hi,

I'd like to implement ignore_interfaces.rx to quiesce the "Could not get interfaces" messages as well.

An additional constraint in our environment however is that we have defined in 

update/update_policy.cf "purge => true" in "body copy_from u_rcp(from,server)", which as I understand it causes any files not present under masterfiles/ on the server to be purged from inputs/ on the client.

If we create /var/cfengine/inputs/ignore_interfaces.rx on a client, it will be purged.  If we distribute it via the server, it will be the same copy on all clients, which is not what we are after either.

Is there a preferred/clean way to keep "purge => true", and have different versions of /var/cfengine/inputs/ignore_interfaces.rx exist on clients?

Or perhaps a way to redefine the location of ignore_interfaces.rx to be outside of the inputs/ directory?, so that it won't be purged?

Thanks.

  - Tim

[Nick Anderson]

On 07/08/2015 04:46 PM, Tim wrote:

> Is there a preferred/clean way to keep "purge => true", and have
> different versions of /var/cfengine/inputs/ignore_interfaces.rx exist on
> clients?
>
> Or perhaps a way to redefine the location of ignore_interfaces.rx to be
> outside of the inputs/ directory?, so that it won't be purged?

I think it would be a good feature request to move it outside of inputs,
or make the location configurable.

What I have seen done is to generate the file from policy, you could
potentially do this at the end of the update policy so that it is there
when the normal policy run starts.

[Ted Zlatanov]

On Wed, 08 Jul 2015 16:52:44 -0500 Nick Anderson <nick.a...@cfengine.com> wrote:

NA> On 07/08/2015 04:46 PM, Tim wrote:
>> Is there a preferred/clean way to keep "purge => true", and have
>> different versions of /var/cfengine/inputs/ignore_interfaces.rx exist on
>> clients?
>>
>> Or perhaps a way to redefine the location of ignore_interfaces.rx to be
>> outside of the inputs/ directory?, so that it won't be purged?

NA> I think it would be a good feature request to move it outside of inputs,
NA> or make the location configurable.

NA> What I have seen done is to generate the file from policy, you could
NA> potentially do this at the end of the update policy so that it is there
NA> when the normal policy run starts.

The interface scanning happens very early so it's hard to move these
settings to a `body common control` attribute.

Together with `inputs`, the location of some system utilities, and
others, these too-early-for-policy settings would benefit from being
obtained from environment variables and also from a optional file like

K1=V1
K2=V2

or in JSON format. As long as the usual sys and const variables are
expanded, I feel this will be very user-friendly.

Another use for this is to add per-system classes instead of specifying
-Dx,y,z on every invocation.

This is fairly easy to implement and would resolve many policy-level
race issues.

Ted

[Ted Zlatanov]

On Thu, 09 Jul 2015 06:35:40 -0400 Ted Zlatanov <t...@lifelogs.com> wrote:

TZ> Together with `inputs`, the location of some system utilities, and
TZ> others, these too-early-for-policy settings would benefit from being
TZ> obtained from environment variables and also from a optional file like

TZ> K1=V1
TZ> K2=V2

TZ> or in JSON format. As long as the usual sys and const variables are
TZ> expanded, I feel this will be very user-friendly.

TZ> Another use for this is to add per-system classes instead of specifying
TZ> -Dx,y,z on every invocation.

TZ> This is fairly easy to implement and would resolve many policy-level
TZ> race issues.

The new def.json facility in 3.7 could be the simplest way to do this.

It currently has this content (as an example):

#+begin_src text
{
"classes":
{
"my_apache": [ "server1", "server2", "redhat.*" ],
"my_other_apache": [ "server[34]", "debian.*" ],
"my_filehost": [ "server3" ],
"my_gateway": [ "server3" ],
"my_yum_role": [ "redhat.*" ],
"my_redhat_role": [ "redhat.*" ],
"my_apt_role": [ "debian.*" ],
"my_debian_role": [ "debian.*" ],
"services_autorun": [ "any" ],
"cfengine_internal_disable_agent_email": [ "enterprise_edition" ]
"cfe_internal_core_watchdog_enabled": [ "any" ],
},

"inputs": [ "$(sys.libdir)/bundles.cf" ],
"vars": {
"acl": [ ".*$(def.domain)", "$(sys.policy_hub)/16" ],

"domain": "mydomain-com",
"mailto": "people@mydomain-com",
"mailfrom": "me@mydomain-com",
"smtpserver": "smtp.mydomain-com",

"input_name_patterns": [ ".*\\.cf",".*\\.dat",".*\\.txt", ".*\\.conf", ".*\\.mustache",
".*\\.sh", ".*\\.pl", ".*\\.py", ".*\\.rb",
"cf_promises_release_id", ".*\\.json", ".*\\.yaml", ".*\\.js" ]
}
}
#+end_src

So, if agents load this in C instead of policy, what can they use here
on startup? All of the classes here could be evaluated after the hard
and persistent classes are loaded in the agent. Then they could be
evaluated again in policy to cover any soft classes.

The inputs can definitely be loaded before any policy is loaded and would
make it much easier to construct the inputs correctly.

Any variables in def.json/vars would go into the "def" variable scope,
so for instance the `acl` value of `def.domain` would resolve to the
`domain` right under it.

Loading def.json in C would also remove a pretty big chunk of policy
from the current masterfiles. It can be validated more carefully, as
well. Finally, using YAML would work as well (internally they both
produce JsonElements), if readability is a concern.

What do you think?

Ted

[Nicolas Charles]

Hi,

Being able to override hardclasses would be of some help in some exotic
systems (so that we can define and undefined classes based on the OS,
rather than on what the agent detects)
Ignore_interface already exists, but it is really unconvenient (and its
location in the inputs folder doesn't really help customize it per host)

As it would run before the parser kicks in, we could also define
behaviour of parser (don't execute commands of execresult if we are only
checking promises validation).

Then there are a lot of hardcoded information that we could modify:
- promises path
- feature activation/deactivation
- commands used for listing the running processes (replacing the complex
ps command in C with something configurable)
- i think there is something we can do for dns also: defining which dns
to query to reverse lookup hostname ?


That's what I'd like, on top of my head; but I'm probably missing use cases

Nicolas

[Nick Anderson]

Right now the augments_file path is specified in policy. It defaults inside of masterfiles making it a global kind of thing, do you propose having a list of places where these augments_files live so that you can also have per host augments?

For sure if the augments_file were loaded by the c directly it would help with inputs.

[Brian Bennett]

It's that I didn't know it was called "augments_file". Things like def.acl or def.trustkeys refer to "override_data_*", but I couldn't find anywhere that "override_data_*" was being defined. Although hindsight being what it is and knowing the file is in json format, I probably should have looked for readjson.

But at any rate, I think if this:

      "augments_file" string => "$(this.promise_dirname)/../../def.json";

were changed to this:

      "augments_file" -> { "override_data" }
            string => "$(this.promise_dirname)/../../def.json";

it would make it easier to find via grep.

Or maybe have $(sys.masterdir)/def.json.example, with some example data.

-- 
Brian Bennett
Looking for CFEngine training?
http://www.verticalsysadmin.com/

--
You received this message because you are subscribed to the Google Groups "help-cfengine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to help-cfengin...@googlegroups.com.
To post to this group, send email to help-c...@googlegroups.com.
Visit this group at http://groups.google.com/group/help-cfengine.
For more options, visit https://groups.google.com/d/optout.

[Ted Zlatanov]

On Wed, 22 Jul 2015 13:54:59 -0700 (PDT) Nick Anderson <nick.a...@cfengine.com> wrote:

NA> Right now the augments_file path is specified in policy. It defaults inside
NA> of masterfiles making it a global kind of thing, do you propose having a
NA> list of places where these augments_files live so that you can also have
NA> per host augments?

It should default to something sensible, definitely. I think we could
look for, in order:

$(sys.masterdir)/def.json
$(sys.workdir)/def.json
$(sys.workdir)/def/$(sys.uqhost).json
$(sys.workdir)/def/$(sys.fqhost).json
$(sys.workdir)/def/$(sys.domain).json
$(sys.workdir)/def/$(sys.ostype).json
$(sys.workdir)/def/$(sys.flavor).json
$(sys.workdir)/def/*.{json,yaml} (sorted lexicographically)

...and merge them from bottom to top, so def.json wins.

On the policy server, I think it would benefit Community and Enterprise
users to make CFEngine compatible with Ansible's inventory system as
specified in https://docs.ansible.com/ansible/intro_inventory.html
instead of inventing a new one. It's a fairly simple standard.

The policy server would parse data files and group_vars, or run and
cache dynamic inventory scripts (like EC2 or OpenStack host discovery)
and create custom def.json output for every client. The dynamic scripts
could follow the standard shown in
https://docs.ansible.com/ansible/intro_dynamic_inventory.html

Ted

[Bruce Schultz]

On 23 July 2015 11:04:02 AM AEST, Ted Zlatanov <t...@lifelogs.com> wrote:
>On Wed, 22 Jul 2015 13:54:59 -0700 (PDT) Nick Anderson
><nick.a...@cfengine.com> wrote:
>
>NA> Right now the augments_file path is specified in policy. It
>defaults inside
>NA> of masterfiles making it a global kind of thing, do you propose
>having a
>NA> list of places where these augments_files live so that you can also
>have
>NA> per host augments?
>
>It should default to something sensible, definitely. I think we could
>look for, in order:
>
>$(sys.masterdir)/def.json
>$(sys.workdir)/def.json
>$(sys.workdir)/def/$(sys.uqhost).json
>$(sys.workdir)/def/$(sys.fqhost).json
>$(sys.workdir)/def/$(sys.domain).json
>$(sys.workdir)/def/$(sys.ostype).json
>$(sys.workdir)/def/$(sys.flavor).json
>$(sys.workdir)/def/*.{json,yaml} (sorted lexicographically)
>
>...and merge them from bottom to top, so def.json wins.

Just a thought.. how about adding $(sys.workdir)/services/def.json to that list? Then all custom policy, including the def overrides, could be contained in one place and managed in git or whatever independently of the standard masterfiles and would make upgrading masterfiles extremely straight forward. Ideally, no customisation of the standard masterfiles would be necessary.

Bruce

--
:b

[Ted Zlatanov]

On Sat, 25 Jul 2015 13:11:22 +1000 Bruce Schultz <bru...@gmail.com> wrote:

BS> On 23 July 2015 11:04:02 AM AEST, Ted Zlatanov <t...@lifelogs.com> wrote:
>> It should default to something sensible, definitely. I think we could
>> look for, in order:
>>
>> $(sys.masterdir)/def.json

Oops, I meant $(sys.inputdir)/def.json

>> $(sys.workdir)/def.json
>> $(sys.workdir)/def/$(sys.uqhost).json
>> $(sys.workdir)/def/$(sys.fqhost).json
>> $(sys.workdir)/def/$(sys.domain).json
>> $(sys.workdir)/def/$(sys.ostype).json
>> $(sys.workdir)/def/$(sys.flavor).json
>> $(sys.workdir)/def/*.{json,yaml} (sorted lexicographically)
>>
>> ...and merge them from bottom to top, so def.json wins.

BS> Just a thought.. how about adding $(sys.workdir)/services/def.json to
BS> that list? Then all custom policy, including the def overrides, could
BS> be contained in one place and managed in git or whatever independently
BS> of the standard masterfiles and would make upgrading masterfiles
BS> extremely straight forward. Ideally, no customisation of the standard
BS> masterfiles would be necessary.

That's basically replacing $(sys.workdir)/def with
$(sys.workdir)/services right?

$(sys.inputdir)/services already exists so it may be confusing to add
this one, one level up from the inputs.

Otherwise I agree with you, the idea is to have a single file in the
masterfiles ($(sys.inputdir)/def.json) and the rest of them outside the
masterfiles.

Ted


Ted

[Bruce Schultz]

Oops, I meant $(sys.inputdir)/services, so that def.json is in the same directory as any custom policy files.

So to set up a policy hub:
- clone the cf-masterfiles to $(sys.workdir)/masterfiles
- clone my organisation policy repo to $(sys.workdir)/masterfiles/services
- clients copy all of masterfiles from the hub to $(sys.inputdir), as is currently the case

So its not much different from the current norms, but the important points are:
1. There's nothing from cf-masterfiles which needs to be changed to get the system to work
2. All of the stuff that I need to modify to define the policy for my organisation is in a single directory, so its easy to manage that in a separate git repository

This I think would make it much easier to explain to folks getting started, and make it easier to handle the upgrades to masterfiles, which seem to get more complex with each release. There's been a few threads here about best practices for updating masterfiles and it generally involves merging the changes and modifying def.cf. Of course, the def.json overrides makes that simpler, but my proposal to keep the masterfiles and services separated at the repository level would make even more simpler, I think.


One problem is that modules typically live in $(sys.workdir)/modules ... is there any technical reason that they could not be in services/modules instead?

Cheers
Bruce

>
>Otherwise I agree with you, the idea is to have a single file in the
>masterfiles ($(sys.inputdir)/def.json) and the rest of them outside the
>masterfiles.
>
>Ted
>
>
>Ted

--
:b

[Ted Zlatanov]

On Sun, 26 Jul 2015 13:00:21 +1000 Bruce Schultz <bru...@gmail.com> wrote:

BS> On 26 July 2015 4:20:48 AM AEST, Ted Zlatanov <t...@lifelogs.com> wrote:
>>>> It should default to something sensible, definitely. I think we could
>>>> look for, in order:
>>>>

>>>> $(sys.inputdir)/def.json
>>>> $(sys.workdir)/def.json
>>>> $(sys.workdir)/def/$(sys.uqhost).json
>>>> $(sys.workdir)/def/$(sys.fqhost).json
>>>> $(sys.workdir)/def/$(sys.domain).json
>>>> $(sys.workdir)/def/$(sys.ostype).json
>>>> $(sys.workdir)/def/$(sys.flavor).json
>>>> $(sys.workdir)/def/*.{json,yaml} (sorted lexicographically)

BS> So to set up a policy hub:
BS> - clone the cf-masterfiles to $(sys.workdir)/masterfiles
BS> - clone my organisation policy repo to $(sys.workdir)/masterfiles/services
BS> - clients copy all of masterfiles from the hub to $(sys.inputdir), as is currently the case

Git doesn't work nicely this way when you try to include one repo inside
another. The way you propose, it would always look dirty, with
uncommitted changes. Git has submodules and subtrees, but ultimately it
becomes a usability nightmare for most users. I would put the
organization's *data* files in `$(sys.workdir)/def` as proposed;
I think we're definitely in agreement on the idea of overlaying the
organization's settings and policy on top of the masterfiles instead of
replacing it.

In practice, organizations can't contain themselves to a single
directory for policy, so they end up customizing all of the masterfiles
to some degree. So I don't think the organization's policy files can be
practically limited to a single directory as you propose. I think their
best approach is to use the `inputs` in def.json to define their own
includes to, for instance, `$(sys.inputdir)/my-services`, and then
manage those in whatever way they like, through update.cf or something
else.

BS> So its not much different from the current norms, but the important points are:
BS> 1. There's nothing from cf-masterfiles which needs to be changed to get the system to work
BS> 2. All of the stuff that I need to modify to define the policy for my organisation is in a single directory, so its easy to manage that in a separate git repository

Yes, definitely, that's the idea. But in addition I propose to parse
def.json before evaluation happens. I hope you agree this is workable.
It will make it much more reliable.

BS> One problem is that modules typically live in $(sys.workdir)/modules ... is there any technical reason that they could not be in services/modules instead?

They can be, if we add a new def.json setting "modules" analogous to the
existing "inputs" and you set it so. There can be good arguments made
for several arrangements of the modules depending on whether you
consider them part of the CFEngine packages, the site policy, or the
machine's local policy.

Ted

[Bruce Schultz]

I had in mind adding services to the gitignore file for masterfiles. But I guess that would break for anyone merging into masterfiles, so it not really a workable solution.

> I would put the
>organization's *data* files in `$(sys.workdir)/def` as proposed;
>I think we're definitely in agreement on the idea of overlaying the
>organization's settings and policy on top of the masterfiles instead of
>replacing it.
>
>In practice, organizations can't contain themselves to a single
>directory for policy, so they end up customizing all of the masterfiles
>to some degree. So I don't think the organization's policy files can be
>practically limited to a single directory as you propose. I think their
>best approach is to use the `inputs` in def.json to define their own
>includes to, for instance, `$(sys.inputdir)/my-services`, and then
>manage those in whatever way they like, through update.cf or something
>else.

My experience was with updating masterfiles from 3.2 to 3.5 ... I had started with a fairly straightforward promises.cf & update.cf in 3.2. The standard masterfiles for 3.5 had grown considerably. Much of the extra stuff seemed applicable to enterprise only. I had customised update.cf and my promises.cf was originally hand written, so merging the updates wasn't so straight forward. Even def.cf had grown up considerably. Now days, I try to avoid modifying anything that comes from the masterfiles repo.

>
>BS> So its not much different from the current norms, but the important
>points are:
>BS> 1. There's nothing from cf-masterfiles which needs to be changed
>to get the system to work
>BS> 2. All of the stuff that I need to modify to define the policy for
>my organisation is in a single directory, so its easy to manage that in
>a separate git repository
>
>Yes, definitely, that's the idea. But in addition I propose to parse
>def.json before evaluation happens. I hope you agree this is workable.
>It will make it much more reliable.

Yes, definitely. I haven't looked at the code yet, but I imagine that the changes added to def.cf to make the overriding of settings possible adds to the complexity.

Keeping as much separation as possible between 'my' code and 'your' code should make future upgrades easier though, I'm sure.

>
>BS> One problem is that modules typically live in
>$(sys.workdir)/modules ... is there any technical reason that they
>could not be in services/modules instead?
>
>They can be, if we add a new def.json setting "modules" analogous to
>the
>existing "inputs" and you set it so. There can be good arguments made
>for several arrangements of the modules depending on whether you
>consider them part of the CFEngine packages, the site policy, or the
>machine's local policy.

How do you define 'machine local policy'? I currently use a config file on each host, which is read and turned into classes by one of my policy bundles. I'm always on the look out for better ideas in this area though.


--
:b

[Ted Zlatanov]

On Tue, 28 Jul 2015 00:47:33 +1000 Bruce Schultz <bru...@gmail.com> wrote:

BS> I had in mind adding services to the gitignore file for masterfiles.
BS> But I guess that would break for anyone merging into masterfiles, so
BS> it not really a workable solution.

Yeah, exactly.

BS> How do you define 'machine local policy'? I currently use a config
BS> file on each host, which is read and turned into classes by one of my
BS> policy bundles. I'm always on the look out for better ideas in this
BS> area though.

I think machine-local policy applies only to the current machine.
Currently that requires a machine-specific change to masterfiles policy
or some CMDB data source, since the masterfiles target the whole site.
That would get easier with my proposal. You still need to put the file
on every machine or use a secondary mechanism to generate it for every
machine (as in the server-side inventory I mentioned).

Ted

[Neil Watson]

What if we did away with the naive masterfiles-only approach that
CFEngine currently has? Instead make masterfiles able to additionally
pull from /var/cfengine/customfiles/. What is pulled could defined by
def.json.

1. Boostrap, copying masterfiles/.
2. Normal run of mastefiles has a build in mechanism, defined by
def.json or similar to also pull from /var/cfengine/customfiles/

--
Neil H Watson
Sr. Partner, Architecture and Infrastructure
CFEngine reporting: https://github.com/evolvethinking/delta_reporting
CFEngine policy: https://github.com/evolvethinking/evolve_cfengine_freelib
CFEngine and vim: https://github.com/neilhwatson/vim_cf3
CFEngine support: http://evolvethinking.com

[Brian Bennett]

I'm thinking it should be an object defined in def.json.

E.g.:

{
"vars": {
"site_dirs": {
"friendly_name1": "/path/to/dir1",
"friendly_name2": "/path/to/dir2"
}
}
}

Then bundle common def in def.cf:

vars:
"site_dirs" slist => getindices("augments[site_dirs]");
"$(site_dirs)" string => translatepath("augments[$(site_dirs)]"),
comment => "Site specified directory paths",
handle => "$(site_dirs)_location";

And bundle server access_rules in cf_server.cf could have:

"$(def.site_dirs)"
handle => "server_access_grant_access_to_$(def.site_inputs)",
comment => "Grant access to path $(def.site_inputs)",
admit => { @(def.acl) };


Later in the policy I could refer to $(def.friendly_name1) for copy_from or inputs promises. Now I can put files in any directory on the filesystem. The only "intrusion" to masterfiles is def.json.

--
Brian Bennett
Looking for CFEngine training?
http://www.verticalsysadmin.com/

[Neil Watson]

On Mon, Jul 27, 2015 at 10:05:44AM -0700, Brian Bennett wrote:
>I'm thinking it should be an object defined in def.json.
>
>E.g.:
> {
> "vars": {
> "site_dirs": {
> "friendly_name1": "/path/to/dir1",
> "friendly_name2": "/path/to/dir2"

Which leads to:
"vars": {
"site_dirs": {
"custom" : "/path/to/custom/policies",
"infosec_dept": "/path/to/infosec/policies",
"EFL" : "/path/to/3rd/party/efl/lib"

Neat and separate.

[Nick Anderson]

On 07/26/2015 04:25 AM, Ted Zlatanov wrote:
> BS> One problem is that modules typically live in $(sys.workdir)/modules ... is there any technical reason that they could not be in services/modules instead?

In the past when I have wanted to ship modules inside of masterfiles, I
just add a policy to copy from masterfiles into /var/cfengine/modules on
the policy hub. The default update policy copies /var/cfengine/modules
from the hub to /var/cfengine/modules on the remote agents.

[Ted Zlatanov]

On Mon, 27 Jul 2015 13:50:55 -0400 Neil Watson <cfen...@watson-wilson.ca> wrote:

NW> Which leads to:
NW> "vars": {
NW> "site_dirs": {
NW> "custom" : "/path/to/custom/policies",
NW> "infosec_dept": "/path/to/infosec/policies",
NW> "EFL" : "/path/to/3rd/party/efl/lib"

I think it should avoid vars, if we want this parsed at the C level.

How about this, at the top level:

version 1, key-value map:

"inputs": {

"custom" : "/path/to/custom/policies",
"infosec_dept": "/path/to/infosec/policies",
"EFL" : "/path/to/3rd/party/efl/lib"
}

version 2, simple list:

"inputs": [ "/my/dir1", "/my/dir2" ]

At the policy level, getindices() and getvalues() will work on both of
these containers. And at the C level if def.json is parsed there, it's
not a problem to support these and throw them into a variable eventually
(but they'll be loaded before the evaluator fires up).

Ted

[Brian Bennett]

The name "inputs" already implies files, not directories. I'd advocate for a search for a better term.

I prefer version 1, so the paths have a defined variable name for reference within policies. Version 2 would make it difficult to assign within policy.

--
Brian Bennett
Looking for CFEngine training?
http://www.verticalsysadmin.com/

[Neil Watson]

On Mon, Jul 27, 2015 at 03:10:07PM -0400, Ted Zlatanov wrote:
>On Mon, 27 Jul 2015 13:50:55 -0400 Neil Watson <cfen...@watson-wilson.ca> wrote:
>
>NW> Which leads to:
>NW> "vars": {
>NW> "site_dirs": {
>NW> "custom" : "/path/to/custom/policies",
>NW> "infosec_dept": "/path/to/infosec/policies",
>NW> "EFL" : "/path/to/3rd/party/efl/lib"
>
>I think it should avoid vars, if we want this parsed at the C level.

I would rather not have another copy everything approach. I'd like to be
able to limit it by class.

[Ted Zlatanov]

On Mon, 27 Jul 2015 15:21:40 -0400 Neil Watson <cfen...@watson-wilson.ca> wrote:

NW> On Mon, Jul 27, 2015 at 03:10:07PM -0400, Ted Zlatanov wrote:
>> On Mon, 27 Jul 2015 13:50:55 -0400 Neil Watson <cfen...@watson-wilson.ca> wrote:
>>
NW> Which leads to:
NW> "vars": {
NW> "site_dirs": {
NW> "custom" : "/path/to/custom/policies",
NW> "infosec_dept": "/path/to/infosec/policies",
NW> "EFL" : "/path/to/3rd/party/efl/lib"
>>
>> I think it should avoid vars, if we want this parsed at the C level.

NW> I would rather not have another copy everything approach. I'd like to be
NW> able to limit it by class.

Astute readers will note that your version doesn't limit by class either :)

On Mon, 27 Jul 2015 12:14:10 -0700 Brian Bennett <brian....@verticalsysadmin.com> wrote:

BB> The name "inputs" already implies files, not directories. I'd advocate for a search for a better term.

It currently implies files, but I don't see why we need a better term
instead of extending the current one. It's obvious, short, and clean.

BB> I prefer version 1, so the paths have a defined variable name for reference within policies. Version 2 would make it difficult to assign within policy.

Merging that with Neil's need for classes:

version 3:

"inputs": {
"custom" : "/path/to/custom/policies",
"platform" : { "redhat": "/p1/redhat", "debian": "/p2/debian" },
"infosec_dept": "/path/to/infosec/policies",
"EFL" : { "eflhost": "/path/to/3rd/party/efl/lib" }
}

I personally would prefer not to add classes into a simple data
structure, so I'm in favor of version 1 (simple key-value).

Ted

[Brian Bennett]

I feel this is getting too complicated compared with what I thought was a simple and elegant solution.

Ted, with your proposal, how would I refer to these directories in policy?
I don't understand how parsing it in C gains anything. My goal was to extend the directories in bundle server access_rules without having to edit policy. All other access_rules are in policy, why push this into C? Why have some access rules in policy while others are in C? I practically wrote the entire policy necessary to implement this in under five minutes with two stanzas. What's wrong with that?

As for "limit it by class", this is the server access rules. Either the server is willing to serve the files or it isn't. The server doesn't care (or even know!) which classes are set on the client. Which files a client decides to access can be controlled on the client with classes, but that has nothing to do with whether or not the server is configured to grant access to that path.

Maybe I've entirely missed the point...

--
Brian Bennett
Looking for CFEngine training?
http://www.verticalsysadmin.com/

[Ted Zlatanov]

On Tue, 28 Jul 2015 01:05:41 -0700 Brian Bennett <brian....@verticalsysadmin.com> wrote:

BB> I feel this is getting too complicated compared with what I thought
BB> was a simple and elegant solution.

If you mean version 3 (inputs with classes) I agree, I like version 1
(simple key-value map) best.

BB> Ted, with your proposal, how would I refer to these directories in
BB> policy?

Probably a key-value map called `sys.inputs`. Iterating over it will
give you the values.

It may have to be a list of files or wildcards, not directories, to be
consistent with the current inputs.

BB> I don't understand how parsing it in C gains anything.

You won't have to rely on evaluation order. I just had to add a
workaround to 3.7 because the def.json inputs were not loaded correctly
in *some* cases because of evaluation.

BB> My goal was to extend the directories in bundle server access_rules
BB> without having to edit policy. All other access_rules are in policy,
BB> why push this into C? Why have some access rules in policy while
BB> others are in C? I practically wrote the entire policy necessary to
BB> implement this in under five minutes with two stanzas. What's wrong
BB> with that?

I never proposed putting access rules in C. The only inputs-related
thing you'll get when parsing def.json in C is to give you a
`sys.inputs` variable and to add those directories and files to the
global inputs.

Contrast that with augmenting inputs in policy today. It requires some
really non-obvious magic with `body file control` and common bundles.
If you were a *new* CFEngine user, would you use that or walk away?

There's nothing wrong with policy. But many settings (like inputs, hard
classes, ignore_interfaces, evaluation control, or the location of some
system utilities) can't be easily overridden or augmented at the policy
level. That's my only goal: not to replace policy but to provide new
functionality that didn't exist before, in a simple way.

Ted

[Neil Watson]

On Tue, Jul 28, 2015 at 01:05:41AM -0700, Brian Bennett wrote:
>I feel this is getting too complicated compared with what I thought was a simple and elegant solution.
>
>Ted, with your proposal, how would I refer to these directories in
>policy? I don't understand how parsing it in C gains anything. My goal
>was to extend the directories in bundle server access_rules without
>having to edit policy. All other access_rules are in policy, why push

Do you know that EFL already does this? See its server bundle:

https://github.com/evolvethinking/evolve_cfengine_freelib/blob/master/masterfiles/lib/3.7/EFL/evolve_freelib.cf#L2287

But, I thought the goal was to get a way from the masterfiles
limitation on both server and client side.

[Ted Zlatanov]

I opened https://dev.cfengine.com/issues/7453 for an implementation of
def.json parsing in C.

Ted