apt package promises: Options= is ignored?

[Moore, Joe]

My cf-agent logs are filled with the output from APT, particularly when I have multiple bundles making new-style package promises.  I know that I can set quiet “2”; in /etc/apt/conf.d/99shaddup and enforce that file via cfengine, but that sets a the flag globally even for an admin who is manually running apt.

 

There is virtually no description of the “options” attribute of a package process other than “It is passed to the module”.  From looking at masterfiles/modules/packages/apt_get, I cannot find anywhere that passed-in options would be used.

 

The package module API (https://docs.cfengine.com/lts/reference-standard-library-package_modules.html) refers to options, but none of the examples have options in the dialog.  I assume to expect multiple line.startswith(“Options=”) for each entry in the slist, including lines for any default_options that are set.  The apt_get module doesn’t handle this case.  I think it’s just a matter of adding those lines to the apt_get_options array.

 

A more concrete and useful (not just cosmetic) example:

  packages:

    “mypackage”

       comment => “Install mypackage from our internal repo.  NB: The Internal repo uses myCA-signed certificate, which is right in this case, but should not be used everywhere”,

       package_policy => “addupdate”,

       package_method => “apt”,

       options        => { “https::CaPath=/usr/local/etc/myCA/” };

 

Thoughts?

 

--Joe

[Nick Anderson]

On 09/14/2016 09:36 AM, Moore, Joe wrote:
> There is virtually no description of the “options” attribute of a
> package process other than “It is passed to the module”. From looking
> at masterfiles/modules/packages/apt_get, I cannot find anywhere that
> passed-in options would be used.

options is a catchall attribute in order to pass arbitrary data into the
package module which is carrying out package operations. It is meant as
a rescue solution when a package module has added functionality which is
not covered by the package promise API. As such there is no official
documentation for this attribute, its usage depends on the package
module in question.

https://docs.cfengine.com/lts/reference-promise-types-packages.html#options

I think that the only package module that supports options currently is
nimclient. So the apt_get module would need to be extended to know what
to do with the options.

> The package module API
> (https://docs.cfengine.com/lts/reference-standard-library-package_modules.html)
> refers to options, but none of the examples have options in the dialog.
> I assume to expect multiple line.startswith(“Options=”) for each entry
> in the slist, including lines for any default_options that are set. The
> apt_get module doesn’t handle this case. I think it’s just a matter of
> adding those lines to the apt_get_options array.
>
>
>
> A more concrete and useful (not just cosmetic) example:
>
> packages:
>
> “mypackage”
>
> comment => “Install mypackage from our internal repo. NB: The
> Internal repo uses myCA-signed certificate, which is right in this case,
> but should not be used everywhere”,
>
> package_policy => “addupdate”,
>
> package_method => “apt”,
>
> options => { “https::CaPath=/usr/local/etc/myCA/” };

So this should come into the module as
options=https::CaPath=/usr/local/etc/myCA.

And then I guess it can be used to extend that apt_get_options array.
https://github.com/cfengine/masterfiles/blob/master/modules/packages/apt_get#L16

[Moore, Joe]

Ok, how about https://github.com/cfengine/masterfiles/pull/802 ?

Seemed pretty straightforward, I did have to move the construction of the cmd_line to after calling package_arguments_builder.

--Joe

--
You received this message because you are subscribed to the Google Groups "help-cfengine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to help-cfengin...@googlegroups.com.
To post to this group, send email to help-c...@googlegroups.com.
Visit this group at https://groups.google.com/group/help-cfengine.
For more options, visit https://groups.google.com/d/optout.

[Nick Anderson]

On 09/14/2016 01:11 PM, Moore, Joe wrote:
> Ok, how about https://github.com/cfengine/masterfiles/pull/802 ?
>
> Seemed pretty straightforward, I did have to move the construction of
> the cmd_line to after calling package_arguments_builder.

I had issues when I gave it a go. Left comment on the PR.

[mike.w...@verticalsysadmin.com]

JM> There is virtually no description of the “options” attribute of a package process other than “It is passed to the module”.

I am working on extending the "yum" package module to allow options to be passed (specifically working to allow "--disablerepo" and "--enablerepo" flags to be set), and I've run into the same lack of documentation.

I understand that options aren't implemented in many package modules, but I'm talking about the documentation of how CFEngine calls the package module, and what the package module should look for.  The current API documentation only states:

"This attribute has no inherent meaning to CFEngine, and will be passed verbatim."

However, it makes no mention of how it will be passed.  As an argument?  (I doubt it.)  On stdin?  (I expect so.)  Before the other text it passes on stdin, or after?  (I could guess it will be before, but that is just a guess.)  With an "Options=" prefix, as Joe Moore guessed?  (I doubt it, given the use of the word "verbatim.")

I tried digging into the C code, but I couldn't tell exactly where and how the options are getting passed to the module.  Nick, could you please clarify that?

Best,

--Mike Weilgart

P.S.: This leads to another question: If the "options" from a CFEngine packages promise are passed verbatim to the package module, can the module differentiate the module protocol that is passed by CFEngine's logic, from fake module protocol input embedded in an "options" flag?  From brief testing, the answer seems to be "yes, it can differentiate these," but I don't see why or how.  Once I know how the options info is actually passed to the module I will better understand the system.

[mike.w...@verticalsysadmin.com]

Further testing shows it does pass the options, at the beginning of the stdin, each one on a line beginning with "options=".  This is how it differentiates options from the rest of the protocol.

Of course, this implies that options must never have embedded newlines, and sure enough, running the following packages promise with neither "mc" nor "zsh" installed will install both of them, but only report installing "mc".  :)

body common control {

  inputs => { "$(sys.libdir)/stdlib.cf" };

}

bundle agent main {

  packages:

    "mc"

      policy => "present",

      package_module => yum,

      options => { "$(const.n)Name=zsh" };

}

;)

Of course, this isn't a bug in any way.  I deliberately abused the "options" flag to fake out the protocol.  However, it is an excellent example to study to understand just how CFEngine passes information to the packages module.

Best,

--Mike Weilgart

Vertical Sysadmin, Inc.

[Moore, Joe]

Yes, you are correct, in https://github.com/cfengine/core/blob/master/cf-agent/package_module.c#L227, the cf-agent passes any values in the options => { } slist down to the underlying process call via STDIN as option=listvalue.  Multiline options => {} as you’ve shown below would need to be interpreted by the module program.  Certainly, you could create a program that interprets the input with a different grammar.  (i.e. Consider anything that does not start with one of the other protocol strings as a continuation of the previous line)

 

Note that it is “option=”, not “Option=”.  The other protocol lines are capitalized (Name, Architecture, Version)

 

You might find my test setup at https://github.com/jpvlsmv/masterfiles/wiki/Troubleshooting-apt_get-module useful.  It’s for the Debian side, but the Vagrantfile can be easily switched to support yum-based installs.  You’d just have to find appropriate options and packages to demonstrate.

 

--Joe

--