yum package module - get info on upgraded packages

[Xander Cage]

hi,

i use the yum package module to manage rpm installs. works well so far, but one thing bothers me. i want to report which packages are going to be upgraded but i can not figure out how to do this

i looked at the modules code but this seems pretty dumb and does not set any applicable classes. it has the obvious "list-upgrade" parameter but i'm not clear how to use this.

wbr

chris

[Xander Cage]

the actual policy:

<pre>

#!/var/cfengine/bin/cf-agent -KI # #body common control #{ # bundlesequence => { "b0026_manage_aixtoolbox" }; # inputs => { # "$(sys.libdir)/stdlib.cf", # "$(sys.inputdir)/itsv/itsv_common_lib.cf", # "$(sys.inputdir)/itsv/hostgroups.cf", # "$(sys.inputdir)/itsv/itsv_stanza_lib.cf", # }; #} # ## would be nicer with regex ("ITSV_ABORT.*") - but Bug prevents this .. #body agent control { # abortbundleclasses => { "ITSV_BUNDLE_ABORT" }; # abortclasses => { "ITSV_ABORT" }; #} # bundle agent b0026_manage_aixtoolbox { # # promise to manage aixtoolbox rpm packages # and yum repo configs # meta: "tags" slist => { "itsv" }; methods: !am_policy_hub:: "do the rpm walk" usebundle => manage_aixtoolbox, ifvarclass => "SPARE_DAILY", action => if_elapsed_day; } bundle agent manage_aixtoolbox { classes: "RPM_ROLLOUT_PLATFORM" expression => or("aix_7_1", "aix_7_2"); #"AIXTOOLBOX_ENABLED" expression => returnszero("/usr/bin/yum -q repolist | /usr/bin/grep -q AIX_Toolbox", "useshell"); # class moved to hostgroups.cf #"EXCLUDED_HOSTS" expression => regcmp("AIXBUILDHOST|AIXTEST01|AIXBUILDHOSTNG", "$(itsv_hostgroups.LPAR_NAME)"); methods: RPM_ROLLOUT_PLATFORM.AIXTOOLBOX_RPMS.!DMZ_HOST.!EXCLUDED_HOSTS:: "use toolbox minion" usebundle => toolbox_minion, comment => "keep all RPM versions the same accross all installations"; reports: "!RPM_ROLLOUT_PLATFORM":: "Only AIX71/AIX72 supported"; } bundle agent toolbox_minion { vars: "wanted_packages_versions" data => '{ "bash": "latest", "bzip2": "latest", "ca-certificates": "latest", "cfengine": "3.12.2-itsv", "cpio": "latest", "coreutils": "latest", "curl": "latest", "db": "latest", "expat": "latest", "file": "latest", "gawk": "latest", "gdbm": "latest", "gettext": "latest", "git": "latest", "gzip": "latest", "info": "latest", "less": "latest", "libiconv": "latest", "lzo": "latest", "make": "latest", "ncurses": "latest", "openldap": "latest", "pcre": "latest", "perl": "5.30.3-1", "pkg-config": "latest", "pysqlite": "latest", "python": "latest", "python-devel": "latest", "python-iniparse": "latest", "python-pycurl": "latest", "python-tools": "latest", "python-urlgrabber": "latest", "python3": "latest", "python3-devel": "latest", "python3-docs": "latest", "python3-test": "latest", "python3-tools": "latest", "readline": "latest", "rlwrap": "latest", "rsync": "latest", "screen": "latest", "sed": "latest", "sqlite": "latest", "tar": "latest", "unzip": "latest", "vim-common": "latest", "vim-enhanced": "latest", "wget": "latest", "xz": "latest", "xz-libs": "latest", "yum": "latest", "yum-metadata-parser": "latest", "zip": "latest", "zlib": "latest", "zlib-devel": "latest" }'; "pkg" slist => getindices( wanted_packages_versions ); "unwanted_packages" slist => { "test-dummy" }; files: "/etc/sudoers_cfesave" copy_from => local_cp("/etc/sudoers"), classes => if_ok("SUDOERS_SAVE_OK"); "/opt/freeware/etc/yum/yum.conf" create => 'true', template_method => 'mustache', edit_defaults => empty, edit_template => '${sys.workdir}/inputs/itsv/templates/yum_conf.mustache'; "/opt/freeware/etc/yum/repos.d/*" delete => tidy; packages: "$(pkg)" policy => "present", package_module => yum, version => "$(wanted_packages_versions[$(pkg)])", options => { "--skip-broken" }, ifvarclass => "YUM_UNINSTALL_OK", classes => if_ok("YUM_INSTALL_OK"); "$(unwanted_packages)" policy => "absent", package_module => yum, classes => if_ok("YUM_UNINSTALL_OK"); SUDOERS_SAVE_OK:: "sudo" policy => "present", package_module => yum, version => "1.8.21p2-1", classes => if_ok("SUDO_INST_OK"); files: SUDO_INST_OK:: "/etc/sudoers" copy_from => local_cp("/etc/sudoers_cfesave"); reports: YUM_INSTALL_OK:: "YUM UPGRADED SOME PACKAGES, SEE /var/log/yum.log"; YUM_UNINSTALL_OK:: "YUM ERASED SOME PACKAGES, SEE /var/log/yum.log"; SUDO_INST_OK:: "SUDO INSTALL IS OK"; DEBUG:: "LPARNAME: '$(itsv_hostgroups.LPAR_NAME)'"; "$(this.bundle) YES THIS ONE"; "$(pkg) SHOULD BE VERSION $(wanted_packages_versions[$(pkg)]) "; }  

</pre>

[Xander Cage]

god, sorry for the bad formatting,  no code formatting in this new interface?

[Nick Anderson]

Xander Cage writes:

Listing 1: Reformatted for humans

#!/var/cfengine/bin/cf-agent -KI

#body common control
#{ # bundlesequence => { "b0026_manage_aixtoolbox" };
# inputs => {
# "$(sys.libdir)/stdlib.cf",
# "$(sys.inputdir)/itsv/itsv_common_lib.cf",
# "$(sys.inputdir)/itsv/hostgroups.cf",
# "$(sys.inputdir)/itsv/itsv_stanza_lib.cf",
# };
#}
# ## would be nicer with regex ("ITSV_ABORT.*") - but Bug prevents this ..
#body agent control {
# abortbundleclasses => { "ITSV_BUNDLE_ABORT" };
# abortclasses => { "ITSV_ABORT" };
#}

bundle agent b0026_manage_aixtoolbox {
#
# promise to manage aixtoolbox rpm packages
# and yum repo configs

meta: "tags" slist => { "itsv" };
  methods:
    !am_policy_hub::
      "do the rpm walk" usebundle => manage_aixtoolbox,
        ifvarclass => "SPARE_DAILY",
        action => if_elapsed_day;
}
bundle agent manage_aixtoolbox {
  classes:
      "RPM_ROLLOUT_PLATFORM" expression => or("aix_7_1", "aix_7_2")

;
      

[Nick Anderson]

Nick Anderson writes:

You didn't ask but …

,-—

#!/var/cfengine/bin/cf-agent -KI

#body common control

#{ # bundlesequence => { "b0026_manage_aixtoolbox" };

# inputs => {

# "$(sys.libdir)/stdlib.cf",

# "$(sys.inputdir)/itsv/itsv_common_lib.cf",

# "$(sys.inputdir)/itsv/hostgroups.cf",

# "$(sys.inputdir)/itsv/itsv_stanza_lib.cf",

# };

#}

Instead of keeping these stubs for direct execution which need to be commented/uncommented, perhaps consider leveraging body file control and bundle agent main.

# ## would be nicer with regex ("ITSV_ABORT.*") - but Bug prevents this ..

#body agent control {

# abortbundleclasses => { "ITSV_BUNDLE_ABORT" };

# abortclasses => { "ITSV_ABORT" };

#}

Is that still true for you?

When I search the tracker I find CFE-223, CFE-961, and CFE-1874 which are all closed.

bundle agent b0026_manage_aixtoolbox {

#

# promise to manage aixtoolbox rpm packages

# and yum repo configs

meta: "tags" slist => { "itsv" };

methods:

!am_policy_hub::

"do the rpm walk" usebundle => manage_aixtoolbox,

ifvarclass => "SPARE_DAILY",

These days if is preferred over ifvarclass.

Personally I am not a fan of the if_ok classes body.

Listing 1: the if_ok classes body

body classes if_ok(x)
# @brief Define the class `x` if the promise is kept or could be repaired
# @param x The name of the class that should be defined
{
      promise_repaired => { "$(x)" };
      promise_kept => { "$(x)" };
}

It's a bit of a misnomer. As you can see it defines the class if the promise is either kept or repaired. But promises can have multiple outcomes, and this doesn't really account for that.

This example illustrates a case where a single promise is both kept and notkept at the same time.

Listing 2: /tmp/multiple-outcomes.cf

#!/var/cfengine/bin/cf-agent -f-
body file control
{
        inputs => { '$(sys.libdir)/stdlib.cf' };
}

bundle agent example_multiple_outcomes
{
  classes:
      "running_as_root" expression => strcmp( "$(sys.user_data[uid])", "0" );

  files:
    running_as_root::
      "/tmp/immutable"
        create => "true",
        handle => "init_immutable", 
        edit_line =>  example_edit_line("Initalized file"),
        classes => results("bundle", "my_id_init"),
        unless => fileexists( "/tmp/immutable" ); 

      "/tmp/immutable"
        create => "true",
        edit_line =>  example_edit_line("Modify"),
        classes => results("bundle", "my_id_modify"),
        depends_on => { "file_immutable" };

  commands:
    running_as_root::
      "chattr +i /tmp/immutable"
        handle => "file_immutable",
        contain => in_shell,
        depends_on => { "init_immutable" }; 

  vars:
    running_as_root::
      "classes" slist => classesmatching(".*my_id.*");

  reports:
    !running_as_root::
      "You need to run this example as root in order for the immutable file to be created";

    running_as_root::
      "Found Class = '$(classes)'";
}

bundle edit_line example_edit_line(str)
{
  insert_lines:
      "$(sys.date) $(str)";
}
bundle agent __main__{methods:"example_multiple_outcomes";}

Listing 3: /tmp/multiple-outcomes.cf

exec 2>&1
cf-agent -KIf /tmp/multiple-outcomes.cf
:

    info: Created file '/tmp/immutable', mode 0600
    info: Inserted the promised line 'Thu Jan 28 10:54:50 2021 Initalized file' into '/tmp/immutable' after locator
    info: insert_lines promise 'Thu Jan 28 10:54:50 2021 Initalized file' repaired
    info: Edited file '/tmp/immutable'
    info: files promise '/tmp/immutable' repaired
    info: Executing 'no timeout' ... 'chattr +i /tmp/immutable'
    info: Completed execution of 'chattr +i /tmp/immutable'
    info: Inserted the promised line 'Thu Jan 28 10:54:50 2021 Modify' into '/tmp/immutable' after locator
    info: insert_lines promise 'Thu Jan 28 10:54:50 2021 Modify' repaired
   error: Can't rename '/tmp/immutable.cf-after-edit' to '/tmp/immutable' - so promised edits could not be moved into place. (rename: Operation not permitted)
   error: Unable to save file '/tmp/immutable' after editing
   error: Errors encountered when actuating files promise '/tmp/immutable'
R: Found Class = 'my_id_init_repaired'
R: Found Class = 'my_id_init_reached'
R: Found Class = 'my_id_modify_failed'
R: Found Class = 'my_id_modify_not_kept'
R: Found Class = 'my_id_modify_error'
R: Found Class = 'my_id_modify_kept'
R: Found Class = 'my_id_modify_reached'
   error: Method 'example_multiple_outcomes' failed in some repairs

reports:

YUM_INSTALL_OK::

"YUM UPGRADED SOME PACKAGES, SEE /var/log/yum.log";

Are you familiar with printfile bodies?

bundle agent example_printfile(file)
{
  reports:
    "The first two lines of '$(file)':"
      printfile => head_n( $(file), 2 );

    "The full content of '$(file)'"
      printfile => cat( $(file) );
}
bundle agent __main__
{
  methods:
      "Print Self"
        usebundle => example_printfile($(this.promise_filename)); 
}

R: The first two lines of '/home/nickanderson/org/cfengine3-gPoHAy':
R: body file control{ inputs => { '$(sys.libdir)/stdlib.cf' };}
R: bundle agent example_printfile(file)
R: The full content of '/home/nickanderson/org/cfengine3-gPoHAy'
R: body file control{ inputs => { '$(sys.libdir)/stdlib.cf' };}
R: bundle agent example_printfile(file)
R: {
R:   reports:
R:     "The first two lines of '$(file)':"
R:       printfile => head_n( $(file), 2 );
R: 
R:     "The full content of '$(file)'"
R:       printfile => cat( $(file) );
R: }
R: bundle agent __main__
R: {
R:   methods:
R:       "Print Self"
R:         usebundle => example_printfile($(this.promise_filename)); 
R: }

For this I guess it would be nice to be able to print the end of the file instead of the whole thing or just the beginning of the file. I created CFE-3558 maybe someone will be nice enough to implement it for us.

[Xander Cage]

hi nick,

lot of good tips  and info, thank you. concerning the aboortbundleclass bug -> https://tracker.mender.io/browse/CFE-3411 the newest one is still open and unassigned.

chris

[Nick Anderson]

Ah! I even filed it, thanks!

So it works, it's just also got annoying output.

[Nick Anderson]

Xander Cage writes:

i use the yum package module to manage rpm installs. works well so far, but one thing bothers me. i want to report which packages are going to be

It's not clear to me exactly what your looking for but two things come to mind.

1. The action_policy attribute in action bodies. Make a warn only promise to upgrade the list of packages, promises not kept should be the ones that want to upgrade.
2. Use the packageupdatesmatching() function to figure out which packages have updates available.

[Xander Cage]

to keep it simple, i want the policy to report which packages were upgraded, best without messing with yum.log but use cfengine internal information. seems action_policy_attribute and packageupdatedmatching are very usefull canditates for this, never heard of them until now.

[Nick Anderson]

With CFEngine Enterprise you don't need to do anything special to be able to report on that kind of thing.

Example screenshot showing changes (promises repaired) for packages promises over the last 7 days::

[Xander Cage]

as we are a pure aix shop and there is no enterprise aix policy server package available, we are forced to toying around with the community edition ;-)

[Xander Cage]

i messed around with action_policy and packagesupdatemathing(), looked promising at first sight, but i was unable to to figure out how to integrate this in my policy, way to complicated for my poor little  brain, thanks anyway.

[Nick Anderson]

Ah yes, you would need to run Linux for the policy server.

[Aleksey Tsalolikhin]

Which I reckon you could do in a VM on an AIX server.  ;)

--
You received this message because you are subscribed to the Google Groups "help-cfengine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to help-cfengin...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/help-cfengine/d9e5e451-9f66-4a1f-8d7f-f9dfc6546588n%40googlegroups.com.

[Xander Cage]

maybe,  we have pc64le linux partitions, but most software is strictly tied to x86 architecture, and this calls for serious headache. ibm has put some effort into the aix toolbox for linux applications eg.  open source rpms recently, lets see where this will get us.

[Nick Anderson]

Xander Cage writes:

i messed around with action_policy and packagesupdatemathing(), looked promising at first sight, but i was unable to to figure out how to integrate this in my policy, way to complicated for my poor little brain, thanks anyway.

What specifically did you find complicated?

• Working with the data structure from packageupdatesmatching()?
• Interpreting the classes that result from a promise to warn or nop?
• Something else?

The basic steps are:

• Figure out what packages have updates available.
  • packageupdatesmatching() returns an array of dicts, so we need to pull an index that we can use to iterate over the datastructure.
• For each of the packages that have updates available, you make a promise that takes no action (warn|nop) and define classes based on the result.
  • Note, promises to warn or nop both emit WARNING messages. It's a bit annoying, CFE-1983 wants to prohibit the messages if nop is used.

body file control
{ 
  inputs => { "$(sys.libdir)/stdlib.cf" };
}
body common control
{
  package_module => "$(package_module_knowledge.platform_default)";
  package_inventory => { $(package_module_knowledge.platform_default) };
}
bundle agent example_packages_version_latest_warn_classes
{
   vars:
     # Get a datastructure containing information about package updates that are available
     "_package_updates" data => packageupdatesmatching( ".*", ".*", ".*", ".*" );

     # packageupdatesmatching() returns an array of dicts, we need to pull the index for iteration
     "_updates_available" slist => getindices( @(_package_updates) );

     # Build a classic array of the packages that have updates available
     # Key on package name, value being the version available for install

     "_update_available[$(_package_updates[$(_updates_available)][name])]" string => "$(_package_updates[$(_updates_available)][version])";
     "_packages_upgradeable" slist => getindices( _update_available );

  packages:
     "$(_packages_upgradeable)"
       policy => "present",
       version => "latest",
       action => policy( "warn" ),
       classes => results( "bundle", "my_pkg_$(_packages_upgradeable)");

   reports:
   "$(_packages_upgradeable) can be upgraded to version '$(_update_available[$(_packages_upgradeable)])'";

   "Classes defined as result of packages promise:$(const.n)$(const.t)$(with)"
     with => join( "$(const.n)$(const.t)", classesmatching( "my_pkg_.*" ) );

}
bundle agent __main__
{
  methods: "example_packages_version_latest_warn_classes";
}

Should result in output something like this:

 warning: Package 'wireshark' should be installed
R: wireshark can be upgraded to version '1.10.14-25.el7'
        my_pkg_wireshark_failed
        my_pkg_wireshark_not_kept
        my_pkg_wireshark_error
        my_pkg_wireshark_reached

[Xander Cage]

this part was (and somwwhat is) thd culprit...

package_module => "$(package_module_knowledge.platform_default)";

package_inventory => { $(package_module_knowledge.platform_default) };

especially the platform part, in my environment (aix, ppc), this does not play well with the default (whatever the default is).  or am i completely on the wrong path...

[Nick Anderson]

Right, for aix i dont think that the default modules include yum, but you could try adding it.

[Xander Cage]

tested further...i changed nimclient to yum in packages.cf to yum.

 aix::

      "platform_default" string => "yum";

now it shows some sings of success...but alaso a lot of error messages

root@aixtest01: /root # /var/cfengine/bin/cf-agent -KI  -f ./manage_aixtoolbox_new.cf

...

 error: Invalid or missing arguments in package_module body 'yum':  query_installed_ifelapsed = -678 query_updates_ifelapsed = -678

   error: New package promise failed sanity check.

   error: Invalid or missing arguments in package_module body 'yum':  query_installed_ifelapsed = -678 query_updates_ifelapsed = -678

   error: New package promise failed sanity check.

   error: Invalid or missing arguments in package_module body 'yum':  query_installed_ifelapsed = -678 query_updates_ifelapsed = -678

   error: New package promise failed sanity check.

   error: Invalid or missing arguments in package_module body 'yum':  query_installed_ifelapsed = -678 query_updates_ifelapsed = -678

...

R: db can be upgraded to version '6.2.38-3'

R: libyaml can be upgraded to version '0.2.2-1'

R: libssh2 can be upgraded to version '1.9.0-1'

R: httpd can be upgraded to version '2.4.46-1'

R: libgcc can be upgraded to version '1:8-1'

R: pcre can be upgraded to version '8.44-1'

R: libunistring can be upgraded to version '0.9.10-1'

R: python-tools can be upgraded to version '2.7.18-1'

...

R: Classes defined as result of packages promise:

        my_pkg_libstdc___reached

        my_pkg_httpd_failed

        my_pkg_libiconv_reached

        my_pkg_sqlite_error

        my_pkg_httpd_error

        my_pkg_pcre_failed

        my_pkg_yum_reached

        my_pkg_curl_error

        my_pkg_liblmdb_not_kept

        my_pkg_bash_not_kept

        my_pkg_pcre_error

        my_pkg_expat_not_kept

        my_pkg_gettext_reached

        my_pkg_python_error

...

 error: Invalid or missing arguments in package_module body 'yum':  query_installed_ifelapsed = -678 query_updates_ifelapsed = -678

   error: New package promise failed sanity check.

   error: Invalid or missing arguments in package_module body 'yum':  query_installed_ifelapsed = -678 query_updates_ifelapsed = -678

   error: New package promise failed sanity check.

   error: Invalid or missing arguments in package_module body 'yum':  query_installed_ifelapsed = -678 query_updates_ifelapsed = -678

   error: New package promise failed sanity check.

   error: Invalid or missing arguments in package_module body 'yum':  query_installed_ifelapsed = -678 query_updates_ifelapsed = -678

   error: New package promise failed sanity check.

   error: Invalid or missing arguments in package_module body 'yum':  query_installed_ifelapsed = -678 query_updates_ifelapsed = -678

   error: New package promise failed sanity check.

   error: Invalid or missing arguments in package_module body 'yum':  query_installed_ifelapsed = -678 query_updates_ifelapsed = -678

   error: New package promise failed sanity check.

   error: Invalid or missing arguments in package_module body 'yum':  query_installed_ifelapsed = -678 query_updates_ifelapsed = -678

   error: New package promise failed sanity check.

   error: Invalid or missing arguments in package_module body 'yum':  query_installed_ifelapsed = -678 query_updates_ifelapsed = -678

so where should i redefine the default_platform for aix as packages.cf seems not to be the best place for this?  where are this error

messages coming from?

[Nick Anderson]

Xander Cage writes:

tested further…i changed nimclient to yum in packages.cf to yum.

aix:: "platform_default" string => "yum";

now it shows some sings of success…but alaso a lot of error messages

root@aixtest01: /root # /var/cfengine/bin/cf-agent -KI -f ./manage_aixtoolbox_new.cf … error: Invalid or missing arguments in package_module body 'yum': query_installed_ifelapsed = -678 query_updates_ifelapsed = -678

Yeah, those values aren't being set. They should be set when you use the full MPF. For my standalone test I also dropped a def.json next to the policy file to set those values.

{
  "vars": {
    "package_module_query_installed_ifelapsed": "0",
    "package_module_query_updates_ifelapsed": "0"
  }
}

[Nick Anderson]

Nick Anderson writes:

Yeah, those values aren't being set. They should be set when you use the full MPF. For my standalone test I also dropped a def.json next to the policy file to set those values.

,-— | { | "vars": { | "package_module_query_installed_ifelapsed": "0", | "package_module_query_updates_ifelapsed": "0" | } | } `-—

Also, this was changed recently so that you don't have to do that if your loading packages.cf.

https://tracker.mender.io/browse/CFE-3504

[Xander Cage]

bingo!

after def.json modification and doing the following everything works...as yum is more and more integrated into aix, should this added to the mpf

in any form? not sure...

body file control
{
  inputs => { "$(sys.libdir)/stdlib.cf",
               "$(sys.inputdir)/itsv/itsv_common_lib.cf",
               "$(sys.inputdir)/itsv/hostgroups.cf",
               "$(sys.inputdir)/itsv/itsv_stanza_lib.cf", };
}

bundle common package_module_knowledge_alternate
{
  vars:
      "platform_default_alternate" string => "yum";

}


body common control
{
  package_module => "$(package_module_knowledge_alternate.platform_default_alternate)";
  package_inventory => { $(package_module_knowledge_alternate.platform_default_alternate) };
}