software_package.csv package parsing for non-standard names

[dknapik]

We are in the process of migrating from CFE2 to CFE3 and I'm trying to figure out an issue that has to do with package management.

BUILD: cfengine-community-3.6.2-1
OS: RHEL5_U11
ARCH: x86_64

What we observe is on every run of cf-agent, a list of packages is always requested to be installed. When the package management gets to it (yum in this case)
it replies back with "It's already installed" and moves on.

This list of problem packages are typically ones that are not named 'properly'. And recently we ran into the issue with the new jdk from Oracle, but also have seen it with libreoffice as well as a few others.

I think we have tracked down the issue to the software_packages.csv cache file that is created by cfengine (/var/cfengine/state/software_packages.csv).
For these problem packages, the entry which is supposed to be something like {name},{version},{arch},{packagemanager?}, it seems to be parsing the entries incorrectly.

For example oracle has built their latest jdk with the following:
name: jdk1.8.0_25
version: 1.8.0_25

and the software_packages.csv is caching the following (spaces added for emphasis):
jdk1,     1.8.0_25-fcs,     8.0_25.x86_64,     rpm

Now, because of the naming scheme Oracle decided to use, we needed to fix the name in the package method to: "jdk1.8.0_25", which is how yum sees it.

And here is where the problem is.
When package promise runs through and tries to compare the requested package name "jdk1.8.0_25" against the software_package.csv list, no match is found because
jdk1 != jdk1.8.0_25, therefore yum tries to install the package which is already installed.


The libreoffice example is worse only because there are multiple packages all called things like:
libreoffice4.3-base-4.3.1.2-2

but software_packages caches the name as just 'libreoffice4' (spaces added for emphasis):
libreoffice4,     4.3.1.2-2,     3-base.x86_64,     rpm


I assume the parsing is trying to grab anything after the '.' and setting that as the arch.

This feels like a bug, but with all of the discussions about nonstandard naming schemes, I don't know if it was addressed earlier or if anyone has suggestions about a workaround.

Thanks,

Dave...

[Neil Watson]

I've given up on packages promises for now. I've returned to user the
package manager from a commands promise. The system load is higher, but
I only promise packages a few times per day. The advantage is not more
version problems. I've included a bundle in EFL that does this:

https://github.com/evolvethinking/evolve_cfengine_freelib/blob/3.5/masterfiles/lib/evolve_freelib.cf#L679-L786

FYI, the 3.5 branch works just fine with 3.6.

--
Neil H Watson
Sr. Partner, Architecture and Infrastructure
CFEngine reporting: https://github.com/evolvethinking/delta_reporting
CFEngine policy: https://github.com/evolvethinking/evolve_cfengine_freelib
CFEngine and vim: https://github.com/neilhwatson/vim_cf3
CFEngine support: http://evolvethinking.com

[Brian Bennett]

Package promises rely heavily on the command used to list installed/available packages and the regex used to match package names and versions.

Over the past 6 months or so there has been a lot of work going into deb, python, pkgsrc, npm and freebsd package methods to use better regular expressions. Based on my observation, rpm hasn't seen a lot of love in this area recently. Third party packages with badly formed names will always be difficult to impossible to support properly.

That being said, we'd greatly appreciate any help in enhancing the rpm package regex patterns.

-- 
Brian Bennett
Looking for CFEngine training?
Seats available for 4-day class beginning Feb 2, 2015
http://www.verticalsysadmin.com/

--
You received this message because you are subscribed to the Google Groups "help-cfengine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to help-cfengin...@googlegroups.com.
To post to this group, send email to help-c...@googlegroups.com.
Visit this group at http://groups.google.com/group/help-cfengine.
For more options, visit https://groups.google.com/d/optout.

[dknapik]

Thank you for your reply. It made me go back and take a closer look at the yum/rpm method we were using.

It looks like the regex we were using was not properly parsing the inputs. I was able to go back into the package.cf library and see what was done for the redhat_knowledge methods. These regex's worked properly for all of the packages I had listed. Once imported into our method, I was able to get the rpm to work and the software_package.csv to update properly.

I still have some work to do to understand the yum method and get it working properly, but at least now I know where to look.

Again, thanks for your reply.

Dave....