[PATCH/puppet 1/1] Fixes #2805 - AIX Package management support including basic tests

[James Turnbull]

Code from Andrew Forgue

Signed-off-by: James Turnbull <ja...@lovedthanlost.net>
---
lib/puppet/provider/package/aix.rb | 110 ++++++++++++++++++++++++++++++++++++
lib/puppet/provider/package/nim.rb | 53 +++++++++++++++++
spec/unit/provider/package/aix.rb | 48 ++++++++++++++++
spec/unit/provider/package/nim.rb | 33 +++++++++++
4 files changed, 244 insertions(+), 0 deletions(-)
create mode 100644 lib/puppet/provider/package/aix.rb
create mode 100644 lib/puppet/provider/package/nim.rb
create mode 100755 spec/unit/provider/package/aix.rb
create mode 100755 spec/unit/provider/package/nim.rb

diff --git a/lib/puppet/provider/package/aix.rb b/lib/puppet/provider/package/aix.rb
new file mode 100644
index 0000000..580c6c7
--- /dev/null
+++ b/lib/puppet/provider/package/aix.rb
@@ -0,0 +1,110 @@
+require 'puppet/provider/package'
+require 'puppet/util/package'
+
+Puppet::Type.type(:package).provide :aix, :parent => Puppet::Provider::Package do
+ desc "Installation from AIX Software directory"
+
+ # The commands we are using on an AIX box are installed standard
+ # (except nimclient) nimclient needs the bos.sysmgt.nim.client fileset.
+ commands :lslpp => "/usr/bin/lslpp",
+ :installp => "/usr/sbin/installp",
+ :inutoc => "/usr/sbin/inutoc"
+
+ # AIX supports versionable packages with and without a NIM server
+ has_feature :versionable
+
+ confine :operatingsystem => [ :aix ]
+ defaultfor :operatingsystem => :aix
+
+ def uninstall
+ # Automatically process dependencies when installing/uninstalling
+ # with the -g option to installp.
+ installp "-gu", @resource[:name]
+ end
+
+ def install(useversion = true)
+ unless source = @resource[:source]
+ self.fail "A directory is required which will be used to find packages"
+ end
+
+ pkg = @resource[:name]
+
+ if (! @resource.should(:ensure).is_a? Symbol) and useversion
+ pkg << @resource.should(:ensure)
+ end
+
+ # run inutoc to create a table of contents for the installer.
+ inutoc source
+
+ installp "-acgwXY", "-d", source, pkg
+ end
+
+ def self.pkglist(hash = {})
+ cmd = [command(:lslpp), "-qLc"]
+
+ if name = hash[:pkgname]
+ cmd << name
+ end
+
+ begin
+ list = execute(cmd).split("\n").collect do |set|
+ pkg_info = set.split(":")
+ pkg = {}
+ pkg[:name] = pkg_info[1]
+ pkg[:ensure] = pkg_info[2]
+ pkg[:provider] = self.name
+ pkg
+ end.compact
+ rescue Puppet::ExecutionFailure => detail
+ if hash[:pkgname]
+ return nil
+ else
+ raise Puppet::Error, "Could not list installed Packages: %s" % detail
+ end
+ end
+
+ if hash[:pkgname]
+ return list.shift
+ else
+ return list
+ end
+ end
+
+ def self.instances
+ pkglist.collect do |hash|
+ new(hash)
+ end
+ end
+
+ def latest
+ unless source = @resource[:source]
+ self.fail "A directory is required which will be used to find packages"
+ end
+
+ inutoc source
+ cmd = [ command(:installp), "-L", "-d", source ]
+
+ pkg_latest = "-1"
+ execute(cmd).split("\n").each do |set|
+ pkg_info = set.split(":")
+ if pkg_info[1] == resource[:name]
+ unless Puppet::Util::Package.versioncmp(pkg_latest, pkg_info[2]) == 1
+ pkg_latest = pkg_info[2]
+ end
+ end
+ end
+
+ unless pkg_latest == "-1"
+ return pkg_latest
+ end
+ return nil
+ end
+
+ def query
+ return self.class.pkglist(:pkgname => @resource[:name])
+ end
+
+ def update
+ self.install(false)
+ end
+end
diff --git a/lib/puppet/provider/package/nim.rb b/lib/puppet/provider/package/nim.rb
new file mode 100644
index 0000000..bbeded6
--- /dev/null
+++ b/lib/puppet/provider/package/nim.rb
@@ -0,0 +1,53 @@
+require 'puppet/provider/package'
+require 'puppet/util/package'
+
+Puppet::Type.type(:package).provide :nim, :parent => :aix, :source => :aix do
+ desc "Installation from NIM LPP source"
+
+ # The commands we are using on an AIX box are installed standard
+ # (except nimclient) nimclient needs the bos.sysmgt.nim.client fileset.
+ commands :nimclient => "/usr/sbin/nimclient"
+
+ # If NIM has not been configured, /etc/niminfo will not be present.
+ # However, we have no way of knowing if the NIM server is not configured
+ # properly.
+ confine :exists => "/etc/niminfo"
+
+ def install(useversion = true)
+ unless source = @resource[:source]
+ self.fail "An LPP source location is required in 'source'"
+ end
+
+ pkg = @resource[:name]
+
+ if (! @resource.should(:ensure).is_a? Symbol) and useversion
+ pkg << @resource.should(:ensure)
+ end
+
+ nimclient "-Fo", "reset"
+ nimclient "-o", "cust", "-a", "installp_flags=acgwXY", "-a", "lpp_source=#{source}", "-a", "filesets=#{pkg}"
+ end
+
+ def latest
+ unless source = @resource[:source]
+ self.fail "An LPP source location is required in 'source'"
+ end
+
+ cmd = [ command(:nimclient), "-o", "showres", "-a", "installp_flags=L", "-a", "resource=#{resource[:source]}" ]
+
+ pkg_latest = "-1"
+ execute(cmd).split("\n").each do |set|
+ pkg_info = set.split(":")
+ if pkg_info[1] == resource[:name]
+ unless Puppet::Util::Package.versioncmp(pkg_latest, pkg_info[2]) == 1
+ pkg_latest = pkg_info[2]
+ end
+ end
+ end
+
+ unless pkg_latest == "-1"
+ return pkg_latest
+ end
+ return nil
+ end
+end
diff --git a/spec/unit/provider/package/aix.rb b/spec/unit/provider/package/aix.rb
new file mode 100755
index 0000000..9d5c1f4
--- /dev/null
+++ b/spec/unit/provider/package/aix.rb
@@ -0,0 +1,48 @@
+#!/usr/bin/env ruby
+
+require File.dirname(__FILE__) + '/../../../spec_helper'
+
+provider_class = Puppet::Type.type(:package).provider(:aix)
+
+describe provider_class do
+ before(:each) do
+ # Create a mock resource
+ @resource = stub 'resource'
+
+ # A catch all; no parameters set
+ @resource.stubs(:[]).returns(nil)
+
+ # But set name and source
+ @resource.stubs(:[]).with(:name).returns "mypackage"
+ @resource.stubs(:[]).with(:source).returns "mysource"
+ @resource.stubs(:[]).with(:ensure).returns :installed
+
+ @provider = provider_class.new
+ @provider.stubs(:resource).returns @resource
+ end
+
+ it "should have an install method" do
+ @provider = provider_class.new
+ @provider.should respond_to(:install)
+ end
+
+ it "should have an uninstall method" do
+ @provider = provider_class.new
+ @provider.should respond_to(:uninstall)
+ end
+
+ it "should have an latest method" do
+ @provider = provider_class.new
+ @provider.should respond_to(:latest)
+ end
+
+ it "should have a query method" do
+ @provider = provider_class.new
+ @provider.should respond_to(:query)
+ end
+
+ it "should have an update method" do
+ @provider = provider_class.new
+ @provider.should respond_to(:update)
+ end
+ end
diff --git a/spec/unit/provider/package/nim.rb b/spec/unit/provider/package/nim.rb
new file mode 100755
index 0000000..a3e65b2
--- /dev/null
+++ b/spec/unit/provider/package/nim.rb
@@ -0,0 +1,33 @@
+#!/usr/bin/env ruby
+
+require File.dirname(__FILE__) + '/../../../spec_helper'
+
+provider_class = Puppet::Type.type(:package).provider(:aix)
+
+describe provider_class do
+ before(:each) do
+ # Create a mock resource
+ @resource = stub 'resource'
+
+ # A catch all; no parameters set
+ @resource.stubs(:[]).returns(nil)
+
+ # But set name and source
+ @resource.stubs(:[]).with(:name).returns "mypackage"
+ @resource.stubs(:[]).with(:source).returns "mysource"
+ @resource.stubs(:[]).with(:ensure).returns :installed
+
+ @provider = provider_class.new
+ @provider.stubs(:resource).returns @resource
+ end
+
+ it "should have an install method" do
+ @provider = provider_class.new
+ @provider.should respond_to(:install)
+ end
+
+ it "should have a latest method" do
+ @provider = provider_class.new
+ @provider.should respond_to(:latest)
+ end
+ end
--
1.6.2.5

[Andrew Forgue]

On Nov 19, 5:12 am, James Turnbull <ja...@lovedthanlost.net> wrote:
> +    def latest
> +        unless source = @resource[:source]
> +            self.fail "An LPP source location is required in 'source'"
> +        end
> +
> +        cmd = [ command(:nimclient), "-o", "showres", "-a", "installp_flags=L", "-a", "resource=#{resource[:source]}" ]
> +
> +        pkg_latest = "-1"
> +        execute(cmd).split("\n").each do |set|
> +            pkg_info = set.split(":")
> +            if pkg_info[1] == resource[:name]
> +                unless Puppet::Util::Package.versioncmp(pkg_latest, pkg_info[2]) == 1
> +                    pkg_latest = pkg_info[2]
> +                end
> +            end
> +        end
> +
> +        unless pkg_latest == "-1"
> +            return pkg_latest
> +        end
> +        return nil
> +    end

I'm noticing that the cmd generated (which should be nimclient -o
showres -a installp_flags=L -a lpp_source=resouce[:source]) can take a
lot of time (few seconds per run), but if I want to manage > 10
packages I think this latest() takes too long.

Is there a way to cache the output of the nimclient -o showres...
command within the package provider? I thought about storing it in a
class hash keyed by the value of resource[:source], and expiring that
on each transaction. Is there a better way?

[Luke Kanies]

You should set up prefetching. It basically needs to return a list of
provider instances with the values filled in already; look at the yum
prefetch method to get an example. This avoids any caching internally.

--
I don't want the world, I just want your half.
---------------------------------------------------------------------
Luke Kanies | http://reductivelabs.com | http://madstop.com

[Markus Roberts]

I know close to nothing about AIX, but I have a few observations about the code:

A few questions / concerns here:

1. Why the compact? There won't be any nil entries in the array (though there may be entries with nil :ensure's, which the compact won't catch).
2. Is there an ignored initial field in the results, or should the name be "pkg_info[0]" (and the ensure "pkg_info[1]")?
3. Are there other fields after the ensure that we want to ignore?
4. It would be a little clearer to write something like:

            list = execute(cmd).split("\n").collect do |set|

               if set =~ /(.*):(.*)/
                    {
                    :name => $1,
                   :ensure => $2,
                   :provider => name
                    }
               end
               end.compact



 

+        rescue Puppet::ExecutionFailure => detail
+            if hash[:pkgname]
+                return nil
+            else
+                raise Puppet::Error, "Could not list installed Packages: %s" % detail
+            end
+        end
+
+        if hash[:pkgname]
+            return list.shift
+        else
+            return list
+        end
+    end

So this routine returns nil, an array, or the first element of the array, depending?

Similar comments about the remaining code available on request.

-- Markus

[Andrew Forgue]

Sure thing. I am not a ruby programmer, so these things are mostly
copied and modified from other package providers. That being said,

>    1. Why the compact? There won't be any nil entries in the array (though

>    there may be entries with nil :ensure's, which the compact won't catch).

You're correct, there will never be any nils the command output.

>    2. Is there an ignored initial field in the results, or should the name

>    be "pkg_info[0]" (and the ensure "pkg_info[1]")?

Here's some example output for the gnu.gcc.rte:
root@aixnim:scripts # lslpp -Lc gnu.gcc.rte
#Package Name:Fileset:Level:State:PTF Id:Fix
State:Type:Description:Destination Dir.:Uninstaller:Message
Catalog:Message Set:Message Number:Parent:Automatic:EFIX
Locked:Install Path:Build Date
gnu.gcc:gnu.gcc.rte:4.3.1.0: : :C: :GNU GCC, the gnu compiler
collection runtime: : : : : : :0:0:/:

The first column is the name of the "package", but a package can
contain multiple "fiilesets". I ignore this first field because you
cant install a package, only filesets (this is with the -w option on
installp), otherwise it'll wildcard the requested package to install
all filesets that match, which is probably *not* what you want).

>    3. Are there other fields after the ensure that we want to ignore?

Most of the other fields are useless to puppet, or at least I don't
see any way or usefulness of using more of the fields.

>    4. It would be a little clearer to write something like:

>
>             list = execute(cmd).split("\n").collect do |set|
>                if set =~ /(.*):(.*)/
>                     {
>                     :name => $1,
>                    :ensure => $2,

>                    :provider => name <http://self.name/>
>                     }
>                end
>                end.compact

I agree, I like this idea better.

> > +        rescue Puppet::ExecutionFailure => detail
> > +            if hash[:pkgname]
> > +                return nil
> > +            else
> > +                raise Puppet::Error, "Could not list installed Packages:
> > %s" % detail
> > +            end
> > +        end
> > +
> > +        if hash[:pkgname]
> > +            return list.shift
> > +        else
> > +            return list
> > +        end
> > +    end
>
> So this routine returns nil, an array, or the first element of the array,
> depending?

Yes

Returns nil if a specific package was requested, and not installed,
Returns the first element of the hash if a specific package was
requested (since the command will only return one line)
Returns a list if no specific package was requested (doesn't raise the
exception because the command will return non-zero if a package isn't
installed)
Raises the exception if no specific package was requested and the
command fails.

> Similar comments about the remaining code available on request.

Sure, I'd love to hear it. Like I said I'm just learning ruby and
count on code review to better understand what's going on.

> -- Markus

[Markus Roberts]

So based on your example (and assuming that there was word wrapping, that we want to ignore the header row, and that the header row always starts with a "#"):

    list = execute(cmd).
        scan(/^[^#][^:]*:([^:]*):([^:]*)/).
        collect { |n,e| {:name => n,:ensure => e,:provider => name}}

Thoughts?

-- Markus

[Andrew Forgue]

On Nov 24, 4:42 pm, Luke Kanies <l...@madstop.com> wrote:
> On Nov 20, 2009, at 11:56 AM, Andrew Forgue wrote:
>
> > I'm noticing that the cmd generated (which should be nimclient -o
> > showres -a installp_flags=L -a lpp_source=resouce[:source]) can take a
> > lot of time (few seconds per run), but if I want to manage > 10
> > packages I think this latest() takes too long.
>
> > Is there a way to cache the output of the nimclient -o showres...
> > command within the package provider?  I thought about storing it in a
> > class hash keyed by the value of resource[:source], and expiring that
> > on each transaction.  Is there a better way?
>
> You should set up prefetching.  It basically needs to return a list of  
> provider instances with the values filled in already; look at the yum  
> prefetch method to get an example.  This avoids any caching internally.
>

I do prefetch all the currently installed packages, but I don't see a
way to prefetch the available packages. Is that possible?

[Luke Kanies]

I assume you mean prefetching the list of updates available to
packages you're managing.

In that case, you first the list of all installed packages, then run
the command that gets all available updates, and then you merge that
data into the first list and return the result. I do think that the
yum provider does exactly this, relying on prefetching from rpm.

--
A nation is a society united by delusions about its ancestry and by
common hatred of its neighbors. -- William Ralph Inge

[Andrew Forgue]

I like it. Seems much cleaner to me, I don't like array subscripting
anyway.

A dumb question, should I take this patch, put it in my repository and
then make changes and submit another patch request? The development
lifecycle page doesn't really explain the part where your patch isn't
perfect. I forked the repository on github and branched a few bugs
I'm working on.

[Markus Roberts]

> A dumb question, should I take this patch, put it in my repository
> and then make changes and submit another patch request?  The
> development lifecycle page doesn't really explain the part where
> your patch isn't perfect.  I forked the repository on github and
> branched a few bugs I'm working on.

Not a dumb question at all.  I was at the same point myself a few months ago.

Basic answer:

* checkout your branch,
* make your changes
* commit them with a trivial commit message (typically squash)
* combine the commits (with "git rebase -i xxxx" where "xxxx" is
  the name of the branch you started from (probably "0.25.x" or
  "master"), then change "pick" to "squash" for the changes)
* Proceed as usual (edit the commit message if needed, rake
  mail_patches, etc.)
* You can push it back to the same branch in your repo.

-- Markus

[JoeZ_aix]

Just getting started here to ADD our AIX lpars as Puppet targets
(Clients) - can anyone point out AIX specific documentation and
related puppet downloads????
THANKs - Joe

On Nov 24, 7:11 pm, Andrew Forgue <andrew.for...@gmail.com> wrote:
> On Nov 24, 6:39 pm, Markus Roberts <mar...@reductivelabs.com> wrote:
>
>
>

> > I know close to nothing aboutAIX, but I have a few observations about the

> > code:
>
> > diff --git a/lib/puppet/provider/package/aix.rb
>
> > > b/lib/puppet/provider/package/aix.rb
> > > new file mode 100644
> > > index 0000000..580c6c7
> > > --- /dev/null
> > > +++ b/lib/puppet/provider/package/aix.rb
> > > @@ -0,0 +1,110 @@
> > > +require 'puppet/provider/package'
> > > +require 'puppet/util/package'
> > > +
> > > +Puppet::Type.type(:package).provide :aix, :parent =>
> > > Puppet::Provider::Package do

> > > +    desc "Installation fromAIXSoftware directory"
> > > +
> > > +    # The commands we are using on anAIXbox are installed standard

> > > +    # (except nimclient) nimclient needs the bos.sysmgt.nim.client
> > > fileset.
> > > +    commands    :lslpp => "/usr/bin/lslpp",
> > > +                :installp => "/usr/sbin/installp",
> > > +                :inutoc => "/usr/sbin/inutoc"
> > > +

> > > +    #AIXsupports versionable packages with and without a NIM server