Recommended way to install custom packages, and what does the Service type's manifest attribute do?

[olizilla]

I am writing a module to configure a Nexus maven repository manager service on a centos5 host.

As far as I'm aware, Nexus isn't available as an rpm, so there is a manual install step. 

I'm new to puppet so I tried getting it to copy a shell script over to the host to have it:

- Extract the tar.gz

- symlink the extracted foldername to nexus

- symlink nexus/blah/init.d/nexus /etc/rc.d/init.d/nexus

- add nexus to check config

- enable nexus at boot

- start nexus via service.

The script works fine when run manually, but obviously puppet runs as the puppet user and getting puppet to exec it dissolved into permissions issues.

So I was wondering, what is the recommend puppet way of handling installs of non packaged programs? 

I have similar problem with installing maven on the same box, but I imlemented it as a sequence of Exec steps, which causes much bloating of the puppet config...

And what does the manifest attribute on a Service type do? Is symlinking the init script necessary or can I just specify the init script in the manifest attribute and have puppet work it's magic?

Thanks,

Oli

[jcbollinger]

On Mar 2, 4:41 am, olizilla <oli.ev...@gmail.com> wrote:
> I am writing a module to configure a Nexus maven repository manager service
> on a centos5 host.
>
> As far as I'm aware, Nexus isn't available as an rpm, so there is a manual
> install step.

That no one else provides an RPM doesn't mean *you* cannot prepare
your own. If this is all that is necessary for a manual install:

> I'm new to puppet so I tried getting it to copy a shell script over to the
> host to have it:
> - Extract the tar.gz
> - symlink the extracted foldername to nexus
> - symlink nexus/blah/init.d/nexus /etc/rc.d/init.d/nexus
> - add nexus to check config
> - enable nexus at boot
> - start nexus via service.

then wrapping it in an RPM should be pretty easy. Then not only is
installing it simple, but you get all the management advantages
provided by the RPM system. Personally, I install software on my RPM-
based systems ONLY via RPMs, some of which I prepare myself for the
purpose.

I would omit the last step from the RPM, though, and maybe the last
two. Puppet can and should manage those aspects of the service for
you, and I think it highly inappropriate for RPMs to start services at
installation time.

> The script works fine when run manually, but obviously puppet runs as the
> puppet user and getting puppet to exec it dissolved into permissions issues.

However you approach the issue, something is dreadfully wrong here.
The Puppet agent needs to run with root privilege to do anything
useful (as you have discovered). Normally it runs as root. The
puppet *master* runs as a less-privileged user, typically named
"puppet", but it should not be running your script. Before you do
anything else, you should make sure that the agent (i.e. puppetd) is
running with appropriate privileges.

> So I was wondering, what is the recommend puppet way of handling installs of
> non packaged programs?

My recommendation is to build your own package. I think several
others around here would agree. It is very much worth the effort. I
would also recommend that you put that RPM in your own yum repository
(they are easy to create), rather than trying to install directly from
an RPM file.

> I have similar problem with installing maven on the same box, but I
> imlemented it as a sequence of Exec steps, which causes much bloating of the
> puppet config...

If you find that you can perform the work as a series of Execs, but
not by copying over and running a script of the same steps, then the
problem does not arise from the user that Puppet runs as. It might,
however, arise from one of these sources:

1) You attempt to execute the script as a program, but you did not
assign it execute permission via the File resource you use to copy
it. The File resource should probably specify mode => 700 (or 750, or
even 755).
2) You do not give the correct path to the script.
3) The script relies on environment variables that are absent or have
different values when Puppet runs the script.
4) The client is running SELinux in enforcing mode, and the script's
SELinux attributes do not allow the Puppet agent to run it. The File
resource's sel* properties can help with setting that up correctly.

> And what does the manifest attribute on a Service type do? Is symlinking the
> init script necessary or can I just specify the init script in the manifest
> attribute and have puppet work it's magic?

I am pretty sure that the 'manifest' property is unrelated to
initscripts. In fact, I am confident that the Service resource will
fail utterly if the relevant initscript (or equivalent) is not already
present when the resource is applied. *Possibly* the command or
Puppet manifest named in the 'manifest' property could create the
initscript when necessary, but yuck! I think the 'manifest' property
is about performing additional client configuration prior to managing
the service itself (e.g. managing a file in /etc/sysconfig).

You should not need to create initscript symlinks. That is what
Service's 'enabled' property is about. You should, however, look at
http://projects.puppetlabs.com/issues/2712 for information about some
of Puppet's current limitations in that area.

In fact, Puppet does not know or care about initscripts or runlevel
symlinks specifically. It relies on the client-side tools to know
what to do with the service name you specify. On CentOS 5, that means
Puppet is going to be passing the service name to /sbin/chkconfig and /
sbin/service. Those tools expect an initscript having the given name
to be present in /etc[/rc.d]/init.d.

So, if you follow all of my advice, the relevant Puppet code might
look something like this:

# /etc/puppet/modules/my_module1/manifests/repositories.pp:
class my_module1::repositories {
yumrepo { "my_packages":
baseurl => "http://my_repo_server.my.com/repos/my_repo",
descr => "my_packages",
enabled => 1,
gpgcheck => 0; # 1 if you sign your packages
}
}

# /etc/puppet/modules/nexus/manifests/maven.pp:
class nexus::maven {
include "my_module1::repositories"

package { "nexus-maven":
ensure => latest,
require => Yumrepo["my_packages"]
}

service { "maven":
enable => true,
hasstatus => true, # if the initscript supports 'status'
hasrestart => true, # if the initscript supports 'restart'
ensure => running,
require => Package["nexus-maven"]
}
}

####

The node(s) that should have Maven installed and running then "include
'nexus::maven'". Of course, all the module, class, package, and
service names are examples.

[olizilla]

You sir, are a gent and a scholar! 

I had not even considered creating my own RPM's, assuming that such a thing was beyond the ken of a mere mortal. I shall be reading up on them tonight.

Thanks for bringing the yumrepo type to my attention, I have been adding repos with yet more Exec's which I guess is symptomatic of being new around these puppets. Out of interest, I can't see an apt equivalent in the type reference docs, is that the case?

Your diagnosis of the script issue was correct. It had nothing to do with the puppet user. It was an SELinux. Kicking it into permissive mode fixed it. I plan to get my head around RPM packaging and repositories anyway, as it sounds like the best route. 

I'm totally clear about what the manifest attribute is supposed to be used for, but it seems I don't need it.

Thank you kindly for taking the time to provide such a clear answer.

Oli

[jcbollinger]

On Mar 2, 4:04 pm, olizilla <oli.ev...@gmail.com> wrote:
> You sir, are a gent and a scholar!

At least one of the above. :-)

> I had not even considered creating my own RPM's, assuming that such a thing
> was beyond the ken of a mere mortal. I shall be reading up on them tonight.

There is a learning curve assocated with RPM building, just like with
anything else, but once you get the hang of it, it's not that hard.
There is a fair amount of documentation arond the web. Among the
better are IBM's 3-part article (start at http://www.ibm.com/developerworks/library/l-rpm1/)
and Fedora's draft manual (http://docs.fedoraproject.org/en-US/
Fedora_Draft_Documentation/0.1/RPM_Guide/index.html). Don't overlook
http://www.rpm.org/, but frankly, I don't find it as useful as some of
the others.

> Thanks for bringing the yumrepo type to my attention, I have been adding
> repos with yet more Exec's which I guess is symptomatic of being new around
> these puppets. Out of interest, I can't see an apt equivalent in the type
> reference docs, is that the case?

There is no apt analog of Yumrepo included in the standard
distribution. I would not be surprised if there were one available
from a third party, however. Check Puppetforge.

> Your diagnosis of the script issue was correct. It had nothing to do with
> the puppet user. It was an SELinux. Kicking it into permissive mode fixed
> it.

Score!

> I plan to get my head around RPM packaging and repositories anyway, as
> it sounds like the best route.

I certainly think so, and not just in a Puppet context. Without RPM
it's not the installs that kill you, though having an RPM does make
multiple installs much easier and more consistent. No, it's the
uninstalls, the upgrades, the dependency management, the software
inventory, .... It's just icing on the cake that RPMs are easy for
software such as Puppet do deal with.

> I'm totally clear about what the manifest attribute is supposed to be used
> for, but it seems I don't need it.
>
> Thank you kindly for taking the time to provide such a clear answer.

You're quite welcome.

Cheers,

John

[Dieter De Meyer]

Hi,

it just so happens that I'm attempting to do the same thing, automating the install of a Nexus repository.

I'm quite new to building RPMs, so I was wondering if you would be willing to share your RPM build script and puppet manifests ?

I'm currently writing a puppet module that performs the steps like untar, symlink, creating a service etc..  but an RPM would be much better.

Thanks in advance.

Regards,

Dieter.