Resolving/ Expanding module puppet:/// URI in exec line

[Edd Grant]

Hi All,

I have defined the following module to untar/unzip and copy the Maven
distributable to a convenient location:

class apache-maven-v3 {
exec { "/bin/tar xzf /etc/puppet/modules/apache-maven-v3/files/
apache-maven-3.0.3-bin.tar.gz":
cwd => "/usr/local/java",
creates => "/usr/local/java/apache-maven-3.0.3",
}
...
}

The above definition executes perfectly however in order to keep the
module portable I want to replace the absolute path to the .gz file
with a puppet:/// URI e.g.

exec { "/bin/tar xzf puppet:///modules/apache-maven-v3/apache-maven-3.0.3-bin.tar.gz":

When I change the class to use the puppet:/// URI I get the following
error:

(/Stage[main]/Apache-maven-v3/Exec[/bin
/tar xzf puppet:///modules/apache-maven-v3/apache-maven-3.0.3-bin.tar.gz]/return
s) change from notrun to 0 failed: /bin/tar xzf puppet:///modules/apache-maven-v
3/apache-maven-3.0.3-bin.tar.gz returned 2 instead of one of [0] at /
etc/puppet/
modules/apache-maven-v3/manifests/init.pp:11

It appears to me that the puppet:/// URI is not being resolved in the
exec and this is causing the tar command to operate on the literal
path puppet:///modules/apache-maven-v3/apache-maven-3.0.3-bin.tar.gz
which of course doesn't exist.

Looking at the docs I can't see any examples of puppet:/// being used
in this way, is there anyway I can obtain the resolved absolute path
to pass this in to my exec? Failing that it there a standard approach
for combining a puppet:/// URI with an exec?

Cheers,

Edd

[Nan Liu]

Use a file resource to deploy it to the agent and make the exec depend
on the file resource.

> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To post to this group, send email to puppet...@googlegroups.com.
> To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
>

[Patrick]

This answer works. A move complicated answer for why is that an exec is basically just a command that's run. If tar doesn't know how to use puppet URIs, (which is doesn't) that syntax won't work. You can put the file on the machine using a file resource. Using wget to get the file works too.

[Edd Grant]

Hi Nan,

Thanks for the answer - I'm not 100% clear how I could acheive this,
could you expand on your suggestion a little, perhaps with an example?
Would the file resource point at the .gz file in the module? If so how
would I then reference the file resource in the tar command?

Many thanks,

Edd

[Martin Alfke]

Hi Edd,

here is an example:

class apache-maven-v3 {
# prepare local filesystem
file { 'java_path':
path => "/usr/local/java",
ensure => directory,
}
# copy file from puppet master to local system
file { 'copy_maven_v3':
path => "/usr/local/java/apache-maven-3.0.3-bin.tar.gz",
source =>
"puppet:///modules/apache-maven-v3/apache-maven-3.0.3-bin.tar.gz",
}
# extract local file
exec { 'install_maven_v3':
command => "/bin/tar zxf /usr/local/java/apache-maven-3.0.3-bin.tar.gz",

cwd => "/usr/local/java",
creates => "/usr/local/java/apache-maven-3.0.3",
}

# make sure the order is set properly
File['java_path'] -> File['copy_maven_v3'] -> Exec['install_maven_v3']
}

kind regards,

Martin

[Edd Grant]

many thanks Martin - makes perfect sense!

[Edd Grant]

Hi Martin,

Have tried this out and have noticed that the copied .gz file is left
in /usr/share/java after unpacking. I tried adding another file
resource to delete it but because this points to the same filepath as
the initial resource puppet disallows it:

Example below:

class apache-maven-v3 {

require java-app-base

# copy file from puppet master to local system
file { 'copy_maven_v3':
path => "/usr/local/java/apache-maven-3.0.3-bin.tar.gz",
source => "puppet:///modules/apache-maven-v3/apache-maven-3.0.3-
bin.tar.gz",
}

# extract local file
exec { 'install_maven_v3':
command => "/bin/tar zxf /usr/local/java/apache-maven-3.0.3-
bin.tar.gz",
cwd => "/usr/local/java",
creates => "/usr/local/java/apache-maven-3.0.3",
}

#delete copied archive
# Puppet disallows this...
file { 'delete_copied_archive':
path => "/usr/local/java/apache-maven-3.0.3-bin.tar.gz",
ensure => "absent",

}

# make sure the order is set properly

File['copy_maven_v3'] -> Exec['install_maven_v3'] ->
File['delete_copied_archive']
}

gives the following error:

Could not run Puppet configuration client: Cannot alias
File[copy_maven_v3] to ["/usr/local/java/apache-maven-3.0.3-
bin.tar.gz"]; resource ["File", ["/usr/local/java/apache-maven-3.0.3-
bin.tar.gz"]
] already exists

Is there an elegant puppetesque way of dealing with this? I'm trying
to avoid resorting to exec commands if possible!

Cheers,

Edd

On May 3, 10:45 am, Martin Alfke <tux...@gmail.com> wrote:

[Martin Alfke]

Hi Edd,

puppet insists in unique resources.
Therefore you can not define the file resource for tar archive twice.

Another option would be to fetch the file via exec (wget/curl/scp),
create a flagfile afterwards and remove the archive after extraction.

Additionally you can set the unless parameter on the exec copy resource
to run only if the destination diretory does not exist.

Example:

class apache-maven-v3 {

# fetch from storage
exec { 'copy_maven_v3':
command => "curl http://...../apache-maven-v3-bin.tar.gz -o
/usr/local.... && touch /usr/local/java/copy_finished",
creates => '/usr/local/java/copy_finished',
# run only if extracted path does not exist
unless => "test -d /usr/local/java/apache-maven-v3",
}

#extract
exec { 'install_maven_v3:
...
}

# remove archive
file { 'delete_copied_archive':
path => '/usr/local/java/...',
ensure => absent,
}

}

kind regards,

Martin

[Russell Howe]

> On 05/03/2011 08:37 PM, Edd Grant wrote:
> > Hi Martin,
> >
> > Have tried this out and have noticed that the copied .gz
> file is left in /usr/share/java after unpacking.

Is this so bad? I would probably do something like:

$tarball_dir = "/usr/local/src"
$maven_version = "1.2.3"

file { "$tarball_dir/apache-maven-$maven_version.tar.gz":
source => "puppet:///..."
[..]
}

exec { "extract maven archive" :
command => "/usr/bin/tar xzf $tarball_dir/apache-maven-$maven_version.tar.gz -C /usr/share/java",
require => File["$tarball_dir/apache-maven-$maven_version.tar.gz"]
}

Or somesuch and leave the tarball where it is.

If you want to prune old tarballs you could do something ugly like

exec { "cleanup old maven tarballs":
command => "/bin/find $tarball_dir -name 'apache-maven-*.tar.gz'|/bin/grep -v apache-maven-$maven_version.tar.gz|/bin/xargs rm -f"
}

or do it the puppet way with a load of file { "foo": ensure => absent }

or have the first file be file { ".../apache-maven.tar.gz": source => "puppet:///.../apache-maven-$maven_version.tar.gz" } so that the filename is invariant, but the contents get replaced with whichever version you pick. This method has the advantage that you don't get a buildup of old tarballs on the node.

You could even do

exec { "download and extract mvn" :
command => "/usr/bin/curl http://foo/apache-maven-$maven_version.tar.gz|/usr/bin/tar xz -C /usr/share/java",
creates => "[...]",
}

Many ways to crack this egg, and I'm sure people will suggest others.

--
Russell Howe
rh...@moonfruit.com

[Patrick]

On May 3, 2011, at 11:37 AM, Edd Grant wrote:

> Hi Martin,
>
> Have tried this out and have noticed that the copied .gz file is left
> in /usr/share/java after unpacking. I tried adding another file
> resource to delete it but because this points to the same filepath as
> the initial resource puppet disallows it:

In my experience, having the file stick around can be helpful if you keep them somewhere out of the way. I'd download it to something like /usr/local/tar_packages and leave it there. If that's a problem, you can use /tmp, which should be cleaned occasionally by the OS. Just make sure to be careful of permissions if it's sensitive;

[slafr...@b-e-f.org]

Ed, I am having trouble unzipping any tarball via Puppet.  So I installed your module to see how you might have done it.  It runs, it creates the javapath and copies the file... but I get the same error that I get on my modules... can I please ask how you made unzip work?  I am on Ubuntu 10.4.1, and  2.6.4 (Puppet Enterprise 1.0). 

err: /Stage[main]/Apache-maven-v3/Exec[install_maven_v3]/returns: change from notrun to 0 failed: /bin/tar zxf /usr/local/java/apache-maven-3.0.3-bin.tar.gz returned 2 instead of one of [0] at /etc/puppetlabs/puppet/modules/apache-maven-v3/manifests/init.pp:18

[Matthew Burgess]

On 11 September 2013 23:55, <slafr...@b-e-f.org> wrote:

Ed, I am having trouble unzipping any tarball via Puppet.  So I installed your module to see how you might have done it.  It runs, it creates the javapath and copies the file... but I get the same error that I get on my modules... can I please ask how you made unzip work?  I am on Ubuntu 10.4.1, and  2.6.4 (Puppet Enterprise 1.0). 

err: /Stage[main]/Apache-maven-v3/Exec[install_maven_v3]/returns: change from notrun to 0 failed: /bin/tar zxf /usr/local/java/apache-maven-3.0.3-bin.tar.gz returned 2 instead of one of [0] at /etc/puppetlabs/puppet/modules/apache-maven-v3/manifests/init.pp:18

​What happens if you run '/bin/tar zxf /usr/local/java/apache-maven-

3.0.3-bin.tar.gz' from the command line?

Additionally, you could change your manifest so that it executes 'pwd && /bin/tar zxf /usr/local/java/apache-maven-3.0.3-bin.tar.gz' so that you can see what directory the contents of the file are being extracted to.  My suspicion is that you're either extracting to the wrong directory, or the directory you're extracting to has incorrect permissions set on it or the tar file has incorrect permissions set on it.

Regards,

Matt.

[Sean LaFreniere]

From the command line all works fine, only not from Puppet.  BTW, I am not the only one with this complaint online on your forums and elsewhere, people complain that Tar works, but not UnTar.  -S

--
You received this message because you are subscribed to a topic in the Google Groups "Puppet Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/puppet-users/AlqzFLkTS28/unsubscribe.
To unsubscribe from this group and all its topics, send an email to puppet-users...@googlegroups.com.

To post to this group, send email to puppet...@googlegroups.com.

Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

[jcbollinger]

On Monday, September 16, 2013 10:53:38 AM UTC-5, Sean LaFreniere wrote:

From the command line all works fine, only not from Puppet.  BTW, I am not the only one with this complaint online on your forums and elsewhere, people complain that Tar works, but not UnTar.  -S

I see no reason to think that this arises from a bug in Puppet.  Almost certainly it involves issues with how Puppet is being used.  Here are some of the aspects of the execution environment provided by Puppet Exec resources that could be related to such issues:

• Puppet provides a very sparse environment to commands it executes.  Any other desired environment variables need to be specified in the 'environment' property of the Exec resource or set within the command itself.  Environment variables are not inherited from the environment provided to the agent, nor are the shell startup files read.
• The working directory in which the command starts is not defined unless you provide the 'cwd' parameter to the Exec.
• The 'command', 'unless', and 'onlyif' commands of every exec all have independent environments.  You cannot use any of them to modify the environment variables or working directory provided to others.
  
• If you want an executable search path, you must provide it via the 'path' parameter or among the specified environment variables.  Otherwise, you must give the fully-qualified pathname of the command(s) you want to run.
• By default, the command is run with the credentials and security context of the Puppet process.  Even though the agent normally runs as a privileged user, mandatory access controls (e.g. SELinux) may still deny it permission to perform some actions.
• Privileged users on the local system may have less -- or even no -- privilege on remote systems.  This can be an issue, for example, with remote NFS filesystems mounted on the local machine.
  
• By default the command is passed directly to the system to execute, bypassing the shell.  On POSIX clients, setting provider => 'shell' will cause the command to be executed via '/bin/sh', instead, or you can always Exec the shell directly.
• By default, success of an Exec is judged by whether the command exits with code 0.  Additional or different successful exit codes can be specified via the 'returns' parameter.

All of those are intentional and logical, but some may nevertheless take users by surprise, especially users more used to a script-centric model of system administration.  The Puppet model does not work the same way.


John