I can't seem to create mountpoint and change permissions after mounting in 2.7.3

[rvlinden]

Hi,

I'm using puppet 2.7.3 on RHEL/CentOS and I have an issue which is now
a big blocking issue within my environment. What I'm trying to
accomplish wit puppet is a create a mountpoint, mount a filesystem on
it and install an application on that filesystems are set proper user/
group and permissions on it, but it fails big time.

The issues I currently run into are about 3 things

1. The file type does not allow multiple directories to be created at
once ('mkdir -p')
2. exec & file type create autorequire dependensies which creates
dependency cycles
3. Puppet does not allow duplicate resouces

I have a define named "lvm::createfs" and what it does is three things

1. Create the mountpoint with an exec {} which used an mkdir -p to
create multiple directories deep at once
2. Create the logical volume with an exec {}
3. Mount the logical volume from step 2 onto the directory from step 1
via Mount {}

Before the filesystem is mounted, the underlaying directory MUST be
owned by root:root with permissions 755. If this is incorrect, some
linux command which do strange things (like the 'rm -R' command)

Once the filesystem is created and mounted, I use puppet to install
applications

After the applications are installed I need to set the application
files in the filesystem to it's proper owner:group and permissions.

To do this I have another define named "sysconfig::permissions". All
this define does is use file {} to set owner, group, mode and recurse.

Before I moved to puppet 2.7.3, puppet allowed me to do this without
problems, but since 2.7.3 puppet is more strict and now 60% of al my
modules fail as I use filesystems, apps and permissions everywhere.

Based on the documentation I read about exec {}, I can now explain why
I get dep.cycles, but I'mout of options and have no ideas anymore how
to get this 'simple normal unix task' to work in puppet.

These are the relations I created myself within puppet

1. exec mkdir /a/b/c => mount /a/b/c => file /a/b/c
2. exec logical volume => mount /a/b/c

This setup does not result in a cycle, but when puppet adds an
autorequire between exec mkdir /a/b/c and file a/b/c, it causes the
whole thing to cycle.

Quote from the manual

Autorequires: If Puppet is managing an exec’s cwd or the executable
file used in an exec’s command, the exec resource will autorequire
those files. If Puppet is managing the user that an exec should run
as, the exec resource will autorequire that user.

I don't mind if I have to rewrite my code or start from scratch, but I
need help to create my mountpoint location with root:root/755 and
after mounting allow the same location to have a different owner:group/
permissions.

These are the defines I use at this moment

define lvm::createfs (
$mountpath = undef,
$mountpoint = undef,
$lvsize = undef,
$fstype = undef,
$vgname = undef,
$lvname = "${name}",
$requisite = '' ) {

# Load defaults

require lvm::params

# Check mountpath equals root

$rootpath = $mountpath ? {
'/' => undef,
default => $mountpath,
}

# Create directory tree including subdirectories

exec { "${lvm::params::module_label}_mkdir_${mountpath}/$
{mountpoint}":
command => "mkdir -p ${mountpath}/${mountpoint}",
onlyif => "test ! -d ${mountpath}/${mountpoint}",
before => Mount["${rootpath}/${mountpoint}"],
}

if $requisite != '' {
Exec["${lvm::params::module_label}_mkdir_${mountpath}/$
{mountpoint}"] {
require => Mount["${requisite}"],
}
}

# Create logical volume and format filesystem

exec { "${lvm::params::module_label}-${vgname}-${lvname}":
logoutput => false,
command => "lvcreate -n ${lvname} -L ${lvsize} /dev/${vgname} &&
mkfs -t ${fstype} /dev/${vgname}/${lvname}",
unless => "lvs | grep -q '${lvname} .*${vgname}'",
before => Mount["${rootpath}/${mountpoint}"],
}

# Mount filesystem

mount { "${rootpath}/${mountpoint}":
atboot => true,
device => "/dev/${vgname}/${lvname}",
ensure => mounted,
fstype => "${fstype}",
options => 'defaults',
dump => '1',
pass => '2',
}

if $requisite != '' {
Mount["${rootpath}/${mountpoint}"] {
require => Mount["${requisite}"],
}
}

} # End define


define sysconfig::permissions (
$sysconfig_module,
$sysconfig_name,
$sysconfig_recurse = 'false',
$sysconfig_owner,
$sysconfig_group,
$sysconfig_mode = undef ) {

# Load defaults

require sysconfig::params

# Set permissions

file { "${sysconfig_module}_${sysconfig_name}":
name => "${sysconfig_name}",
recurse => "${sysconfig_recurse}",
owner => "${sysconfig_owner}",
group => "${sysconfig_group}",
}

if $sysconfig_mode != undef {
File["${sysconfig_module}_${sysconfig_name}"] {
mode => "${sysconfig_mode}",
}
}

} # End define


I call these define from a application class. In the example below a
piece of the MQ class.

class mq {

# Create filesystems

lvm::createfs {
"${mq::params::module_label}_opt_mqm":
mountpath => '/opt',
mountpoint => 'mqm',
lvname => 'opt_mqm',
lvsize => '1G',
fstype => 'ext3',
vgname => "${mq::params::vgname_opt}";
"${mq::params::module_label}_var_mqm":
mountpath => '/var',
mountpoint => 'mqm',
lvname => 'var_mqm',
lvsize => '2G',
fstype => 'ext3',
vgname => "${mq::params::vgname_var}";
"${mq::params::module_label}_var_mqm_log":
mountpath => '/var/mqm',
mountpoint => 'log',
lvname => 'var_mqm_log',
lvsize => '4G',
fstype => 'ext3',
vgname => "${mq::params::vgname_var}",
require => [
Lvm::Createfs["${mq::params::module_label}_var_mqm"],
];
}

# Overrule ownership and permissions

sysconfig::permissions {
"${mq::params::module_label}_permissions_$
{mq::params::base_directory}":
sysconfig_module => "${mq::params::module_label}",
sysconfig_name => "${mq::params::base_directory}",
sysconfig_owner => "${mq::params::user_name}",
sysconfig_group => "${mq::params::group_name}",
sysconfig_mode => "${mq::params::base_permissions}",
require => [
# Mount["${mq::params::base_directory}"], # creates dep.cycle
User["${mq::params::user_name}"],
];
"${mq::params::module_label}_permissions_$
{mq::params::var_directory}":
sysconfig_module => "${mq::params::module_label}",
sysconfig_name => "${mq::params::var_directory}",
sysconfig_owner => "${mq::params::user_name}",
sysconfig_group => "${mq::params::group_name}",
sysconfig_mode => "${mq::params::var_permissions}",
require => [
# Mount["${mq::params::var_directory}"], # creates dep.cycle
# Mount["${mq::params::log_directory}"], # creates dep.cycle
User["${mq::params::user_name}"],
];
}

}

[Denmat]

Hi,

I don't know the answer to you're problem but have you thought of adding the required perms at the point you call the mkdir -p?

In your exec..
=> 'mkdir -p ... && chmod 755 .. && chown root...',

This way you are not managing the resource perms twice in puppet. That may ease some of your problems.

Cheers,
Den

> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To post to this group, send email to puppet...@googlegroups.com.
> To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
>

[jcbollinger]

On Sep 28, 5:02 pm, rvlinden <rene.vanderlinde...@gmail.com> wrote:
> Hi,
>
> I'm using puppet 2.7.3 on RHEL/CentOS and I have an issue which is now
> a big blocking issue within my environment. What I'm trying to
> accomplish wit puppet is a create a mountpoint, mount a filesystem on
> it and install an application on that filesystems are set proper user/
> group and permissions on it, but it fails big time.
>
> The issues I currently run into are about 3 things
>
> 1. The file type does not allow multiple directories to be created at
> once ('mkdir -p')

Surely that's a convenience issue, not a blocker. In any event, I am
confident that if all the parent directories are intended to have the
same properties then it should be possible to use a recursive defined
type to ensure all are present.

> 2. exec & file type create autorequire dependensies which creates
> dependency cycles

See below.

> 3. Puppet does not allow duplicate resouces

Indeed not. How would that make sense? It does provide virtual
resources, which can address some of the scenarios where people
otherwise would want to duplicate resource declarations. You declare
the resource once, virtually, for all nodes that could possibly need
it, then you realize that resource every place where otherwise you
would declare it.

After reading the rest of your post, however, I suspect the problem is
not really duplicate resources. My guess is that your complaint is
really that Puppet cannot manage both a mount point and the root of
the filesystem mounted on it. These are distinct resources, but the
underlying operating system design makes it difficult to distinguish
them, and Puppet cannot do so. Indeed, the mount point's properties
are invisible -- and thus unmanageable -- when a file system is
mounted on it, so I attribute this issue to the OS, not to Puppet.

[...]

> These are the relations I created myself within puppet
>
> 1. exec mkdir /a/b/c => mount /a/b/c => file /a/b/c
> 2. exec logical volume => mount /a/b/c
>
> This setup does not result in a cycle, but when puppet adds an
> autorequire between exec mkdir /a/b/c and file a/b/c, it causes the
> whole thing to cycle.
>
> Quote from the manual
>
> Autorequires: If Puppet is managing an exec’s cwd or the executable
> file used in an exec’s command, the exec resource will autorequire
> those files. If Puppet is managing the user that an exec should run
> as, the exec resource will autorequire that user.

Right, but how does that explain the behavior you see? Surely /a/b/c
is not the working directory or executable of your Exec. I would
guess that it is instead Mount autorequiring its mount point that is
causing the cycle. That's not documented (which constitutes a
documentation bug if Puppet does this), but it seems sort of logical.
Also, there is a feature request asking for this (http://
projects.puppetlabs.com/issues/5015), though it is marked "Code
Insufficient". If you can verify that behavior, however, then I would
file a bug ticket on it, as at minimum there is a documentation bug.
Alternatively, if you can confirm that the Exec is the source of the
autorequire then I would definitely file a ticket for that, as it
would unquestionably be buggy.

> I don't mind if I have to rewrite my code or start from scratch, but I
> need help to create my mountpoint location with root:root/755 and
> after mounting allow the same location to have a different owner:group/
> permissions.

If autorequirements are indeed the root of the problem then the
easiest thing to do might be to declare explicit relationships in the
correct direction. Autorequirements should always be overridden by
explicitly declared ones. If you find that to not be the case then I
urge you to file a bug ticket about it.


John

[rvlinden]

John,

At this moment I'm trying to reproduce the problem by starting small
and add more modules to it.
I have been able to succesfully create a puppet config that creates a
mountpoint directory with 'mkdir -p' as root,
mount a filesystem on top of that directory and change the ownership
once is has been mounted to another user and permissions.

When I debugged puppet, it reported nothing about autorequire and I
only saw the requires I manaully added.

After that I also added a user module which has another file
definition for the user' homedirectory.
Even with a user' homedir set to the mountpoint, still no issues.

For now I can not reproduce the autorequire I saw before where the
Exec["mkdir_/a/b/c"] was linked to File["/a/b/c"]

Rene

[Chris McDermott]

I haven't used them yet, but isn't this what stages are meant to facilitate?

http://docs.puppetlabs.com/references/stable/metaparameter.html#stage

[Scott Smith]

Not really

On Wed, Oct 5, 2011 at 8:54 PM, Chris McDermott <csmcd...@gmail.com> wrote:

I haven't used them yet, but isn't this what stages are meant to facilitate?

http://docs.puppetlabs.com/references/stable/metaparameter.html#stage

--

[jcbollinger]

On Oct 5, 10:54 pm, Chris McDermott <csmcderm...@gmail.com> wrote:
> I haven't used them yet, but isn't this what stages are meant to facilitate?
>
> http://docs.puppetlabs.com/references/stable/metaparameter.html#stage

Run stages are syntactic sugar. There is nothing you can do with them
that you cannot also do with explicit class and resource
relationships.

A little bit of candy is good for the psyche, and their sugary nature
does not predispose me against run stages. They are nice as an
organizational mechanism, and somewhat for self-documentation. On the
other hand, I don't care for parameterized classes at all, and you
cannot use run stages without them.

In this case, run stages do nothing whatever to address the problem
that you cannot manage properties of a mount point while a filesystem
is mounted on it. You still cannot see the underlying directory.
Moreover, Puppet's prohibition against duplicate resources extends
across all run stages.


John