Puppet module 'issue' with sysctl provider (augeasproviders)

Stefan Heijmans

unread,

Jun 19, 2014, 12:12:24 PM6/19/14

to puppet...@googlegroups.com

Hello,

I've got 'something strange' with a Puppet module, using augeasproviders (v1.2.0 or master branch of today) on Puppet 3.6.2
Below a stripped version of it with the same results as the full version;

It installs an Oracle preinstall rpm which also updates /etc/sysctl.conf settings and use augeasproviders sysctl resource to watch these settings.

Puppet module with all resources in 1 file
===========================================

init:pp
=======
class oracle_t {
include single
}

install.pp
==========
class oracle_t::single {
package { 'oracle-rdbms-server-11gR2-preinstall':
    ensure => present,
}
sysctl { 'kernel.shmall':
    ensure => present,
    value   => '1073741824',
    require => Package['oracle-rdbms-server-11gR2-preinstall'],
}
sysctl { 'kernel.sem':
    ensure => present,
    value   => '250 32000 100 128',
    require => Package['oracle-rdbms-server-11gR2-preinstall'],
}
}

Before; /etc/sysctl.conf
-rw-r--r--. 1 root root 1327 Jun 19 12:49 /etc/sysctl.conf

kernel.core_uses_pid = 1
kernel.msgmax = 65536
kernel.msgmnb = 65536
kernel.shmall = 4294967296
kernel.shmmax = 68719476736
kernel.sysrq = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.ip_forward = 0
net.ipv4.tcp_syncookies = 1
net.ipv6.conf.all.disable_ipv6 = 1

kernel.sem is *not* available in /etc/sysctl.conf
kernel.shmall is available in /etc/sysctl.conf
kernel.shmall = 4294967296

Puppet run;
Info: Applying configuration version '1403175278'
Notice: /Stage[main]/Oracle_t::Single/Package[oracle-rdbms-server-11gR2-preinstall]/ensure: created
Notice: Finished catalog run in 55.02 seconds

After; /etc/sysctl.conf
-rw-r--r--. 1 root root 2679 Jun 19 12:54 /etc/sysctl.conf

fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.core_uses_pid = 1
kernel.msgmax = 65536
kernel.msgmnb = 65536
kernel.sem = 250 32000 100 128
kernel.shmall = 1073741824
kernel.shmmax = 4398046511104
kernel.shmmni = 4096
kernel.sysrq = 1
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.ip_forward = 0
net.ipv4.ip_local_port_range = 9000 65500
net.ipv4.tcp_syncookies = 1
net.ipv6.conf.all.disable_ipv6 = 1

kernel.sem is available in /etc/sysctl.conf
kernel.sem = 250 32000 100 128
kernel.shmall is available in /etc/sysctl.conf
kernel.shmall = 1073741824

Now I remove the rpm and use this module;

Puppet module with all resources in 2 files
===========================================

init:pp
=======
class oracle_t {
include install, sysctl
}

install.pp
==========
class oracle_t::install {
package { 'oracle-rdbms-server-11gR2-preinstall':
ensure => present,
}
}

sysctl.pp
=========
class oracle_t::sysctl {
sysctl { 'kernel.shmall':
    ensure => present,
    value   => '1073741824',
    require => Package['oracle-rdbms-server-11gR2-preinstall'],
}
sysctl { 'kernel.sem':
    ensure => present,
    value   => '250 32000 100 128',
    require => Package['oracle-rdbms-server-11gR2-preinstall'],
}
}

Before: /etc/sysctl.conf
-rw-r--r--. 1 root root 1327 Jun 19 13:03 /etc/sysctl.conf

kernel.core_uses_pid = 1
kernel.msgmax = 65536
kernel.msgmnb = 65536
kernel.shmall = 4294967296
kernel.shmmax = 68719476736
kernel.sysrq = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.ip_forward = 0
net.ipv4.tcp_syncookies = 1
net.ipv6.conf.all.disable_ipv6 = 1

kernel.sem is *not* available in /etc/sysctl.conf
kernel.shmall is available in /etc/sysctl.conf
kernel.shmall = 4294967296

puppet run;
Info: Applying configuration version '1403175921'
Notice: /Stage[main]/Oracle_t::Install/Package[oracle-rdbms-server-11gR2-preinstall]/ensure: created
Notice: /Stage[main]/Oracle_t::Sysctl/Sysctl[kernel.shmall]/value: changed configuration value from '4294967296' to '1073741824'
Notice: /Stage[main]/Oracle_t::Sysctl/Sysctl[kernel.sem]/ensure: created
Notice: Finished catalog run in 46.29 seconds

After; /etc/sysctl.conf
-rw-r--r--. 1 root root 1358 Jun 19 13:05 /etc/sysctl.conf

kernel.core_uses_pid = 1
kernel.msgmax = 65536
kernel.msgmnb = 65536
kernel.sem = 250 32000 100 128
kernel.shmall = 1073741824
kernel.shmmax = 68719476736
kernel.sysrq = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.ip_forward = 0
net.ipv4.tcp_syncookies = 1
net.ipv6.conf.all.disable_ipv6 = 1

kernel.sem is available in /etc/sysctl.conf
kernel.sem = 250 32000 100 128
kernel.shmall is available in /etc/sysctl.conf
kernel.shmall = 1073741824

So,
with Puppet module and resources in 1 file;
- sysctl resources are not set
- /etc/sysctl.conf updates from the rpm are there

with Puppet module and resources in 2 files;
- sysctl resources are set
- /etc/sysctl.conf updates from rpm are somehow removed

And I have no idea why :(

Thanks,
Stefan

Stefan Heijmans

unread,

Jul 16, 2014, 1:20:57 PM7/16/14

to puppet...@googlegroups.com

anyone?

Felix Frank

unread,

Jul 16, 2014, 2:01:15 PM7/16/14

to puppet...@googlegroups.com

On 06/19/2014 02:12 PM, Stefan Heijmans wrote:
> Puppet module with all resources in 1 file
> ===========================================

> ...

> kernel.sem is available in /etc/sysctl.conf
> kernel.sem = 250 32000 100 128
> kernel.shmall is available in /etc/sysctl.conf
> kernel.shmall = 1073741824
>

> Puppet module with all resources in 2 files
> ===========================================

> ...

> kernel.sem is available in /etc/sysctl.conf
> kernel.sem = 250 32000 100 128
> kernel.shmall is available in /etc/sysctl.conf
> kernel.shmall = 1073741824
>
>
> So,
> with Puppet module and resources in 1 file;
> - sysctl resources are not set
> - /etc/sysctl.conf updates from the rpm are there
>
> with Puppet module and resources in 2 files;
> - sysctl resources are set
> - /etc/sysctl.conf updates from rpm are somehow removed
>

I don't get it. The results you posted are identical for both variants,
yes? Moreover, they seem to reflect your manifest to the letter.

What is the issue?

Thanks,
Felix

Stefan Heijmans

unread,

Jul 16, 2014, 2:19:23 PM7/16/14

to puppet...@googlegroups.com

Hi,

the difference is in the /etc/sysctl.conf;'

Puppet module with all resources in 1 file
===========================================

After; /etc/sysctl.conf
-rw-r--r--. 1 root root 2679 Jun 19 12:54 /etc/sysctl.conf

fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.core_uses_pid = 1
kernel.msgmax = 65536
kernel.msgmnb = 65536

kernel.sem = 250 32000 100 128

kernel.shmall = 1073741824
kernel.shmmax = 4398046511104
kernel.shmmni = 4096
kernel.sysrq = 1
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.ip_forward = 0
net.ipv4.ip_local_port_range = 9000 65500
net.ipv4.tcp_syncookies = 1
net.ipv6.conf.all.disable_ipv6 = 1

Puppet module with all resources in 2 files
===========================================

After; /etc/sysctl.conf
-rw-r--r--. 1 root root 1358 Jun 19 13:05 /etc/sysctl.conf

kernel.core_uses_pid = 1
kernel.msgmax = 65536
kernel.msgmnb = 65536

kernel.sem = 250 32000 100 128

kernel.shmall = 1073741824
kernel.shmmax = 68719476736
kernel.sysrq = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.ip_forward = 0
net.ipv4.tcp_syncookies = 1
net.ipv6.conf.all.disable_ipv6 = 1

Stefan

jcbollinger

unread,

Jul 17, 2014, 4:16:51 PM7/17/14

to puppet...@googlegroups.com

On Thursday, June 19, 2014 7:12:24 AM UTC-5, Stefan Heijmans wrote:

with Puppet module and resources in 1 file;
- sysctl resources are not set

Your data indicate otherwise.

- /etc/sysctl.conf updates from the rpm are there
with Puppet module and resources in 2 files;
- sysctl resources are set
- /etc/sysctl.conf updates from rpm are somehow removed

Are you sure the RPM's updates were ever applied in the first place? All you actually demonstrate is that they aren't there at the end. One way you could test would be to add this resource to one of the relevant manifests:

exec { 'Preserve intermediate sysctl.conf':
command => '/bin/cp /etc/sysctl.conf /tmp/sysctl.conf.intermediate',

require => Package['oracle-rdbms-server-11gR2-preinstall'],

} -> Sysctl<||>

There are as many reasons why the RPM might not apply the sysctl changes as there are ways to write a scriptlet to apply them.

John

Felix Frank

unread,

Jul 17, 2014, 9:16:08 PM7/17/14

to puppet...@googlegroups.com

On 06/19/2014 02:12 PM, Stefan Heijmans wrote:

> class oracle_t {
> include install, sysctl
> }

FWIW, I consider this bad form.

You should qualify your class names, even if you include inside the same
module.

include oracle_t::install
include oracle_t::sysctl

It's not much more to type, and is actually more readable/maintainable
(arguably, I guess).

Without the qualification, 'sysctl' could as well be the inital class in
a sysctl module.

This might even be among the reasons for your problem, although I
wouldn't put any money on those odds.

Regards,
Felix

Stefan Heijmans

unread,

Jul 18, 2014, 10:31:47 AM7/18/14

to puppet...@googlegroups.com

Hi John/Felix,

thanks for the suggestions, will try them out next week and come back with the results

Stefan

Stefan Heijmans

unread,

Jul 25, 2014, 8:23:24 AM7/25/14

to puppet...@googlegroups.com

Hi,

Just did some tests with the suggestions;

class oracle_t {

include oracle_t::install
include oracle_t::sysctl
}

class oracle_t::install {
package { 'oracle-rdbms-server-11gR2-preinstall':
ensure => present,
}
}

class oracle_t::sysctl {
sysctl { 'kernel.shmall':
ensure => present,

value => '1073741820',

require => Package['oracle-rdbms-server-11gR2-preinstall'],
}
sysctl { 'kernel.sem':
ensure => present,

value => '250 32000 100 129',

require => Package['oracle-rdbms-server-11gR2-preinstall'],
}

exec { 'Preserve intermediate sysctl.conf':
command => '/bin/cp -pf /etc/sysctl.conf /tmp/sysctl.conf.intermediate',

require => Package['oracle-rdbms-server-11gR2-preinstall'],

} -> Sysctl<||>
}

Before; /etc/sysctl.conf
-rw-r--r--. 1 root root 1327 Jul 25 10:07 /etc/sysctl.conf

kernel.core_uses_pid = 1
kernel.msgmax = 65536
kernel.msgmnb = 65536
kernel.shmall = 4294967296
kernel.shmmax = 68719476736
kernel.sysrq = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.ip_forward = 0
net.ipv4.tcp_syncookies = 1
net.ipv6.conf.all.disable_ipv6 = 1

Puppet run;
Info: Applying configuration version '1406274607'

Notice: /Stage[main]/Oracle_t::Install/Package[oracle-rdbms-server-11gR2-preinstall]/ensure: created

Notice: /Stage[main]/Oracle_t::Sysctl/Exec[Preserve intermediate sysctl.conf]/returns: executed successfully
Notice: /Stage[main]/Oracle_t::Sysctl/Sysctl[kernel.shmall]/value: changed configuration value from '1073741824' to '1073741820' and live value from '1073741824' to '1073741820'
Notice: /Stage[main]/Oracle_t::Sysctl/Sysctl[kernel.sem]/value: changed configuration value from '250 32000 100 128' to '250 32000 100 129' and live value from '250 32000 100 128' to '250 32000 100 129'
Notice: Finished catalog run in 58.47 seconds

Diff;
diff /etc/sysctl.conf /tmp/sysctl.conf.intermediate
48c48
< kernel.sem = 250 32000 100 129
---

> kernel.sem = 250 32000 100 128

55c55
< kernel.shmall = 1073741820
---
> kernel.shmall = 1073741824

After; /etc/sysctl.conf
-rw-r--r--. 1 root root 2679 Jul 25 10:11 /etc/sysctl.conf

fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.core_uses_pid = 1
kernel.msgmax = 65536
kernel.msgmnb = 65536

kernel.sem = 250 32000 100 129
kernel.shmall = 1073741820

kernel.shmmax = 4398046511104
kernel.shmmni = 4096
kernel.sysrq = 1
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.ip_forward = 0
net.ipv4.ip_local_port_range = 9000 65500
net.ipv4.tcp_syncookies = 1
net.ipv6.conf.all.disable_ipv6 = 1

So 'suddenly' this works by adding the exec resource with dependencies to all sysctl resources.

Removing the exec resource and changing the package resource to the following

package { 'oracle-rdbms-server-11gR2-preinstall':
ensure => present,

}-> Sysctl<||>

also works.

Before; /etc/sysctl.conf
-rw-r--r--. 1 root root 1327 Jul 25 10:11 /etc/sysctl.conf

kernel.core_uses_pid = 1
kernel.msgmax = 65536
kernel.msgmnb = 65536
kernel.shmall = 4294967296
kernel.shmmax = 68719476736
kernel.sysrq = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.ip_forward = 0
net.ipv4.tcp_syncookies = 1
net.ipv6.conf.all.disable_ipv6 = 1

Puppet run;
Info: Applying configuration version '1406274883'

Notice: /Stage[main]/Oracle_t::Install/Package[oracle-rdbms-server-11gR2-preinstall]/ensure: created

Notice: /Stage[main]/Oracle_t::Sysctl/Sysctl[kernel.shmall]/value: changed configuration value from '1073741824' to '1073741820' and live value from '1073741824' to '1073741820'
Notice: /Stage[main]/Oracle_t::Sysctl/Sysctl[kernel.sem]/value: changed configuration value from '250 32000 100 128' to '250 32000 100 129' and live value from '250 32000 100 128' to '250 32000 100 129'
Notice: Finished catalog run in 63.95 seconds

After; /etc/sysctl.conf
-rw-r--r--. 1 root root 2679 Jul 25 10:19 /etc/sysctl.conf

fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.core_uses_pid = 1
kernel.msgmax = 65536
kernel.msgmnb = 65536

kernel.sem = 250 32000 100 129
kernel.shmall = 1073741820

kernel.shmmax = 4398046511104
kernel.shmmni = 4096
kernel.sysrq = 1
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.ip_forward = 0
net.ipv4.ip_local_port_range = 9000 65500
net.ipv4.tcp_syncookies = 1
net.ipv6.conf.all.disable_ipv6 = 1