does puppet+augeas actually work? limits.conf busticated

[Gajillion]

All,
I've tried several lenses and several iterations and have never been
able to get augeas and puppet to work together. My latest attempt:

augeas {"memlock":
changes => [
"ins domain after /files/etc/security/limits.conf/domain
[last()]",
"set /files/etc/security/limits.conf/domain[last()]/ *",
"set /files/etc/security/limits.conf/domain[last()]/type
hard",
"set /files/etc/security/limits.conf/domain[last()]/item
memlock",
"set /files/etc/security/limits.conf/domain[last()]/value
unlimited",
]
}

Fails: wombat::setup/Augeas[memlock]/returns: change from need_to_run
to 0 failed: Save failed with return code false

However, stepping through those exact steps cut and paste into augtool
works:

augtool> ins domain after /files/etc/security/limits.conf/domain[last
()]
augtool> set /files/etc/security/limits.conf/domain[last()]/ *
augtool> set /files/etc/security/limits.conf/domain[last()]/type hard
augtool> set /files/etc/security/limits.conf/domain[last()]/item
memlock
augtool> set /files/etc/security/limits.conf/domain[last()]/value
unlimited
augtool> save
Saved 1 file(s)
augtool>
# tail -1 /etc/security/limits.conf
* hard memlock unlimited

I can't see anything even remotely obviously wrong in this. Any
ideas? I've tried several iterations using various contexts and get
the same thing every time.

Full failure code from puppet

debug: Augeas[memlock](provider=augeas): Opening augeas with root /,
lens path , flags 0
debug: Augeas[memlock](provider=augeas): Augeas version 0.5.0 is
installed
debug: Augeas[memlock](provider=augeas): sending command 'ins' with
params ["domain", "after", "/files/etc/security/limits.conf/domain[last
()]"]
debug: Augeas[memlock](provider=augeas): sending command 'set' with
params ["/files/etc/security/limits.conf/domain[last()]/", "*"]
debug: Augeas[memlock](provider=augeas): sending command 'set' with
params ["/files/etc/security/limits.conf/domain[last()]/type", "hard"]
debug: Augeas[memlock](provider=augeas): sending command 'set' with
params ["/files/etc/security/limits.conf/domain[last()]/item",
"memlock"]
debug: Augeas[memlock](provider=augeas): sending command 'set' with
params ["/files/etc/security/limits.conf/domain[last()]/value",
"unlimited"]
err: //Node[...]/wombat::setup/Augeas[memlock]/returns: change from
need_to_run to 0 failed: Save failed with return code false

[Rob McBroom]

On 2009-Jun-3, at 3:56 PM, Gajillion wrote:

> Fails: wombat::setup/Augeas[memlock]/returns: change from need_to_run
> to 0 failed: Save failed with return code fals

This is the same error I just said I wasn't able to reproduce in
another thread. What OS is this? Try installing Augeas itself (not
just the libs and ruby bindings) and see if the error goes away.

--
Rob McBroom
<http://www.skurfer.com/>

Don't try to tell me a thing is important to you if the whole of your
"support" entails forcing other to spend time and money on it.

[Gajillion]

I have the complete augeas install as well as the ruby libs. That's
how I was able to test that the paths and inserts are valid Augeas
commands. The RPMs were pulled from EPEL 4. Just to be clear - it
works fine with just Augeas, but fails under Puppet driven Augeas.

[root@buildtest tests]# cat /etc/redhat-release
Red Hat Enterprise Linux AS release 4 (Nahant Update 6)
[root@buildtest tests]# rpm -qa --queryformat "%{NAME}-%{VERSION}-%
{RELEASE}.%{ARCH} \\n" | grep aug
ruby-augeas-0.2.0-3.el4.x86_64
augeas-libs-0.5.0-2.el4.x86_64
augeas-0.5.0-2.el4.x86_64
[root@buildtest tests]#

[Gajillion]

So, a quick question then. I ran a very verbose strace on puppet
while it was going through this change. What I see is this.

Augeas libs successfully opens and reads in the target file:

26971 access("/etc/security/limits.conf", R_OK) = 0
26971 open("/etc/security/limits.conf", O_RDONLY) = 6
26971 fstat(6, {st_mode=S_IFREG|0640, st_size=2912, ...}) = 0
26971 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|
MAP_ANONYMOUS, -1, 0) = 0x2a97621000
26971 read(6, "# $Id: $\n#\n# DEV & QA User Resource Limits\n#\n#Each
line describes a limit for a user in the for
... 8192) = 2912

...

It then opens up a backup file I'm assuming it will write its changes
to:

26971 read(6, "", 4096) = 0
26971 close(6) = 0
26971 munmap(0x2a97621000, 4096) = 0
26971 open("/etc/security/limits.conf.augnew", O_WRONLY|O_CREAT|
O_TRUNC, 0666) = 6

...

It then writes the contents of the old limits.conf to that file (the
file length of 2912 is the same in the read above and the write):

26971 lstat("/etc/security/limits.conf", {st_mode=S_IFREG|0640,
st_size=2912, ...}) = 0
26971 lchown("/etc/security/limits.conf.augnew", 0, 4) = 0
26971 chmod("/etc/security/limits.conf.augnew", 0100640) = 0
26971 fstat(6, {st_mode=S_IFREG|0640, st_size=0, ...}) = 0
26971 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|
MAP_ANONYMOUS, -1, 0) = 0x2a97621000
26971 write(6, "# $Id: $\n#\n# DEV & QA User Resource Limits\n#\n#Each
line describes a limit for a user in the form ...
2912 <unfinished ...>

But then it goes into some sleep, resumes the write, but never adds
anything.

27015 <... nanosleep resumed> {0, 208251453512}) = 0
27015 nanosleep({0, 10000000}, <unfinished ...>
26971 <... write resumed> ) = 2912
26971 fsync(6) = 0
26971 close(6) = 0
26971 munmap(0x2a97621000, 4096) = 0
26971 unlink("/etc/security/limits.conf.augnew") = 0

So I'm not sure what's happening in that nanosleep, but it appears to
be failing and is just returning nothing.

On Jun 3, 3:25 pm, Rob McBroom <mailingli...@skurfer.com> wrote:

[Bryan Kearney]

What does the file look like before you run this? I get a failure on my
machine:

http://pastebin.com/m70e190b2

I am using augeas .50 on Fedora 10.

-- bk

[Bryan Kearney]

Gajillion wrote:
> So, a quick question then. I ran a very verbose strace on puppet
> while it was going through this change. What I see is this.

> <SNIP LOTS OF LOW LEVEL TRACEY STUFF>
You need to differentiate the plugin and augeas. Augeas writes a temp
file and then copies it to the final location.

Now.. the plugin. The first augeas plugin always ran augeas, which
caused the task to fire even if no change was made to the file. We
changed this by adding a no-op mode to augeas, and having the plugin do
a "test run" of the changes to determine if it needs to run.

So.. you are seeing first he test run, and then the actual run.

-- bk

[Gajillion]

Yes. I understand that. I posted the portion of the strace from the
second run. On the first run, my result code is this:

26971 write(1, "\33[0;37mdebug: Augeas[memlock](provider=augeas):
Files changed, should execute\33[0m", 81) = 81
26971 write(1, "\n", 1) = 1
...

26971 sendto(5, "<31>Jun 4 09:32:46 puppetd[26971]: (Augeas[memlock]
(provider=augeas)) Files changed, should execute", 100, MSG_NOSIGNAL,
NULL, 0) = 100

So Augeas recognizes that it's supposed to do something, but on the
second run, it doesn't actually do it and returns a fail code.

[Gajillion]

Yeah, this assumes you have something already in limits.conf. It will
fail if you don't because augeas fails on the last() call.

[root@buildtest security]# tail /etc/security/limits.conf
@arcadev hard nproc 8192
@arcadev soft nofile 2048
@arcadev hard nofile 8192

# Arca QA
@arcaqa soft nproc 2048
@arcaqa hard nproc 8192
@arcaqa soft nofile 2048
@arcqa hard nofile 8192

[Gajillion]

For whatever it's worth, I've been able to trace this to put_store
(which is being called from create_lens) being called with NULL for
state->value. This happens as Augeas is parsing the existing file and
gets to the last value. For some reason, it just keeps going. I'm
afraid this is as far as I can take it as I don't understand the state
generation that well.

[root@buildtest security]# tail -1 /etc/security/limits.conf
@arcqa hard nofile 8192

And with some loud debugging code thrown in...

calling put_lens
in put_store. value : @arcaq
Going into subtree (null)
calling put_lens
in put_store. value : hard
Coming out of subtree (null)
Going into subtree (null)
calling put_lens
in put_store. value : nofile
Coming out of subtree (null)
Going into subtree (null)
calling put_lens
in put_store. value : 8192
Coming out of subtree (null)
Coming out of subtree (null)
Before return check in create_lens with error 0 and tag 49
in create_lens with error 0 and tag 49
Before return check in create_lens with error 0 and tag 50
in create_lens with error 0 and tag 50
calling create_lens
Before return check in create_lens with error 0 and tag 48
in create_lens with error 0 and tag 48
Before return check in create_lens with error 0 and tag 45
in create_lens with error 0 and tag 45
Before return check in create_lens with error 0 and tag 43
in create_lens with error 0 and tag 43
Calling put_store with error 0 and tag 43
in put_store. value : (null)
errored at null value

[David Lutterkort]

On Wed, 2009-06-03 at 12:56 -0700, Gajillion wrote:
> All,
> I've tried several lenses and several iterations and have never been
> able to get augeas and puppet to work together. My latest attempt:
>
> augeas {"memlock":
> changes => [
> "ins domain after /files/etc/security/limits.conf/domain
> [last()]",
> "set /files/etc/security/limits.conf/domain[last()]/ *",

I bet the problem is the trailing '/' - Augeas' XPath syntax doesn't
allow that. The reason this works in augtool is that augtool will be
helpful and clean up the path you entered and remove the trailing slash.

The Puppet Augeas type does not - but it also does not check the return
value of Augeas.set; the next set in your changes creates an domain node
with no value, which is what save ultimately chokes on.

Clearly, we need better error checking/reporting ...

David

[Bryan Kearney]

I will put in a ticket.

-- bk

[Gajillion]

That didn't work either:

debug: Augeas[memlock](provider=augeas): sending command 'ins' with

params ["domain", "after", "/files/etc/security/limits.conf/files/etc/
security/limits.conf/domain[last()]"]

debug: Augeas[memlock](provider=augeas): sending command 'set' with

params ["/files/etc/security/limits.conf/files/etc/security/
limits.conf/domain[last()]", "*"]

debug: Augeas[memlock](provider=augeas): sending command 'set' with

params ["/files/etc/security/limits.conf/files/etc/security/

limits.conf/domain[last()]/type", "hard"]
debug: Augeas[memlock](provider=augeas): sending command 'set' with

params ["/files/etc/security/limits.conf/files/etc/security/

limits.conf/domain[last()]/item", "memlock"]
debug: Augeas[memlock](provider=augeas): sending command 'set' with

params ["/files/etc/security/limits.conf/files/etc/security/

limits.conf/domain[last()]/value", "unlimited"]

err: //Node[buildtest]/wombat::setup/Augeas[memlock]/returns: change

from need_to_run to 0 failed: Save failed with return code false

[David Lutterkort]

On Fri, 2009-06-05 at 06:02 -0700, Gajillion wrote:
> That didn't work either:

And I was so convinced it would ;)

> debug: Augeas[memlock](provider=augeas): sending command 'ins' with
> params ["domain", "after", "/files/etc/security/limits.conf/files/etc/
> security/limits.conf/domain[last()]"]

Do you have both a context set and use an absolute path in your changes
now ? The prefix /files/etc/security/limits.conf is duplicated in the
paths - if that still doesn't fix it, can you send the whole limits.conf
file and the augeas resource you use in your manifest ?

David

[Gajillion]

Solution!

I owe everyone a big thanks and big apology. It turns out that rpm -e
ruby doesn't really rpm -e ruby... Pretty much everything in /usr/lib/
ruby and /usr/lib64/ruby stayed in place when I did that, so when I
upgraded ruby and puppet from 0.24.5 to 0.24.8 and ruby from 1.8.1 to
1.8.6, it just overlayed the 1.8.6 files on top of the 1.8.1 instead
of removing a bunch of stuff that it should have. In effect, I was
really still running puppet 0.24.5 despite the fact that it reported
itself as 0.24.8. I removed all the rpms, deleted /usr/lib/ruby and /
usr/lib64/ruby and reinstalled and it works as advertised.

And David - you were right. It does fail with the trailing slash :)

Thanks again,

Mark

[Bryan Kearney]

Gajillion wrote:
> Solution!
>
> I owe everyone a big thanks and big apology. It turns out that rpm -e
> ruby doesn't really rpm -e ruby... Pretty much everything in /usr/lib/
> ruby and /usr/lib64/ruby stayed in place when I did that, so when I
> upgraded ruby and puppet from 0.24.5 to 0.24.8 and ruby from 1.8.1 to
> 1.8.6, it just overlayed the 1.8.6 files on top of the 1.8.1 instead
> of removing a bunch of stuff that it should have. In effect, I was
> really still running puppet 0.24.5 despite the fact that it reported
> itself as 0.24.8. I removed all the rpms, deleted /usr/lib/ruby and /
> usr/lib64/ruby and reinstalled and it works as advertised.
>
> And David - you were right. It does fail with the trailing slash :)
>

Great... glad to hear it works!

-- bk