Solaris 11/Sparc with Puppet 2.7.14ish (a47c42d6b9c7ed6022b0e7d37505b0c694a5b8d2).
The Cron provider doesn’t appear to actually provide multiple crons..
Crontab contains all the correct cron entries from say:
cron{ 'zfs_scrubber':
command => 'for x in $( zpool list -H | cut -f 1 ); do zpool scrub "${x}" ; done',
user => 'root',
environment => 'PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin',
hour => '3',
minute => '33',
weekday => '6',
}
[root@mundilfari:/var]# crontab -l root # HEADER: This file was autogenerated at Wed Apr 25 19:10:12 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. #ident "%Z%%M% %I% %E% SMI" # # Copyright 2007 Sun Microsystems, Inc. All rights reserved. # Use is subject to license terms. # # The root crontab should be used to perform accounting data collection. # # 10 3 * * * /usr/sbin/logadm 15 3 * * 0 [ -x /usr/lib/fs/nfs/nfsfind ] && /usr/lib/fs/nfs/nfsfind 30 3 * * * [ -x /usr/lib/gss/gsscred_clean ] && /usr/lib/gss/gsscred_clean # Puppet Name: puppet agent 42 * * * * /opt/csw/bin/puppet agent --onetime --no-daemonize >/dev/null
Now I run puppet on said host:
[root@mundilfari:/var]# time puppet agent -t --environment solarisnagios info: Retrieving plugin info: Loading facts in /var/lib/puppet/lib/facter/concat_basedir.rb info: Caching catalog for mundilfari.example.org info: Applying configuration version '460d9de' notice: /Stage[main]/Zfs::Scrubber/Cron[zfs_scrubber]/ensure: created # HEADER: This file was autogenerated at Wed May 02 13:36:28 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. #ident "%Z%%M% %I% %E% SMI" # # Copyright 2007 Sun Microsystems, Inc. All rights reserved. # Use is subject to license terms. # # The root crontab should be used to perform accounting data collection. # # 10 3 * * * /usr/sbin/logadm 15 3 * * 0 [ -x /usr/lib/fs/nfs/nfsfind ] && /usr/lib/fs/nfs/nfsfind 30 3 * * * [ -x /usr/lib/gss/gsscred_clean ] && /usr/lib/gss/gsscred_clean # Puppet Name: puppet agent 42 * * * * /opt/csw/bin/puppet agent --onetime --no-daemonize >/dev/null # Puppet Name: zfs_scrubber PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin 33 3 * * 6 for x in $( zpool list -H | cut -f 1 ); do zpool scrub "${x}" ; done notice: It is highly recommended that you set 'wgetopts=-nv' in your pkgutil.conf. notice: /Stage[main]/Zfs::Snapshots/Cron[zfs weekly snapshot]/ensure: created # HEADER: This file was autogenerated at Wed May 02 13:36:57 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. #ident "%Z%%M% %I% %E% SMI" # # Copyright 2007 Sun Microsystems, Inc. All rights reserved. # Use is subject to license terms. # # The root crontab should be used to perform accounting data collection. # # 10 3 * * * /usr/sbin/logadm 15 3 * * 0 [ -x /usr/lib/fs/nfs/nfsfind ] && /usr/lib/fs/nfs/nfsfind 30 3 * * * [ -x /usr/lib/gss/gsscred_clean ] && /usr/lib/gss/gsscred_clean # Puppet Name: puppet agent 42 * * * * /opt/csw/bin/puppet agent --onetime --no-daemonize >/dev/null # Puppet Name: zfs_scrubber PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin 33 3 * * 6 for x in $( zpool list -H | cut -f 1 ); do zpool scrub "${x}" ; done # Puppet Name: zfs weekly snapshot PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin 15 2 * * 0 /usr/local/bin/zfs-snapshot.rb -r -c 5 -s weekly notice: /Stage[main]/Zfs::Snapshots/Cron[zfs hourly snapshot]/ensure: created notice: Finished catalog run in 97.97 seconds
I actually snipped a few there, as it was getting lengthy…
But now we run crontab -l root and get…
[root@mundilfari:/var]# crontab -l root # HEADER: This file was autogenerated at Wed Apr 25 19:10:12 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. #ident "%Z%%M% %I% %E% SMI" # # Copyright 2007 Sun Microsystems, Inc. All rights reserved. # Use is subject to license terms. # # The root crontab should be used to perform accounting data collection. # # 10 3 * * * /usr/sbin/logadm 15 3 * * 0 [ -x /usr/lib/fs/nfs/nfsfind ] && /usr/lib/fs/nfs/nfsfind 30 3 * * * [ -x /usr/lib/gss/gsscred_clean ] && /usr/lib/gss/gsscred_clean # Puppet Name: puppet agent 42 * * * * /opt/csw/bin/puppet agent --onetime --no-daemonize >/dev/null
So where did they go?!
So the cronjob, at least as far as my Solaris-fu is concerned, doesn’t appear to exist.
[root@mundilfari:~]# gfind /etc /var -xdev -type f -print0 | xargs -0 ggrep -il zfs_scrubber /var/lib/puppet/client_yaml/catalog/mundilfari.dc1.puppetlabs.net.yaml /var/lib/puppet/state/resources.txt /var/lib/puppet/state/state.yaml /var/lib/puppet/state/last_run_report.yaml /var/adm/messages /var/adm/messages.0
Not use Solaris.
You have received this notification because you have either subscribed to it, or are involved in it. To change your notification preferences, please click here: http://projects.puppetlabs.com/my/account
On Solaris puppet seems to modify the crontab by running the following command (under the target user context)
crontab some_filename_with_content_of_new_crontab
Maybe this is not working for solaris 11 anymore? Or does your zfs_scrubber entry appear after you remove the Cron['puppet agent'] resource? Is it really an issue with having multiple cron entries or a general problem?
I have a super complex cron manifest:
[mundilfari:~]% cat cron.pp
cron{
'foo':
command => 'echo foo',
minute => 1;
'bar':
command => 'echo bar',
minute => 2;
'whizz':
command => 'echo whizz',
minute => 3;
'boing':
command => 'echo boing',
minute => 4;
}
I optimistically apply it:
[mundilfari:~]% crontab -l # I AM BLANK [mundilfari:~]% id uid=1025(ben) gid=600(allstaff) [mundilfari:~]% puppet apply -v cron.pp info: Applying configuration version '1336155424'notice: /Stage[main]//Cron[bar]/ensure: created # HEADER: This file was autogenerated at Fri May 04 11:17:07 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. # Puppet Name: bar 2 * * * * echo bar notice: /Stage[main]//Cron[whizz]/ensure: created # HEADER: This file was autogenerated at Fri May 04 11:17:07 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. # Puppet Name: bar 2 * * * * echo bar # Puppet Name: whizz 3 * * * * echo whizz notice: /Stage[main]//Cron[boing]/ensure: created # HEADER: This file was autogenerated at Fri May 04 11:17:08 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. # Puppet Name: bar 2 * * * * echo bar # Puppet Name: whizz 3 * * * * echo whizz # Puppet Name: boing 4 * * * * echo boing notice: /Stage[main]//Cron[foo]/ensure: created # HEADER: This file was autogenerated at Fri May 04 11:17:08 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. # Puppet Name: bar 2 * * * * echo bar # Puppet Name: whizz 3 * * * * echo whizz # Puppet Name: boing 4 * * * * echo boing # Puppet Name: foo 1 * * * * echo foo notice: Finished catalog run in 1.28 seconds [mundilfari:~]% crontab -l # I AM BLANK
Yeah, it’s on Solaris 11. I haven’t got the Solaris 10 zone up yet to test if it’s broken on that yet.
[mundilfari:~]% uname -a SunOS mundilfari 5.11 11.0 sun4v sparc SUNW,SPARC-Enterprise-T5120
So none of your crontab entries appear in the final crontab. Can you do the following on the commandline:
# cat > /tmp/crontab << EOF 1 * * * * echo foo EOF # crontab /tmp/crontab # echo $?
If a crontab resource is out of sync, Puppet will write the new crontab to a tempfile and will then run crontab new_crontab_file to load the crontab. I suppose that’s not working on solaris 11 but I haven’t found a manpage for crontab and Solaris 11.
It works on the command line, just not via Puppet. I’m currently trying to truss the crontab command in Solaris to find out why it is not working.
I’ll write back when I have more.
Interesting!
# crontab -l ben crontab: can't open your crontab file. # /opt/csw/bin/puppet --version 2.7.14 # /opt/puppet/bin/puppet apply -v ~ben/cron.pp info: Loading facts in /var/lib/puppet/lib/facter/concat_basedir.rb info: Applying configuration version '1336170652' notice: /Stage[main]//Cron[bar]/ensure: created # HEADER: This file was autogenerated at Fri May 04 15:30:56 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. # Puppet Name: bar 2 1 * * * echo bar notice: /Stage[main]//Cron[whizz]/ensure: created # HEADER: This file was autogenerated at Fri May 04 15:30:57 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. # Puppet Name: bar 2 1 * * * echo bar # Puppet Name: whizz 3 1 * * * echo whizz notice: /Stage[main]//Cron[boing]/ensure: created # HEADER: This file was autogenerated at Fri May 04 15:30:57 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. # Puppet Name: bar 2 1 * * * echo bar # Puppet Name: whizz 3 1 * * * echo whizz # Puppet Name: boing 4 1 * * * echo boing notice: /Stage[main]//Cron[foo]/ensure: created # HEADER: This file was autogenerated at Fri May 04 15:30:58 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. # Puppet Name: bar 2 1 * * * echo bar # Puppet Name: whizz 3 1 * * * echo whizz # Puppet Name: boing 4 1 * * * echo boing # Puppet Name: foo 1 1 * * * echo foo notice: Finished catalog run in 4.18 seconds
Which gives me:
# crontab -l ben # HEADER: This file was autogenerated at Fri May 04 15:30:58 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. # Puppet Name: bar 2 1 * * * echo bar # Puppet Name: whizz 3 1 * * * echo whizz # Puppet Name: boing 4 1 * * * echo boing # Puppet Name: foo 1 1 * * * echo foo
Yet if I run it with envpuppet, it does the reported behaviour (and I’ve no idea why). This bug may be entirely wrong!! There’s hope.
can you please run
% puppet apply -e 'notify { fact_check: message => $operatingsystem }'
through your envpuppet environment. Maybe the fact cannot be resolved to solaris and puppet uses the wrong commands to write the crontab.
So, just to make sure. I removed the envpuppet version and the CSW version, and installed puppet via the gem.
Still happens.
I know it’s detecting the operatingsystem, as I was adding debug output to the suntab section of lib/puppet/util/filetype.rb.
mundilfari:~]% crontab -r
[mundilfari:~]% puppet --version
2.7.14
[mundilfari:~]% puppet apply -v cron.pp
info: Applying configuration version '1336521346'
notice: /Stage[main]//Cron[bar]/ensure: created
# HEADER: This file was autogenerated at Tue May 08 16:55:49 -0700 2012 by puppet.
# HEADER: While it can still be managed manually, it is definitely not recommended.
# HEADER: Note particularly that the comments starting with 'Puppet Name' should
# HEADER: not be deleted, as doing so could cause duplicate cron jobs.
# Puppet Name: bar
2 1 * * * echo bar
notice: /Stage[main]//Cron[whizz]/ensure: created
# HEADER: This file was autogenerated at Tue May 08 16:55:50 -0700 2012 by puppet.
# HEADER: While it can still be managed manually, it is definitely not recommended.
# HEADER: Note particularly that the comments starting with 'Puppet Name' should
# HEADER: not be deleted, as doing so could cause duplicate cron jobs.
# Puppet Name: bar
2 1 * * * echo bar
# Puppet Name: whizz
3 1 * * * echo whizz
notice: /Stage[main]//Cron[boing]/ensure: created
# HEADER: This file was autogenerated at Tue May 08 16:55:50 -0700 2012 by puppet.
# HEADER: While it can still be managed manually, it is definitely not recommended.
# HEADER: Note particularly that the comments starting with 'Puppet Name' should
# HEADER: not be deleted, as doing so could cause duplicate cron jobs.
# Puppet Name: bar
2 1 * * * echo bar
# Puppet Name: whizz
3 1 * * * echo whizz
# Puppet Name: boing
4 1 * * * echo boing
notice: /Stage[main]//Cron[foo]/ensure: created
# HEADER: This file was autogenerated at Tue May 08 16:55:50 -0700 2012 by puppet.
# HEADER: While it can still be managed manually, it is definitely not recommended.
# HEADER: Note particularly that the comments starting with 'Puppet Name' should
# HEADER: not be deleted, as doing so could cause duplicate cron jobs.
# Puppet Name: bar
2 1 * * * echo bar
# Puppet Name: whizz
3 1 * * * echo whizz
# Puppet Name: boing
4 1 * * * echo boing
# Puppet Name: foo
1 1 * * * echo foo
notice: Finished catalog run in 1.27 seconds
[mundilfari:~]% crontab -l
crontab: can't open your crontab file.
[mundilfari:~]% facter -p operatingsystem
Solaris
[mundilfari:~]% puppet apply -e 'notify { fact_check: message => $operatingsystem }'
notice: Solaris
notice: /Stage[main]//Notify[fact_check]/message: defined 'message' as 'Solaris'
notice: Finished catalog run in 0.38 seconds
I can only guess that it is some sort of a permission issue. When you are already inside the suntab filetype can you please replace output_file.delete with puts output_file.path inside the write method?
This way you can check the content of the new crontab (was the write successful?) and manually run crontab <filename> as the target user to check if that’ll work, too.
If I replace the call to Puppet::Util.execute with system, it works.
[mundilfari:~]% crontab -r [mundilfari:~]% crontab -l crontab: can't open your crontab file. [mundilfari:~]% puppet apply -v cron.pp info: Applying configuration version '1336590047' notice: /Stage[main]//Cron[bar]/ensure: created # HEADER: This file was autogenerated at Wed May 09 12:00:51 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. # Puppet Name: bar 2 1 * * * echo bar notice: /Stage[main]//Cron[whizz]/ensure: created # HEADER: This file was autogenerated at Wed May 09 12:00:51 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. # Puppet Name: bar 2 1 * * * echo bar # Puppet Name: whizz 3 1 * * * echo whizz notice: /Stage[main]//Cron[boing]/ensure: created # HEADER: This file was autogenerated at Wed May 09 12:00:51 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. # Puppet Name: bar 2 1 * * * echo bar # Puppet Name: whizz 3 1 * * * echo whizz # Puppet Name: boing 4 1 * * * echo boing notice: /Stage[main]//Cron[foo]/ensure: created # HEADER: This file was autogenerated at Wed May 09 12:00:51 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. # Puppet Name: bar 2 1 * * * echo bar # Puppet Name: whizz 3 1 * * * echo whizz # Puppet Name: boing 4 1 * * * echo boing # Puppet Name: foo 1 1 * * * echo foo notice: Finished catalog run in 0.91 seconds [mundilfari:~]% crontab -l # HEADER: This file was autogenerated at Wed May 09 12:00:51 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. # Puppet Name: bar 2 1 * * * echo bar # Puppet Name: whizz 3 1 * * * echo whizz # Puppet Name: boing 4 1 * * * echo boing # Puppet Name: foo 1 1 * * * echo foo
With this massive change:
[enlil:puppet]% git diff
diff --git i/lib/puppet/util/filetype.rb w/lib/puppet/util/filetype.rb
index fdc6ccd..f262caa 100755
--- i/lib/puppet/util/filetype.rb
+++ w/lib/puppet/util/filetype.rb
@@ -233,7 +233,7 @@ class Puppet::Util::FileType
File.chown(Puppet::Util.uid(@path), nil, output_file.path)
begin
- Puppet::Util.execute(["crontab", output_file.path], :uid => @path)
+ system( "crontab #{output_file.path}'" )
rescue => detail
raise Puppet::Error, "Could not write crontab for #{@path}: #{detail}"
end
(I am aware this won’t work for anyone other than my user.)
Maybe puppet has some problems changing the user context. Puppet will not only switch the euid but also the groups (http://puppetlabs.com/security/cve/cve-2012-1053).
I guess that solaris only allows cron for the members of the cron group is that correct? Maybe this is dropped somewhere?
It would be interesting to verify that it’s a group membership issue (I was not able to reproduce the issue on linux and with an unpriviledged user running puppet) so can you reverse your “fix” and try if the following does work:
diff --git a/lib/puppet/util/suidmanager.rb b/lib/puppet/util/suidmanager.rb
index f270c40..aa527ed 100644
--- a/lib/puppet/util/suidmanager.rb
+++ b/lib/puppet/util/suidmanager.rb
@@ -160,7 +160,7 @@ module Puppet::Util::SUIDManager
# if used without root to initgroups of another user.
def initgroups(uid)
pwent = Etc.getpwuid(uid)
- Process.initgroups(pwent.name, pwent.gid)
+# Process.initgroups(pwent.name, pwent.gid)
end
With those two files adjusted accordingly, I get the following:
[mundilfari:~]% crontab -r [mundilfari:~]% crontab -l crontab: can't open your crontab file. [mundilfari:~]% puppet apply -v cron.pp info: Applying configuration version '1337814069' notice: /Stage[main]//Cron[bar]/ensure: created # HEADER: This file was autogenerated at Wed May 23 16:01:13 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. # Puppet Name: bar 2 1 * * * echo bar notice: /Stage[main]//Cron[whizz]/ensure: created # HEADER: This file was autogenerated at Wed May 23 16:01:13 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. # Puppet Name: bar 2 1 * * * echo bar # Puppet Name: whizz 3 1 * * * echo whizz notice: /Stage[main]//Cron[boing]/ensure: created # HEADER: This file was autogenerated at Wed May 23 16:01:13 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. # Puppet Name: bar 2 1 * * * echo bar # Puppet Name: whizz 3 1 * * * echo whizz # Puppet Name: boing 4 1 * * * echo boing notice: /Stage[main]//Cron[foo]/ensure: created # HEADER: This file was autogenerated at Wed May 23 16:01:13 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. # Puppet Name: bar 2 1 * * * echo bar # Puppet Name: whizz 3 1 * * * echo whizz # Puppet Name: boing 4 1 * * * echo boing # Puppet Name: foo 1 1 * * * echo foo notice: Finished catalog run in 1.37 seconds [mundilfari:~]% crontab -l # HEADER: This file was autogenerated at Wed May 23 16:01:13 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. # Puppet Name: bar 2 1 * * * echo bar # Puppet Name: whizz 3 1 * * * echo whizz # Puppet Name: boing 4 1 * * * echo boing # Puppet Name: foo 1 1 * * * echo foo [mundilfari:~]% id uid=1025(ben) gid=600(allstaff)
Which seems better but strange.
I guess we are heading in the right direction and puppet drops to many priviledges before running crontab. The final fix may be to only pass a uid to Puppet::Util.execute when the current user and the target user (the one that is specified in the cron resource) differ.
I tried to simulate the steps of changing priviledges and it failed when not run as root on my linux system. Maybe it does silently fail on solaris too
So running the following as root does work (with ben beeing a different user and 1003 beeing the GID of the user ben)
# irb
irb(main):001:0> Process.groups
=> [0, 1, 2, 3, 4, 6, 10, 11, 26, 27]
irb(main):002:0> Process.initgroups('ben',1003)
=> [16, 1003]
irb(main):003:0> Process.groups
=> [16, 1003]
in the example above the supplimentary groups were changed to the one that ben is in (in my case 1003 is ben’s gid and 16 is the cron group)
running the same as the user ben does not work
% irb
irb(main):001:0> Process.groups
=> [16, 10003]
irb(main):002:0> Process.initgroups('ben',1003)
Errno::EPERM: Operation not permitted
from (irb):2:in `initgroups'
from (irb):2
from /usr/bin/irb:12:in `'
irb(main):003:0> Process.groups
=> [16, 10003]
So with the information above I’d expect your cronresource to fail in a visible fashion. Can you please do the examples in irb like I did above? One time as the root user and a second time as your target user?
[mundilfari:~]% irb
irb(main):001:0> Process.groups
=> [600, 666, 673, 682, 683, 601]
irb(main):002:0> Process.initgroups('ben',601)
Errno::EPERM: Not owner
from (irb):2:in `initgroups'
from (irb):2
irb(main):003:0> Process.groups
=> [600, 666, 673, 682, 683, 601]
and as root
[mundilfari:~]% sudo irb
Couldn't load gem of: no such file to load -- wirble
irb(main):001:0> Process.groups
=> [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 12]
irb(main):002:0> Process.initgroups('ben',1003)
=> [1003, 666, 673, 682, 683, 601]
irb(main):003:0> Process.groups
=> [1003, 666, 673, 682, 683, 601]
Ok I guess I finally understand the problem. The method Puppet::Util.execute accepts several arguments as a hash such as :uid, :gid etc. There are some default arguments one of which is :failonfail => true which causes puppet to raise an exception if the command does not return with a zero exitcode.
The cronfiletype now does run
Puppet::Util.execute(["crontab", output_file.path], :uid => @path)
overwriting the default arguments. Because changing priviledges inside the execute method does fail (as you just showed inside irb), the whole cron command fails. But the argument failonfail is now empty so the error is just ignored and the command treated as successfully run. Can you revert any modifications you did before an just change:
diff --git a/lib/puppet/util/filetype.rb b/lib/puppet/util/filetype.rb
index fdc6ccd..4ecd160 100755
--- a/lib/puppet/util/filetype.rb
+++ b/lib/puppet/util/filetype.rb
@@ -233,7 +233,7 @@ class Puppet::Util::FileType
File.chown(Puppet::Util.uid(@path), nil, output_file.path)
begin
- Puppet::Util.execute(["crontab", output_file.path], :uid => @path)
+ Puppet::Util.execute(["crontab", output_file.path], :failonfail => true, :combine => true, :uid => @path)
rescue => detail
raise Puppet::Error, "Could not write crontab for #{@path}: #{detail}"
end
If you now run puppet you should see an error message that running cron failed. Can you please verify this?
[mundilfari:~]% crontab -r 1 [mundilfari:~]% crontab -l crontab: can't open your crontab file. [mundilfari:~]% puppet apply -v cron.pp 1 info: Applying configuration version '1338321920'notice: /Stage[main]//Cron[bar]/ensure: created # HEADER: This file was autogenerated at Tue May 29 13:05:23 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. # Puppet Name: bar 2 1 * * * echo bar err: /Stage[main]//Cron[bar]: Could not evaluate: Could not write crontab for ben: Execution of 'crontab /tmp/puppet20120529-23320-6wgwxd-0' returned 1: notice: /Stage[main]//Cron[whizz]/ensure: created # HEADER: This file was autogenerated at Tue May 29 13:05:24 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. # Puppet Name: bar 2 1 * * * echo bar # Puppet Name: whizz 3 1 * * * echo whizz err: /Stage[main]//Cron[whizz]: Could not evaluate: Could not write crontab for ben: Execution of 'crontab /tmp/puppet20120529-23320-1odzoz7-0' returned 1: notice: /Stage[main]//Cron[boing]/ensure: created # HEADER: This file was autogenerated at Tue May 29 13:05:24 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. # Puppet Name: bar 2 1 * * * echo bar # Puppet Name: whizz 3 1 * * * echo whizz # Puppet Name: boing 4 1 * * * echo boing err: /Stage[main]//Cron[boing]: Could not evaluate: Could not write crontab for ben: Execution of 'crontab /tmp/puppet20120529-23320-o8z6it-0' returned 1: notice: /Stage[main]//Cron[foo]/ensure: created # HEADER: This file was autogenerated at Tue May 29 13:05:24 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. # Puppet Name: bar 2 1 * * * echo bar # Puppet Name: whizz 3 1 * * * echo whizz # Puppet Name: boing 4 1 * * * echo boing # Puppet Name: foo 1 1 * * * echo foo err: /Stage[main]//Cron[foo]: Could not evaluate: Could not write crontab for ben: Execution of 'crontab /tmp/puppet20120529-23320-1rhhfvk-0' returned 1:
I’ll try to write a fix then
Further to my update in the pull request as to running it as root/puppet agent:
This is what I’m seeing on our Solaris 11 box when running puppet agent -t
[....snip most of run...]
notice: /Stage[main]/Postfix::Mboxcheck/Cron[check_for_mailbozes]/ensure: created
# HEADER: This file was autogenerated at Mon Jun 25 10:51:34 -0700 2012 by puppet.
# HEADER: While it can still be managed manually, it is definitely not recommended.
# HEADER: Note particularly that the comments starting with 'Puppet Name' should
# HEADER: not be deleted, as doing so could cause duplicate cron jobs.
#ident "%Z%%M% %I% %E% SMI"
#
# Copyright 2007 Sun Microsystems, Inc. All rights reserved.
# Use is subject to license terms.
#
# The root crontab should be used to perform accounting data collection.
#
#
10 3 * * * /usr/sbin/logadm
15 3 * * 0 [ -x /usr/lib/fs/nfs/nfsfind ] && /usr/lib/fs/nfs/nfsfind
30 3 * * * [ -x /usr/lib/gss/gsscred_clean ] && /usr/lib/gss/gsscred_clean
# 42 * * * * /opt/csw/bin/puppet agent --onetime --no-daemonize >/dev/null
# Puppet Name: puppet agent
53 3 * * * /opt/csw/bin/puppet agent --onetime --no-daemonize >/dev/null
# Puppet Name: zfs_scrubber
PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin
33 3 * * 6 for x in $( zpool list -H | cut -f 1 ); do zpool scrub "${x}" ; done
# Puppet Name: zfs weekly snapshot
PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin
15 2 * * 0 /usr/local/bin/zfs-snapshot.rb -r -c 5 -s weekly
# Puppet Name: zfs hourly snapshot
PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin
5 * * * * /usr/local/bin/zfs-snapshot.rb -r -c 25 -s hourly
# Puppet Name: zfs daily snapshot
PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin
10 1 * * * /usr/local/bin/zfs-snapshot.rb -r -c 8 -s daily
# Puppet Name: check_for_mailbozes
MAILTO=root
17 2 * * 0 /usr/bin/find /var/mail -type f -size +0 -ls
notice: Finished catalog run in 312.63 seconds
[root@mundilfari:~]# crontab -l
# HEADER: This file was autogenerated at Wed Apr 25 19:10:12 -0700 2012 by puppet.
# HEADER: While it can still be managed manually, it is definitely not recommended.
# HEADER: Note particularly that the comments starting with 'Puppet Name' should
# HEADER: not be deleted, as doing so could cause duplicate cron jobs.
#ident "%Z%%M% %I% %E% SMI"
#
# Copyright 2007 Sun Microsystems, Inc. All rights reserved.
# Use is subject to license terms.
#
# The root crontab should be used to perform accounting data collection.
#
#
10 3 * * * /usr/sbin/logadm
15 3 * * 0 [ -x /usr/lib/fs/nfs/nfsfind ] && /usr/lib/fs/nfs/nfsfind
30 3 * * * [ -x /usr/lib/gss/gsscred_clean ] && /usr/lib/gss/gsscred_clean
# Puppet Name: puppet agent
# 42 * * * * /opt/csw/bin/puppet agent --onetime --no-daemonize >/dev/null
43 3 * * * [ -x /opt/csw/bin/gupdatedb ] && /opt/csw/bin/gupdatedb --prunepaths="/dev /devices /proc /tmp /var/tmp" 1>/dev/null 2>&1 # Added by CSWfindutils
[root@mundilfari:~]# puppet --version
2.7.14
Installed from CSWpuppet.
Have you already applied the patch in the pull request? I thought the issue is when changing the usercontext as an unpriviledged user, but the patch does also solve the general issue that a failed cron command is never reported.
The only idea I have right know is that puppet creates the temporary file in a location where the target user cannot acces it. But this should show up when the patch is applied I hope.
Running puppet agent -t with that filetype.rb I get:
notice: /Stage[main]/Postfix::Mboxcheck/Cron[check_for_mailbozes]/ensure: created
# HEADER: This file was autogenerated at Tue Jun 26 12:15:32 -0700 2012 by puppet.
# HEADER: While it can still be managed manually, it is definitely not recommended.
# HEADER: Note particularly that the comments starting with 'Puppet Name' should
# HEADER: not be deleted, as doing so could cause duplicate cron jobs.
#ident "%Z%%M% %I% %E% SMI"
#
# Copyright 2007 Sun Microsystems, Inc. All rights reserved.
# Use is subject to license terms.
#
# The root crontab should be used to perform accounting data collection.
#
#
10 3 * * * /usr/sbin/logadm
15 3 * * 0 [ -x /usr/lib/fs/nfs/nfsfind ] && /usr/lib/fs/nfs/nfsfind
30 3 * * * [ -x /usr/lib/gss/gsscred_clean ] && /usr/lib/gss/gsscred_clean
# 42 * * * * /opt/csw/bin/puppet agent --onetime --no-daemonize >/dev/null
# Puppet Name: puppet agent
53 3 * * * /opt/csw/bin/puppet agent --onetime --no-daemonize >/dev/null
# Puppet Name: zfs_scrubber
PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin
33 3 * * 6 for x in $( zpool list -H | cut -f 1 ); do zpool scrub "${x}" ; done
# Puppet Name: zfs weekly snapshot
PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin
15 2 * * 0 /usr/local/bin/zfs-snapshot.rb -r -c 5 -s weekly
# Puppet Name: zfs hourly snapshot
PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin
5 * * * * /usr/local/bin/zfs-snapshot.rb -r -c 25 -s hourly
# Puppet Name: zfs daily snapshot
PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin
10 1 * * * /usr/local/bin/zfs-snapshot.rb -r -c 8 -s daily
# Puppet Name: check_for_mailbozes
MAILTO=root
17 2 * * 0 /usr/bin/find /var/mail -type f -size +0 -ls
err: /Stage[main]/Postfix::Mboxcheck/Cron[check_for_mailbozes]: Could not evaluate: Could not write crontab for root: Execution of 'crontab /tmp/puppet_suntab20120626-11894-1s17qw7-0' returned 1: PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin
crontab: error on previous line; unexpected character found in line.
PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin
crontab: error on previous line; unexpected character found in line.
PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin
crontab: error on previous line; unexpected character found in line.
PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin
crontab: error on previous line; unexpected character found in line.
MAILTO=root
crontab: error on previous line; unexpected character found in line.
crontab: errors detected in input, no crontab file generated.
Ben Hughes wrote:
Running puppet agent -t with that filetype.rb I get:
[…] PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin crontab: error on previous line; unexpected character found in line.
Well then I guess setting the environment for a specific cron command is valid for some cron implementations (like vixie cron) but not valid for cron found on solaris 11.
Stefan Schulte wrote:
Well then I guess setting the environment for a specific cron command is valid for some cron implementations (like vixie cron) but not valid for cron found on solaris 11.
This would also explain why https://projects.puppetlabs.com/issues/14283#note-7 worked for you as I don’t see any environment settings here.
From man 5 crontab, Solaris only supports TZ, HOME and SHELL being set. Nothing more, due to it being 1987.
Manifest stolen from out of our repo:
[root@mundilfari:~]# cat snapshots.pp
class zfs::snapshots {
$env = $operatingsystem ? {
'freebsd' => 'PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin',
'solaris' => undef,
'sunos' => undef,
default => undef,
}
cron {
'zfs hourly snapshot':
user => root,
minute => 5,
command => '/usr/local/bin/zfs-snapshot.rb -r -c 25 -s hourly',
environment => $env;
'zfs daily snapshot':
user => root,
minute => 10,
hour => 1,
command => '/usr/local/bin/zfs-snapshot.rb -r -c 8 -s daily',
environment => $env;
'zfs weekly snapshot':
user => root,
minute => 15,
hour => 2,
weekday => 0,
command => '/usr/local/bin/zfs-snapshot.rb -r -c 5 -s weekly',
environment => $env;
}
file {
'/usr/local/bin/zfs-snapshot.sh':
ensure => absent,
}
}
include zfs::snapshots
And run it
[root@mundilfari:~]# puppet apply -v snapshots.pp info: Loading facts in /var/lib/puppet/lib/facter/whereami.rb info: Loading facts in /var/lib/puppet/lib/facter/concat_basedir.rb info: Loading facts in /var/lib/puppet/lib/facter/bacula.rb info: Loading facts in /var/lib/puppet/lib/facter/printers.rb info: Loading facts in /var/lib/puppet/lib/facter/haszfs.rb info: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rb info: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rb info: Loading facts in /var/lib/puppet/lib/facter/certname.rb info: Loading facts in /var/lib/puppet/lib/facter/root_home.rb info: Loading facts in /var/lib/puppet/lib/facter/systemid.rb info: Loading facts in /var/lib/puppet/lib/facter/confenv.rb info: Loading facts in /var/lib/puppet/lib/facter/lastuser.rb info: Applying configuration version '1340833290' notice: /Stage[main]/Zfs::Snapshots/Cron[zfs weekly snapshot]/ensure: created # HEADER: This file was autogenerated at Wed Jun 27 14:41:34 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. #ident "%Z%%M% %I% %E% SMI" # # Copyright 2007 Sun Microsystems, Inc. All rights reserved. # Use is subject to license terms. # # The root crontab should be used to perform accounting data collection. # # 10 3 * * * /usr/sbin/logadm 15 3 * * 0 [ -x /usr/lib/fs/nfs/nfsfind ] && /usr/lib/fs/nfs/nfsfind 30 3 * * * [ -x /usr/lib/gss/gsscred_clean ] && /usr/lib/gss/gsscred_clean # 42 * * * * /opt/csw/bin/puppet agent --onetime --no-daemonize >/dev/null # Puppet Name: puppet agent 43 3 * * * [ -x /opt/csw/bin/gupdatedb ] && /opt/csw/bin/gupdatedb --prunepaths="/dev /devices /proc /tmp /var/tmp" 1>/dev/null 2>&1 # Added by CSWfindutils # Puppet Name: zfs weekly snapshot 15 2 * * 0 /usr/local/bin/zfs-snapshot.rb -r -c 5 -s weekly notice: /Stage[main]/Zfs::Snapshots/Cron[zfs hourly snapshot]/ensure: created # HEADER: This file was autogenerated at Wed Jun 27 14:41:35 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. #ident "%Z%%M% %I% %E% SMI" # # Copyright 2007 Sun Microsystems, Inc. All rights reserved. # Use is subject to license terms. # # The root crontab should be used to perform accounting data collection. # # 10 3 * * * /usr/sbin/logadm 15 3 * * 0 [ -x /usr/lib/fs/nfs/nfsfind ] && /usr/lib/fs/nfs/nfsfind 30 3 * * * [ -x /usr/lib/gss/gsscred_clean ] && /usr/lib/gss/gsscred_clean # 42 * * * * /opt/csw/bin/puppet agent --onetime --no-daemonize >/dev/null # Puppet Name: puppet agent 43 3 * * * [ -x /opt/csw/bin/gupdatedb ] && /opt/csw/bin/gupdatedb --prunepaths="/dev /devices /proc /tmp /var/tmp" 1>/dev/null 2>&1 # Added by CSWfindutils # Puppet Name: zfs weekly snapshot 15 2 * * 0 /usr/local/bin/zfs-snapshot.rb -r -c 5 -s weekly # Puppet Name: zfs hourly snapshot 5 * * * * /usr/local/bin/zfs-snapshot.rb -r -c 25 -s hourly notice: /Stage[main]/Zfs::Snapshots/Cron[zfs daily snapshot]/ensure: created # HEADER: This file was autogenerated at Wed Jun 27 14:41:35 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. #ident "%Z%%M% %I% %E% SMI" # # Copyright 2007 Sun Microsystems, Inc. All rights reserved. # Use is subject to license terms. # # The root crontab should be used to perform accounting data collection. # # 10 3 * * * /usr/sbin/logadm 15 3 * * 0 [ -x /usr/lib/fs/nfs/nfsfind ] && /usr/lib/fs/nfs/nfsfind 30 3 * * * [ -x /usr/lib/gss/gsscred_clean ] && /usr/lib/gss/gsscred_clean # 42 * * * * /opt/csw/bin/puppet agent --onetime --no-daemonize >/dev/null # Puppet Name: puppet agent 43 3 * * * [ -x /opt/csw/bin/gupdatedb ] && /opt/csw/bin/gupdatedb --prunepaths="/dev /devices /proc /tmp /var/tmp" 1>/dev/null 2>&1 # Added by CSWfindutils # Puppet Name: zfs weekly snapshot 15 2 * * 0 /usr/local/bin/zfs-snapshot.rb -r -c 5 -s weekly # Puppet Name: zfs hourly snapshot 5 * * * * /usr/local/bin/zfs-snapshot.rb -r -c 25 -s hourly # Puppet Name: zfs daily snapshot 10 1 * * * /usr/local/bin/zfs-snapshot.rb -r -c 8 -s daily notice: Finished catalog run in 2.43 seconds
And lo and behold:
[root@mundilfari:~]# crontab -l # HEADER: This file was autogenerated at Wed Jun 27 14:41:35 -0700 2012 by puppet. # HEADER: While it can still be managed manually, it is definitely not recommended. # HEADER: Note particularly that the comments starting with 'Puppet Name' should # HEADER: not be deleted, as doing so could cause duplicate cron jobs. #ident "%Z%%M% %I% %E% SMI" # # Copyright 2007 Sun Microsystems, Inc. All rights reserved. # Use is subject to license terms. # # The root crontab should be used to perform accounting data collection. # # 10 3 * * * /usr/sbin/logadm 15 3 * * 0 [ -x /usr/lib/fs/nfs/nfsfind ] && /usr/lib/fs/nfs/nfsfind 30 3 * * * [ -x /usr/lib/gss/gsscred_clean ] && /usr/lib/gss/gsscred_clean # 42 * * * * /opt/csw/bin/puppet agent --onetime --no-daemonize >/dev/null # Puppet Name: puppet agent 43 3 * * * [ -x /opt/csw/bin/gupdatedb ] && /opt/csw/bin/gupdatedb --prunepaths="/dev /devices /proc /tmp /var/tmp" 1>/dev/null 2>&1 # Added by CSWfindutils # Puppet Name: zfs weekly snapshot 15 2 * * 0 /usr/local/bin/zfs-snapshot.rb -r -c 5 -s weekly # Puppet Name: zfs hourly snapshot 5 * * * * /usr/local/bin/zfs-snapshot.rb -r -c 25 -s hourly # Puppet Name: zfs daily snapshot 10 1 * * * /usr/local/bin/zfs-snapshot.rb -r -c 8 -s daily
See comment https://github.com/puppetlabs/puppet/pull/830#issuecomment-7600559
From Stephan:
@joshcooper I cannot say anything about aix but I checked Solaris at $WORK today and "can't open your crontab" does indeed return with a non zero exitcode. But I think we should move the
return "" if output.include?("can't open your crontab")
in the ensure block because “can’t open your crontab” is raised when “the user can access the crontab command but no crontab file exists” http://docs.oracle.com/cd/E19082-01/819-2380/sysrescron-60/index.html so the user should not see a prefetching failed message in this case I guess.
Merged into 2.7.x in https://github.com/puppetlabs/puppet/commit/87f9f30
Running puppet as an unpriviledged user to manage its own crontab caused
two problems under solaris:
The uid argument was passed to the execute method to set the correct uid
before executing the crontab command. The execute method has a default
argumenthash with the key :failonfail set to true. Because the suntab
filetype passed it's own hash, failonfail was nil so if crontab
completes with a non-zero exit code no error will be raised.
This is bad because running execute with :uid not only sets the correct
userid in the child process but also tries to run `Process.initgroups`
to set the correct supplementary groups of that user. This call will most
likely fail as a non priviledged user causing the whole child process to fail.
The fix now makes sure that failonfail is passed to the execute method
so failures can be handled correctly and uid is only passed if necessary
(which is not the case when the target user in the crontab resource is
the same user that is running puppet).
Since the execute method now raises an exception on non-zero exit
codes, we have to handle the case where the crontab file doesn't exist
for the user (and return '') versus other fatal errors.
Also, the aix provider had similar problems and was fixed:
Previously, the AIX crontab filetype had the same issue as the Sun one,
the `:failonfail` argument was being set to nil, so it would not raise
an error when crontab returned a non-zero exit code. The only time the
filetype raised an exception was if the command output contained the
line: 'You are not authorized to use the cron command'
This commit changes the AIX crontab filetype to raise an exception if
crontab returns a non-zero exit code.
If the user doesn't have any crontab entries, then the `#read` method
will return an empty string as is done for on Solaris.
If the command failed because the user doesn't have permission, then the
filetype will raise an exception, as it did before.
This commit also adds tests and an AIX crontab fixture.
The masking of exceptions when crontab returns a non-zero exit code was introduced in https://github.com/puppetlabs/puppet/commit/1e6ba6f9ff74bc1204c59457911621a3ad12e8ae which tried to fix managing non-root crontab #781
Released in Puppet 3.0.0-rc4
Released in Puppet 2.7.20