Exec doesn't work with Ubuntu Server 10.04 (Lucid Lynx) 64bit
kai.steverding
test :
class testmodule {
exec {"TEST-EXEC" :
cwd => "/tmp/",
command =>"/usr/bin/touch /tmp/7777 >/tmp/123 2>&1",
timeout => 5,
logoutput=> on_failure
}
}
This simple thing gets the following output from "puppet --debug --
test"
debug: Loaded state in 0.00 seconds
info: Applying configuration version '1265719507'
debug: //testmodule/Exec[TEST-EXEC]: Changing returns
debug: //testmodule/Exec[TEST-EXEC]: 1 change(s)
debug: //testmodule/Exec[TEST-EXEC]: Executing '/usr/bin/touch /tmp/
7777'
debug: Executing '/usr/bin/touch /tmp/7777'
err: //testmodule/Exec[TEST-EXEC]/returns: change from notrun to 0
failed: Command exceeded timeout at /etc/puppet/modules/testmodule/
manifests/init.pp:6
debug: Finishing transaction 69914685668640 with 1 changes
debug: Storing state
debug: Stored state in 0.01 seconds
debug: Format pson not supported for Puppet::Transaction::Report; has
not implemented method 'from_pson'
debug: Format s not supported for Puppet::Transaction::Report; has not
implemented method 'from_s'
What can I do ? Did i make a mistake, or is exec broken ?
James Turnbull
Hash: SHA1
On 10/02/10 12:06 AM, kai.steverding wrote:
> I installed ruby on the above server and tried with a simple exec-
> test :
What Puppet and Ruby versions?
Regards
James Turnbull
- --
Author of:
* Pro Linux System Administration (http://tinyurl.com/linuxadmin)
* Pulling Strings with Puppet (http://tinyurl.com/pupbook)
* Pro Nagios 2.0 (http://tinyurl.com/pronagios)
* Hardening Linux (http://tinyurl.com/hardeninglinux)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQEVAwUBS3FfxCFa/lDkFHAyAQIdnwf9FVevibgcjvympW3YknpQQlIbgl9MPJIt
LoB0JKwqFOs4LjXY+YdP8j3CyIMTtQUGHKTpz5OpPLCBUIUN3spaIfZ/jSfCvozE
S20adBerNJKEtU7JMk1JGNCYVimOARJIWf31bpWnB9NiRuJltc7Cjcez/uJ0QpI7
VGYw3rkjwzZDpE7joajcwyalwaY8KkYprRXXGdQWcBfawyHlay0kj8WUNB4BSK/s
9I+RxoFtNehfF8yt+r4VkOpuGgYXnKKG+1lTmQQw/0W7WWOuKWkhYYM00wPU46DX
ny3oGioYYIK5eH3YQYVBxVbvH/cD5bLQWgRQT94V2SQ1JkemQiJmPA==
=aQtA
-----END PGP SIGNATURE-----
kai.steverding
The exec call works in 2 CPU machine. It does not work in a single CPU
machine.
I can switch my VM from 2 to 1 CPU's and break puppet with this
change.
Here are the informations you requested :
root@puppet:/etc/puppet/manifests# ruby -v
ruby 1.8.7 (2010-01-10 patchlevel 249) [x86_64-linux]
root@puppet:/etc/puppet/manifests# puppet -V
0.25.4
On 9 Feb., 14:14, James Turnbull <ja...@lovedthanlost.net> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 10/02/10 12:06 AM, kai.steverding wrote:
>
> > I installed ruby on the above server and tried with a simple exec-
> > test :
>
> What Puppet and Ruby versions?
>
> Regards
>
> James Turnbull
>
> - --
> Author of:
> * Pro Linux System Administration (http://tinyurl.com/linuxadmin)
> * Pulling Strings with Puppet (http://tinyurl.com/pupbook)
> * Pro Nagios 2.0 (http://tinyurl.com/pronagios)
> * Hardening Linux (http://tinyurl.com/hardeninglinux)
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.7 (Darwin)
> Comment: Using GnuPG with Mozilla -http://enigmail.mozdev.org/
Nigel Kersten
Kai, something is definitely broken in Lucid.
We're seeing all sorts of process exec issues.
Have you nailed this down at all?
--
nigel
Nigel Kersten
So Kai, we've been doing some experimenting here today, and have
reproduced these hangs in all the Debian Ruby1.8 packages back to
1.8.7.174-2.
1.8.7.174-1 we've been unable to reproduce it on though.
From the changelog I'm wondering if the first entry under 174-2 is
responsible. Note this was later removed after upstream integrated it.
ruby1.8 (1.8.7.174-2) unstable; urgency=medium
[ akira yamada ]
* Added debian/patches/090811_thread_and_select.dpatch: threads may hangup
when IO.select called from two or more threads.
* Added debian/patches/090812_finalizer_at_exit.dpatch: finalizers should be
run at exit (Closes: #534241)
* Added debian/patches/090812_class_clone_segv.dpatch: avoid segv when an
object cloned. (Closes: #533329)
* Added debian/patches/090812_eval_long_exp_segv.dpatch: fix segv when eval
a long expression. (Closes: #510561)
* Added debian/patches/090812_openssl_x509_warning.dpatch: suppress warning
from OpenSSL::X509::ExtensionFactory. (Closes: #489443)
[ Lucas Nussbaum ]
* Removed Fumitoshi UKAI <uk...@debian.or.jp> from Uploaders. Thanks a
lot for the past help! Closes: #541037
[ Daigo Moriwaki ]
* debian/fixshebang.sh: skip non-text files, which works around hanging of
sed on scanning gif images.
* Bumped up Standards-Version to 3.8.2.
--
nigel
Joel Ebel
without pthreads, unless ruby fixes the bug upstream which causes the
hang.
https://bugs.launchpad.net/ubuntu/+source/ruby1.8/+bug/520715
Joel
> * Removed Fumitoshi UKAI <u...@debian.or.jp> from Uploaders. Thanks a
Joel Ebel
optionally chime in with any details you can provide on:
https://bugs.launchpad.net/ubuntu/+source/ruby1.8/+bug/520715
Thanks,
Joel
Nigel Kersten
unstable if the Ruby 1.8.7 p249 package is the problem.
Surely we have some people running Debian testing on the list? Seeing
any weird timeouts with execs?
> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To post to this group, send email to puppet...@googlegroups.com.
> To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
>
>
--
nigel
Joshua Anderson
Here's my attempt:
josh@debian:~$ uname -a
Linux debian 2.6.18.8-x86_64-linode10 #1 SMP Tue Nov 10 16:29:17 UTC 2009 x86_64 GNU/Linux
josh@debian:~$ ruby -v
ruby 1.8.7 (2010-01-10 patchlevel 249) [x86_64-linux]
josh@debian:~$ puppet --version
0.25.4
josh@debian:~$ puppet --debug --trace test.pp
Could not retrieve virtual: Permission denied - /proc/xen/capabilities
Could not retrieve virtual: Permission denied - /proc/xen/capabilities
Could not retrieve virtual: Permission denied - /proc/xen/capabilities
Could not retrieve virtual: Permission denied - /proc/xen/capabilities
debug: Creating default schedules
debug: Failed to load library 'selinux' for feature 'selinux'
debug: Failed to load library 'ldap' for feature 'ldap'
debug: /File[/home/josh/.puppet/ssl]: Autorequiring File[/home/josh/.puppet]
debug: /File[/home/josh/.puppet/var/client_yaml]: Autorequiring File[/home/josh/.puppet/var]
debug: /File[/home/josh/.puppet/ssl/certificate_requests]: Autorequiring File[/home/josh/.puppet/ssl]
debug: /File[/home/josh/.puppet/var/log]: Autorequiring File[/home/josh/.puppet/var]
debug: /File[/home/josh/.puppet/var/lib]: Autorequiring File[/home/josh/.puppet/var]
debug: /File[/home/josh/.puppet/var/state]: Autorequiring File[/home/josh/.puppet/var]
debug: /File[/home/josh/.puppet/var/clientbucket]: Autorequiring File[/home/josh/.puppet/var]
debug: /File[/home/josh/.puppet/ssl/private_keys]: Autorequiring File[/home/josh/.puppet/ssl]
debug: /File[/home/josh/.puppet/ssl/certs]: Autorequiring File[/home/josh/.puppet/ssl]
debug: /File[/home/josh/.puppet/var]: Autorequiring File[/home/josh/.puppet]
debug: /File[/home/josh/.puppet/ssl/private]: Autorequiring File[/home/josh/.puppet/ssl]
debug: /File[/home/josh/.puppet/ssl/public_keys]: Autorequiring File[/home/josh/.puppet/ssl]
debug: /File[/home/josh/.puppet/var/state/graphs]: Autorequiring File[/home/josh/.puppet/var/state]
debug: /File[/home/josh/.puppet/var/facts]: Autorequiring File[/home/josh/.puppet/var]
debug: /File[/home/josh/.puppet/var/run]: Autorequiring File[/home/josh/.puppet/var]
debug: Finishing transaction 23715921915640 with 0 changes
info: Applying configuration version '1266113402'
debug: //testmodule/Exec[TEST-EXEC]: Changing returns
debug: //testmodule/Exec[TEST-EXEC]: 1 change(s)
debug: //testmodule/Exec[TEST-EXEC]: Executing '/usr/bin/touch /tmp/7777 >/tmp/123 2>&1'
debug: Executing '/usr/bin/touch /tmp/7777 >/tmp/123 2>&1'
notice: //testmodule/Exec[TEST-EXEC]/returns: executed successfully
debug: Finishing transaction 23715922698720 with 1 changes
josh@debian:~$
-Josh
Nigel Kersten
<joshua_...@mac.com> wrote:
> I'm afraid that I couldn't reproduce this on a Debian VM with Kai's example.
Joshua, I was just having issues reproducing it as well on a 4 core system.
As soon as I ran 3 instances of:
while : ; do openssl speed; done
to peg 3 of the cores, I could reproduce the same case as Kai initially posted.
exec {"TEST-EXEC" :
cwd => "/tmp/",
command =>"/usr/bin/touch /tmp/7777 >/tmp/123 2>&1",
timeout => 5,
logoutput=> on_failure
}
puppet -v ~/test_exec.pp
err: //Exec[TEST-EXEC]/returns: change from notrun to 0 failed:
Command exceeded timeout at /root/test_exec.pp:6
Nigel Kersten
> On Sat, Feb 13, 2010 at 6:13 PM, Joshua Anderson
> <joshua_...@mac.com> wrote:
>> I'm afraid that I couldn't reproduce this on a Debian VM with Kai's example.
>
> Joshua, I was just having issues reproducing it as well on a 4 core system.
>
> As soon as I ran 3 instances of:
>
> while : ; do openssl speed; done
>
> to peg 3 of the cores, I could reproduce the same case as Kai initially posted.
>
> exec {"TEST-EXEC" :
> cwd => "/tmp/",
> command =>"/usr/bin/touch /tmp/7777 >/tmp/123 2>&1",
> timeout => 5,
> logoutput=> on_failure
> }
>
> puppet -v ~/test_exec.pp
> err: //Exec[TEST-EXEC]/returns: change from notrun to 0 failed:
> Command exceeded timeout at /root/test_exec.pp:6
ahah. cc'ing puppet-dev as they may have suggestions for the best way forward.
So this isn't a Puppet bug at all.
It looks to be a bug in the Ruby Timeout module that seems to be
triggered when most of your cores are busy.
I can reliably reproduce it by firing up openssl speed (n-1) times
where n is the number of cores and then using the Timeout module.
#!/usr/bin/ruby1.8
#
%x{/usr/bin/touch /tmp/7777}
puts "executed without timeout ok"
puts "executing with timeout"
require 'timeout'
status = Timeout::timeout(5) {
%x{/usr/bin/touch /tmp/7777}
}
puts "executed with timeout ok"
which will produce something like:
root@testhost:~# ps auxww|grep [o]penssl
root 22337 99.6 0.0 14616 2028 pts/6 R 15:04 2:52 openssl speed
root 22338 99.9 0.0 14616 2028 pts/6 R 15:04 2:49 openssl speed
root 22339 100 0.0 14616 2024 pts/6 R 15:04 2:49 openssl speed
root@testhost:~# ~/tickle_ruby.rb
executed without timeout ok
executing with timeout
/usr/lib/ruby/1.8/timeout.rb:60: execution expired (Timeout::Error)
from /root/tickle_ruby.rb:11
root@testhost:~# killall openssl
[1] Terminated openssl speed &>/dev/null
[2]- Terminated openssl speed &>/dev/null
[3]+ Terminated openssl speed &>/dev/null
root@testhost:~# ~/tickle_ruby.rb
executed without timeout ok
executing with timeout
executed with timeout ok
This looks to be a problem for all the ruby 1.8.7 p249 variants I've
tried, apart from the MacPorts one, which looks to have a bunch of
patches around these issues.
--
nigel
kai.steverding
for the help. We will see, what the ruby core people can do.
Kai
> ...
>
> Erfahren Sie mehr »
Silviu Paragina
on a single core machine (amd sempron if it matters..).
root@testclient:~# ruby -v
ruby 1.8.7 (2010-01-10 patchlevel 249) [i486-linux]
root@testclient:~# puppetd -V
0.25.1
Any other solutions to this and/or if it has been fixed in ubuntu?
Silviu
Nigel Kersten
> Sorry for reviving such an old thread :"> I've actually hit this today on a
> single core machine (amd sempron if it matters..).
>
>
> root@testclient:~# ruby -v
> ruby 1.8.7 (2010-01-10 patchlevel 249) [i486-linux]
> root@testclient:~# puppetd -V
> 0.25.1
>
>
> Any other solutions to this and/or if it has been fixed in ubuntu?
It hasn't been. I've been kept busy with other things unfortunately.
Feel free to update the Ubuntu bug, or check with ruby upstream to see
if anyone has reproduced it.
The Timeout module is all pure Ruby, so it shouldn't be too hard to
work out what is triggering this.
fwiw, the MacPorts version of p249 doesn't have this issue, but it has
a bunch of patches applied to it.
http://svn.macports.org/repository/macports/trunk/dports/lang/ruby/files/
I was planning to work through them and see what patch was resolving
this, but am kind of strapped this week.
Nigel Kersten
chased it up on behalf of Debian and has just gotten a patch uploaded
to Debian that resolves this problem.
http://redmine.ruby-lang.org/issues/show/2739
--
nigel
James Turnbull
Hash: SHA1
On 13/03/10 7:34 AM, Nigel Kersten wrote:
> For those people following along at home on this issue, Lucas Nussbaum
> chased it up on behalf of Debian and has just gotten a patch uploaded
> to Debian that resolves this problem.
>
> http://redmine.ruby-lang.org/issues/show/2739
Kudos to Lucas! Are you guys going to backport the patch or wait for
the Ruby guys to release?
Regards
James Turnbull
- --
Author of:
* Pro Linux System Administration (http://tinyurl.com/linuxadmin)
* Pulling Strings with Puppet (http://tinyurl.com/pupbook)
* Pro Nagios 2.0 (http://tinyurl.com/pronagios)
* Hardening Linux (http://tinyurl.com/hardeninglinux)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQEVAwUBS5qqUSFa/lDkFHAyAQLZfAf+K7Ts/Dzich9wP+hHWAF5pfcTupHkDELH
dJfgu06J/UVGwuMSTa0la6uxBt7c2i+LJ6G9jmpiTmEb7UUe3fgWiEEjfjcPlnL1
JzBWAdrrEFhzLnP5Lakedjdcb94ZVSboxgxb/fZsU6FlLdyLazazM/FeiDltTO3A
E/nssPvdcTre0GOhZBDRxw+A1v32HZy8fIGgaY5Eb9G5/RbouyT9ypNa+hQXtoxA
YYlMnIpfTfzgVetHfxFXhyLp5bwKthaGA++6U7bOM6UkB5kdwywq5IDteEeNauh+
rbNHBKNlirNV3wV0Wm7te/tSN0KQmP65gX1p8fz0VCA9wRAlV8QOAg==
=wkdB
-----END PGP SIGNATURE-----
Nigel Kersten
<ja...@lovedthanlost.net> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 13/03/10 7:34 AM, Nigel Kersten wrote:
>> For those people following along at home on this issue, Lucas Nussbaum
>> chased it up on behalf of Debian and has just gotten a patch uploaded
>> to Debian that resolves this problem.
>>
>> http://redmine.ruby-lang.org/issues/show/2739
>
> Kudos to Lucas! Are you guys going to backport the patch or wait for
> the Ruby guys to release?
He's already uploaded it to Debian, so it will flow down to Ubuntu as
well, I imagine before Lucid is released....
>
> Regards
>
> James Turnbull
>
> - --
> Author of:
> * Pro Linux System Administration (http://tinyurl.com/linuxadmin)
> * Pulling Strings with Puppet (http://tinyurl.com/pupbook)
> * Pro Nagios 2.0 (http://tinyurl.com/pronagios)
> * Hardening Linux (http://tinyurl.com/hardeninglinux)
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.7 (Darwin)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iQEVAwUBS5qqUSFa/lDkFHAyAQLZfAf+K7Ts/Dzich9wP+hHWAF5pfcTupHkDELH
> dJfgu06J/UVGwuMSTa0la6uxBt7c2i+LJ6G9jmpiTmEb7UUe3fgWiEEjfjcPlnL1
> JzBWAdrrEFhzLnP5Lakedjdcb94ZVSboxgxb/fZsU6FlLdyLazazM/FeiDltTO3A
> E/nssPvdcTre0GOhZBDRxw+A1v32HZy8fIGgaY5Eb9G5/RbouyT9ypNa+hQXtoxA
> YYlMnIpfTfzgVetHfxFXhyLp5bwKthaGA++6U7bOM6UkB5kdwywq5IDteEeNauh+
> rbNHBKNlirNV3wV0Wm7te/tSN0KQmP65gX1p8fz0VCA9wRAlV8QOAg==
> =wkdB
> -----END PGP SIGNATURE-----
>
Silviu Paragina
> For those people following along at home on this issue, Lucas Nussbaum
> chased it up on behalf of Debian and has just gotten a patch uploaded
> to Debian that resolves this problem.
>
> http://redmine.ruby-lang.org/issues/show/2739
>
>
Silviu