err: Could not run Puppet configuration client: execution expired

[Steven]

I am new to using puppet and having trouble when updating an agent
from a master which has a lot of files.
Usually the updates work, but when there are a lot of files I get this
error when running:

sudo puppetd --test --server puppet.example.local --trace --debug
>>>
debug: Failed to load library 'rubygems' for feature 'rubygems'
debug: Failed to load library 'selinux' for feature 'selinux'
debug: Failed to load library 'ldap' for feature 'ldap'
debug: Puppet::Type::User::ProviderLdap: feature ldap is missing
debug: Puppet::Type::User::ProviderUser_role_add: file rolemod does
not exist
debug: Puppet::Type::User::ProviderDirectoryservice: file /usr/bin/
dscl does not exist
debug: Puppet::Type::User::ProviderPw: file pw does not exist
debug: Puppet::Type::File::ProviderMicrosoft_windows: feature
microsoft_windows is missing
debug: /File[/var/lib/puppet/state/graphs]: Autorequiring File[/var/
lib/puppet/state]
debug: /File[/var/lib/puppet/lib]: Autorequiring File[/var/lib/puppet]
debug: /File[/var/lib/puppet/ssl/private]: Autorequiring File[/var/lib/
puppet/ssl]
debug: /File[/var/lib/puppet/ssl/certificate_requests]: Autorequiring
File[/var/lib/puppet/ssl]
debug: /File[/var/lib/puppet/ssl/certs/pavml1.example.local.pem]:
Autorequiring File[/var/lib/puppet/ssl/certs]
debug: /File[/var/lib/puppet/ssl/certs/ca.pem]: Autorequiring File[/
var/lib/puppet/ssl/certs]
debug: /File[/var/lib/puppet/ssl/public_keys/
pavml1.example.local.pem]: Autorequiring File[/var/lib/puppet/ssl/
public_keys]
debug: /File[/var/lib/puppet/state/state.yaml]: Autorequiring File[/
var/lib/puppet/state]
debug: /File[/var/lib/puppet/ssl/public_keys]: Autorequiring File[/var/
lib/puppet/ssl]
debug: /File[/var/lib/puppet/ssl/certs]: Autorequiring File[/var/lib/
puppet/ssl]
debug: /File[/var/lib/puppet/state/classes.txt]: Autorequiring File[/
var/lib/puppet/state]
debug: /File[/var/lib/puppet/facts]: Autorequiring File[/var/lib/
puppet]
debug: /File[/var/lib/puppet/client_data]: Autorequiring File[/var/lib/
puppet]
debug: /File[/var/lib/puppet/ssl/private_keys/
pavml1.example.local.pem]: Autorequiring File[/var/lib/puppet/ssl/
private_keys]
debug: /File[/var/lib/puppet/client_yaml]: Autorequiring File[/var/lib/
puppet]
debug: /File[/var/lib/puppet/ssl/private_keys]: Autorequiring File[/
var/lib/puppet/ssl]
debug: /File[/var/lib/puppet/ssl/crl.pem]: Autorequiring File[/var/lib/
puppet/ssl]
debug: /File[/var/lib/puppet/clientbucket]: Autorequiring File[/var/
lib/puppet]
debug: /File[/etc/puppet/puppet.conf]: Autorequiring File[/etc/puppet]
debug: /File[/var/lib/puppet/state]: Autorequiring File[/var/lib/
puppet]
debug: /File[/var/lib/puppet/ssl]: Autorequiring File[/var/lib/puppet]
debug: Finishing transaction 70094315588060
debug: /File[/var/lib/puppet/ssl/private]: Autorequiring File[/var/lib/
puppet/ssl]
debug: /File[/var/lib/puppet/ssl]: Autorequiring File[/var/lib/puppet]
debug: /File[/var/lib/puppet/ssl/public_keys]: Autorequiring File[/var/
lib/puppet/ssl]
debug: /File[/var/lib/puppet/ssl/crl.pem]: Autorequiring File[/var/lib/
puppet/ssl]
debug: /File[/var/lib/puppet/lib]: Autorequiring File[/var/lib/puppet]
debug: /File[/var/lib/puppet/ssl/certs/ca.pem]: Autorequiring File[/
var/lib/puppet/ssl/certs]
debug: /File[/var/lib/puppet/state]: Autorequiring File[/var/lib/
puppet]
debug: /File[/var/lib/puppet/ssl/certificate_requests]: Autorequiring
File[/var/lib/puppet/ssl]
debug: /File[/var/lib/puppet/ssl/private_keys]: Autorequiring File[/
var/lib/puppet/ssl]
debug: /File[/var/lib/puppet/facts]: Autorequiring File[/var/lib/
puppet]
debug: /File[/var/lib/puppet/ssl/certs]: Autorequiring File[/var/lib/
puppet/ssl]
debug: /File[/var/lib/puppet/ssl/public_keys/
pavml1.example.local.pem]: Autorequiring File[/var/lib/puppet/ssl/
public_keys]
debug: /File[/var/lib/puppet/ssl/certs/pavml1.example.local.pem]:
Autorequiring File[/var/lib/puppet/ssl/certs]
debug: /File[/var/lib/puppet/ssl/private_keys/
pavml1.example.local.pem]: Autorequiring File[/var/lib/puppet/ssl/
private_keys]
debug: Finishing transaction 70094315840980
debug: Using cached certificate for ca
debug: Using cached certificate for pavml1.example.local
debug: Finishing transaction 70094315412980
debug: Loaded state in 0.04 seconds
debug: Executing '/etc/puppet/etckeeper-commit-pre'
debug: Using cached certificate for ca
debug: Using cached certificate for pavml1.example.local
debug: Using cached certificate_revocation_list for ca
debug: catalog supports formats: b64_zlib_yaml dot marshal pson raw
yaml; using pson
info: Caching catalog for pavml1.example.local
debug: Puppet::Type::Service::ProviderGentoo: file /sbin/rc-update
does not exist
debug: Puppet::Type::Service::ProviderRunit: file /usr/bin/sv does not
exist
debug: Puppet::Type::Service::ProviderDaemontools: file /usr/bin/svc
does not exist
debug: Puppet::Type::Service::ProviderRedhat: file /sbin/service does
not exist
debug: Puppet::Type::Service::ProviderLaunchd: file /bin/launchctl
does not exist
debug: Puppet::Type::Package::ProviderFreebsd: file /usr/sbin/
pkg_delete does not exist
debug: Puppet::Type::Package::ProviderFink: file /sw/bin/fink does not
exist
debug: Puppet::Type::Package::ProviderRug: file /usr/bin/rug does not
exist
debug: Puppet::Type::Package::ProviderOpenbsd: file pkg_delete does
not exist
debug: Puppet::Type::Package::ProviderSunfreeware: file pkg-get does
not exist
debug: Puppet::Type::Package::ProviderPortage: file /usr/bin/emerge
does not exist
debug: Puppet::Type::Package::ProviderAptitude: file /usr/bin/aptitude
does not exist
debug: Puppet::Type::Package::ProviderRpm: file rpm does not exist
debug: Puppet::Type::Package::ProviderYum: file rpm does not exist
debug: Puppet::Type::Package::ProviderPkg: file /usr/bin/pkg does not
exist
debug: Puppet::Type::Package::ProviderGem: file gem does not exist
debug: Puppet::Type::Package::ProviderAptrpm: file rpm does not exist
debug: Puppet::Type::Package::ProviderUp2date: file /usr/sbin/up2date-
nox does not exist
debug: Puppet::Type::Package::ProviderPortupgrade: file /usr/local/
sbin/pkg_deinstall does not exist
debug: Puppet::Type::Package::ProviderAix: file /usr/bin/lslpp does
not exist
debug: Puppet::Type::Package::ProviderZypper: file /usr/bin/zypper
does not exist
debug: Puppet::Type::Package::ProviderUrpmi: file rpm does not exist
debug: Puppet::Type::Package::ProviderNim: file /usr/sbin/nimclient
does not exist
debug: Puppet::Type::Package::ProviderSun: file /usr/sbin/pkgrm does
not exist
debug: Puppet::Type::Package::ProviderHpux: file /usr/sbin/swinstall
does not exist
debug: Puppet::Type::Package::ProviderPorts: file /usr/local/sbin/
pkg_deinstall does not exist
debug: Creating default schedules
debug: Loaded state in 0.04 secondseyecon
debug: Prefetching apt resources for package
debug: Executing '/usr/bin/dpkg-query -W --showformat '${Status} $
{Package} ${Version}\n''
debug: Puppet::Type::Package::ProviderApt: Executing '/usr/bin/dpkg-
query -W --showformat '${Status} ${Package} ${Version}\n''
debug: /Stage[main]/Ant_1_8_2/File[/usr/ant/latest]/require: requires
File[/usr/ant/apache-ant-1.8.2]
debug: /Stage[main]/Fire/File[/usr/fire_deploy/artifacts/ant/configs/
pavml1]/require: requires File[/usr/fire_deploy/ant/configs]
debug: /Stage[main]/Ntp/File[/etc/ntp.conf]/notify: subscribes to
Service[ntpd]
debug: /Stage[main]/Ant_1_8_2/File[/usr/ant/antenv.sh]/require:
requires File[/usr/ant/apache-ant-1.8.2]
debug: /Stage[main]/Jboss_4_2_2_ga/File[/usr/jboss/jboss-4.2.2.GA]/
require: requires File[/usr/jboss]
debug: /Stage[main]/Fire/File[/usr/fire_deploy/ant]/require: requires
File[/usr/fire_deploy]
debug: /Stage[main]/Fire/File[/usr/fire_deploy/ant/configs]/require:
requires File[/usr/fire_deploy/ant]
debug: /Stage[main]/Fire/Exec[/usr/ant/latest/bin/ant -f deploy.xml]/
require: requires File[/usr/fire_deploy]
debug: /Stage[main]/Fire/Exec[/usr/ant/latest/bin/ant -f deploy.xml]/
subscribe: subscribes to File[/usr/fire_deploy/artifacts]
debug: /Stage[main]/Fire/Exec[/bin/sh /usr/ant/antenv.sh]/require:
requires File[/usr/ant]
debug: /Stage[main]/Ant_1_8_2/File[/usr/ant/apache-ant-1.8.2]/require:
requires File[/usr/ant]
debug: /Stage[main]/Ntp/Service[ntpd]/require: requires Package[ntp]
debug: /Stage[main]/Ant_1_8_2/File[/usr/ant/latest]: Autorequiring
File[/usr/ant]
debug: /Stage[main]/Ant_1_8_2/File[/usr/ant/antenv.sh]: Autorequiring
File[/usr/ant]
debug: /Stage[main]/Fire/File[/usr/fire_deploy/artifacts]:
Autorequiring File[/usr/fire_deploy]
info: Applying configuration version '1314075033'
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
debug: Storing state
debug: Stored state in 0.37 seconds
debug: Executing '/etc/puppet/etckeeper-commit-post'
/usr/lib/ruby/1.8/timeout.rb:64:in `rbuf_fill'
/usr/lib/ruby/1.8/net/protocol.rb:134:in `rbuf_fill'
/usr/lib/ruby/1.8/net/protocol.rb:116:in `readuntil'
/usr/lib/ruby/1.8/net/protocol.rb:126:in `readline'
/usr/lib/ruby/1.8/net/http.rb:2028:in `read_status_line'
/usr/lib/ruby/1.8/net/http.rb:2017:in `read_new'
/usr/lib/ruby/1.8/net/http.rb:1051:in `request'
/usr/lib/ruby/1.8/net/http.rb:1037:in `request'
/usr/lib/ruby/1.8/net/http.rb:543:in `start'
/usr/lib/ruby/1.8/net/http.rb:1035:in `request'
/usr/lib/ruby/1.8/net/http.rb:772:in `get'
/usr/lib/ruby/1.8/puppet/indirector/rest.rb:77:in `search'
/usr/lib/ruby/1.8/puppet/indirector/indirection.rb:238:in `search'
/usr/lib/ruby/1.8/puppet/indirector.rb:58:in `search'
/usr/lib/ruby/1.8/puppet/type/file.rb:590:in `perform_recursion'
/usr/lib/ruby/1.8/puppet/type/file.rb:556:in `recurse_remote'
/usr/lib/ruby/1.8/puppet/type/file.rb:555:in `collect'
/usr/lib/ruby/1.8/puppet/type/file.rb:555:in `recurse_remote'
/usr/lib/ruby/1.8/puppet/type/file.rb:481:in `recurse'
/usr/lib/ruby/1.8/puppet/type/file.rb:376:in `eval_generate'
/usr/lib/ruby/1.8/puppet/transaction.rb:186:in `send'
/usr/lib/ruby/1.8/puppet/transaction.rb:186:in
`generate_additional_resources'
/usr/lib/ruby/1.8/puppet/transaction.rb:84:in `eval_generate'
/usr/lib/ruby/1.8/puppet/transaction.rb:104:in
`eval_children_and_apply_resource'
/usr/lib/ruby/1.8/puppet/transaction.rb:92:in `eval_resource'
/usr/lib/ruby/1.8/puppet/transaction.rb:143:in `evaluate'
/usr/lib/ruby/1.8/puppet/util.rb:414:in `thinmark'
/usr/lib/ruby/1.8/benchmark.rb:308:in `realtime'
/usr/lib/ruby/1.8/puppet/util.rb:413:in `thinmark'
/usr/lib/ruby/1.8/puppet/transaction.rb:142:in `evaluate'
/usr/lib/ruby/1.8/puppet/transaction.rb:135:in `each'
/usr/lib/ruby/1.8/puppet/transaction.rb:135:in `evaluate'
/usr/lib/ruby/1.8/puppet/resource/catalog.rb:144:in `apply'
/usr/lib/ruby/1.8/puppet/configurer.rb:152:in `run'
/usr/lib/ruby/1.8/puppet/util.rb:175:in `benchmark'
/usr/lib/ruby/1.8/benchmark.rb:308:in `realtime'
/usr/lib/ruby/1.8/puppet/util.rb:174:in `benchmark'
/usr/lib/ruby/1.8/puppet/configurer.rb:151:in `run'
/usr/lib/ruby/1.8/puppet/agent.rb:39:in `run'
/usr/lib/ruby/1.8/puppet/agent/locker.rb:21:in `lock'
/usr/lib/ruby/1.8/puppet/agent.rb:39:in `run'
/usr/lib/ruby/1.8/sync.rb:230:in `synchronize'
/usr/lib/ruby/1.8/puppet/agent.rb:39:in `run'
/usr/lib/ruby/1.8/puppet/agent.rb:103:in `with_client'
/usr/lib/ruby/1.8/puppet/agent.rb:37:in `run'
/usr/lib/ruby/1.8/puppet/application.rb:171:in `call'
/usr/lib/ruby/1.8/puppet/application.rb:171:in `controlled_run'
/usr/lib/ruby/1.8/puppet/agent.rb:35:in `run'
/usr/lib/ruby/1.8/puppet/application/agent.rb:114:in `onetime'
/usr/lib/ruby/1.8/puppet/application/agent.rb:88:in `run_command'
/usr/lib/ruby/1.8/puppet/application.rb:300:in `run'
/usr/lib/ruby/1.8/puppet/application.rb:397:in `exit_on_fail'
/usr/lib/ruby/1.8/puppet/application.rb:300:in `run'
/usr/sbin/puppetd:4
err: Could not run Puppet configuration client: execution expired

And there is no created: /usr/fire_deploy after the run. But other
files do get created as required.
I am using Ubuntu 10.07 with Puppet 2.6.1 on both master and agent.

How should I go about fixing this?

Thanks,
Steven

[Denmat]

Hi,

Puppet running the standard webrick server is not very good at file serving.

http://docs.puppetlabs.com/#tuning-and-scaling
That link provides some recommended options.

Cheers,
Den

> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To post to this group, send email to puppet...@googlegroups.com.
> To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
>

[jblaine]

Sadly, Steven, I ran into this same exact problem in Puppet.  Your solution won't be found in

changing to a different server setup (webrick to Passenger, for instance).  I've tested both

with a large number of files and they both fall on their face.  Puppet simply is not capable

of serving large trees of files the way it is designed.  The only solution I have found is to

make native OS packages to distribute and install (instead of simply transferring the

tree of files).

Coming from a CFengine backround (and perhaps going right back to it, the verdict is still

out), this was a major disappointment to find while evaluating Puppet.

It takes CFengine 2.1.x only 1-2 minutes to transfer 14,000 files totalling 147MB.  Puppet took

many hours and memory growth was enormous.