Could not retrieve local facts: can't create Thread: Resource temporarily unavailable

[Helmut Schneider]

Hi,

after update of puppet-agent on Ubuntu I get the following error:

Debug: Facter: Query is operatingsystem
Debug: Facter: Checking query tokens operatingsystem
Debug: Facter: List of resolvable facts:
[#<Facter::SearchedFact:0x000055f89f7fb640 @name="operatingsystem",
@fact_class=Facts::Linux::Os::Name, @filter_tokens=[],
@user_query="operatingsystem", @type=:legacy, @file=nil>]
Debug: Facter: Resolving fact in parallel
Debug: Facter: Loading external facts
Debug: Facter: User query is: []
Debug: Facter: Resolving fact in parallel
Error: Could not retrieve local facts: can't create Thread: Resource
temporarily unavailable
Error: Failed to apply catalog: Could not retrieve local facts: can't
create Thread: Resource temporarily unavailable
Debug: Resolving service 'report' using Puppet::HTTP::Resolver::Settings

helmut@h2873756:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04.5 LTS
Release: 18.04
Codename: bionic
helmut@h2873756:~$ puppet -V
7.0.0
helmut@h2873756:~$

Any ideas?

Thank you!

[Bogdan Irimie]

Hi Helmut,

Facter 4 resolves each fact on a separate thread, this can lead to a high number of threads being created. Can you please provide the output of `cat /proc/sys/kernel/threads-max` on the system where you see this issue? Does this happen at every run or sporadically?

Please try to run facter from the cli and let me know if you have any issues.

You can disable parallel fact resolution with facter.conf (https://puppet.com/docs/facter/3.11/configuring_facter.html) by adding 

global : {

    sequential          : true

}

Best regards,

Bogdan Irimie

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/rqneve%24slb%241%40ciao.gmane.io.

[bogdan...@puppet.com]

Managed to reproduce the problem by limiting the number of threads to 200 on Ubuntu 18.04

echo 200 > /proc/sys/kernel/threads-max

root@raw-drawl:~# facter

Traceback (most recent call last):

14: from /opt/puppetlabs/puppet/bin/facter:10:in `<main>'

13: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/facter/framework/cli/cli_launcher.rb:23:in `start'

12: from /opt/puppetlabs/puppet/lib/ruby/gems/2.7.0/gems/thor-1.0.1/lib/thor/base.rb:485:in `start'

11: from /opt/puppetlabs/puppet/lib/ruby/gems/2.7.0/gems/thor-1.0.1/lib/thor.rb:392:in `dispatch'

10: from /opt/puppetlabs/puppet/lib/ruby/gems/2.7.0/gems/thor-1.0.1/lib/thor/invocation.rb:127:in `invoke_command'

9: from /opt/puppetlabs/puppet/lib/ruby/gems/2.7.0/gems/thor-1.0.1/lib/thor/command.rb:27:in `run'

8: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/facter/framework/cli/cli.rb:114:in `query'

7: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/facter.rb:390:in `to_user_output'

6: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/facter/framework/core/fact_manager.rb:25:in `resolve_facts'

5: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/facter/framework/core/fact/internal/internal_fact_manager.rb:15:in `resolve_facts'

4: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/facter/framework/core/fact/internal/internal_fact_manager.rb:62:in `start_threads'

3: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/facter/framework/core/fact/internal/internal_fact_manager.rb:62:in `map'

2: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/facter/framework/core/fact/internal/internal_fact_manager.rb:63:in `block in start_threads'

1: from /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/facter/framework/core/fact/internal/internal_fact_manager.rb:63:in `new'

/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/facter/framework/core/fact/internal/internal_fact_manager.rb:63:in `initialize': can't create Thread: Resource

[Helmut Schneider]

Am 08.12.2020 um 13:15 schrieb Bogdan Irimie:

> Facter 4 resolves each fact on a separate thread, this can lead to a
> high number of threads being created. Can you please provide the output
> of `cat /proc/sys/kernel/threads-max` on the system where you see this
> issue? Does this happen at every run or sporadically?

helmut@h2873756:~$ cat /proc/sys/kernel/threads-max
3090194
helmut@h2873756:~$

Every time I run puppet.

> Please try to run facter from the cli and let me know if you have any
> issues.

helmut@h2873756:~$ facter

`initialize': can't create Thread: Resource temporarily unavailable
(ThreadError)
helmut@h2873756:~$

> You can disable parallel fact resolution with facter.conf
> (https://puppet.com/docs/facter/3.11/configuring_facter.html

> <https://puppet.com/docs/facter/3.11/configuring_facter.html>) by adding
>
>
> global : {
>
> sequential: true
>
> }

That helps, yes.

The system is a VM on OpenVZ. Other machines on Hyper-V and VMWare are
not affected.

[Helmut Schneider]

Am 08.12.2020 um 14:45 schrieb Helmut Schneider:
> Am 08.12.2020 um 13:15 schrieb Bogdan Irimie:
>
>> Facter 4 resolves each fact on a separate thread, this can lead to a
>> high number of threads being created. Can you please provide the output
>> of `cat /proc/sys/kernel/threads-max` on the system where you see this
>> issue? Does this happen at every run or sporadically?

> The system is a VM on OpenVZ. Other machines on Hyper-V and VMWare are
> not affected.

facter seems to have an issue anyways:

helmut@h2873756:~$ facter ipaddress

helmut@h2873756:~$ facter networking.primary

helmut@h2873756:~$ facter networking | grep primary
primary => "aPublicIP"
helmut@h2873756:~$ facter networking.interfaces

helmut@h2873756:~$ facter networking | grep -A100 interfaces
interfaces => {
lo => {
bindings => [
{
ip => "10.0.124.17",
ip6 => "fe80::cc7a:d052:3220:e442",
mac => "",
mtu => 1500,
netmask => "255.255.255.255",
netmask6 => "ffff:ffff:ffff:ffff::",
network => "10.0.124.17",
network6 => "fe80::",
scope6 => "link"
},
venet0 => {
bindings => [
{
address => "127.0.0.1",
netmask => "255.255.255.255",
network => "127.0.0.1"
},
{
address => "aPublicIP",
netmask => "255.255.255.255",
network => "aPublicIP"
}
],
bindings6 => [
{
address => "::2",
netmask => "ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff",
network => "::2"
},
{
address => "aPublicIP",
netmask => "ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff",
network => "aPublicIP"
}
],
ip => "aPublicIP",
ip6 => "::2",
mac => "",
mtu => 1500,
netmask => "255.255.255.255",
netmask6 => "ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff",
network => "aPublicIP",
network6 => "::2",
scope6 => "compat,global"
}
},
primary => "aPublicIP"
}
helmut@h2873756:~$ facter networking.hostname
h2873756
helmut@h2873756:~$

helmut@h2873756:~$ facter -v
4.0.46
helmut@h2873756:~$