Storeconfigs connection pool problem

[Daniel Kerwin]

Hi list,

i just enabled storeconfigs and cannot use puppetrun on more than 5
hosts. When i try 6+ i get the error message:

puppetmasterd[16209]: could not obtain a database connection within 5
seconds. The max pool size is currently 5; consider increasing it.

My Mysql setup allows a lot more connections (500). Any suggestions?

Thanks,

Daniel

[Joe McDonagh]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Do you have the proper rubygem for mysql installed? I think on
deb/ubuntu it's something like libmysql-ruby1.8.

- --
Joe McDonagh
AIM: YoosingYoonickz
IRC: joe-mac on freenode
L'ennui est contre-r�volutionnaire

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAkt05xoACgkQRkBieEaRmubRKACglFzvJCGtN1rChmbuZqfJKN7/
yYUAn1/KdvVfD8DURsk+lzc+/VAxb1fG
=lPum
-----END PGP SIGNATURE-----

[Jason Koppe]

I'm having the same problem with the mysql gem installed and the gem appears to be in-use

 

jkoppe@indadm1 in ~ at 2010-06-08 23:58:59
--> gem list mysql

*** LOCAL GEMS ***

mysql (2.8.1)

jkoppe@indadm1 in ~ at 2010-06-08 23:59:00
--> cd /var/lib/puppet/log/

jkoppe@indadm1 in /var/lib/puppet/log at 2010-06-09 00:00:20
--> sudo cat rails.log
[sudo] password for jkoppe:
# Logfile created on Tue Jun 08 09:04:41 -0500 2010 by logger.rb/1.5.2.9

jkoppe@indadm1 in /var/lib/puppet/log at 2010-06-09 00:00:25
-->

jkoppe@indadm1 in /var/lib/puppet/log at 2010-06-09 00:08:03
--> sudo lsof | grep 30870 | grep mysql
puppetmas 30870       root  mem       REG        8,3    2083728    1218568 /usr/lib/libmysqlclient.so.15.0.0
puppetmas 30870       root  mem       REG        8,3     148035    1251352 /usr/lib/ruby/gems/1.8/gems/mysql-2.8.1/lib/mysql_api.so
puppetmas 30870       root    7u     IPv4   45902539                   TCP indadm1.indeed.net:37895->indadm1.indeed.net:mysql (ESTABLISHED)
puppetmas 30870       root    9u     IPv4   45898759                   TCP indadm1.indeed.net:37874->indadm1.indeed.net:mysql (ESTABLISHED)
puppetmas 30870       root   11u     IPv4   45902499                   TCP indadm1.indeed.net:37893->indadm1.indeed.net:mysql (ESTABLISHED)
puppetmas 30870       root   19u     IPv4   45902527                   TCP indadm1.indeed.net:37894->indadm1.indeed.net:mysql (ESTABLISHED)
puppetmas 30870       root   22u     IPv4   45902542                   TCP indadm1.indeed.net:37896->indadm1.indeed.net:mysql (ESTABLISHED)

jkoppe@indadm1 in /var/lib/puppet/log at 2010-06-09 00:08:06
-->

 

On Fri, Feb 12, 2010 at 12:28 AM, Joe McDonagh <joseph.e...@gmail.com> wrote:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Daniel Kerwin wrote:
> Hi list,
>
> i just enabled storeconfigs and cannot use puppetrun on more than 5
> hosts. When i try 6+ i get the error message:
>
> puppetmasterd[16209]: could not obtain a database connection within 5
> seconds.  The max pool size is currently 5; consider increasing it.
>
> My Mysql setup allows a lot more connections (500). Any suggestions?
>
> Thanks,
>
> Daniel
>

Do you have the proper rubygem for mysql installed? I think on
deb/ubuntu it's something like libmysql-ruby1.8.

- --
Joe McDonagh
AIM: YoosingYoonickz
IRC: joe-mac on freenode

L'ennui est contre-révolutionnaire

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAkt05xoACgkQRkBieEaRmubRKACglFzvJCGtN1rChmbuZqfJKN7/
yYUAn1/KdvVfD8DURsk+lzc+/VAxb1fG
=lPum
-----END PGP SIGNATURE-----

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to puppet...@googlegroups.com.
To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

--
Jason Koppe
Jason.Rob...@gmail.com
Cell (210) 445-8242

[Dan Carley]

On 9 June 2010 06:09, Jason Koppe <jason.rob...@gmail.com> wrote:

On Fri, Feb 12, 2010 at 12:28 AM, Joe McDonagh <joseph.e...@gmail.com> wrote:

Daniel Kerwin wrote:
> Hi list,
>
> i just enabled storeconfigs and cannot use puppetrun on more than 5
> hosts. When i try 6+ i get the error message:
>
> puppetmasterd[16209]: could not obtain a database connection within 5
> seconds.  The max pool size is currently 5; consider increasing it.
>
> My Mysql setup allows a lot more connections (500). Any suggestions?

Do you have the proper rubygem for mysql installed? I think on
deb/ubuntu it's something like libmysql-ruby1.8.

I'm having the same problem with the mysql gem installed and the gem appears to be in-use

The gem version is a bit of a red herring, so long are using the gem and not Ruby's own connector. As the warning suggests, ActiveRecord's pool option simply needs raising if it's to be put under any strain at all. There is a bug/feature ticket as #2568 [0] which exposes this option to Puppet's configuration. But it's not due to be delivered until 2.6/rowlf.

I've been slipstreaming the patch into 0.25 myself because storeconfigs grinds to a halt without it in my environment. Given the simplicity of the diff and that it seems to be affecting a number of people I think it probably warrants being brought forward to 0.25.

[0] http://projects.puppetlabs.com/issues/2568

[Jason Koppe]

Queuing support from ActiveMQ doesn't even make a difference for me -- I thought the purpose of adding the queuing support was to queue the data in puppetqd's memory and let that flush out to the database when it could.  I'm still seeing this error from the puppetmaster (not puppetqd).

err: could not obtain a database connection within 5 seconds.  The max pool size is currently 5; consider increasing it.

Am I misunderstanding how puppetmasterd+puppetqd is supposed to function?

--

You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to puppet...@googlegroups.com.
To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

[Christopher Johnston]

I have the same issue as well, seems puppetqd uses sqlite for caching....?

[Jason Koppe]

I'm not sure, I can't seem to find README.queuing or much documentation about how it's supposed to work.  I'll checkout the code this weekend if there aren't responses here.

[Christopher Johnston]

Any findings?  From what I can tell it defaults to sqlite leaving me with an issue of my hosts bombing out when more then 6 runs are happening.  I would prefer to have it use mysql for a cache then let the queuing daemon fwd its data to a central mysql server.

[Jason Koppe]

I didn't read anything over last weekend but I'm anxious to find out how to fix this for my environment. 

What is the OS of your puppetmaster?

I've got CentOS and there's another thread about CentOS5 leaking file descriptors.... which seems to be related.

[Thomas A. McGonagle]

Hey Jason,
I ran into this problem myself. I switched to SQLite3, for my stored
configs. No problems with it, and then I run PuppetDashboard (with a
mysql backend as a secondary stored configs source just in case.
Please let me know if you have any questions or if there is something
I can help with.
-Tom

Thomas A. McGonagle
t...@dataero.com
Google Voice: 617-229-5185

On Tue, Jun 22, 2010 at 1:59 AM, Jason Koppe