Agent thundering herd solutions?

176 views
Skip to first unread message

Ryan Bowlby

unread,
Apr 27, 2012, 6:54:54 PM4/27/12
to Puppet Users
I'm hoping to have all the puppet agents run on a 20 minute interval
with 480sec on either side. I don't want to launch the agent from cron
in order to achieve a level of randomness. Is there a splay/random
option that can be included in the sysconfig file or config file?

I want to continue running the agent as a daemon.

Thanks,
Ryan

Nan Liu

unread,
Apr 27, 2012, 7:18:17 PM4/27/12
to puppet...@googlegroups.com
The puppet options related to this are runinterval, splay, and splaylimit.

Nan

Len Rugen

unread,
Apr 28, 2012, 8:00:52 PM4/28/12
to puppet...@googlegroups.com
When we started with splay, over time we found that puppet runs would flock together.  If there were network or system load issues causing multiple puppet runs to be slow, they would seem to clear at the same time, then be on the same schedule from then on.  As other clients would hit a slow run, they would eventually find themselves in the flock with the others.  The foreman chart of number of runs over past few minutes showed this clearly.  Our solution was to cron a weekly puppet agent restart on all clients, then they re-splay and life is good.

When we put puppet in crontab, we had puppet manage that and used fqnd-rand to randomize the minute value.


Nan

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to puppet...@googlegroups.com.
To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.


Reply all
Reply to author
Forward
0 new messages