template compilation performance

[Matt]

I'm running into an issue where I have hundreds of configuration files generated from templates, which they are apache vhosts for various sites. I'm running into a problem where it can take up to 10 minutes to build them and sometimes clients get server execution expired. I'm also on 2.6 puppet, is there anything I can do to boost performance?

[Gary Larizza]

There's a bit of information you'll need to provide for help:

How many masters?  Specs on master?  Running your master under Webrick or something else (nginx, unicorn, apache/passenger)?  Agents running puppet daemonized?  How long does it take to run `puppet master --compile <agent certname>` on the master?  

Let us know what we're looking at and we can provide some help :)

On Wed, May 16, 2012 at 8:46 AM, Matt <mjb...@gmail.com> wrote:

I'm running into an issue where I have hundreds of configuration files generated from templates, which they are apache vhosts for various sites. I'm running into a problem where it can take up to 10 minutes to build them and sometimes clients get server execution expired. I'm also on 2.6 puppet, is there anything I can do to boost performance?

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/n6OoUxF1bDYJ.
To post to this group, send email to puppet...@googlegroups.com.
To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

--

Gary Larizza

Professional Services Engineer

Puppet Labs

[Matt]

Yeah my bad, lots of things going on right now, here is the information.

I currently have 3 puppet masters with 4 vCPU, 8GB ram, running apache with passenger. Puppet agents are run from mcollective with a concurrency of 20. 

Output from the puppet master compile command

notice: Compiled catalog for xxxxxxxxx in environment production in 143.74 seconds

This is what I got when running time infront of the compile command

real 3m28.472s

user 1m13.430s

sys 0m4.031s

On Wednesday, May 16, 2012 11:49:54 AM UTC-4, Gary Larizza wrote:

There's a bit of information you'll need to provide for help:

How many masters?  Specs on master?  Running your master under Webrick or something else (nginx, unicorn, apache/passenger)?  Agents running puppet daemonized?  How long does it take to run `puppet master --compile <agent certname>` on the master?  

Let us know what we're looking at and we can provide some help :)

On Wed, May 16, 2012 at 8:46 AM, Matt <mjb...@gmail.com> wrote:

I'm running into an issue where I have hundreds of configuration files generated from templates, which they are apache vhosts for various sites. I'm running into a problem where it can take up to 10 minutes to build them and sometimes clients get server execution expired. I'm also on 2.6 puppet, is there anything I can do to boost performance?

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/n6OoUxF1bDYJ.
To post to this group, send email to puppet...@googlegroups.com.

To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

[Trevor Vaughan]

Can you check your logs and see if you're getting a bell curve on
compile time, or are they always that long?

Trevor

>>> puppet-users...@googlegroups.com.

>>> For more options, visit this group at
>>> http://groups.google.com/group/puppet-users?hl=en.
>>
>>
>>
>>
>> --
>>
>> Gary Larizza
>> Professional Services Engineer
>> Puppet Labs
>>

> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/puppet-users/-/2hEh4WX4mHQJ.

>
> To post to this group, send email to puppet...@googlegroups.com.
> To unsubscribe from this group, send email to

> puppet-users...@googlegroups.com.

> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.



--

Trevor Vaughan
Vice President, Onyx Point, Inc
(410) 541-6699
tvau...@onyxpoint.com

-- This account not approved for unencrypted proprietary information --

[Matt]

I would say that it that is close to the average. Not all nodes have the same amount of templates, these are just the web servers that are having the long compile time.

>>> puppet-users+unsubscribe@googlegroups.com.

>>> For more options, visit this group at
>>> http://groups.google.com/group/puppet-users?hl=en.
>>
>>
>>
>>
>> --
>>
>> Gary Larizza
>> Professional Services Engineer
>> Puppet Labs
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/puppet-users/-/2hEh4WX4mHQJ.
>
> To post to this group, send email to puppet...@googlegroups.com.
> To unsubscribe from this group, send email to

> puppet-users+unsubscribe@googlegroups.com.