Puppet / scalability

[Alex Harvey]

Hi list,

I am looking at configuration management tool options.

I have a large fleet (> 3,000 hosts) and highly heterogeneous
(RHEL3-5, CentOS, 5RH7, Solaris 10 LDOMs/zones, Solaris 8-9, AIX 5.3 &
6.1 LPARs, HP-UX & Tru64 + Windows). We care mainly about RHEL and
new versions of Solaris & AIX but ability to compile on older versions
is an advantage. Probably, the Windows will be managed by SCCM.

I have read that Puppet could have scalability problems to a site as
large as ours. To keep this simple, I'd like feedback on whether that
is likely to be true for us.

A rep from CFengine has told me that ours would be the largest Puppet
site in the world (I think that's not true). Could someone confirm?

General feedback also welcome.

Kind regards,
Alex Harvey

[Christopher Johnston]

3k? I know of larger but can't comment on who they are, but they are close to 8-10k nodes AFAIK if not more by now.

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to puppet...@googlegroups.com.
To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

[Nigel Kersten]

There are at least a couple of deployments that are more like 30k hosts than 3k.... 

It's absolutely not true.

 

General feedback also welcome.

Kind regards,
Alex Harvey

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to puppet...@googlegroups.com.
To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

--
Nigel Kersten | http://puppetlabs.com | @nigelkersten

Schedule Meetings at: http://tungle.me/nigelkersten

[Eric Shamow]

Nigel,

Don't we have one or two an order of magnitude larger than that?

http://indico.cern.ch/getFile.py/access?contribId=29&sessionId=3&resId=3&materialId=slides&confId=160737 (warning: large PDF)

I think they're looking at 300K or so.  And I believe we have a few more at that size as well.

-Eric

-- 

Eric Shamow

Professional Services

http://puppetlabs.com/

(c)631.871.6441

[Jeff McCune]

On Sun, May 13, 2012 at 11:53 PM, Alex Harvey <alexh...@gmail.com> wrote:

A rep from CFengine has told me that ours would be the largest Puppet
site in the world (I think that's not true).  Could someone confirm?

Here's a recording of a short talk I gave about some performance testing I did last summer.  These are the numbers I gathered from a real world production deployment of tens of thousands of nodes running against multiple Puppet Masters:

http://www.youtube.com/watch?v=wjXrJjKymM4

The CFengine rep is certainly not speaking the truth on this matter.  I hope they're simply ignorant and not willingly lying.

Cheers,

--

Jeff McCune

[James Turnbull]

Hi Alex

> I have read that Puppet could have scalability problems to a site as
> large as ours. To keep this simple, I'd like feedback on whether that
> is likely to be true for us.

On the actual scaling question... Puppet (2.6.x+) is easy to scale as it
is just HTTP(S) traffic. You can front-end Puppet with a load balancer
and then just cluster master nodes to scale to your needs in the same
way you can grow any HTTP service.

Regards

James Turnbull

--
James Turnbull
Puppet Labs
1-503-734-8571
To schedule a meeting with me: http://tungle.me/jamtur01

[Mister IT Guru]

Hi Alex,

Just a quick point,

Glad to see Team Puppet out in force to show you the light! That CF rep, needs a swift kick in the …. well, I don't want to advocate violence here, I think he can do with being added to your junk mail list.

3000 hosts? No problem! You have to serve one session per client every 30 mins, and your puppet master doesn't process much unless you've changed something. That's about a hundred hits a minute, so use apache instead of webrick, and your good to go!

CERN as I understand it, are the guys trying to "recreate the big bang". Not simulate it, like NASA, or do some complex maths research about it… actually do it. Now, if anyone on the planet needs to be 100% sure that their gazillion or so hosts are all playing nice, it's them. They use Puppet.

Next question!?

[Shawn]

Wanted to point out I manage over 3000 x86 tablets with puppet. This doesn't include production servers.
Sent via BlackBerry

[Steve Traylen]

On Monday, 14 May 2012 18:38:29 UTC+2, Eric Shamow wrote:

Nigel,

Don't we have one or two an order of magnitude larger than that?

http://indico.cern.ch/getFile.py/access?contribId=29&sessionId=3&resId=3&materialId=slides&confId=160737 (warning: large PDF)

I think they're looking at 300K or so.  And I believe we have a few more at that size as well.

  We are currently along way from that with a single master and around 200 nodes

  while we learn puppet and migrate away from quattor. 

  The 300k figures are a couple of years away, we will have a respectable figure

  sometime this year we would hope.

Steve (CERN IT)

 

To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

--
Nigel Kersten | http://puppetlabs.com | @nigelkersten

Schedule Meetings at: http://tungle.me/nigelkersten

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to puppet...@googlegroups.com.

To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.

[Erik Dalén]

On 14 May 2012 18:35, Nigel Kersten <ni...@puppetlabs.com> wrote:
>
>
> On Sun, May 13, 2012 at 11:53 PM, Alex Harvey <alexh...@gmail.com> wrote:
>>
>> Hi list,
>>
>> I am looking at configuration management tool options.
>>
>> I have a large fleet (> 3,000 hosts) and highly heterogeneous
>> (RHEL3-5, CentOS, 5RH7, Solaris 10 LDOMs/zones, Solaris 8-9, AIX 5.3 &
>> 6.1 LPARs, HP-UX & Tru64 + Windows).  We care mainly about RHEL and
>> new versions of Solaris & AIX but ability to compile on older versions
>> is an advantage.  Probably, the Windows will be managed by SCCM.
>>
>> I have read that Puppet could have scalability problems to a site as
>> large as ours.  To keep this simple, I'd like feedback on whether that
>> is likely to be true for us.
>>
>> A rep from CFengine has told me that ours would be the largest Puppet
>> site in the world (I think that's not true).  Could someone confirm?
>
>
> There are at least a couple of deployments that are more like 30k hosts than
> 3k....
>
> It's absolutely not true.
>

However, storeconfigs does have some scalability issues, so I guess
that they aren't using that feature with 30k hosts?

(soon there is of course the puppetdb that should replace it and be
more scalable afaik)

--
Erik Dalén

[Alex Harvey]

Thanks kindly to all for the thoughtful responses.  I'll be looking closely at the idea of using load balanced Puppet servers.

I wonder if anyone has any thoughts on the other problem I'm told I'll probably encounter, namely the difficulty in compiling Ruby and other packages in AIX, HPUX & earlier versions of Solaris? 

[Bill Proud]

On Wednesday, May 16, 2012 4:20:23 PM UTC+2, Alex Harvey wrote:

Thanks kindly to all for the thoughtful responses.  I'll be looking closely at the idea of using load balanced Puppet servers.

I wonder if anyone has any thoughts on the other problem I'm told I'll probably encounter, namely the difficulty in compiling Ruby and other packages in AIX, HPUX & earlier versions of Solaris? 

I'm running puppet with Solaris and I haven't had to compile anything. 

[Nigel Kersten]

Let me firm that up a bit :)

PuppetDB *does* replace the existing storeconfigs, and *is* much more scalable :)

[Alex Harvey]

On 15 May 2012 22:10, Steve Traylen <steve....@cern.ch> wrote:

  We are currently along way from that with a single master and around 200 nodes

  while we learn puppet and migrate away from quattor. 

  The 300k figures are a couple of years away, we will have a respectable figure

  sometime this year we would hope.

Steve (CERN IT)

I have had a careful look at
http://projects.puppetlabs.com/projects/1/wiki/Whos_Using_Puppet

This list is, in fairness, consistent with what the CFEngine rep told me.  If there are sites out there with >3K or even >10K nodes, but not listed at Puppet's website, it is reasonable to suppose that the CFEngine rep doesn't know about them. :)

Is it possible that this list needs updating?  Maybe large sites aren't as keen to share their stories with the world?

From the list, I also get the feeling that sites using Puppet to manage their infrastructure tend to be Linux sites.  Google appears to be using Puppet to manage "all recent Mac OS X and Linux desktops, laptops, servers in the corporate infrastructure".  They don't use it "in production", whatever that means.  For those taking it beyond Linux and Mac OS X, I see Solaris mentioned a bit.  Are Sun/Oracle still using it for their Solaris kit?  I don't see any mention of AIX at all.  I believe I have heard the same rumour of the large company at 10K nodes that uses Solaris, AIX and Linux.

So, I am no longer doubting that Puppet can scale to a site as large as ours - especially given what James Turnbull wrote above - but I'd like to ensure that I get my facts right.

Is it possible that the large ~ 10K site is one of the first of its kind in the world - especially to use it for AIX?  What about the 30K site?  A CFEngine document I downloaded claims, "Puppet has recently retracted claims of running 35000 servers at a major bank."  Is this also true?

Best wishes.