Single Enterprise Puppet Master to support 2 Control Repos

[vishc...@gmail.com]

I have a question about configuring a single Enterprise Puppet Master to support 2 control repos.

There are 2 puppet teams who manage different puppet modules in the organisation that I currently work for.

1. Infrastructure Team(a.k.a Production Support Team) 

    - manage the installation/configuration of nagios, graylog, fail2bahn, supervisor, rkhunter, clamav etc using Puppet/R10K for all projects/applications in all environments.

    - This team also does the production env monitoring.

2. Project Development Team                             

    - The project/application development team with a DevOps resource have produced Puppet modules to install the required                                          

          software stack components(nginx, node, redis, tomcat, java, mongodb) for the specific application  

          deploy it along with the configuration files using Puppet/R10K.

There is one PuppetMaster VM for each environment that is shared by both the teams.

Both the teams  have their own Control-Repos (project containing the puppet manifests/scripts) in 2 different Git Repositories.

This is downloaded manually on the PuppetMaster using R10K during deployment time.

Both the Control-Repos have different roles/profiles configured for each node(VM) in the different environments. (standard puppet roles/profiles pattern)

The distributed Application is hosted on 7 VMs in each environment (Test, UAT, Production) 

I was wondering if there is a standard way to run both team's modules at the same time in Puppet Enterprise 4.x?

Puppet Agent is expected to run once every half an hour in each environment.

[Martin Alfke]

Hi,

> On 28 Nov 2016, at 13:32, vishc...@gmail.com wrote:
>
> I have a question about configuring a single Enterprise Puppet Master to support 2 control repos.
>
> There are 2 puppet teams who manage different puppet modules in the organisation that I currently work for.
>
> 1. Infrastructure Team(a.k.a Production Support Team)
> - manage the installation/configuration of nagios, graylog, fail2bahn, supervisor, rkhunter, clamav etc using Puppet/R10K for all projects/applications in all environments.
> - This team also does the production env monitoring.
>
> 2. Project Development Team
> - The project/application development team with a DevOps resource have produced Puppet modules to install the required
> software stack components(nginx, node, redis, tomcat, java, mongodb) for the specific application
> deploy it along with the configuration files using Puppet/R10K.

You can have multiple control-repos within R10k.
You only need to let r10k prefix the environments:

Team 1:
control-repo (branches)
|- production
|- test

Will result in:
<environmentpath>/
team1_production
team1_test

Team 2:
control-repo (branches)
|- development
|- staging

Will result in
<environmentpath>/
team2_development
team2_staging

see: https://docs.puppet.com/pe/latest/r10k_custom.html#sources

>
> There is one PuppetMaster VM for each environment that is shared by both the teams.
> Both the teams have their own Control-Repos (project containing the puppet manifests/scripts) in 2 different Git Repositories.
> This is downloaded manually on the PuppetMaster using R10K during deployment time.
> Both the Control-Repos have different roles/profiles configured for each node(VM) in the different environments. (standard puppet roles/profiles pattern)
> The distributed Application is hosted on 7 VMs in each environment (Test, UAT, Production)
>
> I was wondering if there is a standard way to run both team's modules at the same time in Puppet Enterprise 4.x?
> Puppet Agent is expected to run once every half an hour in each environment.
>
>
>

> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/93e0ff8b-2f52-48ca-8aba-b8fc133ada44%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Rob Nelson]

This will work but I would caution against it. Only recently has per environment segregation been implemented and there are still some issues present (I believe most fixes showed up in 4.8.0 but not sure). You don't want the same module at two different versions for each group being mixed and matched improperly. But, it's a judgement call if that's more worrisome than an extra PE master.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/F51B202C-F961-4BFE-AAE6-3C66A00CBAE2%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Rob Nelson

[Thomas Müller]

Am Montag, 28. November 2016 19:06:55 UTC+1 schrieb Rob Nelson:

This will work but I would caution against it. Only recently has per environment segregation been implemented and there are still some issues present (I believe most fixes showed up in 4.8.0 but not sure). You don't want the same module at two different versions for each group being mixed and matched improperly. But, it's a judgement call if that's more worrisome than an extra PE master.

I can confirm that the issue with different versions of the same module in different environments with native ruby types/providers is a real problem. Just encountered it with the elasticsearch module which we wanted to upgrade. I utterly failed because new types were added and some types changed.

If multiple independent teams are working on the same master you will likely hit this issue faster than with only one team.

https://docs.puppet.com/puppet/latest/reference/environment_isolation.html

- Thomas

[vishc...@gmail.com]

thanks for the useful inputs,

   A solution that seems to work for us is to use a prefix as mentioned below with R10k.

   Puppet Master Enterprise console seems to support a feature called "Node Groups". It is possible to include a node into 2 node groups, one node group deploys the standard of infrastructure modules(environment=production) and the other node group points to "agent specified environment", which has a different name (r10k prefix).

Luckily, for us all there are no overlapping modules between the teams.

[Martin Alfke]

Hi,

> On 29 Nov 2016, at 13:31, vishc...@gmail.com wrote:
>
> thanks for the useful inputs,
> A solution that seems to work for us is to use a prefix as mentioned below with R10k.
> Puppet Master Enterprise console seems to support a feature called "Node Groups". It is possible to include a node into 2 node groups, one node group deploys the standard of infrastructure modules(environment=production) and the other node group points to "agent specified environment", which has a different name (r10k prefix).

Node Groups are fact based rulesets which can be used to include roles/profiles/classes.
And yes: a node can be a member of multiple node groups as long as they are not environment groups.

>
> Luckily, for us all there are no overlapping modules between the teams.
>
>
>
>
>
> On Monday, 28 November 2016 12:32:53 UTC, vishc...@gmail.com wrote:
> I have a question about configuring a single Enterprise Puppet Master to support 2 control repos.
>
>
>
> There are 2 puppet teams who manage different puppet modules in the organisation that I currently work for.
>
>
>
> 1. Infrastructure Team(a.k.a Production Support Team)
>
> - manage the installation/configuration of nagios, graylog, fail2bahn, supervisor, rkhunter, clamav etc using Puppet/R10K for all projects/applications in all environments.
>
> - This team also does the production env monitoring.
>
>
>
> 2. Project Development Team
>
> - The project/application development team with a DevOps resource have produced Puppet modules to install the required
>
> software stack components(nginx, node, redis, tomcat, java, mongodb) for the specific application
>
> deploy it along with the configuration files using Puppet/R10K.
>
>
>
> There is one PuppetMaster VM for each environment that is shared by both the teams.
>
> Both the teams have their own Control-Repos (project containing the puppet manifests/scripts) in 2 different Git Repositories.
>
> This is downloaded manually on the PuppetMaster using R10K during deployment time.
>
> Both the Control-Repos have different roles/profiles configured for each node(VM) in the different environments. (standard puppet roles/profiles pattern)
>
> The distributed Application is hosted on 7 VMs in each environment (Test, UAT, Production)
>
>
>
> I was wondering if there is a standard way to run both team's modules at the same time in Puppet Enterprise 4.x?
>
> Puppet Agent is expected to run once every half an hour in each environment.
>
>
>
>
>

> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/d5b8f971-f218-496f-b53c-23612fc8a7f7%40googlegroups.com.

[Eric Sorenson]

Hi Thomas, have you tried the `puppet generate type` workflow described on that doc to avoid the problems with environment bleed-through? You said you failed w/the elasticsearch module and I'm wondering if that is because the `generate` stuff is not working, or whether there's another part of the problem.

[Thomas Müller]

Am Samstag, 3. Dezember 2016 18:40:58 UTC+1 schrieb Eric Sorenson:

Hi Thomas, have you tried the `puppet generate type` workflow described on that doc to avoid the problems with environment bleed-through? You said you failed w/the elasticsearch module and I'm wondering if that is because the `generate` stuff is not working, or whether there's another part of the problem.

 

i've tested with Puppet 4.8 and  `puppet generate type` - this works . Unfortunatly we're still preparing the  Puppet 4 upgrade.

- Thomas