r10K - Unable to understand the benefits of r10k dynamic environments..

[vishc...@gmail.com]

We use Puppet, hiera, r10k with a number of forge modules, roles and profiles patterns to perform an automated deployment of a Java/NodeJS Application.

I find r10k to be very useful in fetching all dependent modules from a Puppetfile. However I am confused with the benefits of the r10k dynamic environments.

We have 8 environments, so we have created 8 branches in Git. The important feature is that all these 8 environments are identical except for heira properties.

Whenever we make a change in one of the puppet manifests(roles or profiles), we manually have to merge(sometimes cherry-pick)  these changes from 1 branch to the other, when we are ready to do a new deployment in another environment.

Managing code in 8 branches is appearing to be a time-consuming and  error-prone process to me. 

I am thinking of moving away from r10k dynamic environments usage and have only 1 branch (master). 

Whenever we deploy anything to Production env, we just tag the appropriate commit with a Git Tag.

I am thinking when the business application code that the development team are writing does not have a Git branch for every environment, why should R10K mandate that we create a branch for each environment?

I was wondering if any body has benefited from using R10k dynamic environments for automated deployment of an Application and can share their experience.

[Matthew Kennedy]

There are several ways dynamic environments can be helpful here is one I've used to great effect.

Have three primary branches production, staging and development.  (or release, snapshot and master.  The branches themselves don't matter)

Ensure that all production nodes use the production, staging use staging and everything else uses development.

The general flow of things is that changes are pushed into development then development is merged to staging and finally staging is merged to production. At any time higher branches can be merged down, ie production can always merge to staging.  In a perfect world changes would always follow the normal flow and there won't be any commits in higher branches that are now in all lower ones.  However people do things like push to staging to fix a staging issue its a good idea to have merges to production auto merge to development thereby closing the loop. This means that production is always the oldest branch and all changes eventually make it to it.

Scheduling wise the merge from development to staging could be automated and very frequent, we used every day in the evening. Staging to production occurred once a week and as needed. Pushes the development happened all the time.

Intrinsic to this method though is that the branches are NOT strictly equal to your internal 'environments'.  Many environments can use the development dynamic puppet environment for example. You'll need to ensure you can set a nodes puppet environment.  I know with enterprise there is some enforcement of node environment to puppet environment but it is possible to break this. Because of this we stopped referring to nodes being in an environment they were in a 'tier' and mapped tiers to puppet environments. 

Some benefits to this method.  

Because the different branches are more like point in time versions of your configuration you can do spot checks and/or regression checks against other puppet environments. ie run against the staging puppet environment on a production node or validate a change cutting a branch, pushing your change there and running puppet against that branches puppet environment (cannot understate how useful this is).

Visibility to what changes will occur on production.  We had a weekly 'go through the changes from staging' meeting with Ops, Engineering and rel event dev teams.

Relatively simple. 

Some downsides.

If you are not disciplined and the branches diverge merges become long painful events.  It is vital that the cycle is kept tight else you're going to end up in a situation where the are merge conflicts and more importantly lack of trust that the process works well. Do not allow weeks of work to get 'stuck' at staging for example.

Hints

If possible work on feature/topic branches.  Always branch off of production (unless you cant), because it is the oldest branch it should always be possible to merge it back to any other primary branches without bring along commits from newer branches.  Regularly merge production back into so you don't get out-of-sync and try not to let a branch linger for too long.  Then as you validate things merge to development, then merge to staging and finally merge to production.  At that point all the commits from the branch are in all three primary branches and you should be able to delete the feature/topic branch. There are some issues still like half baked things getting into production with the staging to production merge but this avoids a lot of other issues. 

Alternates

If protecting production from half finished work in staging branch is vital it is possible to invert the flow and work with a purely feature/topic based work flow. The staging and development branches are taken from production then all features start as a branch off of production. Merges from feature branches go to development, then staging and finally production but there are never merges from staging to production or development to staging. Because of this staging and development branches can diverge (have commits that are not part of active features/topic branches and not in production) which is bad. To solve this staging and development branches should be periodically deleted and re-cut from production. Then all active features/topic branches are merged back into them. Ideally this should be automated. In addition all pushes to development, staging and production should be denied (even for ops and admins)... only merge requests should be permitted. 

Anyways sorry for the novel

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/34a74b0b-c4bc-465c-b521-19bdfbedc3f3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Angel L. Mateo]

El 26/05/17 a las 01:34, vishc...@gmail.com escribió:

> We use Puppet, hiera, r10k with a number of forge modules, roles and
> profiles patterns to perform an automated deployment of a Java/NodeJS
> Application.
>
> I find r10k to be very useful in fetching all dependent modules from a
> Puppetfile. However I am confused with the benefits of the r10k dynamic
> environments.
> We have 8 environments, so we have created 8 branches in Git. The
> important feature is that all these 8 environments are identical except
> for heira properties.
>

I think this could be the "problem" (not a real problem). If your 8
environments are the same except hiera properties, then I think you
don't need 8 environments, but just one.

You could create a fact indicating which of these 8 environments the
node belongs to and modify your hiera hierarchy according to this new
fact. This way you could use just one environment for all your nodes.

And you could use dynamic environments when you need a dynamic one or
when the difference between environments is more than just hiera.

--
Angel L. Mateo Martínez
Sección de Telemática
Área de Tecnologías de la Información
y las Comunicaciones Aplicadas (ATICA)
http://www.um.es/atica
Tfo: 868889150
Fax: 868888337