r10k database backend

[Patrick Lesher]

Has anyone thought about or tried to have r10k run against a mysql backend to populate at least the information the puppetfile gives?

Updating each puppetfile for a large number of “Environments” can be burdonsome when we have to update the version on a number of modules.  We already have all of our hiera data in a mysql backend and would like to try to simplify some processes.

Thanks,
Patrick

[Garrett Honeycutt]

> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-users...@googlegroups.com
> <mailto:puppet-users...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/4733e1e7-734b-4a13-8bea-c8942a988c40%40googlegroups.com
> <https://groups.google.com/d/msgid/puppet-users/4733e1e7-734b-4a13-8bea-c8942a988c40%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

Hi Patrick,

Without modifying r10k, you could generate the Puppetfile from data in
your database. Keeping the information in a file under revision control
buys you all the nice things related to using a VCS. What is the
motivation of using a database backend? I'm curious as to the problems
you encounter using the normal method of storing your Puppetfile in a
git repo.

Best regards,
-g

--
Garrett Honeycutt
@learnpuppet
Puppet Training with LearnPuppet.com
Mobile: +1.206.414.8658

[Clayton O'Neill]

On Monday, January 25, 2016 at 6:48:07 PM UTC-5, Patrick Lesher wrote:

Has anyone thought about or tried to have r10k run against a mysql backend to populate at least the information the puppetfile gives?

Updating each puppetfile for a large number of “Environments” can be burdonsome when we have to update the version on a number of modules.  We already have all of our hiera data in a mysql backend and would like to try to simplify some processes.

The Puppetfile is Ruby, so in theory you can just access the database directly form the Puppetfile and loop over the results creating new module entries.  We do something similar to read a list of modules from a YAML file with something like this:

# This loads the YAML file that Jenkins maintains of the latest commits
# approved through Gerrit.
PUPPET_DIR = ENV['PUPPET_DIR'] || '/etc/puppet'
PUPPETFILE_YAML = File.join(PUPPET_DIR, 'Puppetfile.yaml')
if File.readable?(PUPPETFILE_YAML)
  require 'yaml'
  data = YAML.load_file(PUPPETFILE_YAML)
  data['modules'].each_pair do |modulename, moduledata|
    mod modulename, :git => moduledata['git'], :commit => moduledata['ref']
  end
end

[Patrick Lesher]

Thank you!  That's a really interesting idea that I will look into.

Garrett,  the current process of keeping the Puppet file in git works fine, the problem is that after a release we might have to update 5-15 places in 10+  environments.  I have some of it scripted but would like to have a central place to keep the versions for reporting and ease of updating.

Thanks!

--Patrick

[Corey Osman]

Yea, I don't see why you couldn't reach out to a redis instance and grab the data and then generate the puppetfile.   Curious if there is a way to get the current branch that r10k would be using at time of execution?

[Christopher Wood]

On Mon, Jan 25, 2016 at 03:25:03PM -0800, Patrick Lesher wrote:
> Has anyone thought about or tried to have r10k run against a mysql backend
> to populate at least the information the puppetfile gives?

Not quite the same problem or solution, but we use basemodulepath to keep a common set of modules in a single environment. This also helps r10k do less work since it's not rolling out copies of the same module to multiple environments.

basemodulepath = /etc/puppet/r10k/common/modules

In practice this means that some environments drift over time, but only in modules with large amounts of configuration changes (MTA, httpd) and the environments which are principally about those modules' services.

> Updating each puppetfile for a large number of “Environments” can be
> burdonsome when we have to update the version on a number of modules.  We
> already have all of our hiera data in a mysql backend and would like to
> try to simplify some processes.
>
> Thanks,
> Patrick
>

> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to [1]puppet-users...@googlegroups.com.

> To view this discussion on the web visit

> [2]https://groups.google.com/d/msgid/puppet-users/4733e1e7-734b-4a13-8bea-c8942a988c40%40googlegroups.com.
> For more options, visit [3]https://groups.google.com/d/optout.
>
> References
>
> Visible links
> 1. mailto:puppet-users...@googlegroups.com
> 2. https://groups.google.com/d/msgid/puppet-users/4733e1e7-734b-4a13-8bea-c8942a988c40%40googlegroups.com?utm_medium=email&utm_source=footer
> 3. https://groups.google.com/d/optout