r10k, git and .gitignore

[Robert]

Hello List,

I finally managed to migrate our Puppet Master from the standard, simple setup to a bit more advanced configuration with r10k and a Bitbucket server.

I converted all our modules into git repositories (one module per repo), stored them in Bitbucket and configured r10k to deploy the whole environment. It works and it's cool!

Now there is just one question mark left*:

- in some modules, under files, I store some bigger binaries (like apache-maven.tgz or a jdk-1.8.tar.gz) for which I'd like to use .gitignore files in order to leave these out of the repository, since they would consume storage space and are easy to download again so they don't need to be tracked. How could these files still be included in the files directory after the r10k run?

Regards

Rp

* but I'm sure there will be some more later ;)

[Gavin Williams]

Robert

Whilst not directly what you're trying to achieve, we do something similar to distribute Java etc by having a separate Puppet fileserver mount called 'software'. 

Then anything that needs to consume those files can just use that mount point... 

Cheers

Gavin 

[J.T. Conklin]

Robert <robp...@gmail.com> writes:
> Now there is just one question mark left*:
> - in some modules, under files, I store some bigger binaries (like
> apache-maven.tgz or a jdk-1.8.tar.gz) for which I'd like to use .
> gitignore files in order to leave these out of the repository, since
> they would consume storage space and are easy to download again so
> they don't need to be tracked. How could these files still be included
> in the files directory after the r10k run?

At my day job we had a few modules that did the same. We addressed
this by moving the tarballs out of .../files, creating *.rpm and *.deb
packages with fpm (https://github.com/jordansissel/fpm), hosting those
packages in our local package manager (we use JFrog's Artifactory for
development, so it was convenient and available for IT use), and then
using puppet's package resource to manage package installation.

--jtc

[Anderson Mills]

Since v2.4.0, r10k allows data to be directed into a specific :install_path from a git repo in the Puppetfile.

See https://github.com/puppetlabs/r10k/blob/master/doc/puppetfile.mkd#per-item-install-path

If you have a separate repo of your "files", you could load that into a directory in the environment's root directory.

By using the control-branch tracking --

see https://github.com/puppetlabs/r10k/blob/master/doc/puppetfile.mkd#control-repo-branch-tracking

-- you could put a branch-per-environment into a "files" oriented git repo.

Hope that's another interesting solution,
Anderson

[Martijn]

You could use Git LFS (Large File Support). Instead of large files clogging up your Git repo, this will keep them separate, while still allowing you to work with them like they're Git objects. See https://www.atlassian.com/git/tutorials/git-lfs/ for a better explanation.

I've not used it with r10k though.

Regards, Martijn

Op woensdag 14 december 2016 15:20:24 UTC+1 schreef Robert:

[John Gelnaw]

We used Gavin's approach and created a "downloads" mount within Puppet, so Puppet still handles the file transfer, but it's from a different set of directories outside of the git repo(s).

Also, for anyone creating .deb / .rpm package files, if you aren't using 'fpm', you should be.

[Robert]

Hey guys,

I just wanted to give you a follow-up on this topic (maybe somebody will have the same question in the future).

All ideas have been helpful but fpm is awesome :)

I now build rpm's from software which do need dependencies, pre/post(un)install scripts etc., but if there's no need for that,  simply packaging a plain a .zip or .tgz with fpm is so simple and painful that even some other teams are happy with it and adopted the tool.

That way our git repo can be kept at a small size, which is useful if used in combination with r10k and dynamic environments - every environment would take up the whole size of the the checked out modules - so I'm glad I asked - and you replied.

Thanks,

rp

On Mon, Dec 19, 2016 at 5:24 PM, John Gelnaw <jge...@gmail.com> wrote:

We used Gavin's approach and created a "downloads" mount within Puppet, so Puppet still handles the file transfer, but it's from a different set of directories outside of the git repo(s).

Also, for anyone creating .deb / .rpm package files, if you aren't using 'fpm', you should be.

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/69b3bf0e-d8ab-46b3-881b-04cb595aa2ed%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.