How do you keep the forge modules you use up to date (and keep your sanity)

[Karsten Heymann]

Hi there,

once again we are trying to update our set of about 70 external forge
modules our puppet codebase uses but we always end up in dependency
hell, especially when trying to update central modules like
puppetlabs/stdlib or puppetlabs/apt or my special friend
puppetlabs/concat. There are always not that well maintained modules
that have something like puppetlabs-apt < 3.0.0 in their metadata.yaml
that makes upgrading these modules to current versions extremely
annoying. How do you handle this? Do you try to keep your modules up
to date and do you care for the version limits in the modules you use.
So far we often just did puppet module upgrade --force
--ignore-dependencies and usually it works without problem, but that's
not really the best solution for this problem. And maintaining
internal forks of all problematic puppet modules with fixed
dependencies would be a lot of work. Any hints would be appreciated.

Best regards
Karsten

[Bart-Jan Vrielink]

Hi,

I share your pain. Too much time I waste on figuring out what set of dependencies will work. Something that may help is Voxpupuli's ra10ke gem (https://github.com/voxpupuli/ra10ke). This adds a few helpful rake tasks to work with dependencies.

https://voxpupuli.org/plugins/#r10k lists a few other plugins to add to your workflow that may work for you.

-- 
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAL017hCvp4dV%2BRZy%2B5V%2Bt9zW42j3ffscxQB5uzN-%3Da1nj313Hg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Karsten Heymann]

Am Di., 9. Juli 2019 um 10:12 Uhr schrieb Bart-Jan Vrielink
<bar...@vrielink.net>:
> I share your pain.

Thank you :)

> https://github.com/voxpupuli/ra10ke
> https://voxpupuli.org/plugins/#r10k

Thank you for the links. I will check them out, maybe the make the
upgrade task a bit easier.

Best regards
Karsten

[Martin Alfke]

Hi,

we never use the puppet module tool.

Instead we mirror upstream modules on an internal git server (including tags) and reference module, git url and tag in a control-repository Puppetfile.

When we want to upgrade modules we create a branch and veriffy that everything still works as expected.

We sometimes even use the octocatalog-diff tool to verify catalogs build with old and new module versions.

hth,

Martin

--

You received this message because you are subscribed to the Google Groups "Puppet Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAL017hAj0-7R_VQUb9%2B4q%3D3eW27%2Bi185HsuNWTXXgLj%3DUen1dw%40mail.gmail.com.

[Trevor Vaughan]

I agree with the approach proposed by Martin.

Forge for discovery, Git mirrors, pinning, and testing for daily use. This also makes it really easy to mesh in with your own internal module development and add privacy controls across the board without standing a new service.

Trevor

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/29402CD0-6E43-4334-842E-791BF2FCA093%40getmailspring.com.

For more options, visit https://groups.google.com/d/optout.

--

Trevor Vaughan
Vice President, Onyx Point, Inc

(410) 541-6699 x788

-- This account not approved for unencrypted proprietary information --

[Luke Bigum]

On Wednesday, 10 July 2019 18:39:44 UTC+1, Martin Alfke wrote:

Hi,

we never use the puppet module tool.

Instead we mirror upstream modules on an internal git server (including tags) and reference module, git url and tag in a control-repository Puppetfile.

When we want to upgrade modules we create a branch and veriffy that everything still works as expected.

We sometimes even use the octocatalog-diff tool to verify catalogs build with old and new module versions.

hth,

Martin

We do the same as well - we mirror into an internal Git Lab, and use r10k to Git clone the upstream modules onto Puppet Masters like they were our own internal ones.  This also helps with our speed; usually when one of our engineers finds a problem in an upstream module that's not fixed in the latest commits, and they want the fix *now*.  So we patch it internally immediately then submit that patch upstream, where more often than not it sits for 1-2 weeks waiting for someone to review and merge :-)

You still have to pay attention to dependencies and the release notes - sometimes we've done big version jumps and missed some sort of functional change that's bitten us (but that would happen with Forge modules anyway).  There's only a handful of modules we pull directly from the Forge, and those are modules that don't have a one-to-one mapping of Git repo to Forge module (such as R.I.'s Choria agent modules).

-Luke