how were your puppet 3->4 upgrades?

[Christopher Wood]

I am not seeing a large amount of blog entries complaining about this upgrade, how has that gone for you? Is there anything you found particularly painful? Would you have done anything different in retrospect?

I'm staring down a 3.7.2 -> 4.2.1 upgrade and after reading a number of docs the back-of-the-envelope optimal upgrade path is as follows. If any of you have commentary I am quite interested, otherwise I will try it and see what happens. I feel like this might be more time-consuming work but less brainpower effort than just yanking everything to 4.2.

1) rpms up to puppet 3.8

PostgreSQL 8.4 to 9.4
PuppetDB 2.2.2 to 2.3.7
(puppetdb-terminus from 2.2.2 to 2.3.7)
Puppet 3.7.2 to 3.8.2

2) enable the future parser

3) replace puppet 3.8.2, passenger 5 with puppetserver 1.1.1

4) replace puppetserver 1.1.1 with puppetserver 2.1.1

5) upgrade PuppetDB from 2.3.7 to 3.0.2

6) use the puppetlabs-puppet_agent forge module to upgrade agents to 4.2.1


My reading:

https://docs.puppetlabs.com/puppet/latest/reference/

https://docs.puppetlabs.com/puppet/4.2/reference/upgrade_major_pre.html
https://docs.puppetlabs.com/puppet/4.2/reference/upgrade_major_server.html
https://docs.puppetlabs.com/puppet/4.2/reference/upgrade_major_agent.html
https://docs.puppetlabs.com/puppet/4.2/reference/upgrade_major_post.html

https://forge.puppetlabs.com/puppetlabs/puppet_agent

https://docs.puppetlabs.com/puppetdb/2.3/upgrade.html

http://docs.puppetlabs.com/puppetdb/3.0/upgrade.html

http://www.postgresql.org/docs/9.4/static/upgrading.html

(Although I may dump/restore the data via puppetdb since that's the actual api to the data, we do not log in via PostgreSQL.)

[Wyatt Alt]

On 8/24/15 12:59 PM, Christopher Wood wrote:
> (Although I may dump/restore the data via puppetdb since that's the actual api to the data, we do not log in via PostgreSQL.)
>

Others can speak to the rest of the plan, but just to clarify in case
I'm understanding you right: the PuppetDB import/export tools are not
intended to work across different versions of PDB, so an export of 2.x
data will not be importable on 3.x. I can see that this is a hole in our
documentation, which I'll fix up in a minute.

Instead, the necessary data migrations will be handled by the upgrade
itself on the first startup of PuppetDB 3.0. Taking a pg_dump or
PuppetDB export for a backup prior to the upgrade is still a prudent
thing to do in case a downgrade is required, but that will only allow
you to restore from the version you came from (possibly earlier ones too
in the export case, but only by happenstance). For your use case I'd
recommend pg_dump over PuppetDB export since it'll run a lot faster, but
either one will get the job done.

Wyatt

[Christopher Wood]

I will be snapshotting the VM to lower the rollback effort there, but good point at that. Otherwise that saves me a bunch of preprod fiddling, thank you.

> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/55DB8009.6080300%40puppetlabs.com.
> For more options, visit https://groups.google.com/d/optout.

[Fabrice Bacchella]

> Le 24 août 2015 à 21:59, Christopher Wood <christop...@pobox.com> a écrit :
>
> I am not seeing a large amount of blog entries complaining about this upgrade, how has that gone for you? Is there anything you found particularly painful? Would you have done anything different in retrospect?
>
> I'm staring down a 3.7.2 -> 4.2.1 upgrade and after reading a number of docs the back-of-the-envelope optimal upgrade path is as follows. If any of you have commentary I am quite interested, otherwise I will try it and see what happens. I feel like this might be more time-consuming work but less brainpower effort than just yanking everything to 4.2.
>
> 1) rpms up to puppet 3.8
>
> PostgreSQL 8.4 to 9.4
> PuppetDB 2.2.2 to 2.3.7
> (puppetdb-terminus from 2.2.2 to 2.3.7)
> Puppet 3.7.2 to 3.8.2
>
> 2) enable the future parser
>
> 3) replace puppet 3.8.2, passenger 5 with puppetserver 1.1.1
>
> 4) replace puppetserver 1.1.1 with puppetserver 2.1.1
>
> 5) upgrade PuppetDB from 2.3.7 to 3.0.2
>
> 6) use the puppetlabs-puppet_agent forge module to upgrade agents to 4.2.1
>
>

My biggest problem: there isn't any working dashboard working with Puppet 4. The puppetdb APIv4 in puppet db 3.1 is not the same as the APIv4 in puppetdb 2, so all of them are now broken, even the one that thought they uses the v4 API.

I had a lot of trouble with facter, it's rewritten but not tested on older release, a few things were broken on Redhat 5 :
https://tickets.puppetlabs.com/browse/FACT-1152
https://tickets.puppetlabs.com/browse/FACT-1137

If you use mcollective there is a bug that prevent installation of both mcollective and puppet on the same server on Redhat 6 :
https://tickets.puppetlabs.com/browse/PUP-4970
and no one take care for this bug at puppetlabs.

All the path change /etc/puppet, /usr/bin to /etc/puppetlab, /opt/puppetlabs. Minor problems might arise from that, if you don't prepare that on puppet modules, other management scripts.

I hade no problem with embedded PKI, rsync -av /var/lib/puppet/ssl /etc/puppetlabs/puppet/ is enough to keep it.

Check your:
[master]
certname =...

in puppet.conf. Rules changes slightly and I had problem with that.

If you wrote you own modules, expect a lot of warning and broken modules becase of new variable typing.

Once the new server was up and running, I used this simple script on all the nodes:
[ -x /opt/puppetlabs/bin/puppet ] || (\
[ -x $(type -p facter) ] && \
rpm -e puppetlabs-release && \
rpm -Uvh http://.../puppetlabs/$(facter operatingsystemmajrelease)/PC1/x86_64/puppetlabs-release-pc1-0.9.2-1.el$(facter operatingsystemmajrelease).noarch.rpm && \
yum install -y puppet-agent && \
rsync -av /var/lib/puppet/ssl /etc/puppetlabs/puppet/ && \
/opt/puppetlabs/bin/puppet agent -v --onetime --no-daemonize --no-show_diff --pluginsource puppet:///plugins --use_srv_records && \
service mcollective restart

[Stefan Heijmans]

On Wednesday, August 26, 2015 at 10:16:59 AM UTC+2, Fabrice Bacchella wrote:

> Le 24 août 2015 à 21:59, Christopher Wood <christop...@pobox.com> a écrit :
>
My biggest problem: there isn't any working dashboard working with Puppet 4. The puppetdb APIv4 in puppet db 3.1 is not the same as the APIv4 in puppetdb 2, so all of them are now broken, even the one that thought they uses the v4 API.

 

Hopefully PuppetDB 3.x.x will support Puppet 3 clients again, like PuppetServer.

 

In the PuppetDB release notes is says that version 2.3.0 supports Puppet 4 [1], so will this not work together with PuppetExplorer 1.5.0 [2]?

 

1. https://docs.puppetlabs.com/puppetdb/latest/release_notes.html#section-11

2. https://github.com/spotify/puppetexplorer/releases/tag/1.5.0

 

Stefan

 

 

[Fabrice Bacchella]

Le 26 août 2015 à 11:30, Stefan Heijmans <heij...@gmail.com> a écrit :

On Wednesday, August 26, 2015 at 10:16:59 AM UTC+2, Fabrice Bacchella wrote:

> Le 24 août 2015 à 21:59, Christopher Wood <christop...@pobox.com> a écrit :
>
My biggest problem: there isn't any working dashboard working with Puppet 4. The puppetdb APIv4 in puppet db 3.1 is not the same as the APIv4 in puppetdb 2, so all of them are now broken, even the one that thought they uses the v4 API.

 

Hopefully PuppetDB 3.x.x will support Puppet 3 clients again, like PuppetServer.

 

In the PuppetDB release notes is says that version 2.3.0 supports Puppet 4 [1], so will this not work together with PuppetExplorer 1.5.0 [2]?

They don't give a lot of details, that's puppetbd 2.3 release notes any way. The API v4 changed between puppetdb 2 and puppetdb 3. So this release note don't apply any more.

For puppet explorer, good news: 

https://github.com/spotify/puppetexplorer/issues/34

Good news with puppet explorer any way :

https://github.com/spotify/puppetexplorer/releases/tag/2.0.0

 

1. https://docs.puppetlabs.com/puppetdb/latest/release_notes.html#section-11

2. https://github.com/spotify/puppetexplorer/releases/tag/1.5.0

 

Stefan

 

 

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/f359ca60-1b0d-4fac-96c3-dfba460995fc%40googlegroups.com.

[Christopher Wood]

On Wed, Aug 26, 2015 at 10:16:28AM +0200, Fabrice Bacchella wrote:
>
> > Le 24 août 2015 à 21:59, Christopher Wood <christop...@pobox.com> a écrit :
> >
> > I am not seeing a large amount of blog entries complaining about this upgrade, how has that gone for you? Is there anything you found particularly painful? Would you have done anything different in retrospect?
> >
> > I'm staring down a 3.7.2 -> 4.2.1 upgrade and after reading a number of docs the back-of-the-envelope optimal upgrade path is as follows. If any of you have commentary I am quite interested, otherwise I will try it and see what happens. I feel like this might be more time-consuming work but less brainpower effort than just yanking everything to 4.2.
> >
> > 1) rpms up to puppet 3.8
> >
> > PostgreSQL 8.4 to 9.4
> > PuppetDB 2.2.2 to 2.3.7
> > (puppetdb-terminus from 2.2.2 to 2.3.7)
> > Puppet 3.7.2 to 3.8.2
> >
> > 2) enable the future parser
> >
> > 3) replace puppet 3.8.2, passenger 5 with puppetserver 1.1.1
> >
> > 4) replace puppetserver 1.1.1 with puppetserver 2.1.1
> >
> > 5) upgrade PuppetDB from 2.3.7 to 3.0.2
> >
> > 6) use the puppetlabs-puppet_agent forge module to upgrade agents to 4.2.1
> >
> >
>
> My biggest problem: there isn't any working dashboard working with Puppet 4. The puppetdb APIv4 in puppet db 3.1 is not the same as the APIv4 in puppetdb 2, so all of them are now broken, even the one that thought they uses the v4 API.
>
> I had a lot of trouble with facter, it's rewritten but not tested on older release, a few things were broken on Redhat 5 :
> https://tickets.puppetlabs.com/browse/FACT-1152
> https://tickets.puppetlabs.com/browse/FACT-1137
>
> If you use mcollective there is a bug that prevent installation of both mcollective and puppet on the same server on Redhat 6 :
> https://tickets.puppetlabs.com/browse/PUP-4970
> and no one take care for this bug at puppetlabs.

That would have been a bit of a gotcha, but I don't manage repos with release rpms. It interferes with purging /etc/yum.repos.d.

> All the path change /etc/puppet, /usr/bin to /etc/puppetlab, /opt/puppetlabs. Minor problems might arise from that, if you don't prepare that on puppet modules, other management scripts.

I figure just adding some sync'ing before the upgrade would do it. Have to check if the puppet-agent rpm does anything to these files after the sync. Thank you!

> I hade no problem with embedded PKI, rsync -av /var/lib/puppet/ssl /etc/puppetlabs/puppet/ is enough to keep it.
>
> Check your:
> [master]
> certname =...
>
> in puppet.conf. Rules changes slightly and I had problem with that.
>
> If you wrote you own modules, expect a lot of warning and broken modules becase of new variable typing.
>
> Once the new server was up and running, I used this simple script on all the nodes:
> [ -x /opt/puppetlabs/bin/puppet ] || (\
> [ -x $(type -p facter) ] && \
> rpm -e puppetlabs-release && \
> rpm -Uvh http://.../puppetlabs/$(facter operatingsystemmajrelease)/PC1/x86_64/puppetlabs-release-pc1-0.9.2-1.el$(facter operatingsystemmajrelease).noarch.rpm && \
> yum install -y puppet-agent && \
> rsync -av /var/lib/puppet/ssl /etc/puppetlabs/puppet/ && \
> /opt/puppetlabs/bin/puppet agent -v --onetime --no-daemonize --no-show_diff --pluginsource puppet:///plugins --use_srv_records && \
> service mcollective restart
>
>

> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/88164374-6100-425D-B7D1-C4C944B35642%40orange.fr.

[Christopher Wood]

(Following up to myself.)

My current set of puppetdb classes do not have an easy upgrade path, especially given how I macgyvered in the capacity to manage certificate whitelisting. However I'm still not sure I care about past data in (appliance'y) puppetdb enough to upgrade anything. I may just build a new puppetdb server and point the puppetmasters at it.

[Pete Harvey]

On 26 August 2015 at 09:16, Fabrice Bacchella

<fabrice....@orange.fr> wrote:
> My biggest problem: there isn't any working dashboard working with Puppet 4.

This. Our upgrade to 4 is essentially blocked until Foreman supports it.

Pete

[Ellison Marks]

We ran into a couple issues, all on centos6.

Several package conflicts. Had to remove the puppetlabs-release package to get the puppetlabs-release-pc1 package to install. New puppetserver package didn't obsolete the old puppet-server package, had to uninstall puppet-server package first. puppetdb-termini package doesn't appear to obsolete puppetdb-terminus package. These might have been fixed in the meantime.

If you're using mcollective-puppet, mcollective-puppet-agent can't find puppet at new path, needed to add "plugin.puppet.command = /opt/puppetlabs/bin/puppet agent" to the server.cfg on all mcollective servers.

The puppetdb upgrade was fairly painless. Though, the fact that they dropped the v2 and v3 apis and completely altered the v4 api, as compared to the v4-experimental api, so that essentially no existing tools could work with the new puppetdb seems pretty boneheaded. What's the point of having an api if you drop it out the window?

Other than that, there were a few differences between the future parser in 3.8 and the actual implementation in 4, but nothing to difficult to fix.

[Michael Stahnke]

On Wed, Aug 26, 2015 at 10:56 AM, Ellison Marks <gty...@gmail.com> wrote:

We ran into a couple issues, all on centos6.

Several package conflicts. Had to remove the puppetlabs-release package to get the puppetlabs-release-pc1 package to install. New puppetserver package didn't obsolete the old puppet-server package, had to uninstall puppet-server package first. puppetdb-termini package doesn't appear to obsolete puppetdb-terminus package. These might have been fixed in the meantime.

Just to be clear here, this was intentional for two reasons.

1. You can use puppetserver with the 3 series as well. 

2. It was designed to be an opt-in (intentional choice not happening by accident; however puppet-server vs puppetserver is not helping make that extremely obvious :/  )

 

--

You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/b21eff5b-49b4-4e98-8385-7850a96d0fa0%40googlegroups.com.

[Michael Stahnke]

On Wed, Aug 26, 2015 at 9:54 PM, Michael Stahnke <sta...@puppetlabs.com> wrote:

On Wed, Aug 26, 2015 at 10:56 AM, Ellison Marks <gty...@gmail.com> wrote:

We ran into a couple issues, all on centos6.

Several package conflicts. Had to remove the puppetlabs-release package to get the puppetlabs-release-pc1 package to install. New puppetserver package didn't obsolete the old puppet-server package, had to uninstall puppet-server package first. puppetdb-termini package doesn't appear to obsolete puppetdb-terminus package. These might have been fixed in the meantime.

The terminus package gets pulled along for the ride on upgrades (for RPMs) and really just depends on the termini package. This was a work around to allow people to still freshly install puppetdb-2.3.z if they so desired. Basically, due to some crazy rules in the way rpm does obsoletes, we had to keep the package around. It's not hurting anything and is pretty much just an entry in the rpmdb at this point.

[Ellison Marks]

Hmm, perhaps it's that I didn't upgrade puppetdb-terminus before trying to install puppetdb-termini then? I saved this from that time, and I not that the current version of puppetdb-terminus is 3:

Transaction Check Error:
  file /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/face/storeconfigs.rb from install of puppetdb-termini-3.0.1-1.el6.noarch conflicts with file from package puppetdb-terminus-2.3.6-1.el6.noarch
  file /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/face/node/status.rb from install of puppetdb-termini-3.0.1-1.el6.noarch conflicts with file from package puppetdb-terminus-2.3.6-1.el6.noarch
  file /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/puppetdb.rb from install of puppetdb-termini-3.0.1-1.el6.noarch conflicts with file from package puppetdb-terminus-2.3.6-1.el6.noarch
  file /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/facts/puppetdb.rb from install of puppetdb-termini-3.0.1-1.el6.noarch conflicts with file from package puppetdb-terminus-2.3.6-1.el6.noarch
  file /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/node/puppetdb.rb from install of puppetdb-termini-3.0.1-1.el6.noarch conflicts with file from package puppetdb-terminus-2.3.6-1.el6.noarch
  file /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/resource/puppetdb.rb from install of puppetdb-termini-3.0.1-1.el6.noarch conflicts with file from package puppetdb-terminus-2.3.6-1.el6.noarch
  file /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/reports/puppetdb.rb from install of puppetdb-termini-3.0.1-1.el6.noarch conflicts with file from package puppetdb-terminus-2.3.6-1.el6.noarch
  file /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/puppetdb.rb from install of puppetdb-termini-3.0.1-1.el6.noarch conflicts with file from package puppetdb-terminus-2.3.6-1.el6.noarch
  file /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/puppetdb/blacklist.rb from install of puppetdb-termini-3.0.1-1.el6.noarch conflicts with file from package puppetdb-terminus-2.3.6-1.el6.noarch
  file /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/puppetdb/char_encoding.rb from install of puppetdb-termini-3.0.1-1.el6.noarch conflicts with file from package puppetdb-terminus-2.3.6-1.el6.noarch
  file /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/puppetdb/command.rb from install of puppetdb-termini-3.0.1-1.el6.noarch conflicts with file from package puppetdb-terminus-2.3.6-1.el6.noarch
  file /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/puppetdb/config.rb from install of puppetdb-termini-3.0.1-1.el6.noarch conflicts with file from package puppetdb-terminus-2.3.6-1.el6.noarch

[Felix Barbeira]

Same here.

[Jeffrey Watts]

Agreed.

Between the AIO packaging, change in configuration file locations, lack of dashboards, and the many other minor annoyances I do not feel the benefits of 4 outweigh the hassle of upgrading.  2->3 was much, much less painful.

Jeffrey.