Jira (PDB-4373) PDB has issues upgrading to versions later than 6.1 on centos6

26 views
Skip to first unread message

Zachary Kent (JIRA)

unread,
May 16, 2019, 5:38:03 PM5/16/19
to puppe...@googlegroups.com
Zachary Kent created an issue
 
PuppetDB / Bug PDB-4373
PDB has issues upgrading to versions later than 6.1 on centos6
Issue Type: Bug Bug
Assignee: Unassigned
Created: 2019/05/16 2:37 PM
Priority: Normal Normal
Reporter: Zachary Kent

During the PDB upgrade_oldest acceptance tests on centos6 for 6.3.x and master we try to upgrade the PDB package from 5.2.0 to the latest snapshot build 6.3.3-SNAPSHOT. This upgrade fails in our tests and results in PDB ending up in a state where running service puppetdb stop is unable to stop PDB and it must be killed by hand. This problem only appears to affect platforms which are running sysvinit and not systemd. 

 

It appears that due to changes with how PDB is invoked in later versions the main namespace provided via the -m argument has changed. 

In versions prior to PDB 6.1.0 the main namespace was puppetlabs.puppetdb.main in later versions it has changed to puppetlabs.puppetdb.cli.services.

 

When testing upgrades from 5.2.0 -> 6.3.2 on a centos6 by hand I was able to see that the command service puppetdb stop was looking for the pid of PDB using the new main namespace when it was still running under the old name. One possibility is that the newer stop script is being run when trying to upgrade which is resulting in service not finding the old name and the right pid to kill. 

Repo:

    1. install pdb 5.2.0 on centos6

    2. grab a package version later than 6.1

    3. yum install the package while pdb is running (this will put the upgrade in a broken state)

    4. add a set -x to /opt/puppetlabs/server/apps/puppetdb/cli/apps/stop

    5. run service puppetdb stop

    6. compare the output from the pgrep call in the stop script to the main namespace pdb is being run under.

We may need to make changes to lein-ezbake templates that create our stop/start scripts to account for the name change on platforms running sysvinit. see lein-ezbake templates

Workaround in case anyone hits this issue before it is fixed:

If you manually stop PDB before trying to install the upgraded package this issue can be avoided.  

If an upgrade was tried while PDB was running and you're in the state where service puppetdb stop can't find and stop PDB, finding the pid, killing PDB, and restarting it should resolve the issue and allow the upgrade to proceed.

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)
Atlassian logo

Zachary Kent (JIRA)

unread,
May 20, 2019, 12:29:03 PM5/20/19
to puppe...@googlegroups.com
Zachary Kent updated an issue
Change By: Zachary Kent
During the PDB upgrade_oldest acceptance tests on centos6 for 6.3.x and master we try to upgrade the PDB package from 5.2.0 to the latest snapshot build 6.3.3-SNAPSHOT. This upgrade fails in our tests and results in PDB ending up in a state where running *service puppetdb stop* is unable to stop PDB and it must be killed by hand. This problem only appears to affect platforms which are running sysvinit and not systemd. 

 

It appears that due to changes with how PDB is invoked in later versions the main namespace provided via the *-m* argument has changed. 

In versions prior to PDB 6.1.0 the main namespace was *puppetlabs.puppetdb.main* in later versions it has changed to *puppetlabs.puppetdb.cli.services*.

 

When testing upgrades from 5.2.0 -> 6.3.2 on a centos6 by hand I was able to see that the command *service puppetdb stop* was looking for the pid of PDB using the new main namespace when it was still running under the old name. One possibility is that the newer stop script is being run when trying to upgrade which is resulting in service not finding the old name and the right pid to kill. 

Repo Repro :


    1. install pdb 5.2.0 on centos6

    2. grab a package version later than 6.1

    3. yum install the package while pdb is running (this will put the upgrade in a broken state)

    4. add a *set -x* to */opt/puppetlabs/server/apps/puppetdb/cli/apps/stop*

    5. run *service puppetdb stop*


    6. compare the output from the pgrep call in the stop script to the main namespace pdb is being run under.

We may need to make changes to lein-ezbake templates that create our stop/start scripts to account for the name change on platforms running sysvinit. see [lein-ezbake templates|https://github.com/puppetlabs/ezbake/tree/07f50665d078b1503b3ebbc306308fadc81615aa/resources/puppetlabs/lein-ezbake/template/global/ext/cli]


*Workaround in case anyone hits this issue before it is fixed*:


If you manually stop PDB before trying to install the upgraded package this issue can be avoided.  

If an upgrade was tried while PDB was running and you're in the state where *service puppetdb stop* can't find and stop PDB, finding the pid, killing PDB, and restarting it should resolve the issue and allow the upgrade to proceed.

Zachary Kent (JIRA)

unread,
Jun 21, 2019, 1:09:03 PM6/21/19
to puppe...@googlegroups.com

Zachary Kent (JIRA)

unread,
Jun 21, 2019, 1:18:03 PM6/21/19
to puppe...@googlegroups.com
Zachary Kent commented on Bug PDB-4373
 
Re: PDB has issues upgrading to versions later than 6.1 on centos6

The Installer Team hit the same issue in their pe upgrade tests here. We'll start working on an update to ezbake to get this fixed. 

Austin Blatt (JIRA)

unread,
Jun 26, 2019, 6:59:03 PM6/26/19
to puppe...@googlegroups.com

Rob Browning (JIRA)

unread,
Jul 2, 2019, 1:17:03 PM7/2/19
to puppe...@googlegroups.com
Rob Browning assigned an issue to Rob Browning
Change By: Rob Browning
Assignee: Zachary Kent Rob Browning

Austin Blatt (JIRA)

unread,
Jul 22, 2019, 7:10:03 PM7/22/19
to puppe...@googlegroups.com

Rob Browning (JIRA)

unread,
Jul 25, 2019, 11:32:03 AM7/25/19
to puppe...@googlegroups.com

Joshua Partlow (JIRA)

unread,
Aug 28, 2019, 5:11:04 PM8/28/19
to puppe...@googlegroups.com

What build of puppetdb should we expect this changes in in 2019.2.x?

Austin Blatt (JIRA)

unread,
Aug 28, 2019, 7:42:03 PM8/28/19
to puppe...@googlegroups.com
Austin Blatt commented on Bug PDB-4373

We are currently blocked on promotions to Lovejoy due to some changes we made to support building for FIPs. So I don't know if we yet have an exact build. I will follow up with the team tomorrow and see if we can get something in to unblock the pipeline for Lovejoy.

Rob Browning (JIRA)

unread,
Sep 11, 2019, 1:31:04 PM9/11/19
to puppe...@googlegroups.com
Rob Browning updated an issue
 
Change By: Rob Browning
Release Notes Summary: Previously package upgrades on CentOS 6 might fail when upgrading from from older versions of PuppetDB (like 5.2) to more recent versions (6.3 or newer).  That should no longer be the case.
Fix Version/s: PDB 6.6.0

Austin Blatt (JIRA)

unread,
Sep 13, 2019, 1:11:06 PM9/13/19
to puppe...@googlegroups.com

Joshua Partlow (JIRA)

unread,
Sep 13, 2019, 1:22:03 PM9/13/19
to puppe...@googlegroups.com
Joshua Partlow commented on Bug PDB-4373
 
Re: PDB has issues upgrading to versions later than 6.1 on centos6

Hi Austin, do you know what build of pdb this would be in now?

Austin Blatt (JIRA)

unread,
Sep 13, 2019, 1:31:03 PM9/13/19
to puppe...@googlegroups.com
Austin Blatt commented on Bug PDB-4373

The clojure projects are still blocked on jar-jar failures, so nothing as of yet. It will be up before the Platform release goes out, as this is also what's holding up the Platform release.

Austin Blatt (JIRA)

unread,
Sep 13, 2019, 6:11:03 PM9/13/19
to puppe...@googlegroups.com
Austin Blatt commented on Bug PDB-4373

Joshua Partlow This has now been promoted into Lovejoy as of 2019.2.0-rc8-263-gbaa6ea2

Joshua Partlow (JIRA)

unread,
Sep 13, 2019, 6:52:03 PM9/13/19
to puppe...@googlegroups.com

Austin Blatt (JIRA)

unread,
Oct 4, 2019, 2:01:03 PM10/4/19
to puppe...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages