Jira (PDB-3515) Clean up orphaned edges

5 views
Skip to first unread message

Rob Browning (JIRA)

unread,
May 12, 2017, 10:04:02 PM5/12/17
to puppe...@googlegroups.com
Rob Browning created an issue
 
PuppetDB / Bug PDB-3515
Clean up orphaned edges
Issue Type: Bug Bug
Assignee: Unassigned
Created: 2017/05/12 7:03 PM
Priority: Normal Normal
Reporter: Rob Browning

If this is what it looks like (not deleting edges when a node is purged), then we'll probably want to make sure to call delete-edges! during purge, and add a migration to handle existing orphans. We'll also want to add a test to make sure that the addition and deletion of a set of nodes leaves no trace (perhaps via dump, or via "all table" traversal).
Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe)
Atlassian logo

Russell Mull (JIRA)

unread,
May 15, 2017, 11:30:03 AM5/15/17
to puppe...@googlegroups.com
Russell Mull updated an issue
Change By: Russell Mull
Story Points: 2

Russell Mull (JIRA)

unread,
May 15, 2017, 11:30:03 AM5/15/17
to puppe...@googlegroups.com

Russell Mull (JIRA)

unread,
May 15, 2017, 11:31:02 AM5/15/17
to puppe...@googlegroups.com
Russell Mull updated an issue
Change By: Russell Mull
Affects Version/s: PDB 4.2.3
Affects Version/s: PDB 4.2.3.1
Affects Version/s: PDB 4.2.3.2

Russell Mull (JIRA)

unread,
May 15, 2017, 11:31:02 AM5/15/17
to puppe...@googlegroups.com
Russell Mull updated an issue
Change By: Russell Mull
Sprint: PuppetDB 2017-05-31

Adam Bottchen (JIRA)

unread,
May 15, 2017, 12:05:39 PM5/15/17
to puppe...@googlegroups.com
Adam Bottchen commented on Bug PDB-3515

I ran a `puppet node purge` on iebzu95xg1zlr4o.delivery.puppetlabs.net pn 5/12 at 11:58:

2017-05-12 11:58:12.886 PDT [db:pe-puppetdb,sess:59160585.3dd5,pid:15829,vtid:29/23,tid:0] DETAIL:  parameters: $1 = 'iebzu95xg1zlr4o.delivery.puppetlabs.net'
 
2017-05-12 11:58:12.887 PDT [db:pe-puppetdb,sess:59160585.3dd5,pid:15829,vtid:29/23,tid:0] LOG:  execute <unnamed>: UPDATE certnames SET deactivated = $1
 
                              WHERE certname=$2
 
                                AND (deactivated IS NULL OR deactivated < $3)

The delete ran several times with a node-purge-ttl of 1d:

2017-05-15 00:47:13.176 PDT [db:pe-puppetdb,sess:591959a8.1477,pid:5239,vtid:17/1007,tid:0] DETAIL:  parameters: $1 = '2017-05-15 00:47:13.172-07', $2 = '2017-05-08 00:47:13.172-07', $3 = '2017-05-08 00:47:13.172-07', $4 = '2017-05-08 00:47:13.172-07'
 
2017-05-15 00:47:13.177 PDT [db:pe-puppetdb,sess:591959a8.1477,pid:5239,vtid:17/1007,tid:0] LOG:  execute S_1: COMMIT
 
2017-05-15 00:47:13.177 PDT [db:pe-puppetdb,sess:591959a8.1477,pid:5239,vtid:17/1008,tid:0] LOG:  execute <unnamed>: SET SESSION CHARACTERISTICS AS TRANSACTION ISOLATION LEVEL READ COMMITTED
 
2017-05-15 00:47:13.177 PDT [db:pe-puppetdb,sess:591959a8.1477,pid:5239,vtid:17/1009,tid:0] LOG:  execute <unnamed>: SHOW TRANSACTION ISOLATION LEVEL
 
2017-05-15 00:47:13.178 PDT [db:pe-puppetdb,sess:591959a8.1477,pid:5239,vtid:17/1010,tid:0] LOG:  execute <unnamed>: SET SESSION CHARACTERISTICS AS TRANSACTION ISOLATION LEVEL READ COMMITTED
 
2017-05-15 00:47:13.178 PDT [db:pe-puppetdb,sess:591959a8.1477,pid:5239,vtid:17/1011,tid:0] LOG:  execute <unnamed>: BEGIN
 
2017-05-15 00:47:13.178 PDT [db:pe-puppetdb,sess:591959a8.1477,pid:5239,vtid:17/1011,tid:0] LOG:  execute <unnamed>: DELETE FROM certnames WHERE deactivated < $1 OR expired < $2
 
2017-05-15 00:47:13.178 PDT [db:pe-puppetdb,sess:591959a8.1477,pid:5239,vtid:17/1011,tid:0] DETAIL:  parameters: $1 = '2017-05-14 00:47:13.178-07', $2 = '2017-05-14 00:47:13.178-07'

The node is now no longer in the certnames table:

# su - pe-postgres -s /bin/bash -c "/opt/puppetlabs/server/bin/psql -d pe-puppetdb -c \"select * from certnames\""
 
 id |             certname              | latest_report_id | deactivated | expired
 
----+-----------------------------------+------------------+-------------+---------
 
  1 | pe-201621-master.puppetdebug.vlan |             6971 |             |
 
(1 row)

However, it and several other nodes, is still present in the edges table.

# su - pe-postgres -s /bin/bash -c "/opt/puppetlabs/server/bin/psql -d pe-puppetdb -c \"select distinct certname from edges order by certname\""
 
                 certname
 
-------------------------------------------
 
 iebzu95xg1zlr4o.delivery.puppetlabs.net
 
 iuq7zoo9k5ku0qp.delivery.puppetlabs.net
 
 pe-201621-agent.puppetdebug.vlan
 
 pe-201621-master.puppetdebug.vlan
 
 pe-aix-61-support.delivery.puppetlabs.net
 
 wkj27v5mqzzvdfq.delivery.puppetlabs.net
 
(6 rows)

Russell Mull (JIRA)

unread,
May 22, 2017, 1:29:03 PM5/22/17
to puppe...@googlegroups.com
Russell Mull updated an issue
Change By: Russell Mull
Affects Version/s: PDB 4.3.0
Affects Version/s: PDB 4.3.1
Affects Version/s: PDB 4.3.2
Affects Version/s: PDB 4.4.0

Russell Mull (JIRA)

unread,
May 22, 2017, 1:30:03 PM5/22/17
to puppe...@googlegroups.com
Russell Mull assigned an issue to Russell Mull
Change By: Russell Mull
Assignee: Russell Mull

Russell Mull (JIRA)

unread,
May 22, 2017, 1:30:04 PM5/22/17
to puppe...@googlegroups.com
Russell Mull updated an issue
Change By: Russell Mull
Fix Version/s: PDB 5.0.0
Fix Version/s: PDB 4.2.3.3

Russell Mull (JIRA)

unread,
May 25, 2017, 3:31:03 PM5/25/17
to puppe...@googlegroups.com
Russell Mull updated an issue
Change By: Russell Mull
Fix Version/s: PDB 4.4.1

Russell Mull (JIRA)

unread,
May 25, 2017, 3:31:03 PM5/25/17
to puppe...@googlegroups.com
Russell Mull commented on Bug PDB-3515

Nick WalkerThat's an oversight, the PR is currently up against stable.

Karen Van der Veer (JIRA)

unread,
May 31, 2017, 5:09:05 PM5/31/17
to puppe...@googlegroups.com
Karen Van der Veer updated an issue
Change By: Karen Van der Veer
Sprint: Hopper, PuppetDB 2017-05-31

Russell Mull (JIRA)

unread,
Jun 1, 2017, 10:35:02 PM6/1/17
to puppe...@googlegroups.com
Russell Mull updated an issue
Change By: Russell Mull
Affects Version/s: PDB 4.2.3.3

Russell Mull (JIRA)

unread,
Jun 1, 2017, 10:35:03 PM6/1/17
to puppe...@googlegroups.com
Russell Mull updated an issue
Change By: Russell Mull
Fix Version/s: PDB 4.2.3.3
Fix Version/s: PDB 4.2.3.4

Russell Mull (JIRA)

unread,
Jun 6, 2017, 6:44:02 PM6/6/17
to puppe...@googlegroups.com
Russell Mull updated an issue
Change By: Russell Mull
Release Notes: Bug Fix

Russell Mull (JIRA)

unread,
Jun 6, 2017, 6:45:03 PM6/6/17
to puppe...@googlegroups.com
Russell Mull updated an issue
Change By: Russell Mull
Release Notes Summary: PuppetDB was leaving unused 'edge' records in the database. For most users this had very little impact, but they could pile up over time as catalogs changed. The unused edges will now be removed on upgrade and continuously cleaned up when they are no longer needed. 

Nick Walker (JIRA)

unread,
Jun 6, 2017, 6:57:02 PM6/6/17
to puppe...@googlegroups.com
Nick Walker updated an issue
Change By: Nick Walker
Release Notes Summary: PuppetDB was leaving unused 'edge' records in the database. For most users this had very little impact, but they could pile up over time as  catalogs changed  nodes are purged if [node-purge-ttl](https://docs . puppet.com/puppetdb/4.4/configure.html#node-purge-ttl) is non-zero.  The unused edges will now be removed on upgrade and continuously cleaned up when they are no longer needed. 

Zachary Kent (Jira)

unread,
Mar 10, 2021, 3:34:04 PM3/10/21
to puppe...@googlegroups.com
Zachary Kent updated an issue
Change By: Zachary Kent
Sprint: PuppetDB 2017-05-31 , Release Engineering Hopper
This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)
Atlassian logo

Morgan Rhodes (Jira)

unread,
Mar 11, 2021, 2:25:01 PM3/11/21
to puppe...@googlegroups.com
Morgan Rhodes updated an issue
Change By: Morgan Rhodes
Sprint: PuppetDB 2017-05-31, Release Engineering Hopper
Reply all
Reply to author
Forward
0 new messages