Jira (PDB-3911) Resource events table should have a primary key

9 views
Skip to first unread message

Nick Walker (JIRA)

unread,
Apr 19, 2018, 8:06:02 PM4/19/18
to puppe...@googlegroups.com
Nick Walker created an issue
 
PuppetDB / Improvement PDB-3911
Resource events table should have a primary key
Issue Type: Improvement Improvement
Assignee: Unassigned
Created: 2018/04/19 5:05 PM
Labels: cspapercuts
Priority: Normal Normal
Reporter: Nick Walker

The Problem

When running pg_repack on the PuppetDB database it won't repack the resource_events table because it doesn't have a primary key.

As far as I can tell resource_events has a unique index on 4 columns that could just be changed to a primary key and we'd get the unique index as a side effect of the unique index.

Suggested Solution

Drop the unique index and create a primary key on the same columns.

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)
Atlassian logo

Nick Walker (JIRA)

unread,
Apr 24, 2018, 5:24:04 PM4/24/18
to puppe...@googlegroups.com
Nick Walker updated an issue
Change By: Nick Walker
h1.  The Problem


When running pg_repack on the PuppetDB database it won't repack the resource_events table because it doesn't have a primary key.  

As far as I can tell resource_events has a unique index on 4 columns that could just be changed to a primary key and we'd get the unique index as a side effect of the unique index.

h1.  Suggested Solution


Drop the unique index and create a primary key on the same columns.  

h1.  Some commands that work for me

I created a primary key on the same columns as the unique constraint and it worked in my test instance.  However, it does change the property column in resource_events to be "not null" where as in the current schema "not null" is not enforced.  

The question is do we actually ever insert null into the property column or is that a safe thing to change?  If we do insert null into that column could we just update NULL rows to have some other value so we can make the primary key.  

{code}
[root@master201810-centos ~]# su - pe-postgres -s /bin/bash -c "/opt/puppetlabs/server/bin/psql -d pe-puppetdb"
psql (9.6.8)
Type "help" for help.

pe-puppetdb=# \d+ resource_events
                     Table "public.resource_events"
      Column       |           Type           |            Modifiers            | Storage  | Stats target | Description
-------------------+--------------------------+---------------------------------+----------+--------------+-------------
report_id         | bigint                   | not null                     | plain    |              |
certname_id       | bigint                   | not null                     | plain    |              |
status            | text                     | not null                     | extended |              |
timestamp         | timestamp with time zone | not null                     | plain    |              |
resource_type     | text                     | not null                     | extended |              |
resource_title    | text                     | not null                     | extended |              |
property          | text                     |                     | extended |              |
new_value         | text                     |                     | extended |              |
old_value         | text                     |                     | extended |              |
message           | text                     |                     | extended |              |
file              | text                     | default NULL::character varying | extended |              |
line              | integer                  |                     | plain    |              |
containment_path  | text[]                   |                     | extended |              |
containing_class  | text                     |                     | extended |              |
corrective_change | boolean                  |                     | plain    |              |
Indexes:
    "resource_events_unique" UNIQUE CONSTRAINT, btree (report_id, resource_type, resource_title, property)
    "resource_events_containing_class_idx" btree (containing_class)
    "resource_events_property_idx" btree (property)
    "resource_events_reports_id_idx" btree (report_id)
    "resource_events_resource_timestamp" btree (resource_type, resource_title, "timestamp")
    "resource_events_resource_title_idx" btree (resource_title)
    "resource_events_status_for_corrective_change_idx" btree (status) WHERE corrective_change
    "resource_events_status_idx" btree (status)
    "resource_events_timestamp_idx" btree ("timestamp")
Foreign-key constraints:
    "resource_events_report_id_fkey" FOREIGN KEY (report_id) REFERENCES reports(id) ON DELETE CASCADE
               ^
pe-puppetdb=# ALTER TABLE resource_events ADD PRIMARY KEY (report_id, resource_type, resource_title, property);
ALTER TABLE
pe-puppetdb=# \d+ resource_events
                     Table "public.resource_events"
      Column       |           Type           |            Modifiers            | Storage  | Stats target | Description
-------------------+--------------------------+---------------------------------+----------+--------------+-------------
report_id         | bigint                   | not null                     | plain    |              |
certname_id       | bigint                   | not null                     | plain    |              |
status            | text                     | not null                     | extended |              |
timestamp         | timestamp with time zone | not null                     | plain    |              |
resource_type     | text                     | not null                     | extended |              |
resource_title    | text                     | not null                     | extended |              |
property          | text                     | not null                     | extended |              |
new_value         | text                     |                     | extended |              |
old_value         | text                     |                     | extended |              |
message           | text                     |                     | extended |              |
file              | text                     | default NULL::character varying | extended |              |
line              | integer                  |                     | plain    |              |
containment_path  | text[]                   |                     | extended |              |
containing_class  | text                     |                     | extended |              |
corrective_change | boolean                  |                     | plain    |              |
Indexes:
    "resource_events_pkey" PRIMARY KEY, btree (report_id, resource_type, resource_title, property)
    "resource_events_unique" UNIQUE CONSTRAINT, btree (report_id, resource_type, resource_title, property)
    "resource_events_containing_class_idx" btree (containing_class)
    "resource_events_property_idx" btree (property)
    "resource_events_reports_id_idx" btree (report_id)
    "resource_events_resource_timestamp" btree (resource_type, resource_title, "timestamp")
    "resource_events_resource_title_idx" btree (resource_title)
    "resource_events_status_for_corrective_change_idx" btree (status) WHERE corrective_change
    "resource_events_status_idx" btree (status)
    "resource_events_timestamp_idx" btree ("timestamp")
Foreign-key constraints:
    "resource_events_report_id_fkey" FOREIGN KEY (report_id) REFERENCES reports(id) ON DELETE CASCADE

pe-puppetdb=# ALTER TABLE resource_events DROP CONSTRAINT IF EXISTS resource_events_unique;
ALTER TABLE
pe-puppetdb=# \d+ resource_events
                     Table "public.resource_events"
      Column       |           Type           |            Modifiers            | Storage  | Stats target | Description
-------------------+--------------------------+---------------------------------+----------+--------------+-------------
report_id         | bigint                   | not null                     | plain    |              |
certname_id       | bigint                   | not null                     | plain    |              |
status            | text                     | not null                     | extended |              |
timestamp         | timestamp with time zone | not null                     | plain    |              |
resource_type     | text                     | not null                     | extended |              |
resource_title    | text                     | not null                     | extended |              |
property          | text                     | not null                     | extended |              |
new_value         | text                     |                     | extended |              |
old_value         | text                     |                     | extended |              |
message           | text                     |                     | extended |              |
file              | text                     | default NULL::character varying | extended |              |
line              | integer                  |                     | plain    |              |
containment_path  | text[]                   |                     | extended |              |
containing_class  | text                     |                     | extended |              |
corrective_change | boolean                  |                     | plain    |              |
Indexes:
    "resource_events_pkey" PRIMARY KEY, btree (report_id, resource_type, resource_title, property)
    "resource_events_containing_class_idx" btree (containing_class)
    "resource_events_property_idx" btree (property)
    "resource_events_reports_id_idx" btree (report_id)
    "resource_events_resource_timestamp" btree (resource_type, resource_title, "timestamp")
    "resource_events_resource_title_idx" btree (resource_title)
    "resource_events_status_for_corrective_change_idx" btree (status) WHERE corrective_change
    "resource_events_status_idx" btree (status)
    "resource_events_timestamp_idx" btree ("timestamp")
Foreign-key constraints:
    "resource_events_report_id_fkey" FOREIGN KEY (report_id) REFERENCES reports(id) ON DELETE CASCADE
{code}

Nick Walker (JIRA)

unread,
Apr 24, 2018, 5:43:03 PM4/24/18
to puppe...@googlegroups.com
Nick Walker updated an issue
h1.  The Problem

When running pg_repack on the PuppetDB database it won't repack the resource_events table because it doesn't have a primary key.  

As far as I can tell resource_events has a unique index on 4 columns that could just be changed to a primary key and we'd get the unique index as a side effect of the unique index Primary Key .

Nick Walker (JIRA)

unread,
Apr 25, 2018, 6:25:03 PM4/25/18
to puppe...@googlegroups.com

Rob Browning (JIRA)

unread,
Apr 26, 2018, 3:02:03 PM4/26/18
to puppe...@googlegroups.com
Rob Browning updated an issue
Change By: Rob Browning
Sprint: PDB Hopper/ Triage

Robert Roland (JIRA)

unread,
Mar 12, 2019, 1:27:06 PM3/12/19
to puppe...@googlegroups.com
Robert Roland updated an issue
Change By: Robert Roland
Release Notes Summary: This adds a primary key to the resource_events table, which allows the use of pg_repack to reclaim space without taking the table offline, like the current "vacuum full" approach does.

This will rewrite the entire resource_events table, so you will need to have more than the existing table's size available during the upgrade. The upgrade time is relative to the size of your table.
Release Notes: Enhancement

Rob Browning (JIRA)

unread,
Mar 12, 2019, 5:49:03 PM3/12/19
to puppe...@googlegroups.com
Rob Browning updated an issue
Change By: Rob Browning
Affects Version/s: PDB 6.3.0

Austin Blatt (JIRA)

unread,
Mar 20, 2019, 1:22:04 PM3/20/19
to puppe...@googlegroups.com
Austin Blatt updated an issue
Change By: Austin Blatt
Fix Version/s: PDB 6.3.0

Heston Hoffman (JIRA)

unread,
Mar 20, 2019, 9:14:03 PM3/20/19
to puppe...@googlegroups.com
Heston Hoffman updated an issue
Change By: Heston Hoffman
Labels: cspapercuts resolved-issue-added

Zachary Kent (Jira)

unread,
Mar 10, 2021, 3:32:03 PM3/10/21
to puppe...@googlegroups.com
Zachary Kent updated an issue
Change By: Zachary Kent
Sprint:
This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)
Atlassian logo

Zachary Kent (Jira)

unread,
Mar 10, 2021, 3:34:05 PM3/10/21
to puppe...@googlegroups.com
Zachary Kent updated an issue
Change By: Zachary Kent
Sprint: Release Engineering Hopper

Morgan Rhodes (Jira)

unread,
Mar 11, 2021, 2:28:08 PM3/11/21
to puppe...@googlegroups.com
Morgan Rhodes updated an issue
Change By: Morgan Rhodes
Sprint: Release Engineering Hopper
Reply all
Reply to author
Forward
0 new messages