Jira (PDB-3529) Report insertion should determine the latest_report_id without recalculating the latest report from the reports table

[Nick Walker (JIRA)]

Nick Walker created an issue

PuppetDB / PDB-3529 Report insertion should determine the latest_report_id without recalculating the latest report from the reports table Issue Type: Improvement Assignee: Unassigned Created: 2017/05/24 10:01 AM Priority: Normal Reporter: Nick Walker The Problem Looking at the scale testing postgresql logs I can see that we are repeatedly calculating the latest_report_id from the reports table. This happens every time we insert a report but we should not need to recalculate the latest_report every time. https://github.com/puppetlabs/puppetdb/blob/def0dc5cedbc31c0d0315abd95dfb20107e2ea45/src/puppetlabs/puppetdb/scf/storage.clj#L1291-L1295 The Suggestion We should store the latest_report_timestamp ( the producer_timestamp from reports ) in certnames with the latest_report_id. Then when we insert a new report we can compare the producer timestamp to the latest_report_timestamp and update the latest_report_id and _timestamp if it is actually the latest report. This would eliminate an expensive query that sorts all of the reports for a certname just to find the most recent report. Add Comment

This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe)

[Nick Walker (JIRA)]

Nick Walker updated an issue

PuppetDB / PDB-3529 Report insertion should determine the latest_report_id without recalculating the latest report from the reports table

Change By: Nick Walker Attachment: Screen Shot 2017-05-24 at 10.13.52 AM.png

Add Comment

[Nick Walker (JIRA)]

Nick Walker updated an issue

PuppetDB / PDB-3529 Report insertion should determine the latest_report_id without recalculating the latest report from the reports table

Change By: Nick Walker Affects Version/s: PDB 4.4.0

Add Comment

[Russell Mull (JIRA)]

Russell Mull updated an issue

PuppetDB / PDB-3529 Report insertion should determine the latest_report_id without recalculating the latest report from the reports table

Change By: Russell Mull Sprint: Hopper

Add Comment

[Russell Mull (JIRA)]

Russell Mull updated an issue

PuppetDB / PDB-3529 Report insertion should determine the latest_report_id without recalculating the latest report from the reports table

Change By: Russell Mull Fix Version/s: PDB 5.0.0 Fix Version/s: PDB 4.4.1

Add Comment

[Russell Mull (JIRA)]

Russell Mull updated an issue

PuppetDB / PDB-3529 Report insertion should determine the latest_report_id without recalculating the latest report from the reports table

Change By: Russell Mull Release Notes: Bug Fix

Add Comment

[Russell Mull (JIRA)]

Russell Mull updated an issue

PuppetDB / PDB-3529 Report insertion should determine the latest_report_id without recalculating the latest report from the reports table

Change By: Russell Mull Release Notes Summary: Report storage has been optimized by requiring fewer tables to be consulted when updating the record of which report is latest.

Add Comment

[Zachary Kent (Jira)]

Zachary Kent updated an issue

PuppetDB / PDB-3529 Report insertion should determine the latest_report_id without recalculating the latest report from the reports table

Change By: Zachary Kent Sprint:

Add Comment

This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)

[Zachary Kent (Jira)]

Zachary Kent updated an issue

PuppetDB / PDB-3529 Report insertion should determine the latest_report_id without recalculating the latest report from the reports table

Change By: Zachary Kent Sprint: Release Engineering Hopper

Add Comment

[Morgan Rhodes (Jira)]

Morgan Rhodes updated an issue

PuppetDB / PDB-3529 Report insertion should determine the latest_report_id without recalculating the latest report from the reports table

Change By: Morgan Rhodes Sprint: Release Engineering Hopper

Add Comment