Jira (PDB-3625) Use upsert for fact storage

2 views
Skip to first unread message

Russell Mull (JIRA)

unread,
Jul 24, 2017, 6:14:02 PM7/24/17
to puppe...@googlegroups.com
Russell Mull updated an issue
 
PuppetDB / New Feature PDB-3625
Use upsert for fact storage
Change By: Russell Mull
Sprint: Hopper
Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe)
Atlassian logo

Russell Mull (JIRA)

unread,
Jul 24, 2017, 6:14:03 PM7/24/17
to puppe...@googlegroups.com
Russell Mull created an issue
Issue Type: New Feature New Feature
Assignee: Unassigned
Created: 2017/07/24 3:13 PM
Priority: Normal Normal
Reporter: Russell Mull

Storing facts currently (as of PDB 5.x) works like this:

  • Get the value hashes for everything in the stored factset
  • Compute the hashes for everything in the new factset
  • Compare them
  • Store new values for whatever is different or new

Postgres 9.6 has support for upsert (INSERT ON CONFLICT UPDATE), which could let us do things better.

  • Store all values in a single upsert statement, rather than using separate updates and inserts.
  • We may be able to forego the local hash comparison, instead sending all values in a single upsert statement that is set up to only store values if they have changed.

Russell Mull (JIRA)

unread,
Jul 26, 2017, 1:38:03 PM7/26/17
to puppe...@googlegroups.com
Russell Mull updated an issue
Change By: Russell Mull
Sprint: Data Platform 2017-07-27 Hopper

Russell Mull (JIRA)

unread,
Jul 26, 2017, 1:38:03 PM7/26/17
to puppe...@googlegroups.com
Russell Mull updated an issue
Change By: Russell Mull
Sprint: Hopper Data Platform 2017-07-27

Jeff Yarnell (JIRA)

unread,
Oct 11, 2017, 4:48:02 PM10/11/17
to puppe...@googlegroups.com
Jeff Yarnell updated an issue

Moving from Hopper to Backlog because we believe this may be resolved by potential change to how facts are stored (which Russ is exploring now).

Jeff Yarnell (JIRA)

unread,
Oct 11, 2017, 4:48:02 PM10/11/17
to puppe...@googlegroups.com
Jeff Yarnell updated an issue
Change By: Jeff Yarnell
Sprint: Hopper/Triage

Claudia Petty (Jira)

unread,
Jun 21, 2023, 10:56:06 AM6/21/23
to puppe...@googlegroups.com
Claudia Petty updated an issue
Change By: Claudia Petty
Labels: new-feature
This message was sent by Atlassian Jira (v8.20.21#820021-sha1:38274c8)
Atlassian logo
Reply all
Reply to author
Forward
0 new messages