Jira (PDB-3684) PDB performance issues with large structured facts

[Adam Bottchen (JIRA)]

Adam Bottchen created an issue

PuppetDB / PDB-3684 PDB performance issues with large structured facts Issue Type: Bug Affects Versions: PDB 4.4.1 Assignee: Unassigned Components: PuppetDB Created: 2017/09/20 3:21 PM Priority: Major Reporter: Adam Bottchen As outlined in PDB-2631, PuppetDB encounters severe performance degradation in the presence of large structured facts. In support, we have seen this impact with mountpoint or partitions facts of only 20-30 KiB, which is not terribly uncommon. As noted in that ticket, the expectation is that PDB will be able to handle anything Facter might throw at it. This issue was resolved in https://tickets.puppetlabs.com/browse/PDB-3249, but the fix was removed by https://tickets.puppetlabs.com/browse/PDB-3611. We need a solution that will allow large structured facts to be stored in PDB without causing causing query performance issues. Add Comment

This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe)

[Adam Bottchen (JIRA)]

Adam Bottchen updated an issue

PuppetDB / PDB-3684 PDB performance issues with large structured facts

Change By: Adam Bottchen

As outlined in PDB-2631, PuppetDB encounters severe performance degradation in the presence of large structured facts.  In support, we have seen this impact with mountpoint or partitions facts of only 20-30 KiB, which is not terribly uncommon.  As noted in that ticket, the expectation is that PDB will be able to handle anything Facter might throw at it.

This issue was resolved in  https://tickets.puppetlabs.com/browse/ PDB-3249, but the fix was removed by  https://tickets.puppetlabs.com/browse/ PDB-3611.  We need a solution that will allow large structured facts to be stored in PDB without causing causing query performance issues.

Add Comment

[Adam Bottchen (JIRA)]

Adam Bottchen updated an issue

PuppetDB / PDB-3684 PDB performance issues with large structured facts

Change By: Adam Bottchen CS Priority: Needs Priority

Add Comment

[Owen Rodabaugh (JIRA)]

Owen Rodabaugh updated an issue

PuppetDB / PDB-3684 PDB performance issues with large structured facts

Change By: Owen Rodabaugh CS Priority: Needs Priority Critical CS Impact: This is a very common and troubleshooting it has consumed a huge amount of support time. The workaround is to stop sending large structured facts which in some cases impacts their use of puppet.  In one example a custom structured fact collected rpm data which they were using in their code somehow so turning it off wasn't really a workaround but a reduction in functionality. CS Severity: 4 - Major CS Business Value: 5 - $$$$$$ CS Frequency: 4 - 50-90% of Customers

Add Comment

[Owen Rodabaugh (JIRA)]

Owen Rodabaugh updated an issue

PuppetDB / PDB-3684 PDB performance issues with large structured facts

Change By: Owen Rodabaugh CS Impact:

This is a very common and troubleshooting it has consumed a huge amount of support time. The workaround is to stop sending large structured facts which in some cases impacts their use of puppet.  In one example a custom structured fact collected rpm data which they were using in their code somehow so turning it off wasn't really a workaround but a reduction in functionality.

This can be very common on Solaris where there is a mount per user. A 20-30KB structured fact can be easily reached on solaris and cause this issue.

Add Comment

[Martin Ewings (JIRA)]

Martin Ewings updated an issue

PuppetDB / PDB-3684 PDB performance issues with large structured facts

Change By: Martin Ewings Method Found: Customer Feedback

Add Comment

[Jeff Yarnell (JIRA)]

Jeff Yarnell updated an issue

PuppetDB / PDB-3684 PDB performance issues with large structured facts

Change By: Jeff Yarnell Sprint: Data Platform 2017-10-18

Add Comment

[Martin Ewings (JIRA)]

Martin Ewings updated an issue

PuppetDB / PDB-3684 PDB performance issues with large structured facts

Change By: Martin Ewings As _emphasized text_As  outlined in PDB-2631, PuppetDB encounters severe performance degradation in the presence of large structured facts.  In support, we have seen this impact with mountpoint or partitions facts of only 20-30 KiB, which is not terribly uncommon.  As noted in that ticket, the expectation is that PDB will be able to handle anything Facter might throw at it. This issue was resolved in PDB-3249, but the fix was removed by PDB-3611.  We need a solution that will allow large structured facts to be stored in PDB without causing causing query performance issues.

Add Comment

[Martin Ewings (JIRA)]

Martin Ewings updated an issue

PuppetDB / PDB-3684 PDB performance issues with large structured facts Change By: Martin Ewings

_emphasized text_As As  outlined in PDB-2631, PuppetDB encounters severe performance degradation in the presence of large structured facts.  In support, we have seen this impact with mountpoint or partitions facts of only 20-30 KiB, which is not terribly uncommon.  As noted in that ticket, the expectation is that PDB will be able to handle anything Facter might throw at it.

This issue was resolved in PDB-3249, but the fix was removed by PDB-3611.  We need a solution that will allow large structured facts to be stored in PDB without causing causing query performance issues.

Add Comment

[Jeff Yarnell (JIRA)]

Jeff Yarnell updated an issue

PuppetDB / PDB-3684 PDB performance issues with large structured facts

Change By: Jeff Yarnell Sprint: Data Platform 2017-10-18 , Data Platform 2017-11-01

Add Comment

[Nick Walker (JIRA)]

Nick Walker commented on PDB-3684

Re: PDB performance issues with large structured facts A configurable facts blacklist in PDB would mitigate but not solve this issue. https://tickets.puppetlabs.com/browse/PDB-3630

Add Comment

[Jeff Yarnell (JIRA)]

Jeff Yarnell updated an issue

PuppetDB / PDB-3684

PDB performance issues with large structured facts

Change By: Jeff Yarnell Sprint: Hopper/Triage, Data Platform 2017-10-18 , Data Platform 2017-11-01

Add Comment

[Wyatt Alt (JIRA)]

Wyatt Alt commented on PDB-3684

Re: PDB performance issues with large structured facts We believe these issues will be resolved with https://github.com/puppetlabs/puppetdb/pull/2410

Add Comment

This message was sent by Atlassian JIRA (v7.0.2#70111-sha1:88534db)

[Wyatt Alt (JIRA)]

Wyatt Alt updated an issue

PuppetDB / PDB-3684

PDB performance issues with large structured facts

Change By: Wyatt Alt

As outlined in PDB-2631, PuppetDB encounters severe performance degradation in the presence of large structured facts.  In support, we have seen this impact with mountpoint or partitions facts of only 20-30 KiB, which is not terribly uncommon.  As noted in that ticket, the expectation is that PDB will be able to handle anything Facter might throw at it. This issue was resolved in PDB-3249, but the fix was removed by PDB-3611.  We need a solution that will allow large structured facts to be stored in PDB without causing causing query performance issues.

Add Comment

[Wyatt Alt (JIRA)]

Wyatt Alt commented on PDB-3684

Re: PDB performance issues with large structured facts The above has been merged.

Add Comment

[Nick Walker (JIRA)]

Nick Walker updated an issue

PuppetDB / PDB-3684

PDB performance issues with large structured facts

Change By: Nick Walker Fix Version/s: PDB 5.2.0

Add Comment

[Nick Walker (JIRA)]

Nick Walker commented on PDB-3684

Re: PDB performance issues with large structured facts Russell Mull is this resolved by PDB-3723 ? If so we can close this out?

Add Comment

[Jeff Yarnell (JIRA)]

Jeff Yarnell updated an issue

PuppetDB / PDB-3684

PDB performance issues with large structured facts

Change By: Jeff Yarnell Release Notes: Not Needed

Add Comment

This message was sent by Atlassian JIRA (v7.5.1#75006-sha1:7df2574)

[Zachary Kent (Jira)]

Zachary Kent updated an issue

PuppetDB / PDB-3684 PDB performance issues with large structured facts

Change By: Zachary Kent Sprint: Data Platform 2017-10-18 , Release Engineering Hopper

Add Comment

This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)

[Morgan Rhodes (Jira)]

Morgan Rhodes updated an issue

PuppetDB / PDB-3684 PDB performance issues with large structured facts

Change By: Morgan Rhodes Sprint: Data Platform 2017-10-18, Release Engineering Hopper

Add Comment