Jira (PDB-3630) Configurable facts blacklist

[Russell Mull (JIRA)]

Russell Mull created an issue

PuppetDB / PDB-3630 Configurable facts blacklist Issue Type: New Feature Assignee: Unassigned Created: 2017/07/26 11:20 AM Priority: Normal Reporter: Russell Mull Add a place in the puppetdb config file to put a set of facts that are blacklisted on ingestion. This will provide an escape hatch for sites that are dealing with poorly behaving facts (large, very dynamic facts) whose value they don't actually care about but which are knocking over the fact storage subsystem. Add Comment

This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe)

[Russell Mull (JIRA)]

Russell Mull updated an issue

PuppetDB / PDB-3630 Configurable facts blacklist

Change By: Russell Mull Sprint: Data Platform Next Sprint

Add Comment

[Wyatt Alt (JIRA)]

Wyatt Alt commented on PDB-3630

Re: Configurable facts blacklist a blacklist went into facter a while ago to address the same issue this addresses. If there's functionality gaps it'd be worth discussing that with the facter team before putting another blacklist here – having two seems confusing to support.

Add Comment

[Russell Mull (JIRA)]

Russell Mull commented on PDB-3630

Re: Configurable facts blacklist Maggie Dreyer iirc you worked on the facter blacklist in some way; but I also seem to remember it having some kind of limitations. do you think it would be worth having such a mechanism in PDB, or would it be actively bad?

Add Comment

[Maggie Dreyer (JIRA)]

Maggie Dreyer commented on PDB-3630

Re: Configurable facts blacklist I would be more in favor of extending the Facter blocking to address this, or maybe using caching, depending on the use case. Right now, only a small number of facts can be blocked, but we can easily extend this list. We limited it because we didn't want to enable blocking of facts that lots of other facts rely on. So instead we picked a couple known to be problematic, namely "EC2" and "filesystem" facts. Are there other specific cases you'd like to be able to block?

Add Comment

[Wyatt Alt (JIRA)]

Wyatt Alt commented on PDB-3630

Re: Configurable facts blacklist Ideally the blacklist could support custom facts

Add Comment

[Maggie Dreyer (JIRA)]

Maggie Dreyer commented on PDB-3630

Re: Configurable facts blacklist Ah yeah, there's a ticket for that: https://tickets.puppetlabs.com/browse/FACT-1575. Just hasn't been prioritized, but we might be able to move on it if there's demand.

Add Comment

[Russell Mull (JIRA)]

Russell Mull commented on PDB-3630

Re: Configurable facts blacklist There's kind of demand; it's a facility we see being useful to work around problems that we just aren't going to be able to solve in PuppetDB any time soon.

Add Comment

[Jeff Yarnell (JIRA)]

Jeff Yarnell updated an issue

PuppetDB / PDB-3630 Configurable facts blacklist

Change By: Jeff Yarnell Sprint: Bad Sprint

Add Comment

[Jeff Yarnell (JIRA)]

Jeff Yarnell updated an issue

PuppetDB / PDB-3630 Configurable facts blacklist

Change By: Jeff Yarnell Sprint: Hopper

Add Comment

[Nick Walker (JIRA)]

Nick Walker commented on PDB-3630

Re: Configurable facts blacklist Reopening this issue, I think we should prioritize it in the near-term. We're continuing to see customers with large facts run into performance issues and often they do not want to blacklist the fact because they use it in puppet code, however, they don't ever view it in the PE console or via PuppetDB in anyway. Having a fact blacklist for PuppetDB is not duplicating functionality with the facter blacklist since blocking things out of PuppetDB effectively has no effect on puppet itself where as blocking facts from facter does affect puppet. The facter blacklist as discussed above requires block groups of facts which PuppetDB should allow blocking any fact by name. There are differences and implementing a PuppetDB fact blacklist is worthwhile.

Add Comment

[Charlie Sharpsteen (JIRA)]

Charlie Sharpsteen commented on PDB-3630

Re: Configurable facts blacklist +1 to what Nick said above.

Add Comment

[Brett Gray (JIRA)]

Brett Gray commented on PDB-3630

Re: Configurable facts blacklist +1 in support of reopening

Add Comment

[Jeff Yarnell (JIRA)]

Jeff Yarnell updated an issue

PuppetDB / PDB-3630 Configurable facts blacklist

Change By: Jeff Yarnell Sprint: Hopper/Triage Data Platform 2017-10-18

Add Comment

[Jeff Yarnell (JIRA)]

Jeff Yarnell updated an issue

PuppetDB / PDB-3630 Configurable facts blacklist

Change By: Jeff Yarnell Story Points: 1 Team: Data Platform

Add Comment

[Zachary Kent (JIRA)]

Zachary Kent assigned an issue to Zachary Kent

PuppetDB / PDB-3630 Configurable facts blacklist

Change By: Zachary Kent Assignee: Zachary Kent

Add Comment

[Jeff Yarnell (JIRA)]

Jeff Yarnell updated an issue

PuppetDB / PDB-3630 Configurable facts blacklist

Change By: Jeff Yarnell Sprint: Data Platform 2017-10-18 , Data Platform 2017-11-01

Add Comment

[Russell Mull (JIRA)]

Russell Mull updated an issue

PuppetDB / PDB-3630 Configurable facts blacklist

Change By: Russell Mull Fix Version/s: PDB 5.1.2 Fix Version/s: PDB 4.4.2 Fix Version/s: PDB 4.2.3.6

Add Comment

[Kenn Hussey (JIRA)]

Kenn Hussey commented on PDB-3630

Re: Configurable facts blacklist Zachary Kent please add release notes for this issue, if applicable. Thanks!

Add Comment

[Zachary Kent (JIRA)]

Zachary Kent updated an issue

PuppetDB / PDB-3630 Configurable facts blacklist

Change By: Zachary Kent Release Notes: New Feature

Add Comment

[Zachary Kent (JIRA)]

Zachary Kent updated an issue

PuppetDB / PDB-3630 Configurable facts blacklist

Change By: Zachary Kent Release Notes Summary: Added a facts blacklist feature to the PDB config file that allows users to specify a list of facts that will be ignored during ingestion.

Add Comment

[Zachary Kent (JIRA)]

Zachary Kent assigned an issue to Zachary Kent

PuppetDB / PDB-3630 Configurable facts blacklist

Change By: Zachary Kent Assignee: Zachary Kent

Add Comment

[Nick Walker (JIRA)]

Nick Walker commented on PDB-3630

Re: Configurable facts blacklist Zachary Kent Russell Mull What is the behavior of PuppetDB when a fact is already being stored and then is added to the blacklist? Will that fact be deleted from the database or will it stick around and just no longer receive updates?

Add Comment

[Russell Mull (JIRA)]

Russell Mull commented on PDB-3630

Re: Configurable facts blacklist It should get flushed the next time a factset comes in for that node.

Add Comment

[Zachary Kent (JIRA)]

Zachary Kent updated an issue

PuppetDB / PDB-3630 Configurable facts blacklist

Change By: Zachary Kent Fix Version/s: PDB 5.1.2 Fix Version/s: PDB 5.1.3

Add Comment

[Russell Mull (JIRA)]

Russell Mull updated an issue

PuppetDB / PDB-3630 Configurable facts blacklist

Change By: Russell Mull Fix Version/s: PDB 4.2.3.6 Fix Version/s: 4.2.3.7

Add Comment

This message was sent by Atlassian JIRA (v7.0.2#70111-sha1:88534db)

[Russell Mull (JIRA)]

Russell Mull updated an issue

PuppetDB / PDB-3630 Configurable facts blacklist

Change By: Russell Mull Fix Version/s: PDB 4.2.3.7 Fix Version/s: 4.2.3.8

Add Comment

[Susan McNerney (JIRA)]

Susan McNerney commented on PDB-3630

Re: Configurable facts blacklist We will need to defer this to the next LTS z, which will be in Jan/Feb. (2016.4.10), due to lack of time to resolve getting it into 2016.4.9 before our launch window.

Add Comment

[Russell Mull (JIRA)]

Russell Mull updated an issue

PuppetDB / PDB-3630 Configurable facts blacklist

Change By: Russell Mull Fix Version/s: PDB 4.4.2 Fix Version/s: 4.4.3

Add Comment

[Russell Mull (JIRA)]

Russell Mull commented on PDB-3630

Re: Configurable facts blacklist Updated fixversion to 4.4.3, sorry about the mistake

Add Comment

[Claudia Petty (Jira)]

Claudia Petty updated an issue

PuppetDB / PDB-3630 Configurable facts blacklist

Change By: Claudia Petty Labels: new-feature

Add Comment

This message was sent by Atlassian Jira (v8.20.21#820021-sha1:38274c8)