Jira (FACT-1858) Provide an option to disable legacy facts so they are not sent to the master from the agent

[Nick Walker (JIRA)]

Nick Walker created an issue

Facter / FACT-1858 Provide an option to disable legacy facts so they are not sent to the master from the agent Issue Type: Improvement Assignee: Unassigned Created: 2018/05/30 1:50 PM Priority: Normal Reporter: Nick Walker The Problem We came out with structured facts or "modern facts" as we call them long ago, however, we still have legacy facts and we send both the legacy version and the modern version whenever the agent requests a catalog. This means we are allowing users to refer to the legacy fact names in code and potentially creating tech debt if we ever remove the legacy fact names, as we probably should. Additionally we are burdening PuppetDB with all of the duplicate information since legacy facts are represented in the modern facts. Downstream of PuppetDB the PE Console is slowed down by the large amount of unique fact names caused by legacy facts. Suggested Improvement We should provide an option to disable legacy facts altogether so that the agent doesn't send them to the master and then subsequently they do not get stored in PuppetDB. We could provide this as an option, deprecate legacy facts and remove them in a future version. Add Comment

This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)

[Nick Walker (JIRA)]

Nick Walker updated an issue

Facter / FACT-1858 Provide an option to disable legacy facts so they are not sent to the master from the agent

Change By: Nick Walker h1.  The Problem

We came out with structured facts or "modern facts" as we call them long ago, however, we still have legacy facts and we send both the legacy version and the modern version whenever the agent requests a catalog.   This means we are allowing users to refer to the legacy fact names in code and potentially creating tech debt if we ever remove the legacy fact names, as we probably should.   Additionally we are burdening PuppetDB with all of the duplicate information since legacy facts are represented in the modern facts.  Downstream of PuppetDB the PE Console is slowed down by the large amount of unique fact names caused by legacy facts.

h1.  Suggested Improvement

We should provide an option to disable legacy facts altogether so that the agent doesn't send them to the master and then subsequently they do not get stored in PuppetDB.   We could provide this as an option, deprecate legacy facts and remove them in a future version.

h1.  Side thought Given this type of setting it could be plumbed into spec testing so that legacy facts could be disabled and tests could fail if someone refers to a legacy fact name.  This would give early warning to users.

Add Comment

[Nick Walker (JIRA)]

Nick Walker updated an issue

Facter / FACT-1858 Provide an option to disable legacy facts so they are not sent to the master from the agent

Change By: Nick Walker CS Priority: Needs Priority

Add Comment

[Adam Bottchen (JIRA)]

Adam Bottchen updated an issue

Facter / FACT-1858 Provide an option to disable legacy facts so they are not sent to the master from the agent

Change By: Adam Bottchen CS Priority: Needs Priority Reviewed

Add Comment

[Austin Boyd (JIRA)]

Austin Boyd updated an issue

Facter / FACT-1858 Provide an option to disable legacy facts so they are not sent to the master from the agent

Change By: Austin Boyd Labels: jira_escalated

Add Comment

[Austin Boyd (JIRA)]

Austin Boyd updated an issue

Facter / FACT-1858 Provide an option to disable legacy facts so they are not sent to the master from the agent

Change By: Austin Boyd Zendesk Ticket IDs: 30378 Zendesk Ticket Count: 1

Add Comment

[Austin Boyd (JIRA)]

Austin Boyd updated an issue

Facter / FACT-1858 Provide an option to disable legacy facts so they are not sent to the master from the agent

Change By: Austin Boyd Labels: jira_escalated

Zendesk Ticket IDs: 30378 Zendesk Ticket Count: 1

Add Comment

[Scott McClellan (JIRA)]

Scott McClellan assigned an issue to Branan Riley

Facter / FACT-1858 Provide an option to disable legacy facts so they are not sent to the master from the agent

Change By: Scott McClellan Assignee: Branan Riley

Add Comment

[Scott McClellan (JIRA)]

Scott McClellan commented on FACT-1858

Re: Provide an option to disable legacy facts so they are not sent to the master from the agent Branan Riley to update with scope.

Add Comment

[Branan Riley (JIRA)]

Branan Riley updated an issue

Facter / FACT-1858

Provide an option to disable legacy facts so they are not sent to the master from the agent

Change By: Branan Riley Team: Platform OS

Add Comment

[Mihai Buzgau (JIRA)]

Mihai Buzgau updated an issue

Facter / FACT-1858 Provide an option to disable legacy facts so they are not sent to the master from the agent

Change By: Mihai Buzgau Team: Platform OS Night's Watch

Add Comment

[Mihai Buzgau (JIRA)]

Mihai Buzgau updated an issue

Facter / FACT-1858 Provide an option to disable legacy facts so they are not sent to the master from the agent

Change By: Mihai Buzgau Sprint: PR - Triage

Add Comment

[Mihai Buzgau (JIRA)]

Mihai Buzgau updated an issue

Facter / FACT-1858 Provide an option to disable legacy facts so they are not sent to the master from the agent

Change By: Mihai Buzgau Story Points: 2

Add Comment

[Mihai Buzgau (JIRA)]

Mihai Buzgau updated an issue

Facter / FACT-1858 Provide an option to disable legacy facts so they are not sent to the master from the agent

Change By: Mihai Buzgau Sprint: PR NW - Triage 2020-01-22

Add Comment

[Gheorghe Popescu (JIRA)]

Gheorghe Popescu assigned an issue to Gheorghe Popescu

Facter / FACT-1858 Provide an option to disable legacy facts so they are not sent to the master from the agent

Change By: Gheorghe Popescu Assignee: Branan Riley Gheorghe Popescu

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on FACT-1858

Re: Provide an option to disable legacy facts so they are not sent to the master from the agent This would also get rid of the top scope variables representing individual facts I suppose. That will break a lot of code, but is a big improvement otherwise. One issue with many top scope variables is that they are all visible in every context and if ERB templates are used it will need to set them all as instance variables in the context given to ERB and that is quite time consuming (and cannot be changed without forcing people to modify their templates). Removing them also means reducing memory consumption when compiling since facts are now duplicated in memory as well (and they had to be read for each compile as well).

Add Comment

[Nick Walker (JIRA)]

Nick Walker commented on FACT-1858

Re: Provide an option to disable legacy facts so they are not sent to the master from the agent

Henrik Lindberg yes, that's why this ticket is to provide an opt-in way to disable legacy facts.  I believe the next step is to provide a reasonable to run PDK spec tests with legacy facts disabled so users can fix any problems disabling creates and ensure each module they have doesn't regress on that.   We'll also want to provide guidance along the lines of exactly what you are talking about to inform users how they will need to change their puppet code or templates.

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on FACT-1858

Re: Provide an option to disable legacy facts so they are not sent to the master from the agent

Nick Walker thanks, all makes sense! There is a subtle difference wrt legacy facts and top scope variables - since the structured facts also result in top scope variables - not sure we want to get rid of all of them and force people to reference them via $facts at all times.

Add Comment

[Nick Walker (JIRA)]

Nick Walker commented on FACT-1858

Re: Provide an option to disable legacy facts so they are not sent to the master from the agent

I think not having facts as top-scope variables would have to be another opt-in change.  That change provides a different benefit to users than what this ticket is trying to achieve around reducing data in PuppetDB.   So, I think we're in agreement, as part of this ticket we should not change facts being top scope variables.

Add Comment

[Gheorghe Popescu (JIRA)]

Gheorghe Popescu commented on FACT-1858

Re: Provide an option to disable legacy facts so they are not sent to the master from the agent

We are implementing a mechanism that lets the user to block legacy facts using `facter.conf`. When blocking legacy facts, `Facter.to_hash` will not return legacy facts and `Facter.value` will return nil if a legacy fact is queried.

Add Comment

[Nick Walker (JIRA)]

Nick Walker commented on FACT-1858

Re: Provide an option to disable legacy facts so they are not sent to the master from the agent

Gheorghe Popescu to confirm my understanding of this change... The user will be required to add "legacy" to the blocklist in facter.conf.   This will make it so that legacy facts are not sent to the master during catalog compilation and thus not stored in PuppetDB?   I don't understand what is meant by "`Facter.value` will still return the fact value in order not to break puppet"

Add Comment

[Gheorghe Popescu (JIRA)]

Gheorghe Popescu commented on FACT-1858

Re: Provide an option to disable legacy facts so they are not sent to the master from the agent

Nick Walker yes, user will be required to to add "legacy" to the blocklist in facter.conf. Yes, legacy facts will not be set to master There are several places in puppet where legacy facts are used by direct querying using `Facter.value` eg: https://github.com/puppetlabs/puppet/blob/master/lib/puppet/provider/group/groupadd.rb#L11

Add Comment

[Nick Walker (JIRA)]

Nick Walker commented on FACT-1858

Re: Provide an option to disable legacy facts so they are not sent to the master from the agent

Gheorghe Popescu I see.  Should there be another ticket for finding those places in Puppet that use legacy fact names and updating them to use structured fact names?

Add Comment

[Nick Walker (JIRA)]

Nick Walker commented on FACT-1858

Re: Provide an option to disable legacy facts so they are not sent to the master from the agent

created PDK-1591 for PDK support of this option.

Add Comment

[Josh Cooper (JIRA)]

Josh Cooper commented on FACT-1858

Re: Provide an option to disable legacy facts so they are not sent to the master from the agent

Should there be another ticket for finding those places in Puppet that use legacy fact names and updating them to use structured fact names? We should at least be internally consistent. Please file a PUP ticket. Will *_core modules also need to change?

Add Comment

[Gheorghe Popescu (JIRA)]

Gheorghe Popescu commented on FACT-1858

Re: Provide an option to disable legacy facts so they are not sent to the master from the agent

We can have a ticket for that but we can do the changes only when Facter 2.x is no longer used in puppet gem(it only knows legacy facts).

Add Comment

[Mihai Buzgau (JIRA)]

Mihai Buzgau updated an issue

Facter / FACT-1858

Provide an option to disable legacy facts so they are not sent to the master from the agent

Change By: Mihai Buzgau Sprint: NW - 2020-01-22 , NW - 2020-02-05

Add Comment

[Norman Heaney (JIRA)]

Norman Heaney commented on FACT-1858

Re: Provide an option to disable legacy facts so they are not sent to the master from the agent Hi all   This ticket has been brought to the attention of the IA Content team. I want to raise our concerns with the proposed approach. The team are working on gathering data on the potential impact on content.    The IAC team will share our finding ASAP.  Thank you Norman

Add Comment

[Norman Heaney (JIRA)]

Norman Heaney commented on FACT-1858

Re: Provide an option to disable legacy facts so they are not sent to the master from the agent

The IA Content team have been looking at understanding the impact of this proposed new feature.  in summary 29 supported modules affect 101 of 124 voxpupuli modules affected Please reference https://tickets.puppetlabs.com/browse/FM-8864

Add Comment

[Mihai Buzgau (JIRA)]

Mihai Buzgau updated an issue

Facter / FACT-1858

Provide an option to disable legacy facts so they are not sent to the master from the agent

Change By: Mihai Buzgau Sprint: NW - 2020-01-22, NW - 2020-02-05 , NW - 2020-02-19

Add Comment

[Jarret Lavallee (Jira)]

Jarret Lavallee updated an issue

Facter / FACT-1858 Provide an option to disable legacy facts so they are not sent to the master from the agent

Change By: Jarret Lavallee Team: Night's Watch Phoenix

Add Comment

This message was sent by Atlassian Jira (v8.20.2#820002-sha1:829506d)

[Jarret Lavallee (Jira)]

Jarret Lavallee commented on FACT-1858

Re: Provide an option to disable legacy facts so they are not sent to the master from the agent I am reopening this ticket as there was no reason listed for this ticket to be closed and it should still be considered.

Add Comment

[Alvin Rodis (Jira)]

Alvin Rodis updated an issue

Facter / FACT-1858

Provide an option to disable legacy facts so they are not sent to the master from the agent

Change By: Alvin Rodis Labels: jira_escalated

Add Comment

[Alvin Rodis (Jira)]

Alvin Rodis updated an issue

Facter / FACT-1858 Provide an option to disable legacy facts so they are not sent to the master from the agent

Change By: Alvin Rodis Zendesk Ticket Count: 1 Zendesk Ticket IDs: 47590

Add Comment

[Josh Cooper (Jira)]

Josh Cooper assigned an issue to Unassigned

Facter / FACT-1858 Provide an option to disable legacy facts so they are not sent to the master from the agent

Change By: Josh Cooper Assignee: Gheorghe Popescu

Add Comment

[Josh Cooper (Jira)]

Josh Cooper updated an issue

Facter / FACT-1858 Provide an option to disable legacy facts so they are not sent to the master from the agent

Change By: Josh Cooper Epic Link: PA-4664

Add Comment

This message was sent by Atlassian Jira (v8.20.11#820011-sha1:0629dd8)

[Josh Cooper (Jira)]

Josh Cooper commented on FACT-1858

Re: Provide an option to disable legacy facts so they are not sent to the master from the agent We attempted to disable legacy facts many years ago by blocking them in facter. However, that breaks types and providers that confine/defaultfor based on legacy facts, like the builtin "init" service provider: $ puppet apply -e '' Error: Could not autoload puppet/provider/service/init: undefined method `downcase' for nil:NilClass In order to disable legacy facts without breaking agents and provide a graceful migration path, we need to handle this in puppet, not facter. I've filed PUP-11662, which will make it possible to disable legacy facts in puppet 7.x. In puppet 8, we will fix puppet's builtin types/providers to reference structured facts and disable legacy facts by default (PUP-11430). Therefore, I'm going to close this as a duplicate, superseded by the puppet tickets.

Add Comment