Jira (FACT-3138) SPIKE - Add Block Groups to GCE Facts

20 views
Skip to first unread message

Lisa Ross (Jira)

unread,
Jul 28, 2022, 10:16:02 AM7/28/22
to puppe...@googlegroups.com
Ryan DeShone created an issue
 
Facter / Improvement FACT-3138
SPIKE - Add Block Groups to GCE Facts
Issue Type: Improvement Improvement
Affects Versions: FACT 4.2.10
Assignee: Unassigned
Components: Facter 4
Created: 2022/07/28 7:15 AM
Priority: Normal Normal
Reporter: Ryan DeShone

Currently, GCE facts in Facter are all lumped into one big fact group from a blocklist standpoint. From facter --list-block-groups

GCE
- gce

This is problematic, as the data it pulls in is pretty wide ranging. Specifically, I'm experiencing an issue at work where one GCE project has a large number of SSH keys. These keys all get pulled into the GCE fact (gce.instance.project.attributes.ssh-keys) resulting in puppet complaining about a fact being larger than the maximum size (fortunately, this is a soft error at the moment). Since we don't actually use that fact, we would like to be able to disable it, as it just wastes resources (it is also not really a fact about the machine really, but I digress). Sadly, since GCE facts are currently all-or-nothing, we cannot blocklist it because we do use other GCE facts pretty widely.

Add Comment Add Comment
 
This message was sent by Atlassian Jira (v8.20.11#820011-sha1:0629dd8)
Atlassian logo

Lisa Ross (Jira)

unread,
Jul 28, 2022, 10:16:06 AM7/28/22
to puppe...@googlegroups.com
Lisa Ross updated an issue
 
Facter / Task FACT-3138
Change By: Lisa Ross
Issue Type: Improvement Task

Lisa Ross (Jira)

unread,
Jul 28, 2022, 10:18:01 AM7/28/22
to puppe...@googlegroups.com
Lisa Ross updated an issue
Change By: Lisa Ross
Reporter: Ryan DeShone Lisa Ross

Lisa Ross (Jira)

unread,
Jul 28, 2022, 10:18:03 AM7/28/22
to puppe...@googlegroups.com
Lisa Ross updated an issue
SPIKE to capture investigation on:
* Does Facter already support this?
* Is it possible to create a new block list of GCE?  

 

+Description of bug+

Currently, GCE facts in Facter are all lumped into one big fact group from a blocklist standpoint. From facter --list-block-groups

{{GCE}}
{{- gce}}

This is problematic, as the data it pulls in is pretty wide ranging. Specifically, I'm experiencing an issue at work where one GCE project has a large number of SSH keys. These keys all get pulled into the GCE fact (gce.instance.project.attributes.ssh-keys) resulting in puppet complaining about a fact being larger than the maximum size (fortunately, this is a soft error at the moment). Since we don't actually use that fact, we would like to be able to disable it, as it just wastes resources (it is also not really a fact about the machine really, but I digress). Sadly, since GCE facts are currently all-or-nothing, we cannot blocklist it because we do use other GCE facts pretty widely.

Lisa Ross (Jira)

unread,
Jul 28, 2022, 10:20:03 AM7/28/22
to puppe...@googlegroups.com
Lisa Ross updated an issue
Change By: Lisa Ross
Sprint: Phoenix 2022-08-17

Lisa Ross (Jira)

unread,
Jul 28, 2022, 10:20:03 AM7/28/22
to puppe...@googlegroups.com
Lisa Ross updated an issue
SPIKE to capture investigation on:
* Does Facter already support this?
* Is it possible to create a new block list of GCE?  

 

+Description of bug original ticket FACT-3126 +


Currently, GCE facts in Facter are all lumped into one big fact group from a blocklist standpoint. From facter --list-block-groups

{{GCE}}
{{- gce}}

This is problematic, as the data it pulls in is pretty wide ranging. Specifically, I'm experiencing an issue at work where one GCE project has a large number of SSH keys. These keys all get pulled into the GCE fact (gce.instance.project.attributes.ssh-keys) resulting in puppet complaining about a fact being larger than the maximum size (fortunately, this is a soft error at the moment). Since we don't actually use that fact, we would like to be able to disable it, as it just wastes resources (it is also not really a fact about the machine really, but I digress). Sadly, since GCE facts are currently all-or-nothing, we cannot blocklist it because we do use other GCE facts pretty widely.

Lisa Ross (Jira)

unread,
Jul 28, 2022, 10:20:05 AM7/28/22
to puppe...@googlegroups.com
Lisa Ross updated an issue
Change By: Lisa Ross
Team: Phoenix

Nirupama Mantha (Jira)

unread,
Jul 28, 2022, 11:35:03 AM7/28/22
to puppe...@googlegroups.com

Nirupama Mantha (Jira)

unread,
Aug 3, 2022, 12:51:03 PM8/3/22
to puppe...@googlegroups.com
Nirupama Mantha updated an issue
Change By: Nirupama Mantha
Sprint: Phoenix 2022-08- 17 31

Morgan Rhodes (Jira)

unread,
Aug 17, 2022, 12:45:03 PM8/17/22
to puppe...@googlegroups.com
Morgan Rhodes updated an issue
Change By: Morgan Rhodes
Sprint: Phoenix 2022- 08 09 - 31 14

Aria Li (Jira)

unread,
Aug 30, 2022, 2:53:03 PM8/30/22
to puppe...@googlegroups.com
Aria Li assigned an issue to Aria Li
Change By: Aria Li
Assignee: Aria Li

Josh Cooper (Jira)

unread,
Sep 2, 2022, 11:40:02 AM9/2/22
to puppe...@googlegroups.com
Josh Cooper commented on Task FACT-3138
 
Re: SPIKE - Add Block Groups to GCE Facts

Paired with Aria Li on this and the way facter works currently is the block list determines which facts should not be collected. But if a single fact like "gce" returns structured data, then the block list does not filter the data. The reason this works for other structured facts (for example, you can block "memory.swap" while allowing "memory.system") is because swap is an independent fact, which just happens to be "mounted" at that point in the fact tree.

That said, I think it would be possible to apply the block list to whatever structured data we collect. We probably want to filter that data before it is cached as well. Note there are some existing bugs around facter block lists being slow (FACT-2916, FACT-2954, FACT-3080) so we'll need to be mindful of that.

Aria Li (Jira)

unread,
Sep 8, 2022, 8:02:03 PM9/8/22
to puppe...@googlegroups.com
Aria Li commented on Task FACT-3138

Hello! Just an update on what Josh Cooper and I are planning on doing. We will be modifying the pre-existing fact blocking mechanism on Facter so users can block specific fact values from facts with structured data, like gce, which uses a hash for its values. Specifically, this will be done using the block list which will match a value in the fact's value hash and filter it out. For now, we will not be focusing on dynamic facts like network.interfaces.docker0 and instead we will focus on implementing this for static facts.

If you're curious about the work we've done already, feel free to check out this link!

Reply all
Reply to author
Forward
0 new messages