Jira (FACT-769) Add ability to filter facts

8 views
Skip to first unread message

Andre Nathan (JIRA)

unread,
Dec 18, 2014, 12:48:30 PM12/18/14
to puppe...@googlegroups.com
Andre Nathan created an issue
 
Facter / New Feature FACT-769
Add ability to filter facts
Issue Type: New Feature New Feature
Assignee: Eric Sorenson
Created: 2014/12/18 9:48 AM
Priority: Normal Normal
Reporter: Andre Nathan

My use-case is a server with many LXC containers. This may affect other people using other container technologies like Docker too.

The issue is that each of my containers has two network interfaces (eth0 and eth1) which are associated to virtual interfaces in the host (eg. "veth0.mycontainer" and "veth1.mycontainer").

In my specific situation, there are 200 containers, meaning 400 virtual interfaces in the host. This causes facter to take a very long time to run, because it has to create facts for all those interfaces. However, those facts are not interesting to me, so it would be cool to be able to tell facter to ignore interfaces matching a certain pattern (eg. /^veth.+/).

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v6.3.7#6337-sha1:2ed701e)
Atlassian logo

Peter Huene (JIRA)

unread,
Sep 10, 2015, 1:04:03 PM9/10/15
to puppe...@googlegroups.com
Peter Huene assigned an issue to Unassigned
Change By: Peter Huene
Assignee: Eric Sorenson
This message was sent by Atlassian JIRA (v6.4.11#64026-sha1:78f6ec4)
Atlassian logo

Koen Dierckx (Jira)

unread,
Sep 2, 2021, 2:37:03 AM9/2/21
to puppe...@googlegroups.com
Koen Dierckx commented on New Feature FACT-769
 
Re: Add ability to filter facts

Disabling the entire fact block is not an option, as our puppet code uses parts of that block.

 

e.g. 1 the networking block contains information about our primary nics (ens192, eth0, ...), that are being used within our puppet manifests

But as a docker host also that same block is filled with docker virtual nics like veth02b3fa9, veth06e56a9, br-2aa431d10302, br-38a3698b07e8 we do not care about

 

e.g.2 We have the same issue with the mountpoints block, it contains the common mounts like / and /boot we use inside puppet

but it is also polluted with docker mounts /run/docker/netns/... and /var/lib/docker/overlay2/...

 

So a filtering system would solve this, so we could ignore entries we are not interested in

e.g. 1 veth and br- for networking

e.g. 2 /run/docker/netns and /var/lib/docker/overlay2

 

This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)
Atlassian logo

Koen Dierckx (Jira)

unread,
Sep 2, 2021, 2:42:03 AM9/2/21
to puppe...@googlegroups.com
Koen Dierckx updated an issue
 
Change By: Koen Dierckx
Affects Version/s: FACT 4.2.3

Victor Bobosila (Jira)

unread,
Sep 17, 2021, 3:50:04 AM9/17/21
to puppe...@googlegroups.com
Victor Bobosila assigned an issue to Victor Bobosila
Change By: Victor Bobosila
Assignee: Victor Bobosila

Ciprian Badescu (Jira)

unread,
Sep 20, 2021, 6:48:03 AM9/20/21
to puppe...@googlegroups.com
Ciprian Badescu assigned an issue to Unassigned
Change By: Ciprian Badescu
Assignee: Victor Bobosila

Andrew Wrenn (Jira)

unread,
Jan 13, 2022, 3:10:02 PM1/13/22
to puppe...@googlegroups.com
Andrew Wrenn commented on New Feature FACT-769
 
Re: Add ability to filter facts

I don't think this would affect us - I could see how if users decided to strip facts out of facter's output, then we wouldn't be able to see those facts anymore. But this is probably the desired behavior as far as the users are concerned.

This message was sent by Atlassian Jira (v8.20.2#820002-sha1:829506d)
Atlassian logo

Alan Evans (Jira)

unread,
May 20, 2022, 7:50:02 PM5/20/22
to puppe...@googlegroups.com
Alan Evans commented on New Feature FACT-769

TL;DR
Do the built-in fact resolutions have a "names" in Facter 3.x?  If so I think we could use Facter.fact(:foo).resolution('bar') to solve this problem but I can't figure out if/what the names of the core fact resolvers are.

Side note: The method Facter.core_value() is interesting.  But I think it's in Facter 4.x (still using 3.x) and it's private so it might be rude to use it.

Background

I am running into the same problem described in the original post.  I have nodes running containers and the mountpoints fact on these nodes is quite large.  (Additionally I have some DB boxes w/ lots and lots of multipath devices that are also unnecessary) so I started looking at this.  It seemed straight forward based on this article I would just "be clever" and do something like:

 

Facter.add(:mountpoints) do
  has_weight 100
  setcode do
    Facter.value(:mountpoints).reject{|k,v| v['filesystem'] == 'overlay'}
  end
end

But alas the devs have thought of that.  (And reasonably so.)
2022-05-20 16:35:07.738257 ERROR puppetlabs.facter - error while resolving custom fact "mountpoints": cycle detected while requesting value of fact "mountpoints"
What's in a name?

In inspecting Facter::Util::Fact and looking at the Facter 3.x source I found the resolver() method which looks like it might be a way to get the value from the built-in filesystem_resolver w/o triggering the loop detection in the value() method,  I just can't work out what the names of these resolvers are.

 

Tomas Barton (Jira)

unread,
Jan 12, 2023, 6:51:03 AM1/12/23
to puppe...@googlegroups.com
Tomas Barton commented on New Feature FACT-769

I have the same issue, mountpoints and networking.interfaces contain too many unnecessary facts (each container generates at least 5 mount points leading to hundreds of unwanted facts):

The current total number of facts: 3333 exceeds the number of facts limit: 2048

A workaround might be defining a blocklist:

facts : {
  blocklist : ["mountpoints", "networking.interfaces"],
}

And the redefining a different fact with desired filtering:

Facter.add(:mounts) do
  setcode do
          mounts = []
          Facter::Util::Resolvers::FilesystemHelper.read_mountpoints.each do |file_system|
            mount = {}
            get_mount_data(file_system, mount)
            next if mount[:device] =~ %r{^(tmpfs|shm|overlay|nsfs)}
            next if mount[:path] =~ %r{^/(proc|sys)} && mount[:filesystem] != 'tmpfs' || mount[:filesystem] == 'autofs'
 
            get_mount_sizes(mount)
            mounts << mount
          end
 
          @fact_list[:mountpoints] = mounts
          @fact_list[fact_name]
        end
  end
end

unfortunately that would break all code using existing `mountpoints` fact. The alternative is monkey patching Facter::Resolvers::Mountpoints which could easily break on facter update.

This message was sent by Atlassian Jira (v8.20.11#820011-sha1:0629dd8)
Atlassian logo

Josh Cooper (Jira)

unread,
Apr 26, 2023, 5:02:02 PM4/26/23
to puppe...@googlegroups.com
Josh Cooper updated an issue
 
Change By: Josh Cooper
My use-case is a server with many LXC containers. This may affect other people using other container technologies like Docker too.

The issue is that each of my containers has two network interfaces (eth0 and eth1) which are associated to virtual interfaces in the host (eg. "veth0.mycontainer" and "veth1.mycontainer").

In my specific situation, there are 200 containers, meaning 400 virtual interfaces in the host. This causes facter to take a very long time to run, because it has to create facts for all those interfaces. However, those facts are not interesting to me, so it would be cool to be able to tell facter to ignore interfaces matching a certain pattern (eg. /^veth.+/).


Another use case is removing ssh keys in GCE or AWS metadata, e.g. gce.instance.project.attributes.ssh-keys

Another use case is removing some, but not all mountpoints (so we can't block the entire fact).

Josh Cooper (Jira)

unread,
Apr 26, 2023, 5:02:03 PM4/26/23
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Team: Phoenix

Claudia Petty (Jira)

unread,
Jun 21, 2023, 8:44:04 AM6/21/23
to puppe...@googlegroups.com
Claudia Petty updated an issue
Change By: Claudia Petty
Labels: new-feature
This message was sent by Atlassian Jira (v8.20.21#820021-sha1:38274c8)
Atlassian logo
Reply all
Reply to author
Forward
0 new messages