Jira (FACT-2724) Confine blocks behave differently with Facter 4, causing spec tests to suddenly fail

27 views
Skip to first unread message

Nate McCurdy (Jira)

unread,
Jul 27, 2020, 1:57:04 PM7/27/20
to puppe...@googlegroups.com
Nate McCurdy created an issue
 
Facter / Bug FACT-2724
Confine blocks behave differently with Facter 4, causing spec tests to suddenly fail
Issue Type: Bug Bug
Affects Versions: 4.0.30
Assignee: Unassigned
Components: Facter 4
Created: 2020/07/27 10:56 AM
Priority: Normal Normal
Reporter: Nate McCurdy

I have a custom Ruby fact that uses confine blocks to limit when it's run. The spec tests for this fact have worked up until I upgraded to the PDK 1.18 which comes with Facter 4 by default.

Here's the fact:

# Our physical hosts have an alias interface of :1 as the logical one.
Facter.add(:logical_interface) do
  confine kernel: 'Linux'
  confine virtual: 'physical'
 
  setcode do
    networking = Facter.value(:networking)
    primary_alias_iface = "#{networking['primary']}:1"
    unless networking['interfaces'][primary_alias_iface].nil?
      networking['interfaces'][primary_alias_iface]
    end
  end
end
 
# In GCP and Virtualbox, always use eth0.
Facter.add(:logical_interface) do
  confine kernel: 'Linux'
  confine virtual: ['gce', 'virtualbox']
 
  setcode do
    Facter.value(:networking)['interfaces']['eth0']
  end
end

And here's the spec test:

require 'spec_helper'
 
describe 'logical_interface', type: :fact do
  let(:networking_hash) do
    {
      'primary'    => 'eth42',
      'interfaces' => {
        'eth0'    => { 'ip' => '2.2.2.2' },
        'eth42:1' => { 'ip' => '1.1.1.1' },
      },
    }
  end
  let(:networking_hash_no_alias) do
    {
      'primary'    => 'eth0',
      'interfaces' => {
        'eth0' => { 'ip' => '3.3.3.3' },
      },
    }
  end
  # GCE should act the same as on-prem
  let(:networking_hash_gce) do
    networking_hash
  end
 
  context 'On a physical host' do
    before(:each) do
      Facter.clear
      # Workaround for https://github.com/puppetlabs/pdk/issues/694
      if Facter.fact(:networking).nil?
        Facter.add(:networking) {}
        Facter.flush
      end
 
      allow(Facter.fact(:networking)).to receive(:value).and_return(networking_hash)
      allow(Facter.fact(:kernel)).to receive(:value).and_return('Linux')
      allow(Facter.fact(:virtual)).to receive(:value).and_return('physical')
    end
 
    it "returns the primary's sub-interface hash" do
      expect(Facter.fact(:logical_interface).value).to eq('ip' => '1.1.1.1')
    end
  end
 
  context 'On a physical host with no sub-interface' do
    before(:each) do
      Facter.clear
      # Workaround for https://github.com/puppetlabs/pdk/issues/694
      if Facter.fact(:networking).nil?
        Facter.add(:networking) {}
        Facter.flush
      end
 
      allow(Facter.fact(:networking)).to receive(:value).and_return(networking_hash_no_alias)
      allow(Facter.fact(:virtual)).to receive(:value).and_return('physical')
    end
 
    it 'returns nil (undef)' do
      expect(Facter.fact(:logical_interface).value).to be_nil
    end
  end
 
  context 'On a GCE host' do
    before(:each) do
      Facter.clear
      # Workaround for https://github.com/puppetlabs/pdk/issues/694
      if Facter.fact(:networking).nil?
        Facter.add(:networking) {}
        Facter.flush
      end
 
      allow(Facter.fact(:networking)).to receive(:value).and_return(networking_hash_gce)
      allow(Facter.fact(:kernel)).to receive(:value).and_return('Linux')
      allow(Facter.fact(:virtual)).to receive(:value).and_return('gce')
    end
 
    it "returns eth0's hash" do
      expect(Facter.fact(:logical_interface).value).to eq('ip' => '2.2.2.2')
    end
  end
end

When Facter 4 is used by the PDK, the tests fail:

 $ pdk test unit --tests spec/unit/facter/logical_interface_spec.rb
pdk (INFO): Using Ruby 2.5.8
pdk (INFO): Using Puppet 6.17.0
[✔] Preparing to run the unit tests.
 
Failures:
 
  1) logical_interface On a physical host returns the primary's sub-interface hash
     Failure/Error: expect(Facter.fact(:logical_interface).value).to eq('ip' => '1.1.1.1')
 
       expected: {"ip"=>"1.1.1.1"}
            got: nil
 
       (compared using ==)
     # ./spec/unit/facter/logical_interface_spec.rb:35:in `block (3 levels) in <top (required)>'
 
  2) logical_interface On a GCE host returns eth0's hash
     Failure/Error: expect(Facter.fact(:logical_interface).value).to eq('ip' => '2.2.2.2')
 
       expected: {"ip"=>"2.2.2.2"}
            got: nil
 
       (compared using ==)
     # ./spec/unit/facter/logical_interface_spec.rb:60:in `block (3 levels) in <top (required)>'
 
Finished in 3.03 seconds (files took 3.79 seconds to load)
3 examples, 2 failures

You can see that Facter 4.0.30 was used:

 $ grep facter Gemfile.lock
    facter (4.0.30)
    facterdb (1.2.0)
      facter
      facter (> 2.0.1, < 5)
      facterdb (>= 0.4.0)
      facterdb (>= 0.8.1, < 2.0.0)
      facter
      facterdb (>= 0.5.0)

But if I use an older version of PDK that comes with Facter 2, the tests pass:

$ pdk test unit --tests spec/unit/facter/logical_interface_spec.rb
pdk (INFO): Using Ruby 2.5.7
pdk (INFO): Using Puppet 6.13.0
[✔] Preparing to run the unit tests.
...
 
Finished in 3.31 seconds (files took 3.46 seconds to load)
3 examples, 0 failures
 
$ pdk --version
1.17.0
 
$ grep facter Gemfile.lock
    facter (2.5.7)
    facterdb (1.4.0)
      facter (< 4.0.0)
      facter (~> 2.5.1)
      facter (> 2.0.1, < 5)
      facterdb (>= 0.4.0)
      facterdb (>= 0.8.1, < 2.0.0)
      facter
      facterdb (>= 0.5.0)

Add Comment Add Comment
 
This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)
Atlassian logo

Nate McCurdy (Jira)

unread,
Jul 27, 2020, 2:10:04 PM7/27/20
to puppe...@googlegroups.com
Nate McCurdy updated an issue
Change By: Nate McCurdy
I have a custom Ruby fact that uses {{confine}} blocks to limit when it's run. The spec tests for this fact have worked up until I upgraded to the PDK 1.18 which comes with Facter 4 by default.

Here's the fact:
{code:ruby}

# Our physical hosts have an alias interface of :1 as the logical one.
Facter.add(:logical_interface) do
  confine kernel: 'Linux'
  confine virtual: 'physical'

  setcode do
    networking = Facter.value(:networking)
    primary_alias_iface = "#{networking['primary']}:1"
    unless networking['interfaces'][primary_alias_iface].nil?
      networking['interfaces'][primary_alias_iface]
    end
  end
end

# In GCP and Virtualbox, always use eth0.
Facter.add(:logical_interface) do
  confine kernel: 'Linux'
  confine virtual: ['gce', 'virtualbox']

  setcode do
    Facter.value(:networking)['interfaces']['eth0']
  end
end
{code}


And here's the spec test:
{code:ruby}
{code}


When Facter 4 is used by the PDK, the tests fail:
{noformat}

$ pdk test unit --tests spec/unit/facter/logical_interface_spec.rb
pdk (INFO): Using Ruby 2.5.8
pdk (INFO): Using Puppet 6.17.0
[✔] Preparing to run the unit tests.

Failures:

  1) logical_interface On a physical host returns the primary's sub-interface hash
     Failure/Error: expect(Facter.fact(:logical_interface).value).to eq('ip' => '1.1.1.1')

       expected: {"ip"=>"1.1.1.1"}
            got: nil

       (compared using ==)
     # ./spec/unit/facter/logical_interface_spec.rb:35:in `block (3 levels) in <top (required)>'

  2) logical_interface On a GCE host returns eth0's hash
     Failure/Error: expect(Facter.fact(:logical_interface).value).to eq('ip' => '2.2.2.2')

       expected: {"ip"=>"2.2.2.2"}
            got: nil

       (compared using ==)
     # ./spec/unit/facter/logical_interface_spec.rb:60:in `block (3 levels) in <top (required)>'

Finished in 3.03 seconds (files took 3.79 seconds to load)
3 examples, 2 failures
{noformat}


You can see that Facter 4.0.30 was used:
{noformat}

$ grep facter Gemfile.lock
    facter (4.0.30)
    facterdb (1.2.0)
      facter
      facter (> 2.0.1, < 5)
      facterdb (>= 0.4.0)
      facterdb (>= 0.8.1, < 2.0.0)
      facter
      facterdb (>= 0.5.0)
{noformat}


But if I use an older version of PDK that comes with Facter 2, the tests pass:
{noformat}

$ pdk test unit --tests spec/unit/facter/logical_interface_spec.rb
pdk (INFO): Using Ruby 2.5.7
pdk (INFO): Using Puppet 6.13.0
[✔] Preparing to run the unit tests.
...

Finished in 3.31 seconds (files took 3.46 seconds to load)
3 examples, 0 failures

$ pdk --version
1.17.0

$ grep facter Gemfile.lock
    facter (2.5.7)
    facterdb (1.4.0)
      facter (< 4.0.0)
      facter (~> 2.5.1)
      facter (> 2.0.1, < 5)
      facterdb (>= 0.4.0)
      facterdb (>= 0.8.1, < 2.0.0)
      facter
      facterdb (>= 0.5.0)
{noformat}

----

And the reason I've labelled this ticket to be about "confine" blocks is because if I change the confine blocks to this, the tests pass:
{code}
confine do                    
  Facter.fact(:kernel).value == 'Linux' &&
  Facter.fact(:virtual).value == 'physical'    
end                    
{code}

So it appears that the standard method of using:
{code}
confine some_fact: 'some_value'
{code}
And stubbing that in the spec test as:
{code}
allow(Facter.fact(:some_fact)).to receive(:value).and_return('some_value')
{code}

No longer works.


Nate McCurdy (Jira)

unread,
Jul 27, 2020, 2:13:03 PM7/27/20
to puppe...@googlegroups.com

Nate McCurdy (Jira)

unread,
Jul 27, 2020, 2:42:03 PM7/27/20
to puppe...@googlegroups.com
Nate McCurdy commented on Bug FACT-2724

To simplify this down even more, it appears that confining a fact with syntax like:

confine :kernel => 'Linux'    
confine :virtual => 'physical'

can no longer be mocked with stubs like this:

allow(Facter.fact(:virtual)).to receive(:value).and_return('physical')
allow(Facter.fact(:kernel)).to receive(:value).and_return('Linux')

How should we write spec tests for facts and mock other facts to satisfy a confine block?

Nate McCurdy (Jira)

unread,
Jul 27, 2020, 6:16:04 PM7/27/20
to puppe...@googlegroups.com
Nate McCurdy commented on Bug FACT-2724

Update... these tests only fail when I add the webmock gem to the Gemfile: https://github.com/bblimke/webmock
If I remove the webmock gem, all the existing tests pass!

The problem is that I need webmock in this module for some other tests.

It looks like Webmock's hard block on all network requests is somehow interfering with Facter's ability to identify a node's platform.
e.g. https://github.com/puppetlabs/puppet/pull/7767 and https://bugs.launchpad.net/puppet-nova/+bug/1492636

Nate McCurdy (Jira)

unread,
Jul 27, 2020, 6:34:03 PM7/27/20
to puppe...@googlegroups.com
Nate McCurdy commented on Bug FACT-2724

Figured out a workaround. And that was to add: allow(Facter.add(:virtual))

e.g.

before(:each) do
  Facter.clear
  allow(Facter.add(:virtual))
  allow(Facter.fact(:kernel)).to receive(:value).and_return('Linux')
  allow(Facter.fact(:virtual)).to receive(:value).and_return('physical')
end

Bogdan Irimie (Jira)

unread,
Jul 28, 2020, 2:48:04 AM7/28/20
to puppe...@googlegroups.com
Bogdan Irimie updated an issue
 
Change By: Bogdan Irimie
Sprint: ready for triage
Sub-team: ghost
Team: Night's Watch

Bogdan Irimie (Jira)

unread,
Jul 29, 2020, 9:11:03 AM7/29/20
to puppe...@googlegroups.com

Bogdan Irimie (Jira)

unread,
Jul 29, 2020, 9:17:02 AM7/29/20
to puppe...@googlegroups.com

Bogdan Irimie (Jira)

unread,
Jul 29, 2020, 9:17:03 AM7/29/20
to puppe...@googlegroups.com
Bogdan Irimie assigned an issue to Unassigned

Bogdan Irimie (Jira)

unread,
Jul 29, 2020, 9:17:03 AM7/29/20
to puppe...@googlegroups.com

Bogdan Irimie (Jira)

unread,
Jul 29, 2020, 9:17:05 AM7/29/20
to puppe...@googlegroups.com

Bogdan Irimie (Jira)

unread,
Jul 30, 2020, 9:40:04 AM7/30/20
to puppe...@googlegroups.com

Bogdan Irimie (Jira)

unread,
Aug 3, 2020, 10:05:03 AM8/3/20
to puppe...@googlegroups.com
Bogdan Irimie commented on Bug FACT-2724
 
Re: Confine blocks behave differently with Facter 4, causing spec tests to suddenly fail

Nate McCurdy thanks for all the details, they were very useful and allowed us to reproduce the issue. I assume you are using `macOS`, please let me know if this assumption is incorrect.

`virtual` fact for `macOS` was not working correctly on `physical` host. Instead of resolving `virtual` fact to `physical`, it was resolving it to `nil`. Because of this issue, your mock

allow(Facter.fact(:virtual)).to receive(:value).and_return('physical')|

was actually

allow(nil).to receive(:value).and_return('physical')|

When you added

Facter.add(:virtual)

you basically created a custom fact and the mock was made on that custom fact instead of `nil`

 

Please give https://github.com/puppetlabs/facter/pull/2010 a try. It should fix the issue.

Nate McCurdy (Jira)

unread,
Aug 3, 2020, 2:08:03 PM8/3/20
to puppe...@googlegroups.com
Nate McCurdy commented on Bug FACT-2724

Bogdan Irimie Correct, I was running the tests on a Mac laptop.

Looking at the PR, does defaulting to 'physical' make sense considering MacOS can be virtualized? Though I suppose that's better than defaulting to 'nil'

Bogdan Irimie (Jira)

unread,
Aug 4, 2020, 9:05:03 AM8/4/20
to puppe...@googlegroups.com

Nate McCurdy We try to detect the hypervisors (vmware, virtualbox and parallels) and if none are detected we conclude it is a physical machine.

This is similar to Facter 3.x (C Facter) implementation https://github.com/puppetlabs/facter/blob/fd44acffcc64544da913e49acfa02bf0da7be62c/lib/src/facts/resolvers/virtualization_resolver.cc#L43

Nate McCurdy (Jira)

unread,
Aug 7, 2020, 12:46:04 PM8/7/20
to puppe...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages