Jira (PUP-8660) "puppet apply" should not ignore updated custom facts within the module under test

12 views
Skip to first unread message

Scott Pruzinsky (JIRA)

unread,
Apr 12, 2018, 11:15:02 PM4/12/18
to puppe...@googlegroups.com
Scott Pruzinsky created an issue
 
Puppet / Improvement PUP-8660
"puppet apply" should not ignore updated custom facts within the module under test
Issue Type: Improvement Improvement
Affects Versions: PUP 4.10.1
Assignee: Unassigned
Components: CLI, PE
Created: 2018/04/12 8:14 PM
Priority: Normal Normal
Reporter: Scott Pruzinsky

Puppet Version: PE 2016.4.5
Puppet Server Version:  2016.4.5**
OS Name/Version:  RHEL 6.4

When using "puppet apply" to locally smoke-test updates to an existing and already "deployed" module, everything works properly except when the module happens to also contain an updated custom fact.  The updated custom fact never gets loaded; instead, "puppet apply" always reads from the pluginsync'ed cache, regardless of "modulepath" setting.  (In this particular case, the custom fact update added support for a second operating system via a small change to the confine directive.)

The module autoload seems to be pretty deterministic and follows the modulepath (user-definable), so why wouldn’t fact loading do the same?  Since “puppet apply” is totally disconnected and unaware of any master anyway, why does it bother loading custom facts from the cache since it really should not count on them being there in the first place? It should be able to get them all via dependencies in “modulepath”.

Proposed workarounds were to delete the old fact from the cache under /opt/puppetlabs/puppet/cache (which works until pluginsync kicks in again), rename the module and fact under test (kludgey, prone to error, and does not test code as-is; and not conducive to integration testing).

Steps to reproduce (transcribed–not from copy & paste):

file { ‘/opt/softwaredir’:

  ensure => directory,

  owner => ‘root’,

  group  => $::my_fact ?

{     /[a-zA-Z0-9]+/ => $::my_fact,     default            => undef,   }

,

}

 

<mymodule>/lib/facter/my_fact.rb

 

getmyfact = “/usr/bin/groups <pick a user> | /usr/bin/tr ‘ ‘ ‘\012’ | /bin/egrep ‘^(<group1>|<group2>|<group3>)$’ | /usr/bin/head -1”

 

Facter.add(:my_fact) do

  confine kernel: 'Solaris'

  setcode do

    Facter::Core::Execution.exec(getmyfact)

  end

end

 

UPDATE to custom fact:

   confine kernel: ['Solaris', 'Linux']

If you test on Linux, the fact never gets created because it is still 'Solaris'-only in the cache.  However, "facter" --custom-dir ...  works properly.

 

Desired Behavior:  Either "puppet apply" should ignore pluginsync'ed cache completely, write the new custom fact to the cache and use it, follow "modulepath" for loading custom facts, or have some sort of option/flag to override loading custom facts from cache.  (The independent, master-oblivious "puppet apply" should use the files from the module under test, and pull from modulepath if not found.)

Actual Behavior:  "puppet apply" ignores updated custom fact file and reads old/stale data from cache.

Log output is not possible, since systems exist in a closed environment (no Internet).

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)
Atlassian logo

Martin Ewings (JIRA)

unread,
Apr 13, 2018, 4:29:03 AM4/13/18
to puppe...@googlegroups.com
Martin Ewings updated an issue
Change By: Martin Ewings
Zendesk Ticket IDs: 29447
Zendesk Ticket Count: 1

Martin Ewings (JIRA)

unread,
Apr 13, 2018, 4:29:03 AM4/13/18
to puppe...@googlegroups.com
Martin Ewings updated an issue
Change By: Martin Ewings
CS Priority: Needs Priority
Zendesk Ticket IDs: 29447
Zendesk Ticket Count: 1

Henrik Lindberg (JIRA)

unread,
Apr 13, 2018, 4:45:02 AM4/13/18
to puppe...@googlegroups.com

Owen Rodabaugh (JIRA)

unread,
Apr 17, 2018, 11:11:02 AM4/17/18
to puppe...@googlegroups.com
Owen Rodabaugh updated an issue
Change By: Owen Rodabaugh
CS Priority: Needs Priority Minor
CS Impact: This behavior is very surprising to the user and requires significant effort to track down what is happening and figure out how to work around it. Ideally `puppet apply` would not rely on the cache, or provide some sort of switch to force using the modulepath.
CS Severity: 3 - Serious
CS Business Value: 2 - $$$
CS Frequency: 2 - 5-25% of Customers

Rob Braden (JIRA)

unread,
Aug 8, 2018, 4:56:03 PM8/8/18
to puppe...@googlegroups.com

Josh Cooper (Jira)

unread,
Apr 1, 2020, 2:21:03 AM4/1/20
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
*Puppet Version: PE 2016.4.5*
*Puppet Server Version:*  *2016.4.5***
*OS Name/Version:  RHEL 6.4*


When using "puppet apply" to locally smoke-test updates to an existing and already "deployed" module, everything works properly except when the module happens to also contain an updated custom fact.  The updated custom fact never gets loaded; instead, "puppet apply" always reads from the pluginsync'ed cache, regardless of "modulepath" setting.  (In this particular case, the custom fact update added support for a second operating system via a small change to the confine directive.)

The module autoload seems to be pretty deterministic and follows the modulepath (user-definable), so why wouldn’t fact loading do the same?  Since “puppet apply” is totally disconnected and unaware of any master anyway, why does it bother loading custom facts from the cache since it really should not count on them being there in the first place? It should be able to get them all via dependencies in “modulepath”.

Proposed workarounds were to delete the old fact from the cache under /opt/puppetlabs/puppet/cache (which works until pluginsync kicks in again), rename the module and fact under test (kludgey, prone to error, and does not test code as-is; and not conducive to integration testing).


Steps to reproduce (transcribed–not from copy & paste):

{code:puppet}
file { ‘/opt/softwaredir’:

  ensure => directory,

  owner => ‘root’,

  group  => $::my_fact ? {

    /[a-zA-Z0-9]+/ => $::my_fact,

    default            => undef,

  },
}
{code }

 

{code:ruby}
#
<mymodule>/lib/facter/my_fact.rb

 

getmyfact = “/usr/bin/groups <pick a user> | /usr/bin/tr ‘ ‘ ‘\012’ | /bin/egrep ‘^(<group1>|<group2>|<group3>)$’ | /usr/bin/head -1”

 

Facter.add(:my_fact) do

  confine kernel: 'Solaris'

  setcode do

    Facter::Core::Execution.exec(getmyfact)

  end

end

{code}  

UPDATE to custom fact:

{code:ruby}
   confine kernel: ['Solaris', 'Linux']
{code}

If you test on Linux, the fact never gets created because it is still 'Solaris'-only in the cache.  However, "facter" --custom-dir ...  works properly.

 

*Desired Behavior:  Either "puppet apply" should ignore pluginsync'ed cache completely, write the new custom fact to the cache and use it, follow "modulepath" for loading custom facts, or have some sort of option/flag to override loading custom facts from cache.  (The independent, master-oblivious "puppet apply" should use the files from the module under test, and pull from modulepath if not found.)*

*Actual Behavior:  "puppet apply" ignores updated custom fact file and reads old/stale data from cache.*


Log output is not possible, since systems exist in a closed environment (no Internet).
This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)
Atlassian logo

Josh Cooper (Jira)

unread,
Apr 1, 2020, 3:21:04 AM4/1/20
to puppe...@googlegroups.com
Josh Cooper commented on Improvement PUP-8660
 
Re: "puppet apply" should not ignore updated custom facts within the module under test

Ideally puppet apply shouldn't load from the libdir cache (where the agent downloads plugins to), but changing that in a 6.x release would likely break someone. One workaround is to set the libdir setting to a non-existent directory when running puppet apply. So given a custom fact factpath:

Facter.add('factpath') do
  setcode do
    "Loaded from #{__FILE__}"
  end
end

contained in both:

/opt/puppetlabs/puppet/cache/lib/facter/factpath.rb
/etc/puppetlabs/code/environments/production/modules/a/lib/facter/factpath.rb

Puppet apply prefers the libdir version:

[root@brief-hearsay facter]# puppet apply -e 'notice($facts["factpath"])'
Notice: Scope(Class[main]): Loaded from /opt/puppetlabs/puppet/cache/lib/facter/factpath.rb
Notice: Compiled catalog for brief-hearsay.delivery.puppetlabs.net in environment production in 0.01 seconds
Notice: Applied catalog in 0.01 seconds

But can be configured to ignore it:

[root@brief-hearsay facter]# puppet apply -e 'notice($facts["factpath"])' --libdir /dev/null
Notice: Scope(Class[main]): Loaded from /etc/puppetlabs/code/environments/production/modules/a/lib/facter/factpath.rb
Notice: Compiled catalog for brief-hearsay.delivery.puppetlabs.net in environment production in 0.01 seconds
Notice: Applied catalog in 0.01 seconds

Josh Cooper (Jira)

unread,
Apr 1, 2020, 3:23:02 AM4/1/20
to puppe...@googlegroups.com

Josh Cooper (Jira)

unread,
Aug 24, 2020, 1:40:03 PM8/24/20
to puppe...@googlegroups.com

Josh Cooper (Jira)

unread,
Nov 2, 2020, 7:49:02 PM11/2/20
to puppe...@googlegroups.com
Josh Cooper commented on Improvement PUP-8660

This is a hard thing to fix because some people distribute report processors from the primary compiler to secondary compilers, and in that case, you do want the pluginsync'ed report processor to take precedence. This isn't going to make puppet7 as more research is required to find a path forward on this.

Josh Cooper (Jira)

unread,
Nov 2, 2020, 7:49:03 PM11/2/20
to puppe...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages