Jira (PUP-11323) Puppet agent runs in environment specified on CLI when not in an agent specified environment

87 views
Skip to first unread message

Elaine McCloskey (Jira)

unread,
Oct 18, 2021, 6:14:03 AM10/18/21
to puppe...@googlegroups.com
Elaine McCloskey created an issue
 
Puppet / Bug PUP-11323
Puppet agent runs in environment specified on CLI when not in an agent specified environment
Issue Type: Bug Bug
Affects Versions: PUP 6.25.0
Assignee: Unassigned
Created: 2021/10/18 3:13 AM
Priority: High High
Reporter: Elaine McCloskey

Puppet Version: 6.25
Puppet Server Version: 2019.8.9
OS Name/Version: tested on centos7

Puppet agent runs in environment specified on CLI when not in an agent specified environment

Desired Behavior: 

In Puppet 6.24 the behaviour is as expected, if a CLI environment is specified but the agent is not in a agent specified node group the run outputs a notice and continues in the server specified environment:

[root@pe-node-48716e-0 ~]# puppet agent -t --environment=testing
 
Notice: Local environment: 'testing' doesn't match server specified node environment 'production', switching agent to 'production'.

Actual Behavior:

After upgrading to 6.25 it runs in the CLI specified environment:

[root@pe-node-48716e-0 ~]# puppet agent -t --environment=testing
 
Info: Using environment 'testing'

Add Comment Add Comment
 
This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)
Atlassian logo

Gabriel Nagy (Jira)

unread,
Oct 18, 2021, 7:23:03 AM10/18/21
to puppe...@googlegroups.com
Gabriel Nagy commented on Bug PUP-11323
 
Re: Puppet agent runs in environment specified on CLI when not in an agent specified environment

Hi Elaine McCloskey, we shuffled some log messages around as a result of implementing PUP-10539, however in this case PE/classifier should still be authoritative, regardless of what's specified on the CLI.

Can you share the following logs?

puppet agent -t --debug --environment=testing
 
cat $(puppet config print lastrunfile)

Elaine McCloskey (Jira)

unread,
Oct 18, 2021, 9:55:02 AM10/18/21
to puppe...@googlegroups.com
Elaine McCloskey updated an issue
 
Change By: Elaine McCloskey
Attachment: debug.txt

Elaine McCloskey (Jira)

unread,
Oct 18, 2021, 9:56:02 AM10/18/21
to puppe...@googlegroups.com
Elaine McCloskey commented on Bug PUP-11323
 
Re: Puppet agent runs in environment specified on CLI when not in an agent specified environment

Attached the debug log debug.txt

[root@pe-node-48716e-0 ~]# cat $(puppet config print lastrunfile)
 
---
 
version:
 
 config: pe-server-48716e-0-production-979f19487ef
 
 puppet: 6.25.0
 
application:
 
 run_mode: agent
 
 initial_environment: testing
 
 converged_environment: production
 
resources:
 
 changed: 0
 
 corrective_change: 0
 
 failed: 0
 
 failed_to_restart: 0
 
 out_of_sync: 0
 
 restarted: 0
 
 scheduled: 0
 
 skipped: 0
 
 total: 50
 
time:
 
 catalog_application: 0.4192146209970815
 
 config_retrieval: 0.38215963499897043
 
 convert_catalog: 0.15539181500207633
 
 exec: 0.132791731
 
 fact_generation: 1.6690107839967823
 
 file: 0.022901776
 
 filebucket: 0.000116674
 
 ini_setting: 0.009516789999999999
 
 package: 0.000676858
 
 pe_anchor: 0.000150198
 
 plugin_sync: 10.741396951998468
 
 puppet_agent_end_run: 0.000164698
 
 schedule: 0.000512716
 
 service: 0.017519027
 
 startup_time: 0.923621749
 
 total: 14.897905855
 
 transaction_evaluation: 0.40708175100007793
 
 yumrepo: 0.0011062469999999999
 
 last_run: 1634565018
 
changes:
 
 total: 0
 
events:
 
 failure: 0
 
 success: 0
 
 total: 0

MARCIN BOJKO (Jira)

unread,
Oct 19, 2021, 2:30:02 AM10/19/21
to puppe...@googlegroups.com
MARCIN BOJKO commented on Bug PUP-11323

Same behavior on Windows - puppet agent stops reacting on ENC environment settings and goes with default one.

—
version:
config:


puppet: 6.25.0
application:
run_mode: agent

initial_environment: production


converged_environment: production
resources:
changed: 0
corrective_change: 0
failed: 0
failed_to_restart: 0
out_of_sync: 0
restarted: 0
scheduled: 0
skipped: 0

total: 0
time:
fact_generation: 4.154527799999869
plugin_sync: 5.896013200000198
startup_time: 1.261798
total: 13.344219
last_run: 1634548638


changes:
total: 0
events:
failure: 0
success: 0
total: 0

Ciprian Badescu (Jira)

unread,
Oct 19, 2021, 4:17:03 AM10/19/21
to puppe...@googlegroups.com

Elaine McCloskey, the run starts with 'testing' environment, but later on the environment is changed to production:

Notice: Local environment: 'testing' doesn't match server specified environment 'production', restarting agent run with environment 'production'

MARCIN BOJKO , can you also check for the above message or provide logs?

 

Elaine McCloskey (Jira)

unread,
Oct 19, 2021, 5:31:03 AM10/19/21
to puppe...@googlegroups.com

Hi Ciprian Badescu, I had missed that message among the output. The behaviour is still different from 6.24 and it makes changes before reverting them when running in production. 

 

Notice: /File[/opt/puppetlabs/puppet/cache/lib/puppet/util/ini_file]/ensure: removed
 
Notice: /File[/opt/puppetlabs/puppet/cache/lib/puppet/util/ini_file.rb]/ensure: removed
 
Notice: /File[/opt/puppetlabs/puppet/cache/lib/puppet/util/postgresql_validator.rb]/ensure: removed
 
Notice: /File[/opt/puppetlabs/puppet/cache/lib/puppet/util/setting_value.rb]/ensure: removed
 
Notice: /File[/opt/puppetlabs/puppet/cache/lib/puppet_x/bodeco]/ensure: removed
 
Notice: /File[/opt/puppetlabs/puppet/cache/lib/puppet_x/chocolatey]/ensure: removed
 
Notice: /File[/opt/puppetlabs/puppet/cache/lib/puppet_x/puppetlabs/registry.rb]/ensure: removed
 
Notice: /File[/opt/puppetlabs/puppet/cache/lib/pwsh]/ensure: removed
 
Notice: /File[/opt/puppetlabs/puppet/cache/lib/pwsh.rb]/ensure: removed
 
Notice: /File[/opt/puppetlabs/puppet/cache/lib/ruby-pwsh.rb]/ensure: removed
 
Notice: /File[/opt/puppetlabs/puppet/cache/lib/task_helper.rb]/ensure: removed
 
Notice: /File[/opt/puppetlabs/puppet/cache/lib/templates]/ensure: removed
 
Info: Retrieving locales
 
Notice: /File[/opt/puppetlabs/puppet/cache/locales/ja]/ensure: removed
 
Info: Loading facts
 
Notice: Local environment: 'testing' doesn't match server specified environment 'production', restarting agent run with environment 'production'

 

 
Add Comment Add Comment
 

Ciprian Badescu (Jira)

unread,
Oct 19, 2021, 5:41:02 AM10/19/21
to puppe...@googlegroups.com

Elaine McCloskey, it will do pluginsync twice in this specific case, first for testing environment and a second time for production. It is like that by design. Is it causes any trouble?

Elaine McCloskey (Jira)

unread,
Oct 19, 2021, 8:51:02 AM10/19/21
to puppe...@googlegroups.com

Hi Ciprian Badescu, thanks for confirming this was by design. I don't forsee it causing issues since it seems like it's only affecting puppet runs when the environment is specified on the command and I don't think many customers run puppet this way. It is a change in behaviour between minor versions and running pluginsync twice doesn't seem as efficient as the previous method of immediately determining the correct server-side environment.

 

Ciprian Badescu (Jira)

unread,
Oct 19, 2021, 9:10:04 AM10/19/21
to puppe...@googlegroups.com

Yes, in this specific case we pluginsync twice compared to the alternative used in previous version to initially classify the node based on puppetdb facts (which could have other problems e.g. on initial puppet run)

Josh Cooper, any opinion on this?

Josh Cooper (Jira)

unread,
Oct 19, 2021, 12:55:02 PM10/19/21
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-11323

Just confirming that the agent is working as intended.

We changed how this process works to fix several customer reported issues where agent runs sometimes fail when the environment changes.

We now optimize for the case where the environment doesn't change.

If you're running puppet agent -t and the agent is allowed to select its own environment, then it will usually only pluginsync once. The number of times depends on what facts the agent already has at the start of the run and whether classification is based on those facts.

If you're running puppet agent -t but the agent is NOT allowed to switch environments, then it may pluginsync twice.

Josh Cooper (Jira)

unread,
Oct 21, 2021, 12:41:03 PM10/21/21
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-11323

I'm going to close this ticket as the agent is working as intended. Please reopen if I've missed something.
Reply all
Reply to author
Forward
0 new messages