| |
|
|
| Change By: |
Josh Cooper |
| Release Notes: |
Bug Fix |
| Release Notes Summary: |
When running in strict environment mode, we now fail the run early if the requested environment does not exist on the server or if the server does not allow the agent to specify it's own environment. |
| Summary: |
No option to fail fast when agent-specified environment does not exist {brief summary of issue} |
|
h2. The Problem
When using an agent-specified environment workflow and the requested environment does not exist, there is no way to halt the Puppet run early and prevent a catalog compilation.
Additionally, the behavior of automatically switching to the "production" environment is unexpected and not desired in an agent-specified environment workflow.
This behavior exposes multiple issues...
* When not using sctrict_environment_mode Puppet Version :*
* The agent gets a 404 from the {{file_metadatas}} endpoint, but it still submits a catalog request:
**
{noformat}[root@agent7 ~]# puppet agent -t --environment fake --http_debug
Info: Using environment 'fake'
opening connection to server7.vagrant:8140...
opened
starting SSL for server7.vagrant:8140...
SSL established, protocol: TLSv1.3, cipher: TLS_AES_128_GCM_SHA256
<- "GET /puppet/v3/file_metadatas/plugins?recurse=false&links=manage&checksum_type=sha256&source_per
.5-p203 (x86_64-linux)\r\nAccept: application/json, text/pson\r\nAccept-Encoding: gzip;q=1.0,deflate
-> "HTTP/1.1 404 Not Found\r\n"
-> "Date: Mon, 31 Jan 2022 21:47:28 GMT\r\n"
-> "Content-Type: application/json;charset=utf-8\r\n"
-> "X- Puppet - Server Version: 7.14.0\r\n"
-> "Content-Length: 87\r\n"
-> "\r\n"
reading 87 bytes...
-> "{\"message\":\"Not Found: Could not find environment 'fake'\",\"issue_kind\":\"RUNTIME_ERROR\"}"
read 87 bytes
Conn keep-alive
Notice: Environment 'fake' not found on server, skipping initial pluginsync.
<- "POST /puppet/v3/catalog/agent7.vagrant?environment=fake HTTP/1.1\r\nX-Puppet-Version: 7.14.0\r\n{noformat}
*
** This puts unneeded load on the Puppetserver while it compiles a catalog.
* The server responds with a 200, which is odd considering the environment doesn't exist.
**
{noformat}-> "HTTP OS Name / 1.1 200 OK\r\n"
-> "Date: Mon, 31 Jan 2022 21:47:28 GMT\r\n"
-> "Content-Type: application/vnd.puppet.rich+json; charset=utf-8\r\n"
-> "X-Puppet- Version: 7.14.0\r\n" {noformat} *
* The agent then switches to the "production" environment. Apparently because it's server-specified is "production". But Describe your issue in my case the external node classifier (ENC) is NOT specifying any environment at all. as much detail as possible…
**
{noformat}Notice: Local environment: 'fake' doesn't match server specified environment 'production', restarting agent run with environment 'production' {noformat} Describe steps to reproduce…
* Here's my ENC script used for testing Desired Behavior :
* *
{code:bash}#!/bin/bash
cat <<EOF
---
class *Actual Behavior : {} *
parameters: {}
EOF {code}
* The agent then does pluginsync against the production environment Please take a moment and follows up with another catalog request, this time against production.
**
{noformat}<- "POST attach any relevant log output and / puppet/v3/catalog/agent7 or manifests . vagrant?environment=production HTTP/1.1\r\nX-Puppet-Version: 7.14.0\r\nUser-Agent: Puppet/7.14.0 Ruby/2.7.5-p203 (x86_64-linux)\r\nAccept: application/vnd.puppet.rich+json, application/json, text/pson\r\nContent-Type: application/x-www-form-urlencoded\r\nAccept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3\r\nHost: server7.vagrant:8140\r\nContent-Length: 25797\r\n\r\n" {noformat}
* This puts even more unnecessary load on will help us immensely when troubleshooting the Puppetserver issue .
*With {{--strict_environment_mode Examples : }}*
* The agent again gets the 404 from file_metadatas but for some reason it still requests a catalog from the server.
**
{noformat}[root@agent7 ~]# Run puppet agent with - t - test - environment fake - trace - http_debug - -strict_environment_mode debug
Info: Using environment 'fake'
opening connection to server7.vagrant:8140...
opened
starting SSL for server7.vagrant:8140...
SSL established, protocol: TLSv1.3, cipher: TLS_AES_128_GCM_SHA256
<- "GET /puppet/v3/file_metadatas/plugins?recurse=false&links=manage&checksum_type=sha256&source_permissions=ignore&environment=fake HTTP/1.1\r\nX-Puppet-Version: 7.14.0\r\nUser-Agent: Puppet/7.14.0 Ruby/2.7.5-p203 (x86_64-linux)\r\nAccept: application/json, text/pson\r\nAccept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3\r\nHost: server7.vagrant:8140\r\n\r\n"
-> "HTTP/1.1 404 Not Found\r\n"
-> "Date: Mon, 31 Jan 2022 22:03:16 GMT\r\n"
-> "Content-Type: application/json;charset=utf-8\r\n"
-> "X-Puppet-Version: 7.14.0\r\n"
-> "Content-Length: 87\r\n"
-> "\r\n"
reading 87 bytes...
-> " Relevant sections of { \"message\":\"Not Found: Could not find environment 'fake'\",\"issue_kind\":\"RUNTIME_ERROR\"}"
read 87 bytes
Conn keep-alive
Notice: Environment 'fake' not found on server, skipping initial pluginsync.
<- "POST { / puppet var / v3 log / catalog puppetlabs / agent7.vagrant?environment=fake HTTP puppetserver / 1 puppetserver . 1\r\nX-Puppet-Version: 7.14.0\r\nUser-Agent: Puppet/7.14.0 Ruby/2.7.5-p203 (x86_64-linux)\r\nAccept: application/vnd.puppet.rich+json, application/json, text/pson\r\nContent-Type: application/x-www-form-urlencoded\r\nAccept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3\r\nHost: server7.vagrant:8140\r\nContent-Length: 25793\r\n\r\n" {noformat log }
** This puts unneeded load on the Puppetserver as it compiles a catalog.
* The agent receives a 200 } or any applicable logs from the server after the catalog request:
** Again, which is weird considering the environment doesn't exist same directory .
**
{noformat}-> "HTTP/1.1 200 OK\r\n"
-> "Date: Mon, 31 Jan 2022 22:03:16 GMT\r\n"
-> "Content-Type: application/vnd.puppet.rich+json; charset=utf-8\r\n"
-> "X-Puppet-Version: 7.14.0\r\n"
-> "Vary: Accept-Encoding, User-Agent\r\n"
-> "Content-Encoding: gzip\r\n"
-> "Content-Length: 316\r\n"
-> "\r\n" {noformat}
* Then the agent gives For more detailed information turn up with a misleading error:
**
{noformat}Error: Not using catalog because its environment 'production' does not match agent specified environment 'fake' and strict_environment_mode is set {noformat}
** This is misleading because the server is NOT specifying an environment (see logs by upping the ENC script above).
** The real problem is that log level in the "fake" environment doesn server ' t exist s logback . xml
h2. Desired Behavior
When using an agent-specified environment workflow:
* The agent should not request a catalog after the initial 404 to the {{file_metadatas}} API Relevant sections of configurations files ( i puppet . e conf, hiera . when pluginsync failed)
* Error messages for non-existent environments shouldn conf, Server ' t assume you're using a server-specified environment s conf .
The points above may be too specific to the current implementation d , so a more generic way to phrase the desired behavior is something like: defaults/sysconfig)
* There should be an agent-side option to fail the Puppet run fast when the requested environment doesn't exist.
* That option should not cause a catalog compilation on the Puppetserver at all.
* The error message seen when the environment doesn't exist should clearly say that the problem is a non-existent environment.
h2. Related Info
This looks like it might be a regression in behavior related to PUP-10582 and possibly was introduced For memory issues with changes made for PUP-6802 server heap dumps are also helpful .
|
|
|
|
| |
|
