So some testing. First let me explain my current setup:
- agents send all requests to an haproxy
- haproxy forwards requests with /puppet-ca to one specific puppetserver
- all other requests go to another group of puppetservers
the puppetserver for CA requests has the msgpack gem installed and was restarted after it was installed. I purged it on all other puppetservers and restarted them. Log from an existing/working puppet agent, where I also purged the msgpack gem:
Debug: Puppet::Network::Format[msgpack]: feature msgpack is missing |
Debug: catalog supports formats: rich_data_json json rich_data_msgpack pson yaml dot |
Debug: Closing connection for https://*:8140 |
Debug: Creating new connection for https://*:8140 |
Debug: Starting connection for https://*:8140 |
Debug: Using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 |
Debug: HTTP POST https://*:8140/puppet/v3/catalog/*?environment=production returned 200 OK |
Debug: Caching connection for https://*:8140
|
besides that, the run works fine. Now I took the same agent, without msgpack installed, and it was talking directly to the CA server with msgpack installed:
Debug: Puppet::Network::Format[msgpack]: feature msgpack is missing |
Debug: catalog supports formats: rich_data_json json rich_data_msgpack pson yaml dot |
Debug: Closing connection for https://*:8140 |
Debug: Creating new connection for https://*:8140 |
Debug: Starting connection for https://*:8140 |
Debug: HTTP POST https://*:8140/puppet/v3/catalog/*?environment=production returned 200 OK
|
agent run works fine as well. Now I took the same agent, without msgpack installed, and it was talking directly to the CA server with msgpack purged (and puppetserver restarted):
Debug: catalog supports formats: rich_data_json json rich_data_msgpack pson yaml dot |
Debug: Closing connection for https://*:8140 |
Debug: Creating new connection for https://*:8140 |
Debug: Starting connection for https://*:8140 |
Debug: Using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 |
Debug: HTTP POST https://*:8140/puppet/v3/catalog/*?environment=production returned 200 OK |
Debug: Caching connection for https://*:8140
|
rich_data_msgpack should not be listed here. I repeated the debug run and added --http_debug
<- "POST /puppet/v3/catalog/*?environment=production HTTP/1.1\r\nX-Puppet-Version: 6.19.1\r\nUser-Agent: Puppet/6.19.1 Ruby/2.5.8-p224 (x86_64-linux)\r\nAccept: application/vnd.puppet.rich+json, application/json, application/vnd.puppet.rich+msgpack, text/pson\r\nContent-Type: application/x-www-form-urlencoded\r\nAccept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3\r\nHost: *:8140\r\nContent-Length: 38071\r\n\r\n" |
... long text is long... |
-> "HTTP/1.1 200 OK\r\n" |
-> "Server: nginx/1.18.0\r\n" |
-> "Date: Wed, 04 Nov 2020 15:05:37 GMT\r\n" |
-> "Content-Type: application/vnd.puppet.rich+json; charset=utf-8\r\n" |
-> "Transfer-Encoding: chunked\r\n" |
-> "Connection: keep-alive\r\n" |
-> "X-Puppet-Version: 6.19.1\r\n" |
-> "Vary: Accept-Encoding, User-Agent\r\n" |
-> "Content-Encoding: gzip\r\n"
|
- it looks like puppetserver listens on localhost and nginx terminates the TLS. This was setup so long ago that I forgot it
- msgpack is still listed as supported, but msgpack gem isn't present on the puppetserver and also not on the agent
- the run seems to work.
I purged puppet with yum, deleted /opt/puppetlabs/puppet and reinstalled the agent. msgpack isn't installed. It tried to talk to a puppserver where msgpack is installed:
ebug: Puppet::Network::Format[msgpack]: feature msgpack is missing |
Debug: catalog supports formats: rich_data_json json rich_data_msgpack pson yaml dot |
Debug: Closing connection for https://*:8140 |
Debug: Creating new connection for https://*:8140 |
Debug: Starting connection for https://*:8140 |
Debug: Using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 |
Debug: HTTP POST https://*:8140/puppet/v3/catalog/*?environment=production returned 200 OK |
Debug: Caching connection for https://*:8140
|
this passes as well? I will try to get a completley new system in the next days and that with that. Somehow it should be possible to reproduce the initial error I had while opening the ticket. |