Jira (PUP-10363) server_list is processed once per service not once per session

21 views
Skip to first unread message

Josh Cooper (Jira)

unread,
Mar 9, 2020, 6:11:03 PM3/9/20
to puppe...@googlegroups.com
Josh Cooper assigned an issue to Josh Cooper
 
Puppet / Bug PUP-10363
server_list is processed once per service not once per session
Change By: Josh Cooper
Assignee: Josh Cooper
Add Comment Add Comment
 
This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)
Atlassian logo

Josh Cooper (Jira)

unread,
Mar 9, 2020, 6:11:06 PM3/9/20
to puppe...@googlegroups.com
Josh Cooper created an issue
Issue Type: Bug Bug
Assignee: Unassigned
Created: 2020/03/09 3:10 PM
Priority: Normal Normal
Reporter: Josh Cooper

In 6.13.0, if puppet is configured to use server_list, it tries to connect to each entry in the list, stopping at the first HTTP 200 connection to puppetserver's simple status endpoint, and adds the resolved server and port into puppet's context, which is then looked up each time a connection is made. Each time connection attempt may result in a 2 minute timeout for each failed connection. The timeout defaults to 2 minutes, but is configurable using the http_connect_timeout puppet setting.

In 6.14.0, puppet will resolve server_list for puppet, fileserver and report services. This can result in additional delay during the run. So if 6.13.0 was delayed X minutes, then puppet 6.14.0 will be delayed 3X minutes. The exact number depends on how many entries in server_list fail before a successfully connection is made.

Josh Cooper (Jira)

unread,
Mar 9, 2020, 6:12:03 PM3/9/20
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Sprint: Platform Core KANBAN

Josh Cooper (Jira)

unread,
Mar 9, 2020, 7:46:03 PM3/9/20
to puppe...@googlegroups.com

Josh Cooper (Jira)

unread,
Mar 10, 2020, 2:15:04 PM3/10/20
to puppe...@googlegroups.com

Melissa Stone (Jira)

unread,
Mar 10, 2020, 5:02:03 PM3/10/20
to puppe...@googlegroups.com

Josh Cooper (Jira)

unread,
Mar 11, 2020, 1:57:02 PM3/11/20
to puppe...@googlegroups.com

Josh Cooper (Jira)

unread,
Mar 11, 2020, 1:57:03 PM3/11/20
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Release Notes: Bug Fix
Release Notes Summary: Fixes a regression in 6.14.0 that caused puppet agents to process their server_list up to 4 times per agent run, instead of once per run. This restores the behavior that existed in 6.13.0 and before.

Josh Cooper (Jira)

unread,
Mar 11, 2020, 10:39:04 PM3/11/20
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-10363
 
Re: server_list is processed once per service not once per session

This passes CI but is totally broken:

$ bx puppet agent -t --server_list woeful-romantic.delivery.puppetlabs.net
Info: Using configured environment 'production'
Info: Retrieving pluginfacts
Error: /File[/Users/josh/.puppetlabs/opt/puppet/cache/facts.d]: Failed to generate additional resources using 'eval_generate': undefined method `get_file_metadatas' for #<Puppet::HTTP::Service::Compiler:0x00007ffad72a6de8>
Error: /File[/Users/josh/.puppetlabs/opt/puppet/cache/facts.d]: Could not evaluate: Could not retrieve file metadata for puppet:///pluginfacts: undefined method `get_file_metadata' for #<Puppet::HTTP::Service::Compiler:0x00007ffad72a6de8>
Info: Retrieving plugin
Error: /File[/Users/josh/.puppetlabs/opt/puppet/cache/lib]: Failed to generate additional resources using 'eval_generate': undefined method `get_file_metadatas' for #<Puppet::HTTP::Service::Compiler:0x00007ffad72a6de8>
Error: /File[/Users/josh/.puppetlabs/opt/puppet/cache/lib]: Could not evaluate: Could not retrieve file metadata for puppet:///plugins: undefined method `get_file_metadata' for #<Puppet::HTTP::Service::Compiler:0x00007ffad72a6de8>
Info: Retrieving locales
Error: /File[/Users/josh/.puppetlabs/opt/puppet/cache/locales]: Failed to generate additional resources using 'eval_generate': undefined method `get_file_metadatas' for #<Puppet::HTTP::Service::Compiler:0x00007ffad72a6de8>
Error: /File[/Users/josh/.puppetlabs/opt/puppet/cache/locales]: Could not evaluate: Could not retrieve file metadata for puppet:///locales: undefined method `get_file_metadata' for #<Puppet::HTTP::Service::Compiler:0x00007ffad72a6de8>
Info: Caching catalog for XXX.puppet.net
Info: Applying configuration version '1583980590'
Notice: Applied catalog in 0.01 seconds
Error: Could not send report: undefined method `put_report' for #<Puppet::HTTP::Service::Compiler:0x00007ffad72a6de8>

The problem is the server list resolver caches the first *service* that we resolved, not the host & port, so since we first resolve server_list for the node request, we end up caching the "compiler" service and using that for fileserver, report, etc.

Melissa Stone (Jira)

unread,
Mar 13, 2020, 1:27:03 PM3/13/20
to puppe...@googlegroups.com

Josh Cooper (Jira)

unread,
Mar 14, 2020, 4:24:03 PM3/14/20
to puppe...@googlegroups.com

Claire Cadman (Jira)

unread,
Apr 6, 2020, 9:40:03 AM4/6/20
to puppe...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages