Jira (PUP-8900) "puppet facts upload" face only tries first master in server_list

19 views
Skip to first unread message

Jesse Reynolds (JIRA)

unread,
May 31, 2018, 8:24:02 PM5/31/18
to puppe...@googlegroups.com
Jesse Reynolds created an issue
 
Puppet / Bug PUP-8900
"puppet facts upload" face only tries first master in server_list
Issue Type: Bug Bug
Affects Versions: PUP 5.5.1
Assignee: Unassigned
Created: 2018/05/31 5:23 PM
Priority: Normal Normal
Reporter: Jesse Reynolds

Puppet Version: 5.5.1
Puppet Server Version: 5.5.2 (2018.1.0.41)
OS Name/Version: RHEL 7

When a node's puppet.conf specifies multiple masters in server_list the facts upload face only tries the first master in the list. If this fails due to a hostname lookup failure on the first master the face exits with an error, rather than trying the second master.

Steps to reproduce:

  • remove any "server" configs from puppet.conf
  • add the following "server_list" configuration: "foo.example,actual-master.example" (where "actual-master.example" is the working master address, and "foo.example" is not resolvable to an IP address)
  • run "puppet facts upload"

Desired Behavior:

  • the face should gracefully fail over to the second master in the server_list and complete the facts upload successfully

Actual Behavior:

  • the face fails with the following error message:

# puppet facts upload
Notice: Uploading facts for 'pe-201730-master.puppetdebug.vlan' to: 'foo.example'
Error: Failed to open TCP connection to foo.example:8140 (getaddrinfo: Name or service not known)
Error: Try 'puppet help facts upload' for usage

I'm attaching the output of "puppet facts upload --debug --trace" as well.

This face (and the facts endpoint in puppetserver) were (re)added in this PR: https://github.com/puppetlabs/puppet/pull/6434

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)
Atlassian logo

Jesse Reynolds (JIRA)

unread,
May 31, 2018, 8:26:02 PM5/31/18
to puppe...@googlegroups.com
Jesse Reynolds updated an issue
Change By: Jesse Reynolds
Attachment: puppet-facts-upload-server_list-debug-trace.txt

Jesse Reynolds (JIRA)

unread,
May 31, 2018, 8:26:03 PM5/31/18
to puppe...@googlegroups.com

Jesse Reynolds (JIRA)

unread,
May 31, 2018, 8:33:02 PM5/31/18
to puppe...@googlegroups.com

Josh Cooper (JIRA)

unread,
Jun 4, 2018, 2:29:02 PM6/4/18
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-8900
 
Re: "puppet facts upload" face only tries first master in server_list

Thanks Jesse Reynolds. Only the agent correctly handles server_list. Branan Riley do you know if there were plans on adding server_list handling to other applications?

Josh Cooper (JIRA)

unread,
Jun 4, 2018, 2:29:03 PM6/4/18
to puppe...@googlegroups.com

Branan Riley (JIRA)

unread,
Jun 4, 2018, 2:34:02 PM6/4/18
to puppe...@googlegroups.com
Branan Riley commented on Bug PUP-8900
 
Re: "puppet facts upload" face only tries first master in server_list

We intentionally kept the scope to the agent application due to the complexities of how it's plumbed hacked into the networking code right now.

Ideally once we have less-magical networking client code in the agent, with failover as a first-class citizen, we can enable it in other applications.

Branan Riley (JIRA)

unread,
Jun 4, 2018, 2:36:03 PM6/4/18
to puppe...@googlegroups.com
Branan Riley commented on Bug PUP-8900

IIRC we had a ticket somewhere to make the docs more clear that failover only occurred in the agent application. I have no idea what the disposition of that ticket ended up being.

Josh Cooper (JIRA)

unread,
Jun 4, 2018, 4:19:02 PM6/4/18
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-8900

Thanks Branan Riley, makes sense. FYI, we've started on a sane http client implementation which supports server_list failover (and SRV records): https://github.com/puppetlabs/puppet/blob/b8759ca8eedf28b132b2488affec1668bb09994b/lib/puppet/rest/route.rb#L70-L88. I'll add this ticket to the Agent HTTP epic. One of the goals is going to be normalizing the various http implementations and migrating call sites over to use the new http client.

Jesse Reynolds (JIRA)

unread,
Jun 27, 2018, 3:18:03 AM6/27/18
to puppe...@googlegroups.com

Neil Binney (JIRA)

unread,
Jun 27, 2018, 3:32:03 AM6/27/18
to puppe...@googlegroups.com
Neil Binney updated an issue
Change By: Neil Binney
CS Priority: Needs Priority Reviewed

Josh Cooper (JIRA)

unread,
Sep 26, 2018, 9:00:08 PM9/26/18
to puppe...@googlegroups.com

Melissa Stone (JIRA)

unread,
Oct 17, 2018, 2:01:04 PM10/17/18
to puppe...@googlegroups.com
Melissa Stone commented on Bug PUP-8900
 
Re: "puppet facts upload" face only tries first master in server_list

Josh Cooper Maggie Dreyer With the recent changes to the rest client, where do we stand on this ticket?

Melissa Stone (JIRA)

unread,
Oct 22, 2018, 3:06:03 PM10/22/18
to puppe...@googlegroups.com
Melissa Stone commented on Bug PUP-8900

Grooming notes:

Currently, only agent knows about server_list. It seems reasonable that other applications that hit a server also have access to server_list.

things that definitely should not need access to server or server_list: config, describe, device

 

Should we move server_list out of configurer, and into requests, like the srv stuff? Anything that does a request would just get it. As long as nothing that doesn't expect it gets it. This should be fine, except the case where a node has server_list (like in PE), but is relying on something to only hit server and not know about server_list. Have a method that allows applications to override it?

 

If we land this in 5.5, we need to be very specific about which applications are impacted (ie only those that we control). We could have something in base that applications can opt into. We also need a list of which applications we will want to opt into this behavior.

 

We were not able to finish grooming this ticket before we ran out of time in the grooming meeting. Will circle back during the follow up meeting.

Melissa Stone (JIRA)

unread,
Oct 26, 2018, 5:32:08 PM10/26/18
to puppe...@googlegroups.com
Melissa Stone commented on Bug PUP-8900

grooming notes:

 

We need to have a conversation about the future of `server_list`. We are leaning toward deprecating it fully. If this is the goal, we shouldn't put too much effort now into improving it. The main part of this conversation is documentation around expectations and usage, ie in PE.

A lot of the functionality you get from server_list might be better achieved via a load balancer or DNS.

We are holding off on moving this ticket anywhere in favor of additional discussion around the lifecycle of `server_list`.

Josh Cooper (JIRA)

unread,
Dec 13, 2018, 4:12:05 PM12/13/18
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Sprint: Coremunity Grooming Hopper

Melissa Stone (JIRA)

unread,
Mar 14, 2019, 7:19:04 PM3/14/19
to puppe...@googlegroups.com
Melissa Stone updated an issue
Change By: Melissa Stone
Sprint: Coremunity Hopper Grooming

Josh Cooper (JIRA)

unread,
Mar 15, 2019, 5:43:03 PM3/15/19
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-8900
 
Re: "puppet facts upload" face only tries first master in server_list

We should add server_list functionality to the rest client, and migrate applications to the using the rest client instead of the indirector. Moving this to the Agent HTTP epic, as we need the client changes to be implemented first.

Josh Cooper (JIRA)

unread,
Apr 2, 2019, 9:20:03 PM4/2/19
to puppe...@googlegroups.com

Josh Cooper (JIRA)

unread,
Jan 6, 2020, 12:20:04 PM1/6/20
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Fix Version/s: PUP 5.5.z
Fix Version/s: PUP 6.y

Josh Cooper (JIRA)

unread,
Jan 13, 2020, 7:35:04 PM1/13/20
to puppe...@googlegroups.com

Josh Cooper (JIRA)

unread,
Jan 13, 2020, 7:37:04 PM1/13/20
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-8900
 
Re: "puppet facts upload" face only tries first master in server_list

Grooming Notes:

1. Target master
2. Change the puppet facts upload action to use the httpclient, route to the puppet service, and call its put_report method.

Josh Cooper (JIRA)

unread,
Jan 17, 2020, 12:27:03 AM1/17/20
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Fix Version/s: PUP 6.y
Fix Version/s: PUP 6.13.0

Josh Cooper (JIRA)

unread,
Jan 17, 2020, 1:16:04 AM1/17/20
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Sprint: Coremunity Grooming Platform Core KANBAN

Josh Cooper (JIRA)

unread,
Jan 17, 2020, 1:16:05 AM1/17/20
to puppe...@googlegroups.com

Melissa Stone (JIRA)

unread,
Jan 22, 2020, 7:59:04 PM1/22/20
to puppe...@googlegroups.com

Melissa Stone (JIRA)

unread,
Jan 22, 2020, 8:00:05 PM1/22/20
to puppe...@googlegroups.com

Josh Cooper (JIRA)

unread,
Jan 22, 2020, 10:39:04 PM1/22/20
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Release Notes Summary: "puppet facts upload" now tries each server entry in the server_list setting in the same way that "puppet agent" does. Previously it only tried the first entry in the server_list setting.
Release Notes: Bug Fix

Josh Cooper (JIRA)

unread,
Jan 25, 2020, 12:16:05 AM1/25/20
to puppe...@googlegroups.com

Kate Medred (JIRA)

unread,
Feb 18, 2020, 12:06:04 PM2/18/20
to puppe...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages