Jira (PUP-8213) Error message for certificate name mismatch is clear as mud

1 view
Skip to first unread message

Reid Vandewiele (JIRA)

unread,
Nov 28, 2017, 7:42:04 PM11/28/17
to puppe...@googlegroups.com
Reid Vandewiele created an issue
 
Puppet / Bug PUP-8213
Error message for certificate name mismatch is clear as mud
Issue Type: Bug Bug
Assignee: Unassigned
Created: 2017/11/28 4:41 PM
Priority: Normal Normal
Reporter: Reid Vandewiele

A common error in enterprise environments is a mismatch between the server setting on the agent and the common name of the master's cert / its dns alt names.

Today, if this situation occurs the error message the agent prints is as clear as mud. For example:

# You can't tell, but for this run server=master.tld
 
# 
[root@aws-1 ssl]# puppet agent -t
 
Warning: Unable to fetch my node definition, but the agent run will continue:
 
Warning: SSL_connect returned=1 errno=0 state=error: certificate verify failed: [ok for /CN=master-1.tld]
 
Info: Retrieving pluginfacts
 
Error: /File[/opt/puppetlabs/puppet/cache/facts.d]: Failed to generate additional resources using 'eval_generate': SSL_connect returned=1 errno=0 state=error: certificate verify failed: [ok for /CN=master-1.tld]
 
...

This error message is INCREDIBLY unhelpful. There's kinda-sorta a reference to the master's certificate being master-1.tld, but you can't see in the message that the configured server is master.tld. There's also no reference to other valid alt names.

What we want:

The error message should clearly state something along the lines of

Error: Unable to validate server certificate. Expected "master.tld":
 
  server certificate common name "master-1.tld" does not match
 
  server certificate alt name "puppet" does not match
 
  server certificate alt name "master" does not match
 
  server certificate alt name "master-alt.tld" does not match

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.0.2#70111-sha1:88534db)
Atlassian logo

Reid Vandewiele (JIRA)

unread,
Nov 28, 2017, 7:43:03 PM11/28/17
to puppe...@googlegroups.com
Reid Vandewiele updated an issue
Change By: Reid Vandewiele
CS Priority: Needs Priority

Reid Vandewiele (JIRA)

unread,
Nov 28, 2017, 7:56:02 PM11/28/17
to puppe...@googlegroups.com
Reid Vandewiele commented on Bug PUP-8213
 
Re: Error message for certificate name mismatch is clear as mud

Here's a stack trace from when this error is thrown.

Error: /File[/opt/puppetlabs/puppet/cache/facts.d]: Failed to generate additional resources using 'eval_generate': SSL_connect returned=1 errno=0 state=error: certificate [204/1995]
 
led: [ok for /CN=master-of-puppets-aws-1.sj.b2c.nike.com]
 
/opt/puppetlabs/puppet/lib/ruby/2.4.0/net/protocol.rb:44:in `connect_nonblock'
 
/opt/puppetlabs/puppet/lib/ruby/2.4.0/net/protocol.rb:44:in `ssl_socket_connect'
 
/opt/puppetlabs/puppet/lib/ruby/2.4.0/net/http.rb:948:in `connect'
 
/opt/puppetlabs/puppet/lib/ruby/2.4.0/net/http.rb:887:in `do_start'
 
/opt/puppetlabs/puppet/lib/ruby/2.4.0/net/http.rb:882:in `start'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/pool.rb:83:in `borrow'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/pool.rb:25:in `with_connection'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/connection.rb:305:in `with_connection'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/connection.rb:176:in `block in do_request'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/connection.rb:173:in `upto'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/connection.rb:173:in `do_request'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/connection.rb:79:in `get'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/rest.rb:153:in `http_request'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/rest.rb:132:in `http_get'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/rest.rb:208:in `block in search'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/rest.rb:256:in `block in do_request'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/request.rb:220:in `do_request'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/rest.rb:256:in `do_request'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/rest.rb:207:in `search'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/file_metadata/selector.rb:19:in `search'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/indirection.rb:270:in `search'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/type/file.rb:701:in `perform_recursion'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/type/file.rb:670:in `block in recurse_remote_metadata'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/type/file.rb:664:in `collect'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/type/file.rb:664:in `recurse_remote_metadata'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/type/file.rb:646:in `recurse_remote'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/type/file.rb:569:in `recurse'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/type/file.rb:470:in `eval_generate'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/additional_resource_generator.rb:56:in `eval_generate'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:112:in `block in evaluate'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:119:in `traverse'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:169:in `evaluate'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:225:in `block in apply'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/log.rb:159:in `with_destination'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/report.rb:140:in `as_logging_destination'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:224:in `apply'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer/downloader.rb:13:in `evaluate'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer/plugin_handler.rb:18:in `download_plugins'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:445:in `download_plugins'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:118:in `get_facts'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:307:in `run_internal'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:222:in `block in run'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:263:in `override'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:196:in `run'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:46:in `block (4 levels) in run'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent/locker.rb:21:in `lock'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:46:in `block (3 levels) in run'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:110:in `with_client'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:43:in `block (2 levels) in run'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:67:in `run_in_fork'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:42:in `block in run'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:179:in `controlled_run'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:40:in `run'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/agent.rb:365:in `onetime'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/agent.rb:343:in `run_command'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:358:in `block in run'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:666:in `exit_on_fail'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:358:in `run'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:132:in `run'
 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:72:in `execute'
 
/opt/puppetlabs/puppet/bin/puppet:5:in `<main>'

Reid Vandewiele (JIRA)

unread,
Nov 28, 2017, 7:56:03 PM11/28/17
to puppe...@googlegroups.com
Reid Vandewiele updated an issue
Change By: Reid Vandewiele
Comment:
Here's a stack trace from when this error is thrown.

{code}
{code}

Reid Vandewiele (JIRA)

unread,
Nov 28, 2017, 7:57:03 PM11/28/17
to puppe...@googlegroups.com
 
Re: Error message for certificate name mismatch is clear as mud

Here's a stack trace from when this error is thrown:

Error: /File[/opt/puppetlabs/puppet/cache/facts.d]: Failed to generate additional resources using 'eval_generate': SSL_connect returned=1 errno=0 state=error: certificate verify failed: [ok for /CN=master-1.tld]

Charlie Sharpsteen (JIRA)

unread,
Nov 28, 2017, 8:13:02 PM11/28/17
to puppe...@googlegroups.com
Charlie Sharpsteen updated an issue
 
Change By: Charlie Sharpsteen
Affects Version/s: PUP 5.3.3

Josh Cooper (JIRA)

unread,
Nov 28, 2017, 11:25:03 PM11/28/17
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Team: Platform Core

Reid Vandewiele (JIRA)

unread,
Nov 29, 2017, 12:22:03 PM11/29/17
to puppe...@googlegroups.com
 
Re: Error message for certificate name mismatch is clear as mud

It looks like the rescue block that deals with OpenSSL::SSL::SSLError exceptions has an if/elsif/else wherein the "if" part is too generic, and intercepts the type of problem described in this ticket. The "elsif" to print a pretty error already exists, but will never be reached due to the greedy "if".

https://github.com/puppetlabs/puppet/blob/5.3.3/lib/puppet/network/http/connection.rb#L313-L329

Brett Gray (JIRA)

unread,
Nov 30, 2017, 6:54:03 PM11/30/17
to puppe...@googlegroups.com
Brett Gray updated an issue
 
Change By: Brett Gray
CS Frequency: 2 - 5-25% of Customers

Brett Gray (JIRA)

unread,
Nov 30, 2017, 6:55:04 PM11/30/17
to puppe...@googlegroups.com
Brett Gray updated an issue
Change By: Brett Gray
CS Business Value: 2 - $$$

Brett Gray (JIRA)

unread,
Nov 30, 2017, 6:55:04 PM11/30/17
to puppe...@googlegroups.com
Brett Gray updated an issue
Change By: Brett Gray
CS Severity: 2 - Annoyance

Brett Gray (JIRA)

unread,
Nov 30, 2017, 6:57:02 PM11/30/17
to puppe...@googlegroups.com
Brett Gray updated an issue
Change By: Brett Gray
CS Frequency: 2 3  -  5- 25 -50 % of Customers

Brett Gray (JIRA)

unread,
Nov 30, 2017, 6:58:02 PM11/30/17
to puppe...@googlegroups.com
Brett Gray updated an issue
Change By: Brett Gray
CS Impact: Making any error messages more useful is always encouraged.
These errors are common and just feel like we know it, but we need a better error message.

Brett Gray (JIRA)

unread,
Nov 30, 2017, 7:08:02 PM11/30/17
to puppe...@googlegroups.com
Brett Gray updated an issue
Change By: Brett Gray
CS Priority: Needs Priority Major

Brett Gray (JIRA)

unread,
Nov 30, 2017, 7:08:06 PM11/30/17
to puppe...@googlegroups.com
Brett Gray updated an issue
Change By: Brett Gray
CS Priority: Major Normal

Maggie Dreyer (JIRA)

unread,
May 25, 2018, 11:57:03 AM5/25/18
to puppe...@googlegroups.com
Maggie Dreyer updated an issue
Change By: Maggie Dreyer
Sub-team: Coremunity
This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)
Atlassian logo

Maggie Dreyer (JIRA)

unread,
May 25, 2018, 11:57:03 AM5/25/18
to puppe...@googlegroups.com
Maggie Dreyer updated an issue
Change By: Maggie Dreyer
Team: Platform Core Server

Maggie Dreyer (JIRA)

unread,
May 25, 2018, 11:57:04 AM5/25/18
to puppe...@googlegroups.com
Maggie Dreyer updated an issue
Change By: Maggie Dreyer
Fix Version/s: PUP 6.0.0

Maggie Dreyer (JIRA)

unread,
Sep 10, 2018, 5:56:02 PM9/10/18
to puppe...@googlegroups.com

Maggie Dreyer (JIRA)

unread,
Oct 2, 2018, 12:23:05 PM10/2/18
to puppe...@googlegroups.com
Maggie Dreyer commented on Bug PUP-8213
 
Re: Error message for certificate name mismatch is clear as mud

We refactored this in https://github.com/puppetlabs/puppet/commit/9f65101b534398d92359bc93c91f65302dc897c9, but it looks like the conditions on the error matching are still the same. So this is still an issue.

Maggie Dreyer (JIRA)

unread,
Oct 30, 2018, 2:46:05 PM10/30/18
to puppe...@googlegroups.com
Maggie Dreyer updated an issue
Change By: Maggie Dreyer
Team: Server Coremunity

Josh Cooper (JIRA)

unread,
Feb 15, 2019, 6:11:04 PM2/15/19
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-8213
 
Re: Error message for certificate name mismatch is clear as mud

Also openssl 1.1 will perform hostname checking before ruby's post_connection_check, which changes how we need to detect the cert mismatch.

Josh Cooper (JIRA)

unread,
Feb 20, 2019, 11:57:05 AM2/20/19
to puppe...@googlegroups.com
Josh Cooper updated an issue
 
Change By: Josh Cooper
Fix Version/s: PUP 6.4.0
Fix Version/s: PUP 6.0.6
Fix Version/s: PUP 5.5.11

Josh Cooper (JIRA)

unread,
Feb 21, 2019, 2:39:04 AM2/21/19
to puppe...@googlegroups.com
Josh Cooper assigned an issue to Josh Cooper
Change By: Josh Cooper
Assignee: Josh Cooper

Josh Cooper (JIRA)

unread,
Feb 21, 2019, 2:39:06 AM2/21/19
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Sprint: Platform Core KANBAN

Josh Cooper (JIRA)

unread,
Feb 25, 2019, 6:27:04 PM2/25/19
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Release Notes Summary: Puppet agents printed a cryptic error message when connecting to an SSL server whose certificate did not match the hostname the agent tried to connect to. This was a regression when running on ruby 2.4 or later, due to differences in how ruby reports the mismatched certificate. Puppet has been updated so it prints the expected error message.
Release Notes: Bug Fix

Jacob Helwig (JIRA)

unread,
Feb 26, 2019, 2:02:05 PM2/26/19
to puppe...@googlegroups.com

Josh Cooper (JIRA)

unread,
Mar 4, 2019, 4:12:05 PM3/4/19
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-8213

Reverted in 5.5.x in https://github.com/puppetlabs/puppet/commit/1a25a732804087fee3d31fcc4c27b6f23fa67ed9 because it did not report on root or intermediate CA cert errors correctly.

Geoff Nichols (JIRA)

unread,
Mar 12, 2019, 6:54:02 PM3/12/19
to puppe...@googlegroups.com
Geoff Nichols commented on Bug PUP-8213

Josh Cooper, I'm following up on tickets targeted at the upcoming Platform 5.5.11 release. Is there anything left to do before this is resolved?

Josh Cooper (JIRA)

unread,
Mar 12, 2019, 10:46:03 PM3/12/19
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Fix Version/s: PUP 6.0.6
Fix Version/s: PUP 5.5.11
Fix Version/s: PUP 6.0.7
Fix Version/s: PUP 5.5.12

Heston Hoffman (JIRA)

unread,
Mar 16, 2019, 8:13:04 PM3/16/19
to puppe...@googlegroups.com
Heston Hoffman updated an issue
Change By: Heston Hoffman
Labels: resolved-issue-added

Josh Cooper (JIRA)

unread,
Mar 18, 2019, 12:11:04 PM3/18/19
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Fix Version/s: PUP 6.4.0
Fix Version/s: PUP 6.4.1

Josh Cooper (JIRA)

unread,
Apr 2, 2019, 1:10:02 PM4/2/19
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Fix Version/s: PUP 6.0.8
Fix Version/s: PUP 5.5.13
Fix Version/s: PUP 6.0.z
Fix Version/s: PUP 5.5.z

Josh Cooper (JIRA)

unread,
Apr 8, 2019, 6:47:03 PM4/8/19
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-8213
 
Re: Error message for certificate name mismatch is clear as mud

This issue was fixed for puppet network requests in PUP-9457 first released in 6.4.0. However, it is still an issue for puppet 5.5.x and 6.0.x when running with ruby 2.4 or 2.5, which are the versions bundled into the respective puppet-agent packages.. It is also an issue in 6.4.0 for any provider or other extension still calling the legacy Puppet::Network::HttpPool.http_instance and #http_ssl_instance methods. I'll need to submit a different fixes for 5.5.x, 6.0.x and 6.4.x due to the different implementations.

Josh Cooper (JIRA)

unread,
Apr 8, 2019, 7:40:03 PM4/8/19
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Fix Version/s: PUP 6.0.z
Fix Version/s: PUP 5.5.z

Josh Cooper (JIRA)

unread,
Apr 10, 2019, 12:38:03 PM4/10/19
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Fix Version/s: PUP 6.4.1
Fix Version/s: PUP 6.4.z

Kris Bosland (JIRA)

unread,
Apr 12, 2019, 5:20:03 PM4/12/19
to puppe...@googlegroups.com

Sebastian Miclea (JIRA)

unread,
Apr 24, 2019, 4:15:03 AM4/24/19
to puppe...@googlegroups.com
Sebastian Miclea updated an issue
 
Change By: Sebastian Miclea
Fix Version/s: PUP 6.4.2

Josh Cooper (JIRA)

unread,
May 30, 2019, 6:55:03 PM5/30/19
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Fix Version/s: PUP 6.4.z
Reply all
Reply to author
Forward
0 new messages