Puppet 6 regenerate all certs fails with OpenSSL::X509::StoreError

2,344 views
Skip to first unread message

Bret Wortman

unread,
Oct 22, 2018, 7:25:10 AM10/22/18
to Puppet Users
We had an issue where someone removed our puppet server's ssl directory, so we need to regenerate all our certs. I'm following the instructions at https://puppet.com/docs/puppet/6.0/ssl_regenerate_certificates.html but am having difficulties:

# puppetserver ca list -a
Traceback (most recent call last):
     9: from /opt/puppetlabs/server/apps/puppetserver/cli/apps/ca:5 in '<main>'
     8: from /opt/puppetlabs/puppet/lib/ruby/vendor_gems/gems/puppetserver-ca-1.0.0/lib/puppetserver/ca/cli.rb:89: in 'run'
     7: from /opt/puppetlabs/puppet/lib/ruby/vendor_gems/gems/puppetserver-ca-1.0.0/lib/puppetserver/ca/action/list.rb:60: in 'run'
     6: from /opt/puppetlabs/puppet/lib/ruby/vendor_gems/gems/puppetserver-ca-1.0.0/lib/puppetserver/ca/action/list.rb:113: in 'get_all_certs'
     5: from /opt/puppetlabs/puppet/lib/ruby/vendor_gems/gems/puppetserver-ca-1.0.0/lib/puppetserver/ca/action/list.rb:113: in 'new'
     4: from /opt/puppetlabs/puppet/lib/ruby/vendor_gems/gems/puppetserver-ca-1.0.0/lib/puppetserver/ca/certificate_authority.rb:16: in 'initialize'
     3: from /opt/puppetlabs/puppet/lib/ruby/vendor_gems/gems/puppetserver-ca-1.0.0/lib/puppetserver/ca/certificate_authority.rb:16: in 'new'
     2: from /opt/puppetlabs/puppet/lib/ruby/vendor_gems/gems/puppetserver-ca-1.0.0/lib/puppetserver/ca/utils/http_client.rb:19: in 'initialize'
     1: from /opt/puppetlabs/puppet/lib/ruby/vendor_gems/gems/puppetserver-ca-1.0.0/lib/puppetserver/ca/utils/http_client.rb:108: in 'make_store'
/opt/puppetlabs/puppet/lib/ruby/vendor_gems/gems/puppetserver-ca-1.0.0/lib/puppetserver/ca/utils/http_client.rb:109:in 'add_file': system lib (OpenSSL::X509::StoreError)
#

Has anyone encountered this before? Any thoughts on how to regenerate my certs on this system and get us going again?

Note: I have puppet installed on one server and puppetdb on another, in case that matters.

Bret Wortman

unread,
Oct 22, 2018, 8:48:39 AM10/22/18
to Puppet Users
Out of curiosity, I updated the server to 6.0.1. No change.

Johan De Wit

unread,
Oct 22, 2018, 11:36:27 AM10/22/18
to puppet...@googlegroups.com

try puppet cert list --all


That seems to work .... git simalar error using the puppet ca command


--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/7715f962-0e79-44f8-9e25-ade744378c37%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Maggie Dreyer

unread,
Oct 22, 2018, 11:37:10 AM10/22/18
to puppet...@googlegroups.com
Unfortunately that particular docs page was incorrectly updated for Puppet 6. If you are running Puppet 6 master AND agents, you can regenerate your CA by using `puppetserver can setup`. This creates a basic intermediate CA with a self-signed root and a CA signing cert. It will also create a new cert for your puppet master. You can read more about this model here: https://puppet.com/docs/puppetserver/6.0/intermediate_ca.html, and more about the new `puppetserver ca` subcommand here: https://puppet.com/docs/puppetserver/6.0/subcommands.html#ca.

However, please note that if you still have some Puppet 5 agents, you'd be better off just restarting Puppet Server, which will generate a new non-intermediate CA (a self-signed root that also is the CA signing cert that issues node certificates). Puppet 5 agents do not properly support the intermediate CA setup without manual intervention.

Whichever route you take to regenerate your CA and master cert, you will also need to regenerate the certs for your agents. This can be accomplished by starting Puppet Server, deleting the SSL dir on each agent node (and puppetdb), then running `puppet agent -t` to submit a signing request to the server. On a Puppet 6 master, use `puppetserver ca sign --certname <node's certname>` to sign the cert, followed by another `puppet agent -t` on the agent to retrieve it.

We made a series of major CA improvements in Puppet 6, which you can read about in the release notes here and here. While updating the docs for this release, we realized that a major overhaul of the CA and SSL docs was needed, as many of them haven't been touched since the release of Puppet 4. We are in the process of getting that written and published now. We really appreciate feedback like this to help us identify spots that are still wrong or confusing.

Please let me know if anything in here doesn't work right for you!
Maggie

--

Bret Wortman

unread,
Oct 22, 2018, 11:46:31 AM10/22/18
to puppet...@googlegroups.com

That worked like a champ. Now I just need to read up on how to get my puppetserver talking to puppetdb again...

Thanks, Maggie!

You received this message because you are subscribed to a topic in the Google Groups "Puppet Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/puppet-users/YIs8AmLHHMg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAMstjg0R1zUrdj76VFYM36wZaaDYKFvL%2BbYAUbGTy2gG-Um9sA%40mail.gmail.com.

hawaii4...@gmail.com

unread,
Oct 25, 2018, 7:57:20 PM10/25/18
to Puppet Users
I have i don't know what is puppet server is? All i have is visual studio code that's all, it doesn't have any cool features as long as i can practice decode to write code, to apply for a "?".aspx,java

hawaii4...@gmail.com

unread,
Oct 25, 2018, 7:57:20 PM10/25/18
to Puppet Users
On Monday, October 22, 2018 at 1:25:10 AM UTC-10, Bret Wortman wrote:
> We had an issue where someone removed our puppet server's ssl directory, so we need to regenerate all our certs. I'm following the instructions at https://puppet.com/docs/puppet/6.0/ssl_regenerate_certificates.html but am having difficulties:
>
> i have downloaded a java se from oracle and node.js, i tried to jdk1.7.80 but it was hopeless for me if my keyboard on dell laptop isn't working, and plus i got a rain out, oh well i have a Visual studio code with Administrator portal. So what do i have to resolve my problems. I work at Lahaina Fish CO and i tried to set my Android phone to Developers Options to get a bug report but somehow i don't know if any of co-workers toggle to see what i was trying to develop or bug report.
8

chol...@redlands.qld.edu.au

unread,
Oct 25, 2018, 7:57:20 PM10/25/18
to Puppet Users
On Tuesday, 23 October 2018 01:37:10 UTC+10, Maggie Dreyer wrote:
Puppet 5 agents do not properly support the intermediate CA setup without manual intervention.

Hi Maggie,

Could you elaborate on the "manual intervention" required to get a Puppet 5 agent to work with the intermediate CA? I'm getting errors associated with the CRL but can't yet find any discussion of this in the documentation. I'm sure this is the incompatibility you mentioned.

> Warning: SSL_connect returned=1 errno=0 state=error: certificate verify failed: [unable to get certificate CRL for /CN=Puppet CA: puppet.test.redlands.qld.edu.au]

Maggie Dreyer

unread,
Oct 26, 2018, 11:44:12 AM10/26/18
to puppet...@googlegroups.com
The information on this page from the Puppet 5 docs will probably help: https://puppet.com/docs/puppetserver/5.3/intermediate_ca_configuration.html

Assuming your intermediate CA was set up using `puppetserver ca setup`, the important bits are:
1) Delete the SSL dir on the agent
2) Set CRL checking on the agent to "leaf"
3) Copy the CA bundle from the master to the agent:
(master) /etc/puppetlabs/puppet/ssl/ca/ca_crt.pem -> (agent) /etc/puppetlabs/puppet/ssl/certs/ca.pem
4) Copy the CRL bundle from the master to the agent:
(master) /etc/puppetlabs/puppet/ssl/ca/ca_crl.pem -> (agent) /etc/puppetlabs/puppet/ssl/crl.pem
5) Do an agent run to generate a CSR and proceed as usual

You shouldn't have to create any bundles or do any work server-side if you used our CLI to build your intermediate CA (either with `setup` or `import).

These steps are necessary because the agent used to not correctly save PEM files with more than one artifact in them (like the cert and CRL bundles here), and it also didn't used to properly iterate over a chain of CRLs even if it had one (hence setting CRL checking to leaf). Both of these issues are fixed in Puppet 6.

Let me know if you have any more issues. I'll look into getting some of this information linked from the Puppet 6 version of the intermediate CA docs.

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

Craig Holyoak

unread,
Oct 29, 2018, 7:07:52 PM10/29/18
to puppet...@googlegroups.com
On Sat, 27 Oct 2018 at 01:44, Maggie Dreyer <mag...@puppet.com> wrote:
> Assuming your intermediate CA was set up using `puppetserver ca setup`, the important bits are:
> 1) Delete the SSL dir on the agent
> 2) Set CRL checking on the agent to "leaf"
> 3) Copy the CA bundle from the master to the agent:
> (master) /etc/puppetlabs/puppet/ssl/ca/ca_crt.pem -> (agent) /etc/puppetlabs/puppet/ssl/certs/ca.pem
> 4) Copy the CRL bundle from the master to the agent:
> (master) /etc/puppetlabs/puppet/ssl/ca/ca_crl.pem -> (agent) /etc/puppetlabs/puppet/ssl/crl.pem
> 5) Do an agent run to generate a CSR and proceed as usual

Thanks, that has worked perfectly.

--
Craig Holyoak
IT Support @ Redlands College
chol...@redlands.qld.edu.au
http://www.redlands.qld.edu.au/

--

Reply all
Reply to author
Forward
0 new messages