Puppet CA corruption

[shoerner]

Hello everyone,

Just getting my first puppet master set up and I am having a problem that I just do not know how to get past. For some reason, my certificate store keeps getting corrupted. Basically what happens is that the server will issue itself a valid certificate (after removing the 'bad' cert) and will run just fine. When I start puppetDB (I am pretty sure it happens around here) on the system though, running the command 'puppet ca list --all' on the PuppetMaster, I get the following:

Error: The certificate retrieved from the master does not match the agent's private key.
Certificate fingerprint: <fingerprint removed>
To fix this, remove the certificate from both the master and the agent and then start a puppet run, which will automatically regenerate a certficate.
On the master:
  puppet cert clean puppetmaster.site
On the agent:
  rm -f /var/lib/puppet/ssl/certs/puppetmaster.site.pem
  puppet agent -t

Error: Try 'puppet help ca list' for usage

I have tried following said instructions which did not work at all. Eventually I was able to build it down to the following steps to regenerate the certificate store:
# service puppetmaster stop
# service puppetdb stop
# service puppet stop
# find $(puppet master --configprint ssldir) -name "$(puppet master --configprint certname).pem" -delete
# rm -rf /var/lib/puppet/ssl
# puppet master --no-daemonize --debug --verbose --trace (kill it when it says "starting puppet")
# /usr/sbin/puppetdb-ssl-setup
# service puppetmaster start
# puppet ca list --all (lists the certs installed)
# service puppetdb start
# puppet ca list --all (prints error message above with new fingerprint)

The master is running Fedora 16 with Puppet 3.0.1 (along with PuppetDB 1.0.2 and Puppet Dashboard). I realize that the solution is only made more difficult by the inclusion of db and dashboard, but the project scope grew too quickly and resulted in attempts to combine services. I am out of ideas save for re-installing the service; after messing with this install for so long, I doubt many people here will want to support this decision.

[tas]

I have the same exact problem over and over and over, so I gave up on Puppet.

But maybe you can try resolve this by checking if times are in sync?

try ntpdate on master and slave.

check date on both machines very quickly to see the time sync

[Deepak Giridharagopal]

That's odd...puppetdb-ssl-setup doesn't actually modify anything in puppet's ssl directory, it should only be reading files contained in that directory and importing them into a keystore (in a completely different dir). If that script messed something up, i'd expect the issue to come up the first time you ran "puppet ca list". When you start the PuppetDB daemon, nothing in /var/lib/puppet/ssl is involved...it doesn't even look at those files.

Are the checksums of your certs and puppet config files the same before and after running puppetdb-ssl-setup, and before and after starting the puppetdb daemon? If, instead of using "service start puppet", you use the puppetdb-foreground script, does the problem persist?

deepak

[shoerner]

Got a little further by running puppetdb-foreground. Now I can get as far as node certificate generation.

Once I start the node certificate generation, I get a new certificate, but then PuppetDB blows up with "javax.net.ssl.SSLException: Received fatal alert: decrypt_error"

Immediately after that puppet ca list --all stops working as mentioned at first.

[shoerner]

@tas: Tried this already, both times were within a few seconds of each other as best as I can tell. Additionally, both systems are synced to US NTP Time pool 1.

[Dennis Hoppe]

Hello Shawn,

Am 29.11.2012 22:52, schrieb shoerner:
> Just getting my first puppet master set up and I am having a problem
> that I just do not know how to get past. For some reason, my certificate
> store keeps getting corrupted. Basically what happens is that the server
> will issue itself a valid certificate (after removing the 'bad' cert)
> and will run just fine. When I start puppetDB (I am pretty sure it
> happens around here) on the system though, running the command 'puppet
> ca list --all' on the PuppetMaster, I get the following:
>
> Error: The certificate retrieved from the master does not match the
> agent's private key.

> Certificate fingerprint: *<fingerprint removed>*

which command did you used to sign your client certificates?

At Puppet 2.7 i was using "puppetca list" and "puppetca sign
host.example.net".

So i thought that i could use "puppet ca list" and "puppet ca sign
host.example.net" at Puppet 3.0.1.

But, when the first Puppet client did a "puppet agent --test", the
puppet master created a private key for host.example.net at
"/var/lib/puppet/ssl/private_keys".

So i assume that there is a difference between "puppet ca" and "puppet
cert".

When i use "puppet cert list" and "puppet cert sign host.example.net",
there will be no private key created and the commands "puppet ca list
--all" and "puppet cert list --all" are working as expected.

Best regards, Dennis

[shoerner]

Hey Dennis,

You are right - there is a difference. Think I tried the cert list command once and dismissed it when it did not work for 'ca list' instead.

Running the command 'puppet cert list --all' does indeed work as intended. Thank you!

Off to go fix puppetdb now that I have ripped the SSL configurations out of it too many times.