client certs won't remain relevant after foreman install
Tim Dunphy
Hey all,
I've just got foreman setup today. And I've had to recreate my certs on the client hosts in order to get that to happen.
However I'm facing an usual issue with my client ssl certs since installing foreman.
I previously had a puppet server that was working well in my environment. But I wanted to get a working web gui onto that setup for a while. So I ended up having to blow away my current puppet config on the puppet server (it's ok because I had it both backed up traditionally and stored in SVN). I then proceeded with the foreman install. Everything went well, until...
It seems that when I first certify a client host it works as anticipated. But on the second run, the newly created cert seems invalid and the error message confusingly asks you to remove the puppet servers cert. As if the client itself thinks it's the puppet server. Weird.
To make this a ltlle more clear I'm going to walk through this scenario which is happening on every one of my servers since the foreman install.
Trying to give this server a fresh start I find it's cert on the puppet server:
[root@puppet:~] #puppet cert --list --all | grep beta
+ "beta.mydomain.com" (E5:82:A8:CC:8D:AE:F9:3C:85:36:00:E6:3D:10:CD:F6)
Then remove the cert from the puppet server:
[root@puppet:~] #puppet cert clean beta.mydomain.com
notice: Revoked certificate with serial 21
notice: Removing file Puppet::SSL::Certificate beta.mydomain.com at '/var/lib/puppet/ssl/ca/signed/beta.mydomain.com.pem'
notice: Removing file Puppet::SSL::Certificate beta.mydomain.com at '/var/lib/puppet/ssl/certs/beta.mydomain.com.pem'
Back on the client host I remove the certs:
[root@beta:~] #find /var/lib/puppet/ssl -type f -exec rm -f {} \;
And when I go to get a new cert an error pops up complaining that the cert is invalid. And it asks me to remove the puppet server's cert on the client host.
[root@beta:~] #puppet agent --test --waitforcert 60 --server puppet.mydomain.cominfo: Creating a new SSL key for puppet.mydomain.com
info: Caching certificate for ca
info: Caching certificate for puppet.mydomain.com ##<--why puppet.mydomain.com?
err: Could not request certificate: The certificate retrieved from the master does not match the agent's private key.
Certificate fingerprint: BB:F6:61:88:56:AD:CD:63:74:62:3B:BA:1A:B3:BD:CD
To fix this, remove the certificate from both the master and the agent and then start a puppet run, which will automatically regenerate a certficate.
On the master:
puppet cert clean puppet.mydomain.com ## <--why puppet.mydomain.com
On the agent:
rm -f /var/lib/puppet/ssl/certs/puppet.mydomain.com.pem ## <--why puppet.mydomain.com
puppet agent -t
And still on the client host I look for a cert named after the puppet server (not the client) it is indeed there:
[root@beta:~] #ls -l /var/lib/puppet/ssl/certs/puppet.mydomain.com.pem
-rw-r----- 1 puppet puppet 1976 Feb 12 23:48 /var/lib/puppet/ssl/certs/puppet.mydomain.com.pem
So my question at this point is, why at this point is this process creating an invalid cert named after the puppet server on the client host? And how can I remedy this rather odd situation.
Thanks,
Tim
--
GPG me!!
gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
Rafael Cristaldo
like that:
# rm -rf /var/lib/puppet/ssl
Check again at the puppet master:
# puppet cert --all and --list |grep hostname
You is using autosign.conf ?
Tim Dunphy
I suggest you to remove all the files and directories at /var/lib/puppet/ssl
like that:
# rm -rf /var/lib/puppet/ssl
The command I showed in the earlier email should do the same thing:
[root@beta:~] #find /var/lib/puppet/ssl -type f -exec rm -f {} \;
But just for the sake of argument I did try the exact command you showed:
[root@beta:~] #rm -rf /var/lib/puppet/ssl
Check again at the puppet master:
# puppet cert --all and --list |grep hostname
[root@puppet:~] #puppet cert --list --all | grep beta
Yup! no client cert here!
And even tho I've already verified that there is no cert for this client on my puppet server, on the FIRST run of puppet --agent --test --waitforcert on that client seems to immediately produce a cert on the client named after the puppet server!
[root@beta:~] #puppet agent --test --waitforcert 60 --server puppet.mydomain.cominfo: Creating a new SSL key for puppet.mydomain.com
info: Caching certificate for ca
info: Caching certificate for puppet.mydomain.com
err: Could not request certificate: The certificate retrieved from the master does not match the agent's private key.
Certificate fingerprint: BB:F6:61:88:56:AD:CD:63:74:62:3B:BA:1A:B3:BD:CD
To fix this, remove the certificate from both the master and the agent and then start a puppet run, which will automatically regenerate a certficate.
On the master:
puppet cert clean puppet.mydomain.com
On the agent:
rm -f /var/lib/puppet/ssl/certs/puppet.mydomain.com.pem
puppet agent -t
info: Retrieving plugin
err: /File[/var/lib/puppet/lib]: Failed to generate additional resources using 'eval_generate: The certificate retrieved from the master does not match the agent's private key.
Certificate fingerprint: BB:F6:61:88:56:AD:CD:63:74:62:3B:BA:1A:B3:BD:CD
To fix this, remove the certificate from both the master and the agent and then start a puppet run, which will automatically regenerate a certficate.
On the master:
puppet cert clean puppet.mydomain.com
On the agent:
rm -f /var/lib/puppet/ssl/certs/puppet.mydomain.com.pem
puppet agent -t
err: /File[/var/lib/puppet/lib]: Could not evaluate: SSL_CTX_use_PrivateKey:: key values mismatch Could not retrieve file metadata for puppet://puppet.mydomain.com/plugins: SSL_CTX_use_PrivateKey:: key values mismatch
err: Could not retrieve catalog from remote server: SSL_CTX_use_PrivateKey:: key values mismatch
warning: Not using cache on failed catalog
err: Could not retrieve catalog; skipping run
err: Could not send report: SSL_CTX_use_PrivateKey:: key values mismatch
It's almost as if the puppet server THINKS that the remote host is named 'puppet.mydomain.com' instead of 'beta.mydomain.com'. What you see above is a FIRST RUN after rm -rf of the /var/lib/puppet/ssl directory.
And if I check the presence of the certificate file named after the puppet server (not the puppet client) it is there on the remote puppet client:
[root@beta:~] #ls -l /var/lib/puppet/ssl/certs/puppet.mydomain.com.pem
-rw-r----- 1 puppet puppet 1976 Feb 13 08:02 /var/lib/puppet/ssl/certs/puppet.mydomain.com.pem
Why on earth a certificate named 'puppet.mydomain.com' exists on the remote host instead of 'beta.mydomain.com' is what I need to figure out and how to correct it.
You is using autosign.conf ?
Nope! Autosign is turned off.
Thanks for your help!
Tim
--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/a9ea5f44-e2dd-4c5a-8d8f-087ab04c551c%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.