Connecting to https://puppet:8140

[Jeff Abrahamson]

I set up a puppetserver.  It's, well, not generating errors outright.

On the same host, I want to run a puppet agent.  But the agent doesn't seem to want to use the FQDN of the host, and so it fails to connect.

[T] jeff@loire:~ $ puppet agent --debug --test
...
Debug: Finishing transaction 27923380
Debug: Creating new connection for https://puppet:8140
Error: Could not request certificate: getaddrinfo: Name or service not known
Exiting; failed to retrieve certificate and waitforcert is disabled
1,[T] jeff@loire:~ $
1,[T] jeff@loire:~ $ cat /etc/puppet/puppet.conf
[main]
logdir=/var/log/puppet
vardir=/var/lib/puppet
ssldir=/var/lib/puppet/ssl
rundir=/var/run/puppet
factpath=$vardir/lib/facter
certname = puppet
dns_alt_names = puppet,puppet.example.com

[master]
# These are needed when the puppetmaster is run by passenger
# and can safely be removed if webrick is used.
ssl_client_header = SSL_CLIENT_S_DN
ssl_client_verify_header = SSL_CLIENT_VERIFY

[agent]
server = puppet.example.com
[T] jeff@loire:~ $ cat /etc/hosts
127.0.0.1    localhost
127.0.1.1    ubuntu

# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

139.162.147.68    loire.example.com loire
[T] jeff@loire:~ $ netstat -a | grep 8140
tcp6       0      0 [::]:8140               [::]:*                  LISTEN    
[T] jeff@loire:~ $

Of course, s/example/my-real-domain-name/g.  Note that loire is the host's A record, there's a CNAME called puppet.  (Note that telnetting to loire.example.com 8140 connects: the tcp6 is a netstat artefact as far as I know.)

Reflection, poking, and googling are drawing blanks for me.  Any suggestions ?

Jeff Abrahamson
+33 6 24 40 01 57

+44 7920 594 255    <-- only when I'm in the UK
http://ml-week.com/     prochaine edition, 2016

http://jeff.purple.com/
http://blog.purple.com/jeff/

[Jeff Abrahamson]

Aha, I can answer half of my own question after another hour of work.  The issue was the certname field in the config.  Removing it makes that all work.  I also removed the short hostname from dns_alt_names, since it seemed unnecessary.  That leaves me only with the famous certificate mismatch problem.

...
Debug: Finishing transaction 5407140
Info: Creating a new SSL key for loire.example.com
Debug: Using cached certificate for ca
Debug: Using cached certificate for ca
Debug: Creating new connection for https://puppet.example.com:8140
Info: Caching certificate for loire.example.com
Error: Could not request certificate: The certificate retrieved from the master does not match the agent's private key.
Certificate fingerprint: EF:0B:DB:FD:8F:18:AB:DC:1F:82:BF:8B:A0:92:CB:D3:4F:0D:72:DC:F2:23:B8:A0:74:76:33:56:6E:32:1F:50
To fix this, remove the certificate from both the master and the agent and then start a puppet run, which will automatically regenerate a certficate.
On the master:
  puppet cert clean loire.example.com
On the agent:
  1a. On most platforms: find /var/lib/puppet/ssl -name loire.example.com.pem -delete
  1b. On Windows: del "/var/lib/puppet/ssl/loire.example.com.pem" /f
  2. puppet agent -t

Exiting; failed to retrieve certificate and waitforcert is disabled
1,[T] jeff@loire:~ $

Jeff Abrahamson
+33 6 24 40 01 57

+44 7920 594 255    <-- only when I'm in the UK
http://ml-week.com/     prochaine edition, 2016

http://jeff.purple.com/
http://blog.purple.com/jeff/

[Jeff Abrahamson]

Restarting the puppetserver seems to sort the certificate mismatch.  Note that my host's A record is loire.example.com with CNAME puppet.example.com.  In /etc/puppet/puppet.conf, I refer to the host as puppet:

    [main]
    logdir=/var/log/puppet
    vardir=/var/lib/puppet
    ssldir=/var/lib/puppet/ssl
    rundir=/var/run/puppet
    factpath=$vardir/lib/facter

    dns_alt_names = puppet.example.com

    [master]
    # These are needed when the puppetmaster is run by passenger
    # and can safely be removed if webrick is used.
    ssl_client_header = SSL_CLIENT_S_DN
    ssl_client_verify_header = SSL_CLIENT_VERIFY

    [agent]
    server = puppet.example.com

(Recall that s/example/mydomain/g.) The problem them transforms into certificate verification failure.

    Warning: Unable to fetch my node definition, but the agent run will continue:
    Warning: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [unable to get local issuer certificate for /CN=loire.example.com]

Comparing certificates confirms they aren't quite the same.  Recall that my master and agent are running on the same host: I want puppet to handle configuration for the master!

    [T] jeff@loire:~ $ openssl s_client -connect loire.example.com:8140 -showcerts > /tmp/cert
    [T] jeff@loire:~ $ sudo openssl verify -CAfile /var/lib/puppet/ssl/certs/ca.pem /tmp/cert
    /tmp/cert: CN = loire.example.com
    error 20 at 0 depth lookup:unable to get local issuer certificate
    2,[T] jeff@loire:~ $

I tried removing all the certificates I could find and restarting the server.

    [T] jeff@loire:~ $ sudo find /var/lib/puppet/ssl -name '*.pem' -delete
    [T] jeff@loire:~ $ sudo rm -rf /etc/puppet/ssl
    [T] jeff@loire:~ $ sudo puppet cert list --all
    Notice: Signed certificate request for ca
    [T] jeff@loire:~ $ sudo puppet cert list --all
    [T] jeff@loire:~ $

(I've never been clear on why certificate lists sometimes become empty on a second request.)  But now on restarting the server, I see this error:

    java.lang.IllegalArgumentException: Expected a KeyPair or PrivateKey, got org.bouncycastle.openssl.PEMEncryptedKeyPair@315e8e4

It looks like I forgot some certificate somewhere.  O, how I would have liked a long entry that says "I see that /path/to/A and /path/to/B are not the same."

I'm temporarily out of ideas and search queries.  If anyone can shed any light, I'd much appreciate it.  If not, this all at least serves to document for those who search these things later.

Jeff Abrahamson
+33 6 24 40 01 57

+44 7920 594 255    <-- only when I'm in the UK
http://ml-week.com/     prochaine edition, 2016

http://jeff.purple.com/
http://blog.purple.com/jeff/

[Jeff Abrahamson]

A final note on this:

I wiped the host, reinitialized (because I'm at that stage of setting things up that this is reasonable and required test), and these problems went away.  It bothers me that I don't understand how and why the certificates were messed up, but I suspect it had to do with my playing with certname, etc.

I'm aware that this thread has become a bit helper-unfriendly.  I've followed on really only for future readers who may find the question and wonder whether anything further was known.

Jeff Abrahamson
+33 6 24 40 01 57

+44 7920 594 255    <-- only when I'm in the UK
http://ml-week.com/     prochaine edition, 2016

http://jeff.purple.com/
http://blog.purple.com/jeff/

[warron.french]

Jeff, I think I ran into this last weekend while setting up my own Puppet Master.  It truly sounds like the exact same issue.

I believe I had to add an extra CNAME record for the name puppet, either that or I had to go and alter the A-record  in my DNS from puppetmaster to puppet.

--------------------------
Warron French

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAM4Y7zxZJqbFo95e57fJOKjtBfXM20sEfJgYracAceoFMqrGXw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.