Interesting "Bad Certificate" Problem

[Aaron Blew]

All,

I'm having an interesting certificate problem with a host I provisioned today.  The host was provisioned and puppet was installed as part of the post-os provisioning process.  After I signed the certificate I see the following on the client side:

[root@client ~]# puppetd --verbose --no-daemonize

notice: Starting Puppet client version 0.25.4

err: Could not retrieve catalog from remote server: certificate verify failed

notice: Using cached catalog

err: Could not retrieve catalog; skipping run

On the puppetmaster side I see this in the web log:

[2010-07-01 13:26:05] client.domain.name - - [01/Jul/2010:13:26:05 PDT] "GET /production/certificate/ca HTTP/1.1" 200 765

[2010-07-01 13:26:05] - -> /production/certificate/ca

[2010-07-01 13:26:05] client.domain.name - - [01/Jul/2010:13:26:05 PDT] "GET /production/certificate/client.domain.name HTTP/1.1" 404 49

[2010-07-01 13:26:05] - -> /production/certificate/client.domain.name

[2010-07-01 13:26:05] client.domain.name - - [01/Jul/2010:13:26:05 PDT] "GET /production/certificate_request/client.domain.name HTTP/1.1" 404 57

[2010-07-01 13:26:05] - -> /production/certificate_request/client.domain.name

[2010-07-01 13:26:05] client.domain.name - - [01/Jul/2010:13:26:05 PDT] "PUT /production/certificate_request/client.domain.name HTTP/1.1" 200 5

[2010-07-01 13:26:05] - -> /production/certificate_request/client.domain.name

[2010-07-01 13:26:05] client.domain.name - - [01/Jul/2010:13:26:05 PDT] "GET /production/certificate/client.domain.name HTTP/1.1" 404 49

[2010-07-01 13:26:05] - -> /production/certificate/client.domain.name

[2010-07-01 13:26:05] client.domain.name - - [01/Jul/2010:13:26:05 PDT] "GET /production/certificate/client.domain.name HTTP/1.1" 404 49

[2010-07-01 13:26:05] - -> /production/certificate/client.domain.name

[2010-07-01 13:27:05] client.domain.name - - [01/Jul/2010:13:27:05 PDT] "GET /production/certificate/client.domain.name HTTP/1.1" 200 847

[2010-07-01 13:27:05] - -> /production/certificate/client.domain.name

[2010-07-01 13:27:05] client.domain.name - - [01/Jul/2010:13:27:05 PDT] "GET /production/certificate_revocation_list/ca HTTP/1.1" 200 508

[2010-07-01 13:27:05] - -> /production/certificate_revocation_list/ca

[2010-07-01 13:27:05] ERROR OpenSSL::SSL::SSLError: sslv3 alert bad certificate

/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:44:in `accept'

/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:44:in `listen'

/usr/lib/ruby/1.8/webrick/server.rb:173:in `call'

/usr/lib/ruby/1.8/webrick/server.rb:173:in `start_thread'

/usr/lib/ruby/1.8/webrick/server.rb:162:in `start'

/usr/lib/ruby/1.8/webrick/server.rb:162:in `start_thread'

/usr/lib/ruby/1.8/webrick/server.rb:95:in `start'

/usr/lib/ruby/1.8/webrick/server.rb:92:in `each'

/usr/lib/ruby/1.8/webrick/server.rb:92:in `start'

/usr/lib/ruby/1.8/webrick/server.rb:23:in `start'

/usr/lib/ruby/1.8/webrick/server.rb:82:in `start'

/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:42:in `listen'

/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:41:in `initialize'

/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:41:in `new'

/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:41:in `listen'

/usr/lib/ruby/1.8/thread.rb:135:in `synchronize'

/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:38:in `listen'

/usr/lib/ruby/site_ruby/1.8/puppet/network/server.rb:131:in `listen'

/usr/lib/ruby/site_ruby/1.8/puppet/network/server.rb:146:in `start'

/usr/lib/ruby/site_ruby/1.8/puppet/daemon.rb:128:in `start'

/usr/lib/ruby/site_ruby/1.8/puppet/application/puppetmasterd.rb:122:in `main'

/usr/lib/ruby/site_ruby/1.8/puppet/application.rb:226:in `send'

/usr/lib/ruby/site_ruby/1.8/puppet/application.rb:226:in `run_command'

/usr/lib/ruby/site_ruby/1.8/puppet/application.rb:217:in `run'

/usr/lib/ruby/site_ruby/1.8/puppet/application.rb:306:in `exit_on_fail'

/usr/lib/ruby/site_ruby/1.8/puppet/application.rb:217:in `run'

/usr/sbin/puppetmasterd:66

[2010-07-01 13:27:24] ERROR OpenSSL::SSL::SSLError: sslv3 alert bad certificate

/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:44:in `accept'

/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:44:in `listen'

/usr/lib/ruby/1.8/webrick/server.rb:173:in `call'

/usr/lib/ruby/1.8/webrick/server.rb:173:in `start_thread'

/usr/lib/ruby/1.8/webrick/server.rb:162:in `start'

/usr/lib/ruby/1.8/webrick/server.rb:162:in `start_thread'

/usr/lib/ruby/1.8/webrick/server.rb:95:in `start'

/usr/lib/ruby/1.8/webrick/server.rb:92:in `each'

/usr/lib/ruby/1.8/webrick/server.rb:92:in `start'

/usr/lib/ruby/1.8/webrick/server.rb:23:in `start'

/usr/lib/ruby/1.8/webrick/server.rb:82:in `start'

/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:42:in `listen'

/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:41:in `initialize'

/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:41:in `new'

/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:41:in `listen'

/usr/lib/ruby/1.8/thread.rb:135:in `synchronize'

/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:38:in `listen'

/usr/lib/ruby/site_ruby/1.8/puppet/network/server.rb:131:in `listen'

/usr/lib/ruby/site_ruby/1.8/puppet/network/server.rb:146:in `start'

/usr/lib/ruby/site_ruby/1.8/puppet/daemon.rb:128:in `start'

/usr/lib/ruby/site_ruby/1.8/puppet/application/puppetmasterd.rb:122:in `main'

/usr/lib/ruby/site_ruby/1.8/puppet/application.rb:226:in `send'

/usr/lib/ruby/site_ruby/1.8/puppet/application.rb:226:in `run_command'

/usr/lib/ruby/site_ruby/1.8/puppet/application.rb:217:in `run'

/usr/lib/ruby/site_ruby/1.8/puppet/application.rb:306:in `exit_on_fail'

/usr/lib/ruby/site_ruby/1.8/puppet/application.rb:217:in `run'

/usr/sbin/puppetmasterd:66

[2010-07-01 13:27:31] ERROR OpenSSL::SSL::SSLError: SSL_write:: internal error

/usr/lib/ruby/1.8/openssl/buffering.rb:178:in `syswrite'

/usr/lib/ruby/1.8/openssl/buffering.rb:178:in `do_write'

/usr/lib/ruby/1.8/openssl/buffering.rb:197:in `<<'

/usr/lib/ruby/1.8/webrick/httpresponse.rb:324:in `_write_data'

/usr/lib/ruby/1.8/webrick/httpresponse.rb:296:in `send_body_string'

/usr/lib/ruby/1.8/webrick/httpresponse.rb:187:in `send_body'

/usr/lib/ruby/1.8/webrick/httpresponse.rb:104:in `send_response'

/usr/lib/ruby/1.8/webrick/httpserver.rb:79:in `run'

/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:45:in `listen'

/usr/lib/ruby/1.8/webrick/server.rb:173:in `call'

/usr/lib/ruby/1.8/webrick/server.rb:173:in `start_thread'

/usr/lib/ruby/1.8/webrick/server.rb:162:in `start'

/usr/lib/ruby/1.8/webrick/server.rb:162:in `start_thread'

/usr/lib/ruby/1.8/webrick/server.rb:95:in `start'

/usr/lib/ruby/1.8/webrick/server.rb:92:in `each'

/usr/lib/ruby/1.8/webrick/server.rb:92:in `start'

/usr/lib/ruby/1.8/webrick/server.rb:23:in `start'

/usr/lib/ruby/1.8/webrick/server.rb:82:in `start'

/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:42:in `listen'

/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:41:in `initialize'

/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:41:in `new'

/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:41:in `listen'

/usr/lib/ruby/1.8/thread.rb:135:in `synchronize'

/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:38:in `listen'

/usr/lib/ruby/site_ruby/1.8/puppet/network/server.rb:131:in `listen'

/usr/lib/ruby/site_ruby/1.8/puppet/network/server.rb:146:in `start'

/usr/lib/ruby/site_ruby/1.8/puppet/daemon.rb:128:in `start'

/usr/lib/ruby/site_ruby/1.8/puppet/application/puppetmasterd.rb:122:in `main'

/usr/lib/ruby/site_ruby/1.8/puppet/application.rb:226:in `send'

/usr/lib/ruby/site_ruby/1.8/puppet/application.rb:226:in `run_command'

/usr/lib/ruby/site_ruby/1.8/puppet/application.rb:217:in `run'

/usr/lib/ruby/site_ruby/1.8/puppet/application.rb:306:in `exit_on_fail'

/usr/lib/ruby/site_ruby/1.8/puppet/application.rb:217:in `run'

/usr/sbin/puppetmasterd:66

It seems like the certificate might be bad but I've run puppetca --revoke/puppetca --clean and re-generated the certificate on the client side a few times.  I'm kind of at a loss.

Thanks all!

-Aaron

[Jeff McCune]

On Thu, Jul 1, 2010 at 1:36 PM, Aaron Blew <aaro...@gmail.com> wrote:
> All,
> I'm having an interesting certificate problem with a host I provisioned
> today.

Have you checked your clocks? Is the client in sync with the server?

--
Jeff McCune
http://www.puppetlabs.com/