x509 certificate signed by unknown authority on known good SSL certificate

[Leidrin]

Hello,

We just stood up an instance of blackbox exporter, and are checking 3 public URLs we have published.

2 of these urls (our main website [www.example.com] and our public cloud [cloud.example.com]) use the same certificate, and both validate in a web browser, but only one fails our https probe.

-------------------

On the successful probe (www.example.com):

ts=2021-12-04T22:59:13.687241146Z caller=main.go:320 module=http_2xx target=https://www.example.com level=info msg="Beginning probe" probe=http timeout_seconds=9.5

ts=2021-12-04T22:59:13.687512254Z caller=http.go:335 module=http_2xx target=https://www.example.com level=info msg="Resolving target address" ip_protocol=ip4

ts=2021-12-04T22:59:13.705337932Z caller=http.go:335 module=http_2xx target=https://www.example.com level=info msg="Resolved target address" ip=9.9.9.9

ts=2021-12-04T22:59:13.705528263Z caller=client.go:251 module=http_2xx target=https://www.example.com level=info msg="Making HTTP request" url=https://9.9.9.9 host=www.example.com

ts=2021-12-04T22:59:14.082224421Z caller=main.go:130 module=http_2xx target=https://www.example.com level=info msg="Received HTTP response" status_code=200

ts=2021-12-04T22:59:14.230206404Z caller=main.go:130 module=http_2xx target=https://www.example.com level=info msg="Response timings for roundtrip" roundtrip=0 start=2021-12-04T22:59:13.705719316Z dnsDone=2021-12-04T22:59:13.705719316Z connectDone=2021-12-04T22:59:13.753485467Z gotConn=2021-12-04T22:59:13.861778963Z responseStart=2021-12-04T22:59:14.082125293Z tlsStart=2021-12-04T22:59:13.753563355Z tlsDone=2021-12-04T22:59:13.861722668Z end=2021-12-04T22:59:14.230171423Z

ts=2021-12-04T22:59:14.23049714Z caller=main.go:320 module=http_2xx target=https://www.example.com level=info msg="Probe succeeded" duration_seconds=0.543175273

-------------------

On the unsuccessful probe:

ts=2021-12-04T23:11:20.08588326Z caller=main.go:320 module=http_2xx target=https://cloud.example.com level=info msg="Beginning probe" probe=http timeout_seconds=9.5

ts=2021-12-04T23:11:20.08660784Z caller=http.go:335 module=http_2xx target=https://cloud.example.com level=info msg="Resolving target address" ip_protocol=ip4

ts=2021-12-04T23:11:20.109536839Z caller=http.go:335 module=http_2xx target=https://cloud.example.com level=info msg="Resolved target address" ip=9.10.9.10

ts=2021-12-04T23:11:20.109953128Z caller=client.go:251 module=http_2xx target=https://cloud.example.com level=info msg="Making HTTP request" url=https://9.10.9.10 host=cloud.example.com

ts=2021-12-04T23:11:20.155121601Z caller=main.go:130 module=http_2xx target=https://cloud.example.com level=error msg="Error for HTTP request" err="Get \"https://9.10.9.10\": x509: certificate signed by unknown authority"

ts=2021-12-04T23:11:20.155282857Z caller=main.go:130 module=http_2xx target=https://cloud.example.com level=info msg="Response timings for roundtrip" roundtrip=0 start=2021-12-04T23:11:20.110415083Z dnsDone=2021-12-04T23:11:20.110415083Z connectDone=2021-12-04T23:11:20.128115064Z gotConn=0001-01-01T00:00:00Z responseStart=0001-01-01T00:00:00Z tlsStart=2021-12-04T23:11:20.128236858Z tlsDone=2021-12-04T23:11:20.155045139Z end=0001-01-01T00:00:00Z

ts=2021-12-04T23:11:20.155445558Z caller=main.go:320 module=http_2xx target=https://cloud.example.com level=error msg="Probe failed" duration_seconds=0.069415226

-------------------

I can visit both sites in browser and see the valid certificate, which appears identical on both sites.

Redacted hostnames/IPs but happy to send in private if needed for diagnosis.

Running Blackbox from docker using Latest build, also tested against current Master.

Any ideas?

PS. Is there any way to increase the size of the "New Conversation" window here? The default ~1/8th of my screen proved tough to compose in, I ended up doing most of this in notepad++ :P

[Brian Candler]

Recent versions of Go ignore the CommonName (CN) in the cert, and only honour SubjectAltName.  However, if that were the problem, the error wouldn't say "certificate signed by unknown authority".

If your URLs are reachable from the Internet, then I suggest you post the real URLs here (in the spirit of the PowerDNS support policy) and it could be answered in seconds.  If you're not prepared to do that, then all I can do is give you some general pointers.

How sure are you that it's "the same certificate"?  Have you tried:

openssl s_client -connect www.example.com:443 -servername www.example.com

openssl s_client -connect cloud.example.com:443 -servername cloud.example.com

In each case, copy the section between "-----BEGIN CERTIFICATE-----" and "-----END CERTIFICATE-----" inclusive to a file (say "www.example.com.pem" and "cloud.example.com.pem")

Check that the files are exactly identical.

diff -u www.example.com.pem cloud.example.com.pem

If they are not, decode the contents:

openssl x509 -in www.example.com.pem -noout -text

openssl x509 -in cloud.example.com.pem -noout -text

And if they are the same, look at the full chains returned by your server(s) for both names:

openssl s_client -connect www.example.com:443 -servername www.example.com -showcerts

openssl s_client -connect cloud.example.com:443 -servername cloud.example.com -showcerts

and see what's different between them.  Since you're implying that they resolve to two different IP addresses, it could be that one server is returning the intermediate certs properly, and the other is not.