Blackbox exporter probe_ssl_earliest_cert_expiry giving wrong value in grafana

1,585 views
Skip to first unread message

James Eduard Andaya

unread,
Apr 23, 2020, 5:26:44 AM4/23/20
to Prometheus Users

I have configured and installed blackbox exporter to monitor application url and SSL expiry in grafana dashboard , I see the SSL expiry is showing wrong

probe_ssl_earliest_cert_expiry{instance=~"$target"} -time()

but on priometheus panel i was able to get the correct data of expiration.


https://github.com/prometheus/blackbox_exporter/issues/607

Murali Krishna Kanagala

unread,
Apr 23, 2020, 6:05:24 AM4/23/20
to James Eduard Andaya, Prometheus Users
Hi James,

Multiply the expiry value you receive from probe_ssl_earliest_cert_expiry  with thousand when you use it on Grafana. We can directly apply the units on Grafana without subtracting the time() from it.

Thanks,
Murali Kanagala

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/77995789-75a7-4d14-93df-9c7d20d97221%40googlegroups.com.

James Eduard Andaya

unread,
Apr 23, 2020, 6:33:43 AM4/23/20
to Prometheus Users
Hi sir thanks for the input how, im using those on grafana

probe_ssl_earliest_cert_expiry{instance=~"$target"} - time()


probe.PNG



Regards,
James


On Thursday, April 23, 2020 at 6:05:24 PM UTC+8, Murali Krishna Kanagala wrote:
Hi James,

Multiply the expiry value you receive from probe_ssl_earliest_cert_expiry  with thousand when you use it on Grafana. We can directly apply the units on Grafana without subtracting the time() from it.

Thanks,
Murali Kanagala

On Thu, Apr 23, 2020, 4:26 AM James Eduard Andaya <jamesedu...@gmail.com> wrote:

I have configured and installed blackbox exporter to monitor application url and SSL expiry in grafana dashboard , I see the SSL expiry is showing wrong

probe_ssl_earliest_cert_expiry{instance=~"$target"} -time()

but on priometheus panel i was able to get the correct data of expiration.


https://github.com/prometheus/blackbox_exporter/issues/607

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to promethe...@googlegroups.com.

Julius Volz

unread,
Apr 23, 2020, 6:35:37 AM4/23/20
to James Eduard Andaya, Prometheus Users
In what ways is the expiry being shown incorrectly in Grafana? Can you provide a screenshot including the panel output in Grafana as well?

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/35888b1a-4c08-4f9c-8db3-2ff8e8416104%40googlegroups.com.

James Eduard Andaya

unread,
Apr 23, 2020, 6:56:33 AM4/23/20
to Prometheus Users

The status on grafana is incorrect based on actual ssl certificate expiry, pelase refer below on the screenshoot.


vhan.PNG

vhan2.PNG

vhancert.PNG





On Thursday, April 23, 2020 at 6:35:37 PM UTC+8, Julius Volz wrote:
In what ways is the expiry being shown incorrectly in Grafana? Can you provide a screenshot including the panel output in Grafana as well?

On Thu, Apr 23, 2020 at 12:33 PM James Eduard Andaya <jamesedu...@gmail.com> wrote:
Hi sir thanks for the input how, im using those on grafana

probe_ssl_earliest_cert_expiry{instance=~"$target"} - time()


probe.PNG



Regards,
James

On Thursday, April 23, 2020 at 6:05:24 PM UTC+8, Murali Krishna Kanagala wrote:
Hi James,

Multiply the expiry value you receive from probe_ssl_earliest_cert_expiry  with thousand when you use it on Grafana. We can directly apply the units on Grafana without subtracting the time() from it.

Thanks,
Murali Kanagala

On Thu, Apr 23, 2020, 4:26 AM James Eduard Andaya <jamesedu...@gmail.com> wrote:

I have configured and installed blackbox exporter to monitor application url and SSL expiry in grafana dashboard , I see the SSL expiry is showing wrong

probe_ssl_earliest_cert_expiry{instance=~"$target"} -time()

but on priometheus panel i was able to get the correct data of expiration.


https://github.com/prometheus/blackbox_exporter/issues/607

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to promethe...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/77995789-75a7-4d14-93df-9c7d20d97221%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to promethe...@googlegroups.com.

Murali Krishna Kanagala

unread,
Apr 23, 2020, 7:10:45 AM4/23/20
to James Eduard Andaya, Prometheus Users
Grafana query:
probe_ssl_earliest_cert_expiry{instance=~"$target"} * 1000

Grafana unit: any date unit 

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/35888b1a-4c08-4f9c-8db3-2ff8e8416104%40googlegroups.com.

Brian Brazil

unread,
Apr 23, 2020, 7:24:22 AM4/23/20
to James Eduard Andaya, Prometheus Users
On Thu, 23 Apr 2020 at 11:56, James Eduard Andaya <jamesedua...@gmail.com> wrote:

The status on grafana is incorrect based on actual ssl certificate expiry, pelase refer below on the screenshoot.


vhan.PNG

vhan2.PNG

 

vhancert.PNG


This is only one cert, the blackbox exporter looks at everything in the chain for  probe_ssl_earliest_cert_expiry. So most likely some other cert in the chain is close to expiring.

Brian
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/4b5ac84a-fe3c-4998-9cb4-7e0f8c6f3fd6%40googlegroups.com.


--

James Eduard Andaya

unread,
Apr 23, 2020, 7:38:42 AM4/23/20
to Prometheus Users
Hi Brian yes its correct but the grafana display status that is not accurate based on the actual ceritificate expiration, sorry just newbie here in prometheus, any help on querry to display the right expiration date



sslcert.PNG



Regards,
James




On Thursday, April 23, 2020 at 7:24:22 PM UTC+8, Brian Brazil wrote:

Julius Volz

unread,
Apr 23, 2020, 8:45:18 AM4/23/20
to James Eduard Andaya, Prometheus Users
In your Grafana panel under Visualization -> Units, what is the unit set to for this graph? It looks like it might be set to micro- or nanoseconds, but you need to set it to seconds.

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/4b5ac84a-fe3c-4998-9cb4-7e0f8c6f3fd6%40googlegroups.com.

James Eduard Andaya

unread,
Apr 23, 2020, 9:00:27 AM4/23/20
to Prometheus Users
Hi Julius it was set to  Duration (s)

durationsec.PNG




Regards,
James


On Thursday, April 23, 2020 at 8:45:18 PM UTC+8, Julius Volz wrote:
In your Grafana panel under Visualization -> Units, what is the unit set to for this graph? It looks like it might be set to micro- or nanoseconds, but you need to set it to seconds.

James Eduard Andaya

unread,
Apr 23, 2020, 9:16:30 AM4/23/20
to Prometheus Users

@brian Brazil can you check this https://github.com/greg-solutions/blackbox_exporter/commit/b869cef1368e2728bbbb5ac8f74f61f1b4ce3a75 can we apply the fixes or update ?


Regards,
James 

 

Brian Brazil

unread,
Apr 23, 2020, 9:20:40 AM4/23/20
to James Eduard Andaya, Prometheus Users
On Thu, 23 Apr 2020 at 14:16, James Eduard Andaya <jamesedua...@gmail.com> wrote:

@brian Brazil can you check this https://github.com/greg-solutions/blackbox_exporter/commit/b869cef1368e2728bbbb5ac8f74f61f1b4ce3a75 can we apply the fixes or update ?


As I already indicated on the PR, this change is incorrect and does not address the relevant issue. 

--

James Eduard Andaya

unread,
Apr 23, 2020, 9:48:21 AM4/23/20
to Prometheus Users
Ok noted, so is there any work around to add the request like display the actual expiration based on the certificate actual status?

Regards,
James

On Thursday, April 23, 2020 at 9:20:40 PM UTC+8, Brian Brazil wrote:

Per Carlson

unread,
Apr 24, 2020, 7:25:26 AM4/24/20
to James Eduard Andaya, Prometheus Users
Ok noted, so is there any work around to add the request like display the actual expiration based on the certificate actual status?

You could try https://github.com/ribbybibby/ssl_exporter to track the expiry of every certificate in the chain.

But as many have told you in this thread, you should really check the expiry of the root and intermediate certs in the chain. If any of those expire you have a server certificate no one will trust, regardless of the expiry date of the server cert.

James Eduard Andaya

unread,
Apr 24, 2020, 7:42:59 AM4/24/20
to Prometheus Users
Hi Per thanks for the input, issue was already resolved, i was able to configured and make some changes on SSL Monitoring, thank you guys

Regards,
James
Reply all
Reply to author
Forward
0 new messages