Using Blackbox_Exporter

[yongem...@gmail.com]

Dear All,

I Am Currently Tasked With Using Prometheus, Blackbox Exporter To Monitor Website and Have Some Questions

1. How Can I Monitor The Request Time and Response Time Of The Website? What Are The Appropriate Metrics?

2. Am I Doing It Correctly By Using The Metric probe_duration_seconds To Monitor The Ping? 

The Results Is Very Different From Pinging From Server To Website vs. Using This Metric.

Thank You In Advance

[Ben Kochie]

You might want to consider using https://latency.at/

On Sat, May 6, 2017 at 4:52 AM, <yongem...@gmail.com> wrote:

Dear All,

I Am Currently Tasked With Using Prometheus, Blackbox Exporter To Monitor Website and Have Some Questions

1. How Can I Monitor The Request Time and Response Time Of The Website? What Are The Appropriate Metrics?

probe_duration_seconds will give you the response time of the blackbox probe.

If you want to find out how long the server thinks it took to handle the request, you will have to use whitebox monitoring on the server.  Setting this up highly depends on what kind of webserver software you're using, what kind of application software, and how your server is configured.

 

2. Am I Doing It Correctly By Using The Metric probe_duration_seconds To Monitor The Ping? 

The Results Is Very Different From Pinging From Server To Website vs. Using This Metric.

 probe_duration_seconds is correct.  If when you say "ping", you're talking about ICMP ping, yes, they will be different.  ICMP does not involve sending anything to the webserver software.   Where as with blackbox_exporter, you're probably probing with http.

Also note that the blackbox_exporter by default does a DNS lookup for every probe.  If your server doesn't have a local DNS proxy, you could get much higher and inconsistent latency results.  I recommend using something like dnsmasq installed locally on the server running the blackbox_exporter to cache DNS lookups.

Thank You In Advance

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/320f04eb-8033-412d-8cd8-c5c5b16da296%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Emmanuel Yong]

I am looking at deploying it in-house and not on the cloud.

Are there any other ways to fulfill?

Also, is my configuration below correct to monitor the ping of the website

Here are my configurations and my understanding

1. probe_duration_seconds metrics is measured in seconds

To convert it to milliseconds, i use probe_duration_seconds(job="ping_all"} * 1000

Also i configured the step to be 60s and resolution of 1/1

Axes i show Left Y with Unit as milliseconds (ms) and Scale is linear

Not sure whether the above is the proper way of doing it

On Saturday, May 6, 2017 at 2:44:30 PM UTC+8, Ben Kochie wrote:

You might want to consider using https://latency.at/

On Sat, May 6, 2017 at 4:52 AM, <yongem...@gmail.com> wrote:

Dear All,

I Am Currently Tasked With Using Prometheus, Blackbox Exporter To Monitor Website and Have Some Questions

1. How Can I Monitor The Request Time and Response Time Of The Website? What Are The Appropriate Metrics?

probe_duration_seconds will give you the response time of the blackbox probe.

If you want to find out how long the server thinks it took to handle the request, you will have to use whitebox monitoring on the server.  Setting this up highly depends on what kind of webserver software you're using, what kind of application software, and how your server is configured.

 

2. Am I Doing It Correctly By Using The Metric probe_duration_seconds To Monitor The Ping? 

The Results Is Very Different From Pinging From Server To Website vs. Using This Metric.

 probe_duration_seconds is correct.  If when you say "ping", you're talking about ICMP ping, yes, they will be different.  ICMP does not involve sending anything to the webserver software.   Where as with blackbox_exporter, you're probably probing with http.

Also note that the blackbox_exporter by default does a DNS lookup for every probe.  If your server doesn't have a local DNS proxy, you could get much higher and inconsistent latency results.  I recommend using something like dnsmasq installed locally on the server running the blackbox_exporter to cache DNS lookups.

Thank You In Advance

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To post to this group, send email to promethe...@googlegroups.com.

[Ben Kochie]

It seems like you're using Grafana for the dashboard visualization, I will respond assuming this.

On Sat, May 6, 2017 at 8:49 AM, Emmanuel Yong <yongem...@gmail.com> wrote:

I am looking at deploying it in-house and not on the cloud.

Are there any other ways to fulfill?

Also, is my configuration below correct to monitor the ping of the website

Here are my configurations and my understanding

1. probe_duration_seconds metrics is measured in seconds

To convert it to milliseconds, i use probe_duration_seconds(job="ping_all"} * 1000

This is unnecessary in Grafana.  Under the "Axes" tab, set the "unit" to "time" and "Seconds (s)".  This will correctly display milliseconds when appropriate.

Also i configured the step to be 60s and resolution of 1/1

Do not set the step, Grafana will automatically scale this.

Setting the resolution is fine.

 

Axes i show Left Y with Unit as milliseconds (ms) and Scale is linear

Not sure whether the above is the proper way of doing it

On Saturday, May 6, 2017 at 2:44:30 PM UTC+8, Ben Kochie wrote:

You might want to consider using https://latency.at/

On Sat, May 6, 2017 at 4:52 AM, <yongem...@gmail.com> wrote:

Dear All,

I Am Currently Tasked With Using Prometheus, Blackbox Exporter To Monitor Website and Have Some Questions

1. How Can I Monitor The Request Time and Response Time Of The Website? What Are The Appropriate Metrics?

probe_duration_seconds will give you the response time of the blackbox probe.

If you want to find out how long the server thinks it took to handle the request, you will have to use whitebox monitoring on the server.  Setting this up highly depends on what kind of webserver software you're using, what kind of application software, and how your server is configured.

 

2. Am I Doing It Correctly By Using The Metric probe_duration_seconds To Monitor The Ping? 

The Results Is Very Different From Pinging From Server To Website vs. Using This Metric.

 probe_duration_seconds is correct.  If when you say "ping", you're talking about ICMP ping, yes, they will be different.  ICMP does not involve sending anything to the webserver software.   Where as with blackbox_exporter, you're probably probing with http.

Also note that the blackbox_exporter by default does a DNS lookup for every probe.  If your server doesn't have a local DNS proxy, you could get much higher and inconsistent latency results.  I recommend using something like dnsmasq installed locally on the server running the blackbox_exporter to cache DNS lookups.

Thank You In Advance

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To post to this group, send email to promethe...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/320f04eb-8033-412d-8cd8-c5c5b16da296%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/a0a3d82f-a530-42f1-bf18-f70192fc6513%40googlegroups.com.

[Emmanuel Yong]

Hi Ben,

Thank You for Feedback, Definitely i will have this blogged with the appropriate credit and share for everyone.

Is this the correct metrics to measure ping? It seems different from the server output vs. using this exporter

Also, any exporter we can use to measure the request time and response time and the latency or packet loss when monitoring website?

[Ben Kochie]

On Sat, May 6, 2017 at 1:54 PM, Emmanuel Yong <yongem...@gmail.com> wrote:

Hi Ben,

Thank You for Feedback, Definitely i will have this blogged with the appropriate credit and share for everyone.

Is this the correct metrics to measure ping? It seems different from the server output vs. using this exporter

Like I said, ICMP Ping and HTTP "ping" are going to return different results.

 

Also, any exporter we can use to measure the request time and response time and the latency or packet loss when monitoring website?

There are lots of things to monitor, but without more information about your infrastructure, I don't know what to recommend.

Maybe hop on our IRC chat (https://prometheus.io/community/), See the Riot link for a web interface.

 

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/78e2ac9d-c330-41b5-aefa-4a840f4ffd5b%40googlegroups.com.

[Emmanuel Yong]

Hi Ben,

Thank You For Your Wonderful Feedback, Just A Short Summary.

I think what was discuss here, will be worth sharing to everyone as it really clarifies a lot on blackbox exporter

1. Metric probe_duration_seconds:

- Provides the response time of the blackbox probe

- Blackbox exporter utilizes http ping as the prober is based on http

Blackbox_exporter by default does a DNS lookup for every probe so this lookup time is already included in the metric probe_duration_seconds

To prevent getting inconsistent latency results if the server do not have any local DNS proxy, recommendation will be to have something like dnsmasq installed locally on the server running the blackbox_exporter to cache DNS lookups.

Also, if i wanted to measure the RTT, request and response time of the website this is not possible with blackbox? I need to use whitebox? Or i am wrong on this statement?

What i wanted to achieve was

- Display Total Ping Response Time from blackbox_exporter server to a particular website (Example: google.com)

- Display Total Packet Loss from blackbox_exporter server to a particular website (Example: google.com)

- Display How Long Does It Take In Time format request from blackbox_exporter server to a particular website (Example: google.com) and How long does it take for the website to response to my request

[Emmanuel Yong]

Dear All,

Can help me understand the following warning message?

time="2017-05-06T10:33:22+08:00" level=warning msg="Timeout reading from socket for www.google.com: read ip4 0.0.0.0: i/o timeout" source="icmp.go:143" 

time="2017-05-06T10:34:03+08:00" level=warning msg="Timeout reading from socket for www.icbc.com.cn: read ip4 0.0.0.0: i/o timeout" source="icmp.go:143" 

time="2017-05-06T10:35:03+08:00" level=warning msg="Timeout reading from socket for www.icbc.com.cn: read ip4 0.0.0.0: i/o timeout" source="icmp.go:143" 

time="2017-05-06T10:35:20+08:00" level=warning msg="Timeout reading from socket for www.abchina.com: read ip4 0.0.0.0: i/o timeout" source="icmp.go:143" 

time="2017-05-06T10:35:52+08:00" level=warning msg="Timeout reading from socket for www.cmbc.com.cn: read ip4 0.0.0.0: i/o timeout" source="icmp.go:143"