http 200 wouldn't work if target is behind cloudflare

1,000 views
Skip to first unread message

caar...@gmail.com

unread,
Jun 8, 2018, 9:47:53 AM6/8/18
to Prometheus Users
I'm using blackbox_exporter to monitor some sites, and it seems that if the target is behind cloudflare CDN, it blackbox just returns a `0` status code and the probe fails.

My config is:

  - job_name: blackbox
    metrics_path: /probe
    params:
      module: [http_2xx]
    static_configs:
      - targets:
        - https://site1.com
        - https://site2.com
        - https://etc.com
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: blackbox_exporter:9115
     

The blackbox_exporter config is the default of the docker image.

I'm using blackbox_exporter v0.12.0.

Logs don't show anything but:

level=info ts=2018-05-31T18:27:04.425031477Z caller=main.go:213 msg="Starting blackbox_exporter" version="(version=0.12.0, branch=HEAD, revision=4a22506cf0cf139d9b2f9cde099f0012d9fcabde)"
level
=info ts=2018-05-31T18:27:04.42819512Z caller=main.go:220 msg="Loaded config file"
level
=info ts=2018-05-31T18:27:04.428468953Z caller=main.go:324 msg="Listening on address" address=:9115


Is this expected? Can I work around it somehow?

Thanks

Brian Brazil

unread,
Jun 8, 2018, 9:49:20 AM6/8/18
to caar...@gmail.com, Prometheus Users
Cloudflare has all sorts of DDoS protection, and an automated request like a blackbox probe has a good chance of falling foul to that. Do you have the debug output for the probe?

Brian
 

Thanks

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/3ca51dfa-c7ac-497d-b65a-f8e23635994d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--

caar...@gmail.com

unread,
Jun 8, 2018, 10:04:12 AM6/8/18
to Prometheus Users
Yes:


ts=2018-06-08T13:52:57.26797122Z caller=main.go:174 module=http_2xx target=http://example.com level=debug msg="Beginning probe" probe=http timeout_seconds=9.5
ts=2018-06-08T13:52:57.268212721Z caller=main.go:174 module=http_2xx target=http://example.com level=debug msg="Resolving target address" preferred_ip_protocol=ip6
ts=2018-06-08T13:52:57.420379029Z caller=main.go:174 module=http_2xx target=http://example.com level=debug msg="Resolved target address" ip=2400:cb00:2048:1::6814:3968
ts=2018-06-08T13:52:57.420648853Z caller=main.go:174 module=http_2xx target=http://example.com level=debug msg="Making HTTP request" url=http://[2400:cb00:2048:1::6814:3968] host=example.com
ts=2018-06-08T13:52:57.420966247Z caller=main.go:174 module=http_2xx target=http://example.com level=debug msg="Error for HTTP request" err="Get http://[2400:cb00:2048:1::6814:3968]: dial tcp [2400:cb00:2048:1::6814:3968]:80: connect: network is unreachable"
ts=2018-06-08T13:52:57.421127718Z caller=main.go:174 module=http_2xx target=http://example.com level=debug msg="Response timings for roundtrip" roundtrip=0 start=2018-06-08T13:52:57.420856611Z dnsDone=2018-06-08T13:52:57.420856611Z connectDone=2018-06-08T13:52:57.420918738Z gotConn=0001-01-01T00:00:00Z responseStart=0001-01-01T00:00:00Z end=2018-06-08T13:52:57.420932323Z
ts=2018-06-08T13:52:57.421266013Z caller=main.go:174 module=http_2xx target=http://example.com level=debug msg="Probe failed" duration_seconds=0.153208083


Indeed the ipv6 addr doesn't work from that machine, but works from my machine... 
I whitelisted the blackbox machine on cloudflare, still blocked it seems, probably will take some time to apply. I'll report back if it works so other people benefit :)

Thanks
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To post to this group, send email to promethe...@googlegroups.com.



--

Brian Brazil

unread,
Jun 8, 2018, 10:13:18 AM6/8/18
to caar...@gmail.com, Prometheus Users
On 8 June 2018 at 15:04, <caar...@gmail.com> wrote:
Yes:


ts=2018-06-08T13:52:57.26797122Z caller=main.go:174 module=http_2xx target=http://example.com level=debug msg="Beginning probe" probe=http timeout_seconds=9.5
ts=2018-06-08T13:52:57.268212721Z caller=main.go:174 module=http_2xx target=http://example.com level=debug msg="Resolving target address" preferred_ip_protocol=ip6
ts=2018-06-08T13:52:57.420379029Z caller=main.go:174 module=http_2xx target=http://example.com level=debug msg="Resolved target address" ip=2400:cb00:2048:1::6814:3968
ts=2018-06-08T13:52:57.420648853Z caller=main.go:174 module=http_2xx target=http://example.com level=debug msg="Making HTTP request" url=http://[2400:cb00:2048:1::6814:3968] host=example.com
ts=2018-06-08T13:52:57.420966247Z caller=main.go:174 module=http_2xx target=http://example.com level=debug msg="Error for HTTP request" err="Get http://[2400:cb00:2048:1::6814:3968]: dial tcp [2400:cb00:2048:1::6814:3968]:80: connect: network is unreachable"
ts=2018-06-08T13:52:57.421127718Z caller=main.go:174 module=http_2xx target=http://example.com level=debug msg="Response timings for roundtrip" roundtrip=0 start=2018-06-08T13:52:57.420856611Z dnsDone=2018-06-08T13:52:57.420856611Z connectDone=2018-06-08T13:52:57.420918738Z gotConn=0001-01-01T00:00:00Z responseStart=0001-01-01T00:00:00Z end=2018-06-08T13:52:57.420932323Z
ts=2018-06-08T13:52:57.421266013Z caller=main.go:174 module=http_2xx target=http://example.com level=debug msg="Probe failed" duration_seconds=0.153208083


Indeed the ipv6 addr doesn't work from that machine, but works from my machine... 
I whitelisted the blackbox machine on cloudflare, still blocked it seems, probably will take some time to apply. I'll report back if it works so other people benefit :)

This is a simpler problem, you don't have a working v6 setup. Try setting preferred_ip_protocol to ip4.

Brian
 
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/11731eb8-3d4d-4155-882a-6b8c1763fd4b%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--

caar...@gmail.com

unread,
Jun 8, 2018, 10:24:38 AM6/8/18
to Prometheus Users
Oh indeed, I forgot to change that!

It is working now, thanks Brian!

Have a great day!

On Friday, June 8, 2018 at 11:13:18 AM UTC-3, Brian Brazil wrote:
Reply all
Reply to author
Forward
0 new messages