Blackbox to telnet

38 views
Skip to first unread message

David Aylward

unread,
May 8, 2020, 7:52:16 AM5/8/20
to Prometheus Users

HI,

We using BlackBox to do our monitoring.
We've noticed that when BlackBox is pointed at telnet or SNTP with the TCP module it closes the connection with a reset instead of a graceful close, which it does for other TCP ports. These resets show up as errors in our other systems, is there a config or something we can add to have this close with a fin and not a reset.


WhatsApp Image 2020-05-08 at 12.42.07.jpeg

Brian Brazil

unread,
May 8, 2020, 8:03:45 AM5/8/20
to David Aylward, Prometheus Users
Can you share the debug output?
 
--

David Aylward

unread,
May 8, 2020, 8:40:18 AM5/8/20
to Prometheus Users
ts=2020-05-08T12:29:52.877070815Z caller=main.go:116 module=tcp_connect target=100.100.0.39:23 level=info msg="Beginning probe" probe=tcp timeout_seconds=9.5
ts=2020-05-08T12:29:52.87760679Z caller=utils.go:42 module=tcp_connect target=100.100.0.39:23 level=info msg="Resolving target address" preferred_ip_protocol=ip6
ts=2020-05-08T12:29:52.877828711Z caller=utils.go:52 module=tcp_connect target=100.100.0.39:23 level=warn msg="Resolution with preferred IP protocol failed, attempting fallback protocol" fallback_protocol=ip4 err="address 100.100.0.39: no suitable address found"
ts=2020-05-08T12:29:52.877996813Z caller=utils.go:65 module=tcp_connect target=100.100.0.39:23 level=info msg="Resolved target address" ip=100.100.0.39
ts=2020-05-08T12:29:52.878092926Z caller=tcp.go:55 module=tcp_connect target=100.100.0.39:23 level=info msg="Dialing TCP without TLS"
ts=2020-05-08T12:29:52.943090024Z caller=tcp.go:99 module=tcp_connect target=100.100.0.39:23 level=info msg="Successfully dialed"
ts=2020-05-08T12:29:52.943316164Z caller=main.go:127 module=tcp_connect target=100.100.0.39:23 level=info msg="Probe succeeded" duration_seconds=0.065955569



Metrics that would have been returned:
# HELP probe_dns_lookup_time_seconds Returns the time taken for probe dns lookup in seconds
# TYPE probe_dns_lookup_time_seconds gauge
probe_dns_lookup_time_seconds 0.000272136
# HELP probe_duration_seconds Returns how long the probe took to complete in seconds
# TYPE probe_duration_seconds gauge
probe_duration_seconds 0.065955569
# HELP probe_failed_due_to_regex Indicates if probe failed due to regex
# TYPE probe_failed_due_to_regex gauge
probe_failed_due_to_regex 0
# HELP probe_ip_protocol Specifies whether probe ip protocol is IP4 or IP6
# TYPE probe_ip_protocol gauge
probe_ip_protocol 4
# HELP probe_success Displays whether or not the probe was a success
# TYPE probe_success gauge
probe_success 1



Module configuration:
prober: tcp

Brian Brazil

unread,
May 8, 2020, 9:33:47 AM5/8/20
to David Aylward, Prometheus Users
Hmm, so the blackbox exporter is gracefully closing its half of the connection with a FIN. I'd say what's happening is that the blackbox exporter is closing its end connection as it no longer cares about the connection at all, then some additional data is received which causes it to RST as it never reads anything in this scenario. Given this is telnet, I'd suggest using the query_response feature to close the connection at the telnet level or just eat that response.

Brian

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/f26d012e-681c-4b93-b4d4-cc17800ada51%40googlegroups.com.


--
Reply all
Reply to author
Forward
0 new messages