Additional phase metrics for TCP probes?

[Hugo Slabbert]

Would folks be open to additional phase metrics for TCP probes, similar to the resolve, connect, etc. phases in http prober and the resolve etc. phase metrics in the icmp prober?

I've got a case where it would be helpful to have insight into the connect (TCP connection established) vs. closed phases.  On low latency connections, the initial TCP connection 3-way handshake hits about 2-3 ms for me on a set of targets (I'm assuming it's iniital flow entry, like conntrack), whereas the connection close (FIN+ACK both ways, then final ACK) is significantly quicker (around 0.3 ms).  So the connection close phase is closer to the real network RTT, whereas the initial TCP connection establishment has about an order of magnitude overhead.

I could try to add ICMP probes on for these hosts as well, and hopefully with low enough frequency could bypass that initial overhead as later ICMP probes reuse existing flow/session entries, but it would be ideal to be able to get this additional info from the existing TCP probes.

I'm just starting to dive into golang, but if it's not something that others would want to take up I could try my hand at it, modeling after the trace model used in the http prober, if this would be something useful for others and that has a shot at getting approved as a PR.

[Hugo Slabbert]

Sorry; this was for blackbox_exporter specifically.

I skimmed quickly and missed that this is the general Prometheus-users lists rather than just blackbox-specific. 

--
You received this message because you are subscribed to a topic in the Google Groups "Prometheus Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/prometheus-users/4pAxiUUSCqw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/b149ab13-d629-4889-9423-08b8a16fc39cn%40googlegroups.com.

[Marcelo Magallón]

Sounds useful to me.

You'll probably have to do some experimentation with the code, as from Go it's possible to determine the time it takes for the initial connection to be created. I'm not 100% sure what you get if you time the duration of the net.Conn Close() call.

You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CABTdvP%2BLsKmVszhyBqhfu2S_LqXB67cWHZ1gYP%2BtbvO4446QUA%40mail.gmail.com.

--

Marcelo Magallón

[Hugo Slabbert]

Great, thanks. I'll play around. In my somewhat naïve view I figured that DialTCP() should wrap the SYN, SYN+ACK, ACK, though I'd have to look a bit at possibly breaking out the TLS pieces. I figure the conn.Close() should wrap the closing FIN+ACK, FIN+ACK, ACK wrap-up, though I'd have to sort out how to time that given that it's deferred. 

[Marcelo Magallón]

Sounds good.

You could have something like

defer func() {

    start := time.Now()

    err := conn.Close()

    if err != nil {

        elapsed := time.Since(start)

        durationGaugeVec.WithLabelValues("close").Add(elapsed)

    } else {

        ...

    }

}()

--

Marcelo Magallón

[Hugo Slabbert]

Right! Yea, the thought came to wrap a function there. I wasn't sure if that would be copacetic, but I also hadn't thought to make it a literal. That looks clean and sensible there, thanks!