NTP monitoring with ntp collector

[Hugo Evelyn]

Hi Guys,

With reading the info from official website https://github.com/prometheus/node_exporter/blob/master/docs/TIME.md, here comes 2 questions about the metrics.

1. I want to check if my servers are synchronized with NTP server or not. 

From the source code https://github.com/beevik/ntp/blob/master/ntp.go#L504, for my understanding, node_ntp_offset compares local system clock with NTP server. But from the following definition "ntp.org always sets NTPD time to local clock instead of relaying remote NTP time, so this offset is irrelevant for this NTPD.", I'm a little confuse whether this metrics can be used or not.

node_ntp_offset

Clock offset between local time and NTPD time.

ntp.org always sets NTPD time to local clock instead of relaying remote NTP time, so this offset is irrelevant for this NTPD.

This value is used in sanity check as part of causality violation estimate.

2. What the usage of node_ntp_sanity?

From the definition

node_ntp_sanity

Aggregate NTPD health including stratum, leap flag, sane freshness, root distance being less than collector.ntp.max-distance and causality violation being less than collector.ntp.local-offset-tolerance.

Causality violation is lower bound estimate of clock error done using SNTP, it's calculated as positive portion of abs(node_ntp_offset) - node_ntp_rtt / 2.

From the source code I found this comment in front of the source code of sanity check. So this metrics is to measure which situation? 

"// Here is SNTP packet sanity check that is exposed to move burden of

// configuration from node_exporter user to the developer."

From the lines of code https://github.com/prometheus/node_exporter/blob/master/collector/ntp.go#L169, I think node_ntp_sanity is checking resp.Validate(), resp.RootDistance <= *ntpMaxDistance and resp.MinError <= maxerr, if all the 3 conditions are met, then sanity check reponse 1, which means, the current situation is ok. 

[Hugo Evelyn]

Hi,

I want to check if my local servers are sychronized with NTP servers. Can I use ntp collector? And can I use  node_ntp_offset to know the offset between local servers' system clock and NTP servers

From https://github.com/prometheus/node_exporter/blob/master/docs/TIME.md

node_ntp_offset

Clock offset between local time and NTPD time.

ntp.org always sets NTPD time to local clock instead of relaying remote NTP time, so this offset is irrelevant for this NTPD.

This value is used in sanity check as part of causality violation estimate.

From source code https://github.com/beevik/ntp/blob/master/ntp.go#L504

func offset(org, rec, xmt, dst ntpTime) time.Duration {

   // local clock offset

   // offset = ((rec-org) + (xmt-dst)) / 2

   a := rec.Time().Sub(org.Time())

   b := xmt.Time().Sub(dst.Time())

   return (a + b) / time.Duration(2)

}

[Ben Kochie]

The node_exporter NTP collector is not really useful for what you're trying to do. It uses the NTP protocol, which isn't really how you monitor the tracking state of an NTP client. I wish I could just remove that collector as it's pretty useless and doesn't really do what people think it does.

The timex collector is a bit more useful, there is node_timex_sync_status. But not all NTP clients update adjtimex the same way, so you have to validate it for your environment.

Better, is to actually monitor your NTP client directly. For example, you can use the chrony_exporter[0] to monitor chrony NTP tracking status. This will give you direct status of how the NTP client is working.

[0]: https://github.com/SuperQ/chrony_exporter

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/48c393ba-21b7-4378-b1d0-47b7b22caa57n%40googlegroups.com.

[Hugo Evelyn]

Thanks for your reply. According to my researching result, timex collector is used to sync system clock from RTC(Real time clock) clock. Can it monitor the sync status between local servers and upstream NTP servers?

[Hugo Evelyn]

Sorry for my above words, correct like this  timex collector is used to monitor sync status between sync system clock from RTC(Real time clock) clock. Can it monitor the sync status between local servers and upstream NTP servers?