Prometheus Query/alerting rules related to NFS Detach mount using node-exporter mountstats and nfs collector does not work
25 views
Skip to first unread message
Satyam Vishnoi
Jun 26, 2020, 6:50:21 AM6/26/20
to Prometheus Users
I should get alert when below given /mapr nfs mount-point get detached .
I am using following 2 metrics provided by node-exporter collector mountstats and nfs .
Query-1 absent(node_filesystem_size_bytes { job="exporter_node",fstype="nfs" ,mountpoint="/mapr",instance=~".*:9100" }) ==1
Query-2 rate(node_mountstats_nfs_age_seconds_total {export="localhost:/mapr"}[5m]) < 1
My first query is providing me desired output but i have to hard-code instance value on it. the regex used above does not work. My second query with rate doesn't give me a stable value to put on alert .
Am i doing something wrong here? Please provide any solution to resolve my problem .
Christian Hoffmann
Jun 30, 2020, 1:52:29 AM6/30/20
to Satyam Vishnoi, Prometheus Users
Hi,
On 6/26/20 12:50 PM, Satyam Vishnoi wrote:
> I should get alert when below given /mapr nfs mount-point get detached .
>
>
> I am using following 2 metrics provided by node-exporter collector
> mountstats and nfs .
>
>
> Query-1 absent(node_filesystem_size_bytes {
> job="exporter_node",fstype="nfs" ,mountpoint="/mapr",instance=~".*:9100"
> }) ==1
>
>
> Query-2 rate(node_mountstats_nfs_age_seconds_total
> {export="localhost:/mapr"}[5m]) < 1
>
>
> My first query is providing me desired output but i have to hard-code
> instance value on it. the regex used above does not work. My second
> query with rate doesn't give me a stable value to put on alert .
Right, absent will only work if you know what to look for. However,
maybe Prometheus already has the required information in the form of
other series?
If you want to alert if any of your instances lacks a specific
mountpoint, you can try this:
up == 1 unless on(instance)
node_filesystem_size_bytes{fstype="nfs",mountpoint="/mapr"}
This selects all online nodes [1] and discards all nodes which do have
the mountpoint. What's left is all online nodes without the mountpoint,
which should be exactly what you were after.
Kind regards,
Christian
[1] You can also skip the == 1 part to check all nodes. However, it
usually makes little sense to place such an alert for offline nodes as I
assume you will alert on up != 1 anyway.
On 6/26/20 12:50 PM, Satyam Vishnoi wrote:
> I should get alert when below given /mapr nfs mount-point get detached .
>
>
> I am using following 2 metrics provided by node-exporter collector
> mountstats and nfs .
>
>
> Query-1 absent(node_filesystem_size_bytes {
> job="exporter_node",fstype="nfs" ,mountpoint="/mapr",instance=~".*:9100"
> }) ==1
>
>
> Query-2 rate(node_mountstats_nfs_age_seconds_total
> {export="localhost:/mapr"}[5m]) < 1
>
>
> My first query is providing me desired output but i have to hard-code
> instance value on it. the regex used above does not work. My second
> query with rate doesn't give me a stable value to put on alert .
maybe Prometheus already has the required information in the form of
other series?
If you want to alert if any of your instances lacks a specific
mountpoint, you can try this:
up == 1 unless on(instance)
node_filesystem_size_bytes{fstype="nfs",mountpoint="/mapr"}
This selects all online nodes [1] and discards all nodes which do have
the mountpoint. What's left is all online nodes without the mountpoint,
which should be exactly what you were after.
Kind regards,
Christian
[1] You can also skip the == 1 part to check all nodes. However, it
usually makes little sense to place such an alert for offline nodes as I
assume you will alert on up != 1 anyway.
Satyam Vishnoi
Jun 30, 2020, 1:54:58 AM6/30/20
to Prometheus Users
Hi Team ,
Please provide any solution to my above problem .. I am stuck with this.
Reply all
Reply to author
Forward
0 new messages