How does altermanager handle null value from prometheus

[anton...@gmail.com]

Hello everyone

     I have a alert rule as below:

     groups:

         - name: Circuits alerts

           rules:

  

          - alert: NET-Alerts

            expr: rttMonLatestRttOperSense{circuit_src="RCBC"} !=1

            for: 30s

            annotations:

                    Summary: '{{$labels.circuit_name}}' Circuit Is Down"

I have a metric  rttMonLatestRttOperSense{circuit_src="RCBC",target='1.1.1.1'} , and its value is  4.Then i receive an email shows that the circuit is down, until now all the things are normal.

But after a fews seconds i received a resolved email .When i check the prometheus , the value is null. and then the alertmanager keep firing and resolved , i receive lots of emails.Why when the value is null the alertmanager 

thought the alert had resolved ?

[Goutham Veeramachaneni]

Hi,

You also need to check for existence of the metric in the alert then. expr: rttMonLatestRttOperSense{circuit_src="RCBC"} !=1 or absent(rttMonLatestRttOperSense)

Basically, when there is no metric, metric != 1 would evaluate to nothing, which is the same as the expression result when the metric exists with value 1.

Thanks,

Goutham.

[anton...@gmail.com]

Hi Goutham

      Thanks for you reply, maybe i need to clarify the requirement.

       Actually the function asent() can return 1 when there is no data, but in my environment i need to set the null value as the value of last capture.So that i can  get the correct alert 

在 2018年8月22日星期三 UTC+8下午2:14:57，Goutham Veeramachaneni写道：

[Matthias Rampke]

I don't think we have a function to do that. The closest you can get is to use one of the `X_over_time` functions[0] with a suitable time window. Extending the last value forever does not fit well into the Prometheus metrics model (which expects targets to come and go) so you will always have to chose some time window.

/MR

[0] https://prometheus.io/docs/prometheus/latest/querying/functions/#%3Caggregation%3E_over_time()

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To post to this group, send email to promethe...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/87e0bb65-262e-4cb0-ae7f-64126fc888b2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[anton...@gmail.com]

ok,this is noted, thank you , maybe i will try  another way to avoid the null value 

在 2018年8月23日星期四 UTC+8下午3:41:18，Matthias Rampke写道：