alert manager and blackbox exporter

[francesco marchesini]

Hi,

I want confiured an alert only when 3 instance are down on process success. So I do this promql query

scalar(sum without(instance) (probe_success{instance=~"192.168.0.118|192.168.0.119|192.168.50.237"})) < bool 1

 the resul is 0 (false)

The resul is False (o) because process_seccuss 192.168.50.237

is 1

When i setup the alerm it fire.. I dont know why.

I try a test like that 

1 < bool 1

and it not fire

Why who same result have different behaviur?

thanks

[Brian Candler]

> When i setup the alerm it fire.. I dont know why.

An alert fires whenever an expression has *any* value.  It doesn't matter what the value is, or even if the value is zero.  If you don't want the alert to fire, then don't return any timeseries.

Note that an expression like

foo > 10

is a filter, not a boolean.  It filters all the 'foo' timeseries and returns only those ones with value over 10.  It returns an empty set of timeseries if there is no metric with value over 10; if there are any timeseries with foo > 10, then one or more timeseries will be returned (and it is the presence of these timeseries which triggers the alert, not the value of these timeseries).

In your case, I would simplify your expression to something like this:

count(probe_success{whatever} == 0) >= 3

This will return a single timeseries if the count is 3 or more, and no timeseries if the value is less than 3.