promehteus alertmanger getting duplicate alert in conlsole .

[Prashant Singh]

Dear All,

I am getting duplicate alert in prometheus alertmanager url : http://:9093 when setting continue : true. how can avoid duplicate alert .  `I am using webhook api and smtp based alert. 

alertname="PGDB_Query_alert" 1 alert
alertname="PGDB_Query_alert"  1 alert

alertmanager.yml

  - match:
     job: CES_PGDB_Query
     #severity: warning
    receiver: CES_PGDB_Query_test
    continue: true

thanks,

Prashant Singh

[Brian Candler]

That's what "continue: true" means.  It means you should deliver the alert to the receiver at this level (in this case "CES_PGDB_Query_test"), *and then continue* to process the remainder of the routes at this level in the tree.  If any of those also match the alert, then the alert will get delivered again (usually to different recipients, but that depends on what you've configured)

You'll need to check your whole routing tree, and check what labels are on each of your alerts, and then configure it to do what you want.

There's an interactive tool for testing your routing tree here:

https://prometheus.io/webtools/alerting/routing-tree-editor/

Routes can be nested, i.e. they form a tree. As well as a receiver, you can give a child set of 'routes'. At each level in the tree, the 'receiver' is only used if none of the child 'routes' have matched - regardless of whether continue: true was set or not.

So for example, if you want a single rule which sends to three receivers, you can write it like this:

    - matchers:
        - 'job = CES_PGDB_Query'
      routes: [ {receiver: FOO, continue: true}, {receiver: BAR, continue: true}, {receiver: BAZ} ]

You might find it easier to understand if you write it out in long form:

    - matchers:
        - 'job = CES_PGDB_Query'
      routes:

        - receiver: FOO    # no match conditions = match everything

          continue: true

        - receiver: BAR

          continue: true

        - receiver: BAZ