pd integartion

[deln...@gmail.com]

There's specific use case where we want to ensure that Pagerduty will re-alert if a user marks the PD alert resolved , but initial alert  is still active. There's a chance that user might be not aware that the issue is not resolved. According to resolve_timeout it might take few hours before the alert is triggered again. According to https://community.pagerduty.com/forum/t/re-trigger-incidents-after-theyve-been-resolved/1706/3 this is possible. Does anyone use this feature? Been trying to set this feature - but with no luck. Also what's preferable way to use for integration - Prometheus or Events API v2?

[Nemanja Delic]

I'll try to bring more details here:
Actions:
1. alertmanager triggers pagerduty incident,

2. acknowledging the incident in PD console, 
3. fixing the problem 
4. manually resolve the incident - again in PD console

5. the real problem is still there - an alert in alertmanager is still in active state ( have not fixed it actually)

6. timeout_interval in alertmanager.yaml is set to 1h

7. what happens is that in a random interval ( it's less than 1h after I've resolved the incident) - I do receive a totally new PD incident on the initial issue. 

According to this link this is desired behavior or no? If it's desired - how come the incident in not triggered right after resolving?
If it's not desired - why do I receive the alert at all?

I'm running the latest version of Alertmanager.

On Wed, Oct 28, 2020 at 1:39 PM deln...@gmail.com <deln...@gmail.com> wrote:

There's specific use case where we want to ensure that Pagerduty will re-alert if a user marks the PD alert resolved , but initial alert  is still active. There's a chance that user might be not aware that the issue is not resolved. According to resolve_timeout it might take few hours before the alert is triggered again. According to https://community.pagerduty.com/forum/t/re-trigger-incidents-after-theyve-been-resolved/1706/3 this is possible. Does anyone use this feature? Been trying to set this feature - but with no luck. Also what's preferable way to use for integration - Prometheus or Events API v2?

--
You received this message because you are subscribed to a topic in the Google Groups "Prometheus Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/prometheus-users/MxEHLyg-w7M/unsubscribe.
To unsubscribe from this group and all its topics, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/585fa5f8-95db-48c1-bbf7-f4f0cf4bc8bbn%40googlegroups.com.