I want use mtail to alerting end-users when a pattern match into the logs.
Mtail is used as exporter with prometheus/alertmanager. Currently, i use alertmanager receivers that are not very advanced (mattermost, mail).
When mtail matchs a pattern, a metric of type gauge is updated to 1. An alerting is send from Alertmanager. Then the metric is updated to 0 because no new recording. So alertmanager close the issue.
If several logs match during the following minutes, an alert will be generated almost every time and the end-user will be flooded of messages.
Do you know how I can deal with this situation? For example, is it possible with mtail to keep my metric with 1 as value during a defined period of time (1 hour for example) without take into account new record?
Otherwise, have you already used a specific configuration of Prometheus/Alertmanager to take into account this use case ?