Getting the Age of alerts.

42 views
Skip to first unread message

Yagyansh S. Kumar

unread,
Mar 17, 2020, 5:33:47 AM3/17/20
to Prometheus Users
Hi. I want to extract the age of the alerts(i.e from when the alert is Critical or Warning or even Resolved). Is this possible?

Christian Hoffmann

unread,
Mar 17, 2020, 5:34:05 PM3/17/20
to Yagyansh S. Kumar, Prometheus Users
Hi,

On 3/17/20 10:33 AM, Yagyansh S. Kumar wrote:
> Hi. I want to extract the age of the alerts(i.e from when the alert is
> Critical or Warning or even Resolved). Is this possible?

Where are you looking for that?

Within Prometheus? You could try deriving that from the ALERTS meta
metric which gets generated automatically.

From the Alertmanager API? You should be able to get a startsAt value there.

Kind regards,
Christian

Julien Pivotto

unread,
Mar 17, 2020, 5:36:54 PM3/17/20
to Christian Hoffmann, Yagyansh S. Kumar, Prometheus Users
On 17 Mar 22:33, Christian Hoffmann wrote:
> Hi,
>
> On 3/17/20 10:33 AM, Yagyansh S. Kumar wrote:
> > Hi. I want to extract the age of the alerts(i.e from when the alert is
> > Critical or Warning or even Resolved). Is this possible?
>
> Where are you looking for that?
>
> Within Prometheus? You could try deriving that from the ALERTS meta
> metric which gets generated automatically.

Hi,

Within Prometheus you have the timestamp in ALERTS_FOR_STATE. However
that is more an "internal" metric that is there for keeping the state
of alerts across restarts, so use with care.

regards,

>
> From the Alertmanager API? You should be able to get a startsAt value there.
>
> Kind regards,
> Christian
>
> --
> You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/9d8806d2-9ee8-5c65-670b-d92811869810%40hoffmann-christian.info.

--
(o- Julien Pivotto
//\ Open-Source Consultant
V_/_ Inuits - https://www.inuits.eu
signature.asc

Yagyansh S. Kumar

unread,
Mar 20, 2020, 6:22:47 AM3/20/20
to Prometheus Users
Thanks for the help. But what is the format of this value that we are getting from ALERTS_FOR_STATE

ALERTS_FOR_STATE{alertname="OutOfMemoryWarn",cluster="cluster",instance="instance",job="job"severity="WARNING",source="prometheus"} 1584698633

What does this value represent.

On Wednesday, March 18, 2020 at 3:06:54 AM UTC+5:30, Julien Pivotto wrote:
On 17 Mar 22:33, Christian Hoffmann wrote:
> Hi,
>
> On 3/17/20 10:33 AM, Yagyansh S. Kumar wrote:
> > Hi. I want to extract the age of the alerts(i.e from when the alert is
> > Critical or Warning or even Resolved). Is this possible?
>
> Where are you looking for that?
>
> Within Prometheus? You could try deriving that from the ALERTS meta
> metric which gets generated automatically.

Hi,

Within Prometheus you have the timestamp in ALERTS_FOR_STATE. However
that is more an "internal" metric that is there for keeping the state
of alerts across restarts, so use with care.

regards,

>
> From the Alertmanager API? You should be able to get a startsAt value there.
>
> Kind regards,
> Christian
>
> --
> You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to promethe...@googlegroups.com.

Julien Pivotto

unread,
Mar 20, 2020, 6:28:59 AM3/20/20
to Yagyansh S. Kumar, Prometheus Users
On 20 Mar 03:22, Yagyansh S. Kumar wrote:
> Thanks for the help. But what is the format of this value that we are
> getting from ALERTS_FOR_STATE
>
> ALERTS_FOR_STATE{alertname="OutOfMemoryWarn",cluster="cluster",instance="instance",job="job"severity="WARNING",source="prometheus"}
> *1584698633*


It is a unix timestamp

https://en.wikipedia.org/wiki/Unix_time

>
> What does this value represent.
>
> On Wednesday, March 18, 2020 at 3:06:54 AM UTC+5:30, Julien Pivotto wrote:
> >
> > On 17 Mar 22:33, Christian Hoffmann wrote:
> > > Hi,
> > >
> > > On 3/17/20 10:33 AM, Yagyansh S. Kumar wrote:
> > > > Hi. I want to extract the age of the alerts(i.e from when the alert is
> > > > Critical or Warning or even Resolved). Is this possible?
> > >
> > > Where are you looking for that?
> > >
> > > Within Prometheus? You could try deriving that from the ALERTS meta
> > > metric which gets generated automatically.
> >
> > Hi,
> >
> > Within Prometheus you have the timestamp in ALERTS_FOR_STATE. However
> > that is more an "internal" metric that is there for keeping the state
> > of alerts across restarts, so use with care.
> >
> > regards,
> >
> > >
> > > From the Alertmanager API? You should be able to get a startsAt value
> > there.
> > >
> > > Kind regards,
> > > Christian
> > >
> > > --
> > > You received this message because you are subscribed to the Google
> > Groups "Prometheus Users" group.
> > > To unsubscribe from this group and stop receiving emails from it, send
> > an email to promethe...@googlegroups.com <javascript:>.
> > > To view this discussion on the web visit
> > https://groups.google.com/d/msgid/prometheus-users/9d8806d2-9ee8-5c65-670b-d92811869810%40hoffmann-christian.info.
> >
> >
> > --
> > (o- Julien Pivotto
> > //\ Open-Source Consultant
> > V_/_ Inuits - https://www.inuits.eu
> >
>
> --
> You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/dfe78f8e-b039-44a0-a6b0-20bd477c8757%40googlegroups.com.
signature.asc
Reply all
Reply to author
Forward
0 new messages