How to find if a label is missing from a metric

60 views
Skip to first unread message

Arnav Bose

unread,
May 26, 2020, 3:46:08 PM5/26/20
to Prometheus Users

Hi,


I have a metric:


DS_Info{desc="GOT",index="312",name="EC80",exported_namespace="s905"}


Is there anyway I can alert if any one of these labels go missing. Not the metric, just one of the labels.


Thanks

Arnav Bose

unread,
May 26, 2020, 3:59:57 PM5/26/20
to Arnav Bose, Prometheus Users
No, not that way. What I mean is there is a possibility that the label itself will be missing. Say the label desc will be completely missing from the metric. Instead of 4 labels I will get only 3 labels.  I want to alert at that time. 



On Tue, May 26, 2020, 3:54 PM Julien Pivotto <roidel...@prometheus.io> wrote:
Hi,

Assuming you know the labels:

DS_Info{desc=""} or DS_Info{index=""} or DS_Info{name=""} or DS_Info{exported_namespace=""}

regards

>
>
> Thanks
>
> --
> You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/662678d9-b7d0-4d8a-a0d0-c6d0e37fda80%40googlegroups.com.


--
Julien Pivotto
@roidelapluie

Arnav Bose

unread,
May 26, 2020, 4:20:21 PM5/26/20
to Arnav Bose, Prometheus Users
I don't get it. Do you mean these are the same?

DS_Info{desc="GOT",index="312",name="EC80",exported_namespace="s905"}

DS_Info{index="312",name="EC80",exported_namespace="s905"}

The second metric is missing the label desc.

The label itself is not there. desc="" will not work in this case. 

On Tue, May 26, 2020, 4:15 PM Julien Pivotto <roidel...@prometheus.io> wrote:
On 26 May 15:59, Arnav Bose wrote:
> No, not that way. What I mean is there is a possibility that the label
> itself will be missing. Say the label desc will be completely missing from
> the metric. Instead of 4 labels I will get only 3 labels.  I want to alert
> at that time.

Empty label and missing label are the same in this Promql example.

Brian Candler

unread,
May 26, 2020, 5:22:14 PM5/26/20
to Prometheus Users
{label=""} means 'the label does not exist', and {label!=""} means 'the label exists with any value'.

Labels cannot exist with empty string value.

Try it in PromQL expression browser and you'll see.  For example:

up

up{flurblywombat=""}

- they will give the same result.

Note: it's the metric name plus bag of labels which is the unique identity of the timeseries.  When labels change or "go missing", you have created a completely different timeseries.

Julien Pivotto

unread,
May 26, 2020, 6:58:28 PM5/26/20
to Arnav Bose, Prometheus Users
On 26 May 15:59, Arnav Bose wrote:
> No, not that way. What I mean is there is a possibility that the label
> itself will be missing. Say the label desc will be completely missing from
> the metric. Instead of 4 labels I will get only 3 labels. I want to alert
> at that time.

Empty label and missing label are the same in this Promql example.

>
>
>
> On Tue, May 26, 2020, 3:54 PM Julien Pivotto <roidel...@prometheus.io>
> wrote:
>
> > On 26 May 12:46, Arnav Bose wrote:
> > >
> > >
> > > Hi,
> > >
> > >
> > > I have a metric:
> > >
> > >
> > > DS_Info{desc="GOT",index="312",name="EC80",exported_namespace="s905"}
> > >
> > >
> > > Is there anyway I can alert if any one of these labels go missing. Not
> > the
> > > metric, just one of the labels.
> >
> > Hi,
> >
> > Assuming you know the labels:
> >
> > DS_Info{desc=""} or DS_Info{index=""} or DS_Info{name=""} or
> > DS_Info{exported_namespace=""}
> >
> > regards
> >
> > >
> > >
> > > Thanks
> > >
> > > --
> > > You received this message because you are subscribed to the Google
> > Groups "Prometheus Users" group.
> > > To unsubscribe from this group and stop receiving emails from it, send
> > an email to prometheus-use...@googlegroups.com.
> > > To view this discussion on the web visit
> --
> You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.

Julien Pivotto

unread,
May 26, 2020, 6:58:28 PM5/26/20
to Arnav Bose, Prometheus Users
On 26 May 22:25, Julien Pivotto wrote:
> On 26 May 16:20, Arnav Bose wrote:
> > I don't get it. Do you mean these are the same?
> >
> > DS_Info{desc="GOT",index="312",name="EC80",exported_namespace="s905"}
> >
> > DS_Info{index="312",name="EC80",exported_namespace="s905"}
> >
> > The second metric is missing the label desc.
> >
> > The label itself is not there. desc="" will not work in this case.
>
> Yes desc="" will work.

From our documentation: https://prometheus.io/docs/concepts/data_model/

A label with an empty label value is considered equivalent to a label
that does not exist.
> > To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAJO1_QQMORmGj-VpcVANdp5u7rzCTu3A%3DV9-SqD%3DjdZu36gX%3DA%40mail.gmail.com.
>
> --
> Julien Pivotto
> @roidelapluie

--
Julien Pivotto
@roidelapluie

Julien Pivotto

unread,
May 26, 2020, 6:58:28 PM5/26/20
to Arnav Bose, Prometheus Users
On 26 May 16:20, Arnav Bose wrote:
> I don't get it. Do you mean these are the same?
>
> DS_Info{desc="GOT",index="312",name="EC80",exported_namespace="s905"}
>
> DS_Info{index="312",name="EC80",exported_namespace="s905"}
>
> The second metric is missing the label desc.
>
> The label itself is not there. desc="" will not work in this case.

Yes desc="" will work.

Julien Pivotto

unread,
May 26, 2020, 6:58:29 PM5/26/20
to Arnav Bose, Prometheus Users
On 26 May 12:46, Arnav Bose wrote:
>
>
Hi,

Assuming you know the labels:

DS_Info{desc=""} or DS_Info{index=""} or DS_Info{name=""} or DS_Info{exported_namespace=""}

regards

>
>
> Thanks
>
> --
> You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.

Arnav Bose

unread,
May 28, 2020, 12:53:28 PM5/28/20
to Prometheus Users
It worked. Thanks a lot.
Reply all
Reply to author
Forward
0 new messages