How to find newly added indicators

[li yun]

Hello everyone, I have encountered some problems in the process of using prometheus. There are many monitoring indicators. I used the service discovery method. I need to know which indicators are newly added by service discovery in a certain period of time. May I ask this prometheus Can it be done?

[li yun]

For example, the following situation

isphone{name="user",exname~"13"}

This exname program will continuously collect a lot of monitoring indicators, but I want to know which indicators have been added in a certain period of time

[Christian Hoffmann]

Hi,

On 10/13/20 12:22 PM, li yun wrote:
> For example, the following situation

> *isphone{name="user",exname~"13"}*

> This exname program will continuously collect a lot of monitoring
> indicators, but I want to know which indicators have been added in a
> certain period of time
> 在2020年10月13日星期二 UTC+8 下午6:19:15<li yun> 写道：
>
> Hello everyone, I have encountered some problems in the process of
> using prometheus. There are many monitoring indicators. I used the
> service discovery method. I need to know which indicators are newly
> added by service discovery in a certain period of time. May I ask
> this prometheus Can it be done?

You can ask Prometheus to return those metrics which do exist now and
haven't existed some time ago. Example for 1 hour:


isphone{name="user",exname~"13"} unless
isphone{name="user",exname~"13"} offset 1h

Note: It sounds like you might be putting rather dynamical data
(exname?) into labels. This may lead to cardinality issues. Be sure to
implement some safeguards and/or beware of the potential resource
requirements.

Kind regards,
Christian

[li yun]

I tried it and it didn’t work. For example, the indicator I’m monitoring now includes isphone{name="user",exname="13"},
  isphone{name="user",exname="x14"}
  isphone{name="user",exname="a15"}

At this time, the program returns a new indicator
  isphone{name="user",exname="d2"}, I hope to find this newly added indicator quickly in the chart, because I have a lot of indicators.

Christian Hoffmann <ma...@hoffmann-christian.info> 于2020年10月14日周三 下午10:50写道：

[li yun]

I have a lot of indicators, I want to know which indicator is newly added suddenly, and I can find it quickly

li yun <linux...@gmail.com> 于2020年10月16日周五 上午10:30写道：

[Brian Candler]

What happens if you try simply:

        isphone unless isphone offset 1h

?

[li yun]

Because my data is collected every 5 minutes, I need to use sum_over_time(isphone{name="user",exname~".*"}[5m] unleass sum_over_time(isphone{name="user",exname~".*" }[5m], but there will be grammatical errors, is there any better way

Brian Candler <b.ca...@pobox.com> 于2020年10月16日周五 下午3:30写道：

What happens if you try simply:

        isphone unless isphone offset 1h

?

--
You received this message because you are subscribed to a topic in the Google Groups "Prometheus Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/prometheus-users/XpzVmYZ49gc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/5d531065-05a7-445d-b653-5af736bc5c2do%40googlegroups.com.

[Christian Hoffmann]

On 10/17/20 2:28 AM, li yun wrote:
> Because my data is collected every 5 minutes

If possible, try fixing that. The maximum sane scrape interval is 2m. If
nothing else helps, you may consider hiding the problem by using
recording rules (with sum_over_time) or changing --query.lookback-delta.