Filtering data based on unique values

[Rohit Bharati]

Hi Everyone,

I am setting up monitoring for my bamboo Job(CI/CD tool). For which I am using a nodeJS exporter ( I am new to it) which returns the following feilds:

------------------------------------------------------------------------------------------------------------------------------------

# HELP build_number build number for plan.

# TYPE build_number gauge

build_number{planName="Some_name"} 2839

# HELP successful_test_count number of test cases passed for the build number.

# TYPE successful_test_count gauge

successful_test_count{planName=" Some_name  "} 3

# HELP failed_test_count number of test cases failed for the build number.

# TYPE failed_test_count gauge

failed_test_count{planName=" Some_name  "} 0

# HELP test_count_percentage Percentage = (success/(success + failure))*100 

# TYPE test_count_percentage gauge

test_count_percentage{planName=" Some_name  "} 100

----------------------------------------------------------------------------------------------------------------------------------------------

as this plan is set to be built every 5 mins therefore I have set my Prometheus to scrape this endpoint every 5 mins.  

but in real case some times the plan takes more than 5 mins to build , hence the data updated after 5 mins so my Prometheus shows the duplicate of the same metric in grafana

What i wished

I wanted to see results based on  build_number{planName=" Some_name  "} 2839   which will be always unique and keep on increasing.

How can I achieve this so that I can see only unique values for a metric?

or how to make a metric value unique? either in promethues.yml or exporter

Thanks

[Stuart Clark]

Prometheus should be set to scrape at least every 2 minutes, otherwise time series will be marked as stale and not be returned by queries.

It is perfectly normal for metrics not to have changed between scrapes for some applications, so nothing special needs to be done. Just set the scrape interval to 2 minutes and let it keep fetching the latest values. There are no worries around storage due to the way that the Prometheus TSDB works.

Ideally you would use counters rather than gauges which count things like successes in total (rather than for a particular run) to give greater meaning & flexibility to the queries you can do. One thing to remember is that Prometheus is a metrics system, not an events system, so is designed to show how metrics change over time. There is no concept of "a build" so it is not possible to find the number of successful tests for a particular build, but it is possible to graph how that number varies over time. If you do want to be able to lookup values based on a build you should use an events system (such as Elasticsearch) or a standard database (e.g. MySQL).

If you can describe what you are wanting to be able to display in Grafana we might be able to give some more advice.