promQL: getting data from multiple metrics in single query

[kiran]

Hello all

My use case is to present to the user data of each instance in a table format where each column indicates a metric, the first column being instance id. And I have to refresh the table data every minute. How do I handle getting data from multiple queries? 

Is not there an elegant way of getting information from multiple metrics in just one query which I believe is efficient rather than running one query per metric and putting the data together. Running multiple queries every minute(or even lesser interval) seems inefficient theoretically.

[Brian Candler]

You can do a PromQL query like {__name__="foo|bar"} but that's messy if you also want to filter on different labels for metrics foo and bar.

If you're only interested in the current values of each metric, then you can query the /federate endpoint where you can provide the match[] parameter multiple times with multiple queries.

https://prometheus.io/docs/prometheus/latest/federation/#configuring-federation

e.g.

curl -g 'prometheus:9090/federate?match[]=up&match[]={__name__=~"scrape_.*"}'

In any case, I wouldn't worry too much about inefficiency of separate queries. The main time taken is reading out the data and formatting it; whether that's done in a single query or spread over two queries isn't going to make much difference.  If you want to ensure that the table data lines up to the same instant in time, then pick an instant and provide the same time=xxx parameter to each separate query.

[kiran]

Thank you Brian. 

In my use case I have custom metrics that I will be sending to victoria metrics and will be using PormQL. 

The metrics constitute application level metadata and some high level metrics for each application. 

I may have 20-25 such data points for each application.

Ultimately I need to present all the data for each application in tables morethan graphs/charts. 

Since I have control over the data structure, I was thinking instead of sending vertical data(one time series per metric per function), I could send flat structure e.g metrics as labels. Is it recommended this way or this would increase cardinality? Or is there any better way I am not able to visualize. 

Typical way:

Metric1{appname=‘app1’} value timepoint

Metric2{appname=‘app1’} value timepoint

Metric3{appname=‘app1’} value timepoint

.

.

Metric25{appname=‘app1’} value timepoint

Metric1{appname=‘app2’} value timepoint

Metric2{appname=‘app2’} value timepoint

Metric3{appname=‘app2’} value timepoint

.

.

Metric25{appname=‘app2’} value timepoint

Alternative way I am thinking of(one time series per function):

Application metadata does not change that often, so I am thinking to send only once in 24 hrs and other metrics in 1 min interval. 

I am thinking I can have 2 metrics(not really metrics, but one for application metadata and one for actual application metrics).

app_meta{appname="app1", metric1="value", metric2="value", metric3="value"............metric25="value"} 1 timepoint            [Here 1 is just a dummy value]

app_meta{appname="app2", metric1="value", metric2="value", metric3="value"............metric25="value"} 1 timepoint

app_meta{appname="app3", metric1="value", metric2="value", metric3="value"............metric25="value"} 1 timepoint

app_metric{appname="app1", metric1="value", metric2="value", metric3="value"............metric25="value"} 1 timepoint            [Here 1 is just a dummy value]

app_metric{appname="app2", metric1="value", metric2="value", metric3="value"............metric25="value"} 1 timepoint

app_metric{appname="app3", metric1="value", metric2="value", metric3="value"............metric25="value"} 1 timepoint

I may be completely wrong with my approach. Please suggest.

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/bd76547c-e534-4133-a54f-28615df90ff2o%40googlegroups.com.

[Brian Candler]

On Saturday, 7 November 2020 14:12:53 UTC, kiran wrote:

In my use case I have custom metrics that I will be sending to victoria metrics and will be using PormQL. 

Using remote_write from prometheus, or your applicatino directly writing to VictoriaMetrics?  VM supports import in a bunch of different formats, such as influxdb line protocol and CSV, so you can choose whatever you find most convenient.

Since I have control over the data structure, I was thinking instead of sending vertical data(one time series per metric per function), I could send flat structure e.g metrics as labels. Is it recommended this way or this would increase cardinality?

Definitely not.  The metrics need to be the metrics, not the labels.  The entire storage mechanism depends on this: it's the bag of labels which defines "what is a timeseries", and all the metrics for that timeseries are stored adjacent to each other, for efficient compression and retrieval.  Plus: all the PromQL functions which operate on numbers, like sum() and rate() and so on, operate on the value and not the labels.