Set variable for multi datasource Prometheus in Grafana

[Miền Nguyễn]

Hi everyone,

I want to draw metrics of many Prometheus clusters in a Grafana dashboard. But I can't set variable for all datasource Prometheus to filter on the top dashboard.

I'm using Grafana version 7.0. How way merge multi datasource Prometheus to one datasource and query metrics all datasource in a Metrics line of Grafana?

Can you help me, please?

Thanks and Best regards,

Mien Nguyen

[Brian Candler]

Works for me in Grafana 6.6.2.  Dashboard Settings > Variables:

Panel Query:

The "source" dropdown appears in the top row of the dashboard.

I suggest you ask on the Grafana mailing list if you are having problems with Grafana.

[Miền Nguyễn]

Hi Brian Candler,

Thank you. Before, I did as you but I only query metric when I choose "Query" cell is "$source" match with source variable on the top dashboard. I want to merge multi datasource Prometheus together on Grafana. Can this be done?

Example:

When I choose "Query" cell is "$source", I can query any metrics in any datasource Prometheus.

Vào Th 5, 21 thg 5, 2020 vào lúc 17:36 Brian Candler <b.ca...@pobox.com> đã viết:

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/f995994b-56c7-4616-8ac9-f8cb2441a298%40googlegroups.com.

--

Best Regards,

Miền Nguyễn

[Brian Candler]

This is a really a question about grafana rather than prometheus - as far as I know, grafana can't do that.

I believe you can install promxy, and point grafana at promxy, and promxy will combine results from multiple prometheus backends.  I haven't deployed it myself.

The approach I use is to have multiple prometheus servers perform remote_write into a single VictoriaMetrics instance, and point Grafana at that.  That means that "global" queries don't need to touch multiple backends.

[Julien Pivotto]

On 21 May 18:18, Miền Nguyễn wrote:
> Hi Brian Candler,
>
> Thank you. Before, I did as you but I only query metric when I choose
> "Query" cell is "$source" match with source variable on the top dashboard.
> I want to merge multi datasource Prometheus together on Grafana. Can this
> be done?
>
> Example:
> When I choose "Query" cell is "$source", I can query any metrics in any
> datasource Prometheus.

You can use: "Mixed datasource" in grafana.

>
>
> Vào Th 5, 21 thg 5, 2020 vào lúc 17:36 Brian Candler <b.ca...@pobox.com>
> đã viết:
>
> > Works for me in Grafana 6.6.2. Dashboard Settings > Variables:
> >
> > [image: img1.png]
> >
> >
> > Panel Query:
> >
> > [image: img2.png]
> >
> > The "source" dropdown appears in the top row of the dashboard.
> >
> > I suggest you ask on the Grafana mailing list if you are having problems
> > with Grafana.
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Prometheus Users" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to prometheus-use...@googlegroups.com.
> > To view this discussion on the web visit
> > https://groups.google.com/d/msgid/prometheus-users/f995994b-56c7-4616-8ac9-f8cb2441a298%40googlegroups.com

> > <https://groups.google.com/d/msgid/prometheus-users/f995994b-56c7-4616-8ac9-f8cb2441a298%40googlegroups.com?utm_medium=email&utm_source=footer>
> > .

> >
>
>
> --
> Best Regards,
> Miền Nguyễn
>

> --
> You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CANYhk1_wLNY2ksgGa3Bsio3xQz9x5a%3DCbr8MwanB06Cd3naWTw%40mail.gmail.com.




--
Julien Pivotto
@roidelapluie

[Kevin Cameron]

I have done the same thing as Brian, I have several Prometheus instances remote_writing to a single VictoriaMetrics server per data center.  I have a longer retention policy on the VictoriaMetrics server and it provides isolation between any queries from the dashboarding and the Prometheus servers that drive our alerting and data collection.

Kevin