Cloudwatch exporter scraping from multiple aws accounts

[rs vas]

Hi, would like to know if cloudwatch exporter installed in one AWS account can scrape cloudwatch metrics from multiple AWS accounts using multiple roleARNs in the configuration? Anything specific needs to be configured in-order to support multiple accounts at the same time?

I could not find any way to do this as of now? Anyone has any luck doing this? Appreciate your help!

Thanks!

vas

[Brian Brazil]

On Sat, 20 Jun 2020 at 09:11, rs vas <rsvas...@gmail.com> wrote:

Hi, would like to know if cloudwatch exporter installed in one AWS account can scrape cloudwatch metrics from multiple AWS accounts using multiple roleARNs in the configuration? Anything specific needs to be configured in-order to support multiple accounts at the same time?

You need one exporter per account and region.

Brian

 

I could not find any way to do this as of now? Anyone has any luck doing this? Appreciate your help!

Thanks!

vas

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAPs_Afhj04AYaayHpqVnHu0wUnfddGq2vJ0NaSoXstnacz-p%3Dw%40mail.gmail.com.

--

Brian Brazil

www.robustperception.io

[rs vas]

thanks Brian for the quick response. Is there any possibility we can enhance to support multi account and region scraping? 

Or any other options/libraries provide that functionality. Appreciate any pointers or suggestions on the same!

[rs vas]

Just checking on this again! Is it something by design that is not possible?

Or is it just a question of priority and someone doing the work to support multiple accounts and regions by cloudwatch exporter?

https://github.com/prometheus/cloudwatch_exporter/issues/123 If it is later, Me/someone can definitely spend some time enhancing it. please let me know!

[Stuart Clark]

The general design of most exporters is that they deal with a single instance of the thing being monitored (single server, database, region, account), with multiple instances being handled by running multiple copies of the exporter.

One reason this often works well is because you generally need different credentials and find that different underlying instances could fail independently - which makes failures easier to handle.

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[rs vas]

Thanks Suart for your response.I agree with you most of the cases - for example: nodeExporter (instance specific metrics), JMXExpoter(JVM specific metrics).

But I believe Cloudwatch Exporter is different - It is currently responsible/limited to scrape region specific metrics. Is there any design limitation expanding this and having flexibility to scrape multiple regions and accounts? That way each metric should have two more additional tags probably account and region to identify the unique metric. Please share your thoughts!

I found another workaround: YACE - yet another cloudwatch exporter (which allow scraping from multiple regions and accounts) - Any ideas on this how widely YACE is used?

• https://github.com/ivx/yet-another-cloudwatch-exporter
• https://aws.amazon.com/blogs/opensource/monitor-aws-services-used-by-kubernetes-with-prometheus-and-promcat

Appreciate your help!

[Brian Candler]

> That way each metric should have two more additional tags probably account and region to identify the unique metric.

You simply have multiple scrape targets, one for each account/region, and add labels specific to each target. Even file_sd_config or static_sd_config can do that.