How can I actively push prometheus metrics to cloudwatch?

2,576 views
Skip to first unread message

lea...@gmail.com

unread,
Oct 4, 2017, 3:49:39 PM10/4/17
to Prometheus Developers
There is an exporter for cloudwatch which exports the cloudwatch data to prometheus, but is there an opposite solution? While I collect data from other source to prometheus and publish it to cloudwatch?

Brian Brazil

unread,
Oct 4, 2017, 5:02:17 PM10/4/17
to lea...@gmail.com, Prometheus Developers
On 4 October 2017 at 20:49, <lea...@gmail.com> wrote:
There is an exporter for cloudwatch which exports the cloudwatch data to prometheus, but is there an opposite solution? While I collect data from other source to prometheus and publish it to cloudwatch?

I'm not aware of anyone creating one of those yet, but I can't imagine it to be too hard to use with remote write. 

--

lea...@gmail.com

unread,
Oct 4, 2017, 5:12:13 PM10/4/17
to Prometheus Developers
Thanks for replying, is there any document for "remote write"?

Brian Brazil

unread,
Oct 4, 2017, 5:15:29 PM10/4/17
to lea...@gmail.com, Prometheus Developers
 

--
You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-developers+unsub...@googlegroups.com.
To post to this group, send email to prometheus-developers@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/82424b03-ed33-4de6-a8c2-15b55b4b58fb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--

lea...@gmail.com

unread,
Oct 5, 2017, 1:55:31 PM10/5/17
to Prometheus Developers
On Wednesday, October 4, 2017 at 2:15:29 PM UTC-7, Brian Brazil wrote:
> On 4 October 2017 at 22:12, <lea...@gmail.com> wrote:
>
>
> On Wednesday, October 4, 2017 at 2:02:17 PM UTC-7, Brian Brazil wrote:
>
> > On 4 October 2017 at 20:49,  <lea...@gmail.com> wrote:
>
> > There is an exporter for cloudwatch which exports the cloudwatch data to prometheus, but is there an opposite solution? While I collect data from other source to prometheus and publish it to cloudwatch?
>
> >
>
> >
>
> >
>
> > I'm not aware of anyone creating one of those yet, but I can't imagine it to be too hard to use with remote write. 
>
> >
>
> > --
>
> >
>
> >
>
> > Brian Brazil
>
> > www.robustperception.io
>
>
>
> Thanks for replying, is there any document for "remote write"?
>
>
>
> https://www.robustperception.io/using-the-remote-write-path/
>
>  
>
>
> --
>
> You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
>
> To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-devel...@googlegroups.com.
>
> To post to this group, send email to prometheus...@googlegroups.com.
>
> To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/82424b03-ed33-4de6-a8c2-15b55b4b58fb%40googlegroups.com.
>
>
>
> For more options, visit https://groups.google.com/d/optout.
>
>
>
>
>
> --
>
>
> Brian Brazil
> www.robustperception.io

Thanks very much.

Zintori

unread,
Nov 29, 2017, 7:39:28 PM11/29/17
to Prometheus Developers
Hello,

I came across this blog post and I am experimenting posting Prometheus metrics to AWS CloudWatch with remote write: https://www.robustperception.io/using-the-remote-write-path/

Theoretically speaking, I should be able to configure the CloudWatch endpoint in the Prometheus configuration file: https://forums.aws.amazon.com/message.jspa?messageID=266173

While it sounds good in theory, it doesn't 'just work' because of these two main issues:
- The API calls to AWS CloudWatch needs to be signed with the secret access key and this adds another layer of complexity to manage these keys safely.
- The format of Prometheus output is not the same as the expected parameters of these CloudWatch API calls.

It looks like there need to be an intermediate layer between Prometheus and CloudWatch for this to work and it will leverage the AWS SDK to manage the API calls and signing of API requests. Essentially, I will end up building something similar to this library

It is probably not terribly difficult, but not exactly 'just works'. Am I missing something here?

On Wednesday, October 4, 2017 at 2:15:29 PM UTC-7, Brian Brazil wrote:
On 4 October 2017 at 22:12, <lea...@gmail.com> wrote:

On Wednesday, October 4, 2017 at 2:02:17 PM UTC-7, Brian Brazil wrote:
> On 4 October 2017 at 20:49,  <lea...@gmail.com> wrote:
> There is an exporter for cloudwatch which exports the cloudwatch data to prometheus, but is there an opposite solution? While I collect data from other source to prometheus and publish it to cloudwatch?
>
>
>
> I'm not aware of anyone creating one of those yet, but I can't imagine it to be too hard to use with remote write. 
>
> --
>
>
> Brian Brazil
> www.robustperception.io

Thanks for replying, is there any document for "remote write"?

 

--
You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-developers+unsub...@googlegroups.com.
To post to this group, send email to prometheus...@googlegroups.com.



--

Brian Brazil

unread,
Nov 30, 2017, 3:20:51 AM11/30/17
to Zintori, Prometheus Developers
On 30 November 2017 at 00:39, Zintori <irene...@gmail.com> wrote:
Hello,

I came across this blog post and I am experimenting posting Prometheus metrics to AWS CloudWatch with remote write: https://www.robustperception.io/using-the-remote-write-path/

Theoretically speaking, I should be able to configure the CloudWatch endpoint in the Prometheus configuration file: https://forums.aws.amazon.com/message.jspa?messageID=266173

While it sounds good in theory, it doesn't 'just work' because of these two main issues:
- The API calls to AWS CloudWatch needs to be signed with the secret access key and this adds another layer of complexity to manage these keys safely.
- The format of Prometheus output is not the same as the expected parameters of these CloudWatch API calls.

It looks like there need to be an intermediate layer between Prometheus and CloudWatch for this to work and it will leverage the AWS SDK to manage the API calls and signing of API requests. Essentially, I will end up building something similar to this library

It is probably not terribly difficult, but not exactly 'just works'. Am I missing something here?

Yes, you'd need to create an adapter.

Brian
 

On Wednesday, October 4, 2017 at 2:15:29 PM UTC-7, Brian Brazil wrote:
On 4 October 2017 at 22:12, <lea...@gmail.com> wrote:

On Wednesday, October 4, 2017 at 2:02:17 PM UTC-7, Brian Brazil wrote:
> On 4 October 2017 at 20:49,  <lea...@gmail.com> wrote:
> There is an exporter for cloudwatch which exports the cloudwatch data to prometheus, but is there an opposite solution? While I collect data from other source to prometheus and publish it to cloudwatch?
>
>
>
> I'm not aware of anyone creating one of those yet, but I can't imagine it to be too hard to use with remote write. 
>
> --
>
>
> Brian Brazil
> www.robustperception.io

Thanks for replying, is there any document for "remote write"?

 

--
You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-developers+unsubscri...@googlegroups.com.
To post to this group, send email to prometheus...@googlegroups.com.



--

--
You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-developers+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--

berni...@gmail.com

unread,
Apr 30, 2018, 4:45:34 PM4/30/18
to Prometheus Developers
On Wednesday, 4 October 2017 12:49:39 UTC-7, 于海潮 wrote:
> There is an exporter for cloudwatch which exports the cloudwatch data to prometheus, but is there an opposite solution? While I collect data from other source to prometheus and publish it to cloudwatch?

Hello,

I just found this new tool that does what you asked for: https://github.com/cloudposse/prometheus-to-cloudwatch
Reply all
Reply to author
Forward
0 new messages