Prometheus alert trigger AWS Lambda for disk-resize

[David Friedman]

Hi all

I have Prometheus set up with AlertManager and I want to set up that when I have a disk that gets to 90 % it will trigger an AWS Lambda that will enlarge my ec2 disk 

I see that I can set up a AWS SNS with AlertManager and that can trigger the Lambda 

What will be the best way to do this ?

Thanks 

[sayf eddine Hammemi]

Hello,

For my knowledge, the best way is to set up an ApiGateway in front of the lambda and use alertmanager webhooks to trigger it.

Btw I don't feel comfortable acting automatically on an alertmanager payload. I ld add more logic in the lambda function to recheck the disk (size and content (in case a faulty process is filling the disk with useless data)) before sending the resize command, to prevent any faulty data or flapping alert, also to not keep resize to the infinity

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/4fdfee6d-8714-4b81-85e6-765425b38444n%40googlegroups.com.