Simple "Moving Average"?

4,842 views
Skip to first unread message

Dave Cadwallader

unread,
Jul 30, 2018, 3:51:42 PM7/30/18
to Prometheus Users
Looking for some advice on smoothing out a spikey graph based on gauges.

Given these two metrics:
  • instances_total{state="busy"}
  • instances_total
We want to plot the ratio of "busy" instances over time.

This query produces the expected result, broken down by AWS region:

sum(instances_total{state="busy"}) by (region) / sum(instances_total) by (region)

But the graph is pretty spikey.  What would you recommend to apply a smoothing factor to make it more readable over a longer interval?  We looked at the holt_winters function, but it seemed to require a range vector and our query produces an instant vector.  Ideally we're looking for something simple like a "5 min moving average".

Thanks!
Dave

Ben Kochie

unread,
Jul 30, 2018, 4:54:08 PM7/30/18
to Dave Cadwallader, Prometheus Users
If you're using Grafana, there's a handy $__interval variable

It seems like your instances_total is a gauge. We recommend _total be reserved for counters to make them easy to identify.

If it is a gauge, you can do something like this:

sum(avg_over_time(instances_total{state="busy"}[$__interval])) by (region)

That should give you the rolling average of the gage, matched to the step in Grafana.

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To post to this group, send email to promethe...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/f9d706fc-dbf3-4768-9e75-f1f2e72099d5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Dave Cadwallader

unread,
Aug 1, 2018, 12:47:29 AM8/1/18
to Prometheus Users
I've been using Grafana for 4 years, and I never knew about $__interval!  This was a mind-blowing answer.  Your solution worked perfectly.  Thanks for being awesome, Ben!

Tristan Colgate

unread,
Aug 1, 2018, 4:11:21 AM8/1/18
to Dave Cadwallader, Prometheus Users
It's also helpful to make sure you set the min step when using $__interval. This usually results in less things breaking when you zoom in too close.

Reply all
Reply to author
Forward
0 new messages