Can prometheus detect the periodicity of a metric?

[Jónás Jurásek]

I want to create alerts based on if a periodic signal(not perfectly periodic) has changed significantly from its previous behaviour. For example: A size of a table in a db is growing a certain amount every hour, but then every day it goes back to the size it was a day before. I want to alert if this table grew twice as much, or hasn't grown at all, or if if doesn't go back to it's original size. But rather than me giving the 1 day periodicity, I want prometheus to detect it, because different tables can have different periods. 

Is there any way to do that with prometheus? Or is the any other way to detect the change of the behaviour?

[Stuart Clark]

That's definitely something that can be done _using_ Prometheus, but not something done _within_ it.

You'd have an application which uses the query API (or is sent live data via remote write) to fetch metrics and then does whatever calculations are needed (for example using machine learning methods). You might then expose back new metrics to represent the outcome of those calculations (via scraping, remote read or remote write) which can then be used for visualisation and alerting.

Broadly what you are talking about is anomaly detection, which have a lot of ideas, PoCs, tools and blog articles created - you will find various ideas and ways to achieve such things from talks given at PromCon over the past years for example.

On 17 April 2023 07:51:21 BST, "Jónás Jurásek" <jurasek...@gmail.com> wrote:

I want to create alerts based on if a periodic signal(not perfectly periodic) has changed significantly from its previous behaviour. For example: A size of a table in a db is growing a certain amount every hour, but then every day it goes back to the size it was a day before. I want to alert if this table grew twice as much, or hasn't grown at all, or if if doesn't go back to it's original size. But rather than me giving the 1 day periodicity, I want prometheus to detect it, because different tables can have different periods. 

Is there any way to do that with prometheus? Or is the any other way to detect the change of the behaviour?

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[Jónás Jurásek]

Thank you for your help. 

I was almost certain that this would be the case, but I wanted to make sure. I will check out the ideas you mentioned.