Aggregating histograms with different bucket boundaries

[Adam Sydnor]

Hey all,

I was wondering if it is a bad idea to aggregate histograms that have different bucket boundaries. If so, why?

Or can Prometheus handle this and produce reliable metrics like quartiles?

I want to dynamically resize buckets boundaries in a histogram based on observed data but still be able to aggregate it with other histograms that have done the same.

Thanks!

Adam

[Bjoern Rabenstein]

On 29.04.23 08:09, Adam Sydnor wrote:
>
> I was wondering if it is a bad idea to aggregate histograms that have
> different bucket boundaries. If so, why?
>
> Or can Prometheus handle this and produce reliable metrics like quartiles?

Generally, Prometheus cannot handle this.

You could play some tricks and only add or remove buckets while
keeping enough buckets the same, and then carefully craft your queries
so that only buckets that are shared by all involved histograms are
taken into account.

But that's a lot hassle and not really recommended.

The new Native Histograms (still an experimental feature, and the only
instrumentation client with full support is client_golang) have
everything you need. You might want to explore them. We still need to
write a good "getting started" guide, but there are a number of talks
available on Youtube. Maybe start here:

- https://promcon.io/2022-munich/talks/native-histograms-in-prometheus/
- https://promcon.io/2022-munich/talks/promql-for-native-histograms/

--
Björn Rabenstein
[PGP-ID] 0x851C3DA17D748D03
[email] bjo...@rabenste.in