Is the +Inf bucket optional?

[Fabian Stäber]

Hi,

I'm having trouble figuring out if the +Inf bucket is optional.

The client_model protobuf says it's optional:

https://github.com/prometheus/client_model/blob/63fb9822ca3ba7a4ba5184071fb8f2ea000a99ef/io/prometheus/client/metrics.proto#LL71C89-L71C114

The OpenMetrics protobuf says buckets are optional

https://github.com/OpenObservability/OpenMetrics/blob/1386544931307dff279688f332890c31b6c5de36/proto/openmetrics_data_model.proto#L140

The OpenMetrics specification says "Histogram MetricPoints MUST have one bucket with an +Inf threshold.".

https://github.com/OpenObservability/OpenMetrics/blob/1386544931307dff279688f332890c31b6c5de36/specification/OpenMetrics.md

So, does it depend on the format (required in text format, optional in protobuf)?

Bonus question: What about buckets for native histograms, are they optional?

Fabian

[Bjoern Rabenstein]

On 17.01.23 10:01, 'Fabian Stäber' via Prometheus Developers wrote:
>
> I'm having trouble figuring out if the +Inf bucket is optional.
>
> The client_model protobuf says it's optional:
>

> <goog_966920514>
> https://github.com/prometheus/client_model/blob/63fb9822ca3ba7a4ba5184071fb8f2ea000a99ef/io/prometheus/client/metrics.proto#LL71C89-L71C114

Yes, it's optional in the protobuf format.

> The OpenMetrics protobuf says buckets are optional
>
> https://github.com/OpenObservability/OpenMetrics/blob/1386544931307dff279688f332890c31b6c5de36/proto/openmetrics_data_model.proto#L140

I guess this part of the OM protobuf spec hasn't been checked with the
same scrutiny as most of the OM text format.

> The OpenMetrics specification says "Histogram MetricPoints MUST have one
> bucket with an +Inf threshold.".
>
> https://github.com/OpenObservability/OpenMetrics/blob/1386544931307dff279688f332890c31b6c5de36/specification/OpenMetrics.md

Yes, that's pretty clear, and I guess the intention is to make as few
moving parts optional as possible.

Before OM, the +Inf bucket was considered redundant as it is
essentially the same as the _count. There is the weird edge case of
NaN observations, where one might argue they should be counted in
_count but not in the +Inf bucket, but this case is of fairly low
practical relevance (and I read the OM spec as "still count them in
the +Inf bucket"). I would assume Prometheus will happily tolerate
the absence of the +Inf bucket during ingestion and simply create it
from the _count value.

> So, does it depend on the format (required in text format, optional in
> protobuf)?

Yes, that sounds about right.

OM text format is clear about requiring it. OM protobuf is weird, as
you have discovered. The old Prometheus formats (both text and proto)
are blissfully underspecified. For a precise answer, we needed to
double-check what Prometheus is doing upon ingestion. I'm sure that
Prometheus 1.x added the +Inf time series in any case, but Prometheus
2.x might stick closer to what it sees in the text exposition (and
thus not create the +Inf time series if it is missing in the
exposition).

> Bonus question: What about buckets for native histograms, are they optional?

Yes. A native histogram with zero observations will also have zero
buckets.

Which leads to the interesting edge case of how to recognize if a
histogram is meant to be a native one or not if it hasn't received any
observations yet, see
https://github.com/prometheus/client_golang/issues/1127

--
Björn Rabenstein
[PGP-ID] 0x851C3DA17D748D03
[email] bjo...@rabenste.in