Binary operations between range vectors and scalars

[Brian Candler]

I don't know if this has been proposed before, so I'd like to raise it here before taking it to github or prometheus-developers.

There are cases where binary operators could act between range vectors and scalars, but this is not currently allowed today (except by using subqueries, which end up resampling the timeseries). Simple examples:

up[10m] == 1    # filtering

foo[10m] * 8    # arithmetic

I think the semantics of these are obvious. The operation would take place between each value in the range vector and the scalar, and the timestamp of each result value would equal the timestamp of each value in the input range vector.

Use cases would include things like:

count_over_time(up[10m] == 1)    # how many times was there a successful scrape in last 10 minutes?

[Chris Siebenmann]

I like this as a general idea and as you note it would deal with a lot
of situations where one needs to use subqueries today. However, I think
there may be semantic issues with feeding filtered range vectors to
operations like rate() or delta(). For instance, if you do
delta(metric[1h] > 0), does delta() extrapolate using the timestamps of
the first and last time series in the original range vector or the
filtered one?

(You might say 'why are you filtering delta()?' and one plausible answer
is 'the metrics source sometimes glitches out and returns impossible
results, we want to filter them out to get the delta of the real
readings'.)

- cks
PS: speaking of timestamps, it would be nice if timestamp() worked on a
range vector and yielded its own range vector of the timestamps of every
element of the range vector, basically replacing the original values in
the range vector with the corresponding timestamps. But this is probably
outside PromQL's processing model.

[Brian Candler]

On Monday 29 January 2024 at 15:24:26 UTC Chris Siebenmann wrote:

For instance, if you do
delta(metric[1h] > 0), does delta() extrapolate using the timestamps of
the first and last time series in the original range vector or the
filtered one?

I would expect it to use the timestamps of the first and last points the filtered one.

 

PS: speaking of timestamps, it would be nice if timestamp() worked on a
range vector and yielded its own range vector of the timestamps of every
element of the range vector, basically replacing the original values in
the range vector with the corresponding timestamps. But this is probably
outside PromQL's processing model.

I agree with that.

I'd also like to be able to answer the question "what was the timestamp of the most recent successful scrape"? One option would be

timestamp(last_over_time(up[10m] == 1))

... but as far as I can see, this currently wouldn't work: timestamp(last_over_time(up[10m])) just shows the current eval time, unlike timestamp(up). Maybe last_over_time() could be changed to preserve the timestamp?

Another option would be

max(timestamp(up[10m]))

which requires your proposed change to allow timestamp(rangevector).