PromQL one-to-many matching with different label sets

[Simon Let]

I have some metrics with different label sets, for example:

```

label_metric{foo="a", bar="b"} 0

label_metric{foo="a", bar="c"} 0

my_metric{foo="a", bar="b"} 5

my_metric{foo="a"} 3
```

When I use the following aggregation:

```

min by (foo, bar) (

  my_metric

)
```

I get an expected result:

```

my_metric{foo="a", bar="b"} 5

my_metric{foo="a"} 3
```

But when I instead use one-to-many operation:

```

label_metric

+ on (foo, bar) group_left()

my_metric
```

I would expect a following result:

```

{foo="a", bar="b"} 5

{foo="a", bar="c"} 3
```

as it would be consistent with the aggregation operation behavior.

But instead, I get the following result:

```

{foo="a", bar="b"} 5

```

This behavior seems inconsistent with the aggregation operation to me. 
Is there a reason why `group_left()` silently drops part of the data instead of matching it with the right-hand side? 

It looks like one-to-many matching only works with the largest label set and drops the rest of the data. This is very confusing and dangerous imho.

Am I missing something? 


Thanks in advance
Simon Let

[Julius Volz]

Hi Simon,

Aggregation and binary operators are quite different beasts and only sometimes happen to be usable to reach the same query outcome...

The "on(foo, bar)" label matching modifier you provided tells PromQL to match series from the left and right sides only when they have exactly the same values for *both* of those labels. But in your case, the left series has {foo="a", bar="c"} and the right series has only {foo="a"}, which is equivalent to {foo="a", bar=""}, so a different (non-existent) value for the bar label. Thus, those two series do not find each other as a match at all and don't become part of the output. The "group_left()" modifier only helps to allow one-to-many matching in situations where your matching criteria end up matching each series from the left with multiple series from the right, but it cannot create matches that weren't there in the first place.

Regards,

Julius

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/97e321fd-846f-4bf2-b380-ca72924af0f3n%40googlegroups.com.

--

Julius Volz

PromLabs - promlabs.com