PromQL: many-to-many matching not allowed: matching labels must be unique on one side

[Ramprasath A]

Hi,

I have an expression of a recording rule named node:node_num_cpu:sum and I am planning to add this rule to Thanos-Rule component but it fails in Thanos-Query itself with the error message: many-to-many matching not allowed: matching labels must be unique on one side (attached screenshot here with too. Please find the attachment.)

Here is the query:
count by (cluster, node) (sum by (node, cpu) (
node_cpu_seconds_total{job="node-exporter"}
* on (namespace, pod) group_left(node)
node_namespace_pod:kube_pod_info:
)) 

Similarly, there are around about 50 other recording and alerting rules which are affected.

Thank you for your time and consideration.

Sincerely,

Ramprasath A

[Julius Volz]

Hi Ramprasath,

When you do a N-to-1 (or many-to-one) match, you need to ensure that your matching labels (in your case, "namespace" and "pod") don't produce multiple matches on the "many" (right) side for any entry on the "one" (left) side. The error message lists two series from the right side which have identical namespace + pod labels, but differ on at least the "replica" label ("thanos-receive-prod-0" vs. "thanos-receive-poc-0"). Maybe you only wanted to select series from one Thanos receive replica on the right side and that would fix the issue? Or you could aggregate away the extra label somehow before applying the binary operator.

Regards,

Julius

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/c100c985-58ec-40e6-b15b-10d6d704b1bfn%40googlegroups.com.

--

Julius Volz

PromLabs - promlabs.com

[Ramprasath A]

Thank you for the clarification. I will check this out and get back to you.

[Ramprasath A]

Hi Julius,

It is an honor and my pleasure to receive a reply from you.

Thanks a lot for pointing me in the right direction.

The query worked flawlessly when I added the label: clusterName.

The modified query looks like this:

count by (cluster, node) (sum by (node, cpu) (

        node_cpu_seconds_total{job="node-exporter",clusterName="cluster1"}

      * on (namespace, pod) group_left(node)

        node_namespace_pod:kube_pod_info:{clusterName="cluster1"}
      ))

Sincerely,

Ramprasath A