Prometheus metrics repeat, and cause the promql to be unavailable.

[sowhat7]

I add a label for node 10.193.32.44, and I get repeat metrics from prometheus at a certain moment.
And then my promql is unavailable, because many to many are not allowed. 

 

prometheus version: 2.7.2
node-exporter version: 0.17.0  

[Christian Hoffmann]

Hi,

On 2020-12-11 04:36, sowhat7 wrote:
> I add a label for node 10.193.32.44, and I get repeat metrics from
> prometheus at a certain moment.

Is this a permanent issue or does it affect the time around your change
only?
I suspect the latter. In that case, you could use some kind of
aggregation to work around that (e.g. topk(1, ...), min()). You could
also try to delete the overlapping data (probably plus some minutes due
to staleness) using the DELETE API.

If the former, then there might be some issue with service discovery and
you would need to share your relevant configs in order for someone to be
able to help.

> prometheus version: 2.7.2
> node-exporter version: 0.17.0

These versions sound really, really old. You should consider updating.

Kind regards,
Christian