How to get hostname ( or nodename) to show up in an alert.

[Tony Kirk]

Hello All,

  I'm semi-new to prometheus, and I have inherited a large multi site setup to learn and admin. This system hasn't been touched in 6 months since the person who set up the whole system left 6 months ago. I have been asked to replace the ip:port (instance) in the alert body with the hostname. While I'm still learning, I haven't come across any easy solution for that.

  nodename is the closest thing I've seen so far, but I can't figure out how to pull that key/value out of node_uname_info for this purpose.

  Anyone have a straight forward way to do this? Or even a not straight forward way that I can try to understand?

[Ben Kochie]

On Wed, Jun 21, 2023 at 11:25 PM Tony Kirk <tony...@gmail.com> wrote:

Hello All,

  I'm semi-new to prometheus, and I have inherited a large multi site setup to learn and admin. This system hasn't been touched in 6 months since the person who set up the whole system left 6 months ago. I have been asked to replace the ip:port (instance) in the alert body with the hostname. While I'm still learning, I haven't come across any easy solution for that.

  nodename is the closest thing I've seen so far, but I can't figure out how to pull that key/value out of node_uname_info for this purpose.

Chicken and egg problem. How do you get the hostname if the scrape fails? It's basically impossible. How would `up` know what the hostname is is it can't scrape the target?

 

  Anyone have a straight forward way to do this? Or even a not straight forward way that I can try to understand?

What you need is to have your hostnames added at discovery time. How are you discovering your targets? That's the straight forward way. Tell Prometheus the hostnames via service discovery.

 

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/e96e1be2-6add-4363-b244-2ff0815a0ac8n%40googlegroups.com.

[Brian Candler]

On Wednesday, 21 June 2023 at 21:25:14 UTC Tony Kirk wrote:

  I'm semi-new to prometheus, and I have inherited a large multi site setup to learn and admin. This system hasn't been touched in 6 months since the person who set up the whole system left 6 months ago. I have been asked to replace the ip:port (instance) in the alert body with the hostname. While I'm still learning, I haven't come across any easy solution for that.

As Ben says, the best way to do that is by setting the instance label to just the hostname.

Reference:

https://www.robustperception.io/controlling-the-instance-label

A simple example:

https://groups.google.com/forum/#!topic/prometheus-users/I4dVtwI5YrM