Prometheus dashboard configuration

[monika samal]

Hello All, 

I have enabled prometheus port monitoring using blackbox exporter, so i wanted to know can we configure target directly on dashboard instead of .yml file and how can we connect to target without opening port on target machine.

[Brian Candler]

Prometheus doesn't have a dashboard as such.  Maybe you're thinking of something else like Grafana?

You configure prometheus via prometheus.yml, which uses a "service discovery" mechanism to find out targets to scrape.  The "static" service discovery mechanism lists the targets within the prometheus.yml file; the "file" service discovery lists them in separate files; and there are others which may make sense depending on your environment (e.g. getting lists of targets from Kubernetes, from AWS, from consul etc).

How you integrate this is up to you.  For example, I use Netbox to generate a file sd config for prometheus.

> how can we connect to target without opening port on target machine.

That depends on what you're talking about. 

You said you were talking about blackbox_exporter.  That means there are two bits of communication: prometheus to blackbox_exporter, and blackbox_exporter to target.  If you co-locate the blackbox_exporter with prometheus, you'll have to open ports for blackbox_exporter to be able to test the targets.  But you can always put the blackbox_exporter somewhere else closer to the targets, and instead open the HTTP scrape between prometheus and blackbox exporter.

If you're talking about node_exporter, then this sits on each node being monitored.  Yes, you need to open the port to allow prometheus to node_exporter, because prometheus is a "pull" monitoring system.  There is Pushprox but you really don't want to use that unless you have no other possibility.  You can secure communication to node_exporter with TLS and client certificates (actually this will be a new feature in 1.0.0, but it's currently still on release candidate 1.0.0-rc.0)