Reloading config appears to not actually reload.

876 views
Skip to first unread message

tper...@gmail.com

unread,
Dec 16, 2016, 10:41:31 PM12/16/16
to Prometheus Users
Hi folks,

Run into an interesting issue with Prometheus v1.4.1. I have it deployed with docker, supervised by a systemd unit. The configurations and such are mounted into the container using a bind volume mount(s). I've tried the recommended ways of reloading the configuration, but im not sure if what i'm seeing is expected or not: specifically, i see stuff like this in the log:

```
Dec 16 19:17:41 prometheus01 docker[11475]: {"level":"info","msg":"Loading configuration file /etc/prometheus/prometheus.yml","source":"main
```

But when i visit http://$myhost:9090/config the old values still display, and I don't get any change in behaviour, indicating that in-fact, the reload of config did not actually take place. I've tried using:

```
curl -s -XPOST localhost:9090/-/reload
```

Which as mentioned, triggers that log-line. I've also tried:

```
docker exec -it prometheus sh -c 'killall -HUP prometheus'
```

Both result in the aforementioned log-line, but neither seems to effectively reload the configuration. Am I doing something wrong?

Thanks in advance - prometheus has been working really well for us thus far.

-- T

Yosef Yudilevich

unread,
Dec 17, 2016, 3:50:41 AM12/17/16
to Prometheus Users
same thing without docker
it happens not only 1.4 but also before
i just restart to reload configs

Julius Volz

unread,
Dec 17, 2016, 2:03:45 PM12/17/16
to Yosef Yudilevich, Prometheus Users
Hm, it works for me at 1.4.1. With Docker, you have to be careful to not bind-mount just a single file (prometheus.yml), but an entire directory containing that file. Otherwise, an update to the file on the host will not result in an update in the container. With a directory, it works for me though - does it work for you?


--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/b0bd2ba8-96b1-40ae-955f-45626cfdfdf9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Timothy Perrett

unread,
Dec 17, 2016, 2:08:14 PM12/17/16
to Prometheus Users, yosef.yu...@gmail.com
Oh interesting - currently I am bind-mounting the specific file. I'll try the directory later today and get back to you :-)

Thanks!

On Saturday, 17 December 2016 11:03:45 UTC-8, Julius Volz wrote:
Hm, it works for me at 1.4.1. With Docker, you have to be careful to not bind-mount just a single file (prometheus.yml), but an entire directory containing that file. Otherwise, an update to the file on the host will not result in an update in the container. With a directory, it works for me though - does it work for you?
On Sat, Dec 17, 2016 at 9:50 AM, Yosef Yudilevich <yosef.yu...@gmail.com> wrote:
same thing without docker
it happens not only 1.4 but also before
i just restart to reload configs

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To post to this group, send email to promethe...@googlegroups.com.

Timothy Perrett

unread,
Dec 18, 2016, 2:42:35 PM12/18/16
to Prometheus Users, yosef.yu...@gmail.com
Huzzah, this actually works for me - the config did seem to reload when mounting the whole dir :-) 

-- T

Julius Volz

unread,
Dec 18, 2016, 2:56:10 PM12/18/16
to Timothy Perrett, Prometheus Users, Yosef Yudilevich
Nice :)

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/fbc5dd0c-a188-4121-ad55-3ec94c60f97b%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages