Trying to start alertmanager using simple.yaml

[robotun...@gmail.com]

Attempting to deploy a POC test with prometheus, alertmanager, and collectd-explorer.  For alertmanager using alertmanager-0.15.0-rc.1.linux-amd64.tar.gz.

When I attempt to start I see the following 

$ sudo ./alertmanager --config.file=simple.yaml

level=info ts=2018-04-04T22:37:26.091853599Z caller=main.go:140 msg="Starting Alertmanager" version="(version=0.15.0-rc.1, branch=HEAD, revision=acb111e812530bec1ac6d908bc14725793e07cf3)"

level=info ts=2018-04-04T22:37:26.091933484Z caller=main.go:141 build_context="(go=go1.10, user=root@f278953f13ef, date=20180323-13:05:10)"

level=error ts=2018-04-04T22:37:26.106113741Z caller=main.go:160 msg="Unable to initialize gossip mesh" err="create memberlist: Failed to get final advertise address: No private IP address found, and explicit IP not provided"

I have not modified simple.yaml

From what I am reading, it appears that alertmanager is often configured in clustered mode via the --cluster flag, and it appears that my failure is related. Can anyone advise me on the proper startup procedure?

[Simon Pasquier]

The problem is with the memberlist library that manages the cluster. If you're not deploying AlertManager in a cluster mode, you can disable it explicitly by adding the '--cluster.listen-address=""' option.
But it would be interesting to know more why the error happens. Can you start the AlertManager with debug log level (--log.level=debug)?

Also please share the output of "ip addr".

Simon

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/9fa8128d-c431-46b4-87d3-323dbe84f323%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[d...@danwalker.com]

+1 I'm also getting this issue with the release candidate for 0.15, using the 0.14 alertmanager doesn't give me this problem.

[Christopher Paquin]

Simon, here is the output you requested. My POC pretty much a very simple VM running Prometheus, Alertmanager, and Collect Exporter.

# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:b1:09:f4 brd ff:ff:ff:ff:ff:ff
    inet 96.239.250.149/24 brd 96.239.250.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:feb1:9f4/64 scope link 
       valid_lft forever preferred_lft forever

--
You received this message because you are subscribed to a topic in the Google Groups "Prometheus Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/prometheus-users/-5wd-P13xCI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to prometheus-users+unsubscribe@googlegroups.com.

To post to this group, send email to prometheus-users@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/caa6d630-2dc2-442d-b6c9-c0b1cde6efde%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Christopher Paquin
770-906-7646

chris....@comcast.net

[Simon Pasquier]

On Fri, Apr 6, 2018 at 6:41 PM, Christopher Paquin <chris....@comcast.net> wrote:

Simon, here is the output you requested. My POC pretty much a very simple VM running Prometheus, Alertmanager, and Collect Exporter.

# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:b1:09:f4 brd ff:ff:ff:ff:ff:ff
    inet 96.239.250.149/24 brd 96.239.250.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:feb1:9f4/64 scope link 
       valid_lft forever preferred_lft forever

The error happens because the memberlist library (used for the AlertManager clustering) fails to find a private IP address (technically any local IP address listed in the RFC 6890 [1] would work).

The solution is either to pass "--cluster.advertise-address=96.239.250.149" to AlertManager or configure an additional private IP address on your machine.

This is probably the same problem for Dan.

 
[1] https://tools.ietf.org/html/rfc6890

On Fri, Apr 6, 2018 at 8:52 AM, <d...@danwalker.com> wrote:

+1 I'm also getting this issue with the release candidate for 0.15, using the 0.14 alertmanager doesn't give me this problem.

On Wednesday, April 4, 2018 at 11:43:10 PM UTC+1, robotun...@gmail.com wrote:

Attempting to deploy a POC test with prometheus, alertmanager, and collectd-explorer.  For alertmanager using alertmanager-0.15.0-rc.1.linux-amd64.tar.gz.

When I attempt to start I see the following 

$ sudo ./alertmanager --config.file=simple.yaml

level=info ts=2018-04-04T22:37:26.091853599Z caller=main.go:140 msg="Starting Alertmanager" version="(version=0.15.0-rc.1, branch=HEAD, revision=acb111e812530bec1ac6d908bc14725793e07cf3)"

level=info ts=2018-04-04T22:37:26.091933484Z caller=main.go:141 build_context="(go=go1.10, user=root@f278953f13ef, date=20180323-13:05:10)"

level=error ts=2018-04-04T22:37:26.106113741Z caller=main.go:160 msg="Unable to initialize gossip mesh" err="create memberlist: Failed to get final advertise address: No private IP address found, and explicit IP not provided"

I have not modified simple.yaml

From what I am reading, it appears that alertmanager is often configured in clustered mode via the --cluster flag, and it appears that my failure is related. Can anyone advise me on the proper startup procedure?

--
You received this message because you are subscribed to a topic in the Google Groups "Prometheus Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/prometheus-users/-5wd-P13xCI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/caa6d630-2dc2-442d-b6c9-c0b1cde6efde%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Christopher Paquin
770-906-7646

chris....@comcast.net

--

You received this message because you are subscribed to the Google Groups "Prometheus Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.

To post to this group, send email to prometheus-users@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAPZocUwvfyrxSCXz_ncoc0s7Pc4sa2J9oN54XfryoR6oRYwi4Q%40mail.gmail.com.

[cabatan...@gmail.com]

i also have same issue. Where you able to resolve the issue?

level=info ts=2018-09-26T06:48:21.001327851Z caller=main.go:174 msg="Starting Alertmanager" version="(version=0.15.2, branch=HEAD, revision=d19fae3bae451940b8470abb680cfdd59bfa7cfa)"

level=info ts=2018-09-26T06:48:21.001407716Z caller=main.go:175 build_context="(go=go1.10.3, user=root@3101e5b68a55, date=20180814-10:53:39)"

level=error ts=2018-09-26T06:48:21.001443723Z caller=main.go:199 msg="unable to initialize gossip mesh" err="invalid advertise address: address 146.40.10.22: missing port in address"

To unsubscribe from this group and all its topics, send an email to prometheus-use...@googlegroups.com.
To post to this group, send email to promethe...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/caa6d630-2dc2-442d-b6c9-c0b1cde6efde%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Christopher Paquin
770-906-7646

chris....@comcast.net

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To post to this group, send email to promethe...@googlegroups.com.

[Simon Pasquier]

As the error message says, you need to pass the port number too:

./alertmanager --cluster.advertise-address=1.2.3.4:8888 ...

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/0fd22c9b-9ac3-43c6-a090-f0fe6ab644a7%40googlegroups.com.

[Eudexter Cabatan]

Is port 8888 default port? I am only testing one box. Should i use the ip address of my box for the cluster.advertise-address

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.

To post to this group, send email to promethe...@googlegroups.com.

[Simon Pasquier]

The default port is 9094. If you're testing a single box setup (I assume only one AlertManager instance), don't specify the "cluster.advertise-address" parameter and AlertManager will listen on all available interfaces.

PS: I see where your confusion came from. My earlier response to Christopher was wrong as it missed the ":port" part.

[Eudexter Cabatan]

Thanks for the clarification.