Troubleshoot Suggestions

25 views
Skip to first unread message

Mitchell Tuckness

unread,
Nov 21, 2022, 1:05:35 AM11/21/22
to Prometheus Users
Hi All,

I just put Prometheus on a virtual machine following this guide. The service is running; I don't see any errors in my Syslog that I can pin directly to Prometheus. I was hoping I could get advice on where to look; what might be wrong? I am leaning toward the config file, but as far as I can tell, it should be OK for just this basic setup.

My issue is I don't get a webpage when I try to access the server. UFW is allowing that port. Prometheus service is running. 

The only thing I see that is odd in the Syslog is:

Nov 15 16:17:01 prometheus CRON[1131]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Nov 15 17:00:27 prometheus prometheus[770]: ts=2022-11-15T17:00:27.963Z caller=compact.go:519 level=info component=tsdb
msg="write block" mint=1668520812852 maxt=1668528000000 ulid=01GHY3R9TKDCF0H656G28KP10J duration=103.526641ms
Nov 15 17:00:27 prometheus prometheus[770]: ts=2022-11-15T17:00:27.966Z caller=head.go:1213 level=info component=tsdb ms
g="Head GC completed" caller=truncateMemory duration=2.790302ms
Nov 15 17:03:38 prometheus kernel: [13219.657945] [UFW BLOCK] IN=enp0s3 OUT= MAC=01:00:5e:00:00:01:40:f5:20:0c:4b:f3:08:
00 SRC=192.168.1.7 DST=224.0.0.1 LEN=32 TOS=0x00 PREC=0x00 TTL=1 ID=4 PROTO=2
Nov 15 17:03:38 prometheus kernel: [13219.658284] [UFW BLOCK] IN=enp0s3 OUT= MAC=01:00:5e:00:00:01:40:f5:20:0c:4b:f3:08:
00 SRC=192.168.1.7 DST=224.0.0.1 LEN=32 TOS=0x00 PREC=0x00 TTL=1 ID=4 PROTO=2
Nov 15 17:17:01 prometheus CRON[1145]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Nov 15 18:17:01 prometheus CRON[1157]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)

Ben Kochie

unread,
Nov 21, 2022, 1:13:43 AM11/21/22
to Mitchell Tuckness, Prometheus Users
PROTO=2 is IGMP, which has nothing to do with the TCP connection required for Prometheus.

Sounds like you have a typo in your UFW rules.

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/8ce61c16-3dae-4ddc-a0dc-50cb8416faffn%40googlegroups.com.

Brian Candler

unread,
Nov 21, 2022, 3:41:02 AM11/21/22
to Prometheus Users
> My issue is I don't get a webpage when I try to access the server

What do you get instead - an error response (if so, what error)? A timeout?

You can check whether the packets are arriving on your prometheus server:

tcpdump -i eth0 -nn -s0 -X tcp port 9090

Replace eth0 with your primary interface name if it's different. (ip addr list)

purc...@maladomini.com

unread,
Nov 22, 2022, 10:32:51 AM11/22/22
to Ben Kochie, Prometheus Users

The UFW rule looks OK, as far as I can tell.

 

Status: active

 

     To                         Action      From

     --                         ------      ----

[ 1] 22/tcp                     ALLOW IN    Anywhere                   # SSH

[ 2] 9090/tcp                   ALLOW IN    Anywhere                   # Prometheus

[ 3] 22/tcp (v6)                ALLOW IN    Anywhere (v6)              # SSH

[ 4] 9090/tcp (v6)              ALLOW IN    Anywhere (v6)              # Prometheus

purc...@maladomini.com

unread,
Nov 22, 2022, 10:33:13 AM11/22/22
to Brian Candler, Prometheus Users

I just figured it out. It is something on the firewall rules in my IPSEC tunnel between my two networks. I tried it from a local machine on that network and I could hit the webpage!

 

From: promethe...@googlegroups.com <promethe...@googlegroups.com> On Behalf Of Brian Candler
Sent: Monday, November 21, 2022 1:41 AM
To: Prometheus Users <promethe...@googlegroups.com>
Subject: Re: [prometheus-users] Troubleshoot Suggestions

 

> My issue is I don't get a webpage when I try to access the server

--
You received this message because you are subscribed to a topic in the Google Groups "Prometheus Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/prometheus-users/LH9o1R7B5QU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/2090410d-14fa-42d9-8195-9107e0da011cn%40googlegroups.com.

Reply all
Reply to author
Forward
0 new messages