exclude monitoring system ip from counters

51 views
Skip to first unread message

Yosef Yudilevich

unread,
Dec 17, 2018, 10:14:50 PM12/17/18
to mtail-users
hi i need help to exclude monitoring system checks <remote_addr> of 192.168.1.1
from the counter

may be someone may help pls



############
counter http_requests_total by request_method,status


# log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
#                '$status $body_bytes_sent "$http_referer" '
#                '"$http_user_agent" "$http_x_forwarded_for" '
#                '$request_time $upstream_response_time $pipe';


/^/ +
/(?P<remote_addr>[0-9A-Za-z\.-:]+) / +
/- / +
/(?P<remote_user>[0-9A-Za-z-]+) / +
/(?P<time_local>\[\d{2}\/\w{3}\/\d{4}:\d{2}:\d{2}:\d{2} [-|\+]\d{4}\]) / +
/"(?P<request_method>[A-Z]+) (?P<request_uri>\S+) (?P<http_version>HTTP\/[0-9\.]+)" / +
/(?P<status>\d{3}) / +
/(?P<bytes_sent>\d+) / +
/"(?P<http_referer>\S+)" / +
/"(?P<http_user_agent>[[:print:]]+)" / +
/"(?P<x_forwarded_for>[0-9A-Za-z\.-]+)" / +
#/(?P<request_seconds>\d+)\.(?P<request_milliseconds>\d+) / +
#/(?P<upstream_seconds>\d+)\.(?P<upstream_milliseconds>\d+) / +
/(?P<pipe>.*)/ +

/$/ {
        http_requests_total[tolower($request_method)][$status]++
}
###########################

Jamie Wilkinson

unread,
Dec 20, 2018, 10:23:42 PM12/20/18
to Yosef Yudilevich, mtail-users
Hello Yosef,

If you only want to count the request method and status of each log line, then you can discard most of the regular expression in the match part.

So you want to not increment a counter if the remote_addr part is a specific IP address, try this condition around your counter increment:

 ... {
    $remote_addr != "192.168.1.1" {
       http_reuests_total[...]++
    }
}

--
You received this message because you are subscribed to the Google Groups "mtail-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mtail-users...@googlegroups.com.
To post to this group, send email to mtail...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mtail-users/afc079a6-6ba5-4adb-b936-51a8a7b8b389%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Krishna Khatana

unread,
Jul 2, 2021, 12:15:04 AM7/2/21
to mtail-users
I have the same program/configuration but when i start mtail its gives me error as below :
***********************
Jul 02 04:11:55 krashan-kumar-back.int.dev.mykronos.com systemd[1]: Started mtail.
Jul 02 04:11:55 krashan-kumar-back.int.dev.mykronos.com mtail[18848]: I0702 04:11:55.244335   18848 main.go:114] mtail version 3.0.0-rc46 git revision 76fadc0d5ef06dd83ad5129f1f14fa46f95b31a9 go version go1.16.4 go arch amd64 go os linux
Jul 02 04:11:55 krashan-kumar-back.int.dev.mykronos.com mtail[18848]: I0702 04:11:55.246101   18848 main.go:115] Commandline: ["/usr/local/bin/mtail/mtail" "--address" "10.3.50.108" "--port" "9101" "--emit_prog_label=false" "--alsologtostderr" "--progs" "/etc/mtail/" "--logs" "/data/nginx/logs/access.log"]
Jul 02 04:11:55 krashan-kumar-back.int.dev.mykronos.com mtail[18848]: I0702 04:11:55.246629   18848 store.go:178] Starting metric store expiry loop every 1h0m0s
Jul 02 04:11:55 krashan-kumar-back.int.dev.mykronos.com mtail[18848]: I0702 04:11:55.248471   18848 runtime.go:119] Compile errors for nginx.mtail:
Jul 02 04:11:55 krashan-kumar-back.int.dev.mykronos.com mtail[18848]: compile failed for nginx.mtail:
Jul 02 04:11:55 krashan-kumar-back.int.dev.mykronos.com mtail[18848]: nginx.mtail:15:1: syntax error: unexpected NL, expecting DIV
Jul 02 04:11:55 krashan-kumar-back.int.dev.mykronos.com mtail[18848]: I0702 04:11:55.248627   18848 tail.go:259] Tailing /data/nginx/logs/access.log
Jul 02 04:11:55 krashan-kumar-back.int.dev.mykronos.com mtail[18848]: I0702 04:11:55.248761   18848 mtail.go:124] Listening on 10.3.50.108:9101

Can anybody help what wrong i am doing ? please help its quite urgent

Jamie Wilkinson

unread,
Jul 2, 2021, 12:20:30 AM7/2/21
to Krishna Khatana, mtail-users
The nginx.mtail program has a syntax error at line 15, position 1.  It was expecting a closing `/` but got a newline instead (probably the preceding line.)

What's the whole program please?

--
You received this message because you are subscribed to the Google Groups "mtail-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mtail-users...@googlegroups.com.

Krishna Khatana

unread,
Jul 2, 2021, 4:28:38 AM7/2/21
to mtail-users
Thank you so much for the help, I corrected that issue and able to start mtail without any issue, now i am having another issue that i am not able to see this counter( http_requests_total ) on the Prometheus server or in the mtail metrics below is the commands how i started the mtail and prometheus .

Mtail start service :
[Unit]
Description=mtail
Wants=network-online.target
After=network-online.target

[Service]
Type=simple
Restart=on-failure
WorkingDirectory=/usr/local/bin/
ExecStart=/usr/local/bin/mtail/mtail \
    --address  127.0.0.1  --port 9101 \
    --emit_prog_label=false --alsologtostderr \
    --progs /etc/mtail/ \
    --logs /data/nginx/logs/access.log

User=mtail
Group=mtail

[Install]
WantedBy=multi-user.target

Prometheus startup code:
global:
  scrape_interval: 5s

scrape_configs:
  - job_name: 'prometheus'
    scrape_interval: 5s
    static_configs:
      - targets: ['127.0.0.1:9090']
  - job_name: 'mtail'
    scrape_interval: 5s
    static_configs:
      - targets: [' 127.0.0.1 :9101']

I am also not able to see the counter variable in metrics of mtail(http:// 127.0.0.1 :9100/metrics)

can you please help me , my baisc question is to how to see the counter(http_requests_total) on the prometheus ?

Reply all
Reply to author
Forward
0 new messages