Haproxy + mtail

[Nikolay]

Hi all! Please tell me, I've been suffering for a week now, I can't configure mtail to download from haproxy. When debugging writes

mtail -one_shot -logtostderr -progs /etc/mtail/haproxy.mtail -logs /var/log/haproxy.log

(used both 54 and 24 version)

I0502 14:15:59.380543  430179 checker.go:254] capture group reference `hostname' at haproxy.mtail:14:3-128 appears to be unused
I0502 14:15:59.380583  430179 checker.go:254] capture group reference `application' at haproxy.mtail:14:3-128 appears to be unused
I0502 14:15:59.380594  430179 checker.go:254] capture group reference `pid' at haproxy.mtail:14:3-128 appears to be unused
I0502 14:15:59.380603  430179 checker.go:254] capture group reference `application' at haproxy.mtail:14:3-128 appears to be unused
I0502 14:15:59.380611  430179 checker.go:254] capture group reference `pid' at haproxy.mtail:14:3-128 appears to be unused


In general, mtail works, but deduces such metrics from usefulness

mtail_vm_line_processing_duration_seconds_bucket
mtail_vm_line_processing_duration_seconds_count
mtail_vm_line_processing_duration_seconds_sum

Mtail is run like this sudo mtail -progs /etc/mtail/haproxy.mtail -logs /var/log/haproxy.log

Use config https://gitlab.com/gitlab-cookbooks/gitlab-mtail/-/blob/master/files/default/mtail/haproxy.mtail

My log-format 

log-format %ci:%cp\ [%t]\ %ft\ %sslv\ %b/%s\ %Tq/%Tw/%Tc/%Tr/%Tt\ %ST\ %B\ %U\ %CC\ %CS\ %tsc\ %ac/%fc/%bc/%sc/%rc\ %sq/%bq\ %hr\ %hs\ %{+Q}r

Example logs:

May  2 13:53:25 test haproxy[402508]: 192.168.1.131:13677 [02/May/2023:13:53:25.221] stats - stats/<NOSRV> 0/-1/-1/-1/0 503 221 129 - - SC-- 1/1/0/0/0 0/0 "GET / HTTP/1.1"
May  2 13:53:25 test haproxy[402508]: 192.168.1.131:13678 [02/May/2023:13:53:25.223] stats - stats/<NOSRV> 0/-1/-1/-1/0 503 221 129 - - SC-- 1/1/0/0/0 0/0 "GET / HTTP/1.1"
May  2 13:53:25 test haproxy[402508]: 192.168.1.131:13679 [02/May/2023:13:53:25.225] stats - stats/<NOSRV> 0/-1/-1/-1/0 503 221 129 - - SC-- 1/1/0/0/0 0/0 "GET / HTTP/1.1"
May  2 13:53:25 test haproxy[402508]: 192.168.1.131:13680 [02/May/2023:13:53:25.227] stats - stats/<NOSRV> 0/-1/-1/-1/0 503 221 129 - - SC-- 1/1/0/0/0 0/0 "GET / HTTP/1.1"

Thank you all in advance for your replies!

[Jamie Wilkinson]

Hi Nikolay!

Are those the only errors that mtail emits?

They're only warnings about the expression from haproxy.mtail line 14, that those named capture groups are never used in the program.  They don't seem related to the problem here.  

The metrics you see are the default mtail debugging metrics.

If you can run with another flag: --vmodule=tailer=2 then we'll get more debug information from the log reader stage.  Adding --vmodule=vm=2 will have more information from the program execution in the virtual machine. 

Also can you provide the full log, not just this snippet?  That will give me more information to help.

What I suspect is happening is the regular expression is not matching those lines, but I can't tell from what you've given.

--
You received this message because you are subscribed to the Google Groups "mtail-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mtail-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mtail-users/45a58ed7-9d20-4101-b5a0-4360d04aaca5n%40googlegroups.com.

[Nikolay]

Good afternoon!  Here are my configs.

If there is anything else you need please let me know.

Thanks!

среда, 3 мая 2023 г. в 01:16:43 UTC+3, Jamie Wilkinson:

[Nikolay]

In general, I want to get metrics for haproxy http requests with the time of each request to build histograms and quantiles.
Here are my metrics so far from mtail.

среда, 3 мая 2023 г. в 09:55:22 UTC+3, Nikolay:

[Nikolay]

Good afternoon! Attached all my configs and logs above.

среда, 3 мая 2023 г. в 01:16:43 UTC+3, Jamie Wilkinson:

Hi Nikolay!

[Jamie Wilkinson]

I can reproduce this and had to add some debugging `glog.Infof`s to figure out what was happening.

You've got `getfilename() != "haproxy.log"` at the top of the file.  It's not doing any further processing because of that conditional.

If you remove that block, then the program proceeds.

None of the patterns internal to the syslog decorated block match the lines in the input though.

To view this discussion on the web visit https://groups.google.com/d/msgid/mtail-users/034b032f-5672-497c-bddf-69f085a1a857n%40googlegroups.com.