mtail named pipes

[Raoul]

Hi,

I am looking to get mtail and named pipes working. Please, correct me if I am wrong or misunderstood something.

My current setup uses CentOS 7 with rsyslog 8.24 and I created an output line in rsyslog.conf like the following:

*.*     |/var/log/pipe

Then I created the fifo using mkfifo -m 644 /var/log/pipe. After starting rsyslog I can use cat < /var/log/pipe and I can see the whole syslog output.

Now, when I try to start mtail (even as root) using mtail -logtostderr -port 9102 -progs /etc/mtail -logs /var/log/pipe, it doesn't fully startup. The port 9102 remains closed and in the log lines on stdout I can only see messages Loaded, Starting and Started program xxx.mtail. No lines Tailing /var/log/pipe or Listening on port 9102.

What am I missing or doing wrong?

Thanks for your help,

Raoul

[Jamie Wilkinson]

I had assumed that this would work. So it sounds like a bug and I'll try to reproduce it.

--
You received this message because you are subscribed to the Google Groups "mtail-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mtail-users...@googlegroups.com.
To post to this group, send email to mtail...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mtail-users/cb0c6701-7551-4359-8646-dc5f4a23b498%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jamie Wilkinson]

There's an existing test https://github.com/google/mtail/blob/master/tests/read_pipe_test.sh that tests this codepath.  What's different?

Can you add the flag --vmodule=tail=2 please?

[Raoul Thill]

Starting mtail with --vmodule=tail=2 add the following output to the log:

I1126 09:29:55.936588 9962 tail.go:146] AddPattern: /var/log/pipe
I1126 09:29:55.936798 9962 tail.go:162] glob matches: [/var/log/pipe]
I1126 09:29:55.936832 9962 tail.go:244] openlogPath /var/log/pipe false false
I1126 09:29:55.936972 9962 tail.go:262] Adding a file watch on
"/var/log/pipe"
I1126 09:29:56.089055 9962 tail.go:328] Event type
watcher.Event{Op:1, Pathname:"/var/log/pipe"}
I1126 09:29:56.089113 9962 tail.go:200] handleLogUpdate /var/log/pipe

mtail is not started, it hangs there.

Raoul

[Jamie Wilkinson]

Great, that gives me a good start on where to look.  Thanks!

To view this discussion on the web visit https://groups.google.com/d/msgid/mtail-users/CALxFJS-daGLadgL82B7M88sF0-Crv9Cdyjtqo2FbCSsuZTg3ow%40mail.gmail.com.

[Jamie Wilkinson]

The file open doesn't succeed until the pipe has some data written to it.  I am not sure why this is, yet, but it seems that you're not getting anything from syslog on that pipe yet?

If you cat something to your pipe, then mtail finishes startup and listens on the port.  This is why the test passes.

One fix to make is to not block the server startup on, I think, a read from the file handle.

[Jamie Wilkinson]

Aha!  named pipes block on open until both sides open it!  The fix is to open the file in nonblocking mode.

Please try the latest mtail from source.