syslog multiline?

[Ilya Trushchenko]

Hi, how can use syslog input plugin with unstructured multiline messages?

I mean I just want to have a new message only if

format_firstline /^NewLineRegex/

matches, all other lines should be appended to the current message

[Mr. Fiber]

Hmm... Syslog RFC doesn't define multiline case.

What the logs generated by syslog with multilne?

Masahiro

--
You received this message because you are subscribed to the Google Groups "Fluentd Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ilya Trushchenko]

Not generated, but forwarded. Many kinds of logs, including java errors with backtraces. For example, for flme, I use syslog-ng with 
source s_flume {
       file("/var/log/flume-ng/flume.log"
               program_override("flume")
               multi-line-mode(regexp)
               multi-line-prefix("[0-9]{2} ... [0-9]{4}")
       );
};

and this is enough for syslog-ng to parse multiline logs of any form. Any string that does not start with a timestamp - is a part of multiline message.

I forward this log to another syslog-ng instance using syslog protocol, and there I do not need any additional options - syslog-ng understands multiline (don't know if it's in RFC)

But when I forward these logs to fluentd - It doesn't understand this multiline format and it does not have an option to set the first line of a multiline message

[Mr. Fiber]

> But when I forward these logs to fluentd - It doesn't understand this multiline format and it does not have an option to set the first line of a multiline message

Yes because you are first person who wants to handle multiline message by Fluentd syslog plugin.

What the message syslog-ng send?

<6>Sep 11 00:00:00 localhost logger: multi

line

log

or

<6>Sep 11 00:00:00 localhost logger: multi

symbol-for-multiline: line

symbol-for-multiline: log

?

And rsyslog or other syslog family follows syslog-ng format?

[Igal Baevsky]

Hi Ilya,

Did you find a solution for this problem? I'm facing exactly the same issue as you are and wondering weather I should do something to flatten those multiline messages with syslog-ng or fluentd.

Thanks,