Multline in log?

[Lucas Gazire]

I finally integrated log4net with fluentd, one of log messages of the system contains a breakline, when i checked the log the message didn't come complete:

{

      "_index" : "log4net-2015.09.02",

      "_type" : "fluentd",

      "_id" : "AU-N5jryMiYcKf52S_Kz",

      "_score" : 1.0,

      "_source":{"thread":"1","level":"INFO","logger":"Log.Log","username":"LoggerClass","hostname":"BRD03858683","message":"2546796 ","@timestamp":"2015-09-02T08:51:57-03:00"}

    }

the field message misses a few lines. Can someone help me how to insert the lines the were breaked?

and the config file:

### Listen on port 5140, module in_syslog ###

<source>

 type syslog

 port 5140

 bind 0.0.0.0

 tag syslog

</source>

### Parsing the events ###

<match syslog.local7.info>

 type parser

 remove_prefix syslog

 format /^(?<thread>[^ ]*) (?<level>[^ ]*) (?<logger>[^ ]*) (?<username>[^ ]*) (?<hostname>[^ ]*) \[\[(?<message>[^*]*)/

 key_name message

</match>

### Write parsed events to ElasticSearch ###

<match local7.info>

 buffer_type file

 buffer_path /mnt/ramdisk/log4net.buff

 buffer_chunk_limit 4m

 buffer_queue_limit 50

 flush_interval 3s

 type elasticsearch

 logstash_format true

 logstash_prefix log4net

 host localhost

 port 9200

</match>

Thanks!

Best Regards!

Lucas Gazire

[Mr. Fiber]

Your case is similar to https://groups.google.com/forum/#!searchin/fluentd/syslog$20multiline/fluentd/FK4OQ4xvEhA/m1rMulXTRL8J

Hmm... some users send multiline logs to syslog,

in_syslog should support multiline.

https://github.com/fluent/fluentd/pull/640

This PR tries to support recent syslog spec.

If your log4net uses latest syslog protocol, it may resolve this problem because

latest syslog protocol has the byte size of message at the head.

Masahiro

--
You received this message because you are subscribed to the Google Groups "Fluentd Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Lucas Gazire]

I'm using the latest version of the log4net and it's odd that the messages with /n are not coming in complete.

[Lucas Gazire]

I don't know if it's relevant but my syslog regex: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.12/lib/fluent/plugin/in_syslog.rb

is like this:

 SYSLOG_REGEXP = /^\<([0-9]+)\>(.*)/

Is that correct?

On Wednesday, September 2, 2015 at 11:51:29 AM UTC-3, repeatedly wrote:

[Mr. Fiber]

Maybe correct.

But in_syslog with tcp uses `\n` for message separation so it is not good for multiline logs.

Using in_syslog with udp may resolve the problem.