[warn]: #0 failed to parse message dataHelp with parsing syslog

[Renan Hingel]

Hi,

Glad to find this community, I'm struggling with parsing syslog from Cisco ASA/Firepower, I've learned Cisco syslog isn't exactly RFC5424 compatible, despite the console stating so.

Symptom:

I'm getting the following logs:

2021-02-03 23:57:56 +0000 [warn]: #0 failed to parse message data="<165>2021-02-03T23:57:55Z FIREWALL_HOSTNAME : %FTD-5-111008: User 'enable_1' executed the 'more system:running-config' command."

Command issued:

 tail -f 1 /var/log/td-agent/td-agent.log

My environment:

td-agent 1.11.2

My configuration:

<source>
  @type syslog
  port 1470
  bind 0.0.0.0
  tag system
<parse>
  @type regexp
  expression /\<(?<priority>[0-9]+)\>(?<time>[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z) (?<host>[^ ]*) : (?<id>[^ ]*) (?<message>.*)/
  time_key time
  time_format %Y-%m-%dT%H:%M:%SZ"
</parse>
</source>

I've tried checking my regex on https://regexr.com/ and also on https://fluentular.herokuapp.com/

According to both websites, the regex I've wrote should match the syslog above, see:

I'm lost and confused, if anyone can help, I'll be thankful.

Thanks in advance,

Renan

[Kentaro Hayashi]

Hi,

> time_format %Y-%m-%dT%H:%M:%SZ"

It seems that time_format contains an extra double quote ("),  isn't it?

Regards,

2021年2月4日木曜日 9:12:30 UTC+9 renan...@gmail.com:

[Mr. Fiber]

In addition, you don't need priority parts in expression because in_syslog removes <pri> parts and passes other parts to parser.

See official article: https://docs.fluentd.org/input/syslog#less-than-parse-greater-than-directive

--
You received this message because you are subscribed to the Google Groups "Fluentd Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fluentd/57cfa5b3-9561-4d31-9173-49920d7caf2fn%40googlegroups.com.

[Renan Hingel]

Ken

Thanks for the tip with the double quotes after the time format, I haven't noticed that!

Reapeatedly

Thanks for pointing that, I have tried to fiddle with those options before, they didn't work for me, but I've updated my conf with the following:

<source>
  @type syslog
  port 1470
  bind 0.0.0.0
  tag system
<parse>
  @type regexp

  with_priority true

  expression /<(?<priority>[0-9]+)>(?<time>[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z) (?<host>[^ ]*) : (?<id>[^ ]*) (?<message>.*)/
  time_key time
  time_format %Y-%m-%dT%H:%M:%SZ
</parse>
</source>

I'm still getting these errors, see:

2021-02-04 11:17:07 +0000 [warn]: #0 failed to parse message data="<165>2021-02-04T11:17:07Z HOSTNAME : %FTD-5-111008: User 'enable_1' executed the 'pager 0' command."

[Mr. Fiber]

I said remove <(?<priority>[0-9]+)> from expression.

See my link. It shows actual example.

To view this discussion on the web visit https://groups.google.com/d/msgid/fluentd/6b84a1b7-90e8-4b25-b14c-804949495826n%40googlegroups.com.

[Renan Hingel]

Removing the priority from regex did the trick!

Thanks!

You received this message because you are subscribed to a topic in the Google Groups "Fluentd Google Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/fluentd/1y1srXM0YFw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to fluentd+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fluentd/CADJXRfbRuAdJpLAK3pXFjz_CyJqc8xqBG7bTs78h_b5EDghC%3DQ%40mail.gmail.com.