Custom decoder for Mikrotik firewall logs. Space makes a difference?

108 views

Skip to first unread message

nullByte

unread,

Jun 13, 2024, 8:10:46 AM6/13/24

to Wazuh | Mailing List

Hello all,

I am ingesting some Mikrotik firewall logs into Wazuh, and have written a custom decoder.

When testing with logtest, it only works if I put a space in front of it?
I am missing something obvious here, but just cannot see it.

Example log:

Jun 13 10:27:12 hex.home.arpa mikrotik-fw: FD_INVALID forward: in:vlan16 out:ether1, connection-state:invalid src-mac cc:2d:e0:ff:de:23, proto TCP (RST), 10.10.10.22:38512->157.240.9.18:443, prio 1->0, len 40

Decoder:

<decoder name="mikrotik">
<prematch>mikrotik-fw</prematch>
</decoder>
<decoder name="mikrotik-fw">
<parent>mikrotik</parent>
<regex type="pcre2"> (\w+[ ]*\d+ \d+:\d+:\d+) hex.home.arpa (mikrotik-fw): (\S+) (\w+): in:([^\s]+) out:(.+), (.+)? src-mac ([^,]+), proto (.+), (\d+.\d+.\d+.\d+):?(\d+)?->(\d+.\d+.\d+.\d+):?(\d+,)? (prio \S+)?, len (\d+)</regex>
<order>timestamp, program_name, mkt_fw_rule, mkt_chain, mkt_iface_in, mkt_iface_out, connstate, srcmac, proto, srcip, srcport, dstip, dstport, mkt_prio, mkt_len</order>
</decoder>

Test (second log paste is with a space in front):

root@wazuh:/home/dalibor# /var/ossec/bin/wazuh-logtest
Starting wazuh-logtest v4.7.5
Type one log per line

Jun 13 10:27:12 hex.home.arpa mikrotik-fw: FD_INVALID forward: in:vlan16 out:ether1, connection-state:invalid src-mac cc:2d:e0:ff:de:23, proto TCP (RST), 10.10.10.22:38512->157.240.9.18:443, prio 1->0, len 40

** Wazuh-Logtest: WARNING: (7605): It is not possible to overwrite 'if_group' value in rule '61634'. The original value is retained.
** Wazuh-Logtest: WARNING: (7605): It is not possible to overwrite 'if_group' value in rule '61638'. The original value is retained.
** Wazuh-Logtest: WARNING: (7605): It is not possible to overwrite 'if_group' value in rule '61618'. The original value is retained.
** Wazuh-Logtest: WARNING: (7605): It is not possible to overwrite 'if_group' value in rule '92058'. The original value is retained.

**Phase 1: Completed pre-decoding.
full event: 'Jun 13 10:27:12 hex.home.arpa mikrotik-fw: FD_INVALID forward: in:vlan16 out:ether1, connection-state:invalid src-mac cc:2d:e0:ff:de:23, proto TCP (RST), 10.10.10.22:38512->157.240.9.18:443, prio 1->0, len 40'
timestamp: 'Jun 13 10:27:12'
hostname: 'hex.home.arpa'
program_name: 'mikrotik-fw'

**Phase 2: Completed decoding.
No decoder matched.

Jun 13 10:27:12 hex.home.arpa mikrotik-fw: FD_INVALID forward: in:vlan16 out:ether1, connection-state:invalid src-mac cc:2d:e0:ff:de:23, proto TCP (RST), 10.10.10.22:38512->157.240.9.18:443, prio 1->0, len 40

**Phase 1: Completed pre-decoding.
full event: ' Jun 13 10:27:12 hex.home.arpa mikrotik-fw: FD_INVALID forward: in:vlan16 out:ether1, connection-state:invalid src-mac cc:2d:e0:ff:de:23, proto TCP (RST), 10.10.10.22:38512->157.240.9.18:443, prio 1->0, len 40'

**Phase 2: Completed decoding.
name: 'mikrotik'
connstate: 'connection-state:invalid'
dstip: '157.240.9.18'
dstport: '443,'
mkt_chain: 'forward'
mkt_fw_rule: 'FD_INVALID'
mkt_iface_in: 'vlan16'
mkt_iface_out: 'ether1'
mkt_len: '40'
mkt_prio: 'prio 1->0'
program_name: 'mikrotik-fw'
proto: 'TCP (RST)'
srcip: '10.10.10.22'
srcmac: 'cc:2d:e0:ff:de:23'
srcport: '38512'
timestamp: 'Jun 13 10:27:12'

^C

Thanks.

Dalibor

Pedro Nicolás Gomez

unread,

Jun 13, 2024, 9:24:12 AM6/13/24

to Wazuh | Mailing List

As you can see in the first part of the wazuh-logtest screenshot you shared, the timestamp, hostname and program_name are captured in the pre-decoding phase, while your parent decoder is trying to match "mikrotik-fw", which fails because it was already extracted in the pre-decode phase:

/home/dalibor# /var/ossec/bin/wazuh-logtest

Starting wazuh-logtest v4.7.5

Type one log per line

** Wazuh-Logtest: WARNING: (7605): It is not possible to overwrite 'if_group' value in rule '61634'. The original value is retained.

** Wazuh-Logtest: WARNING: (7605): It is not possible to overwrite 'if_group' value in rule '61638'. The original value is retained.

** Wazuh-Logtest: WARNING: (7605): It is not possible to overwrite 'if_group' value in rule '61618'. The original value is retained.

** Wazuh-Logtest: WARNING: (7605): It is not possible to overwrite 'if_group' value in rule '92058'. The original value is retained.

**Phase 1: Completed pre-decoding.

full event: 'Jun 13 10:27:12 hex.home.arpa mikrotik-fw: FD_INVALID forward: in:vlan16 out:ether1, connection-state:invalid src-mac cc:2d:e0:ff:de:23, proto TCP (RST), 10.10.10.22:38512->157.240.9.18:443, prio 1->0, len 40'

timestamp: 'Jun 13 10:27:12'

hostname: 'hex.home.arpa'

program_name: 'mikrotik-fw'

**Phase 2: Completed decoding.

No decoder matched.

On the other hand, when you add the initial blank space, nothing is captured in the pre-decoding phase, so your parent decoder does match with "mikrotik-fw" and then the child decoder can extract all the information:

**Phase 1: Completed pre-decoding.

full event: ' Jun 13 10:27:12 hex.home.arpa mikrotik-fw: FD_INVALID forward: in:vlan16 out:ether1, connection-state:invalid src-mac cc:2d:e0:ff:de:23, proto TCP (RST), 10.10.10.22:38512->157.240.9.18:443, prio 1->0, len 40'

Returning to the log without the initial blank space, having that information pre-decoded, we can use the program_name for the parent decoder. The decoders would look like this:

<program_name>mikrotik-fw</program_name>

</decoder>

<parent>mikrotik</parent>

<regex type="pcre2">(\S+) (\w+): in:([^\s]+) out:(.+), (.+)? src-mac ([^,]+), proto (.+), (\d+.\d+.\d+.\d+):?(\d+)?->(\d+.\d+.\d+.\d+):?(\d+,)? (prio \S+)?, len (\d+)</regex>

<order>mkt_fw_rule, mkt_chain, mkt_iface_in, mkt_iface_out, connstate, srcmac, proto, srcip, srcport, dstip, dstport, mkt_prio, mkt_len</order>

</decoder>

In the parent decoder

<prematch>mikrotik-fw</prematch>

was changed to

<program_name>mikrotik-fw</program_name>

And in the child decoder the part where it decodes the timestamp and program_name has been removed ( (\w+[ ]*\d+ \d+:\d+:\d+) hex.home.arpa (mikrotik-fw): )

Here is the output of wazuh-logtest with the first example log:

/home/nico/wazuh# /var/ossec/bin/wazuh-logtest

Starting wazuh-logtest v4.7.4

Type one log per line

**Phase 1: Completed pre-decoding.

timestamp: 'Jun 13 10:27:12'

hostname: 'hex.home.arpa'

program_name: 'mikrotik-fw'

**Phase 2: Completed decoding.

name: 'mikrotik'

connstate: 'connection-state:invalid'

dstip: '157.240.9.18'

dstport: '443,'

mkt_chain: 'forward'

mkt_fw_rule: 'FD_INVALID'

mkt_iface_in: 'vlan16'

mkt_iface_out: 'ether1'

mkt_len: '40'

mkt_prio: 'prio 1->0'

proto: 'TCP (RST)'

srcip: '10.10.10.22'

srcmac: 'cc:2d:e0:ff:de:23'

srcport: '38512'

I hope it helps.

Best regards,

Pedro Nicolas.

nullByte

unread,

Jun 14, 2024, 5:30:35 AM6/14/24

to Wazuh | Mailing List

Hello Pedro,

You explained it perfectly. It finally works as it should.

Thank you for taking the time to respond and fix this!

Regards,

Dalibor

Pedro Nicolás Gomez

unread,

Jun 17, 2024, 3:32:25 AM6/17/24

to nullByte, Wazuh | Mailing List

Hi Dalibor,
I am glad that the issue is resolved. Happy to help.

Best regards,

Pedro Nicolas.

--
You received this message because you are subscribed to the Google Groups "Wazuh | Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/a7d4315f-6f68-4dbe-a6b7-d61e4b27a0f1n%40googlegroups.com.

Reply all

Reply to author

Forward

0 new messages