Only first decoder works

[Nauris Metlans]

Hello,

trying to make decoder for mikrotik and for now it looks like this:

<var name="hostname">NoName</var>

<decoder name="mikrotik">
<prematch>$hostname</prematch>
</decoder>


<decoder name="mikrotik_winbox_login_fail">
<parent>mikrotik</parent>
<regex>^\S+\s+(\S+\s+\d+\s+\S+)\s+\S+\s+login failure for user (\S+) from (\S+)</regex>
<order>time,user,srcip</order>
</decoder>

<decoder name="mikrotik_winbox_logout">
<parent>mikrotik</parent>
<regex>^\S+\s+(\S+\s+\d+\s+\S+)\s+\S+ user (\S+) logged out from (\S+)</regex>
<order>time,user,srcip</order>
</decoder>

<decoder name="mikrotik_winbox_login">
<parent>mikrotik</parent>
<regex>^\S+\s+(\S+\s+\d+\s+\S+)\s+\S+\s+user (\S+) logged in from (\S+)</regex>
<order>time,user,srcip</order>
</decoder>

The problem is that first child decoder only works. Whichever I put first is decoded, but two other are not decoded.

When I copy first child decoder in the end like this:

<var name="hostname">NoName</var>

<decoder name="mikrotik">
    <prematch>$hostname</prematch>
</decoder>


<decoder name="mikrotik_winbox_login_fail">
    <parent>mikrotik</parent>
    <regex>^\S+\s+(\S+\s+\d+\s+\S+)\s+\S+\s+login failure for user (\S+) from (\S+)</regex>
    <order>time,user,srcip</order>
</decoder>

<decoder name="mikrotik_winbox_logout">
    <parent>mikrotik</parent>
    <regex>^\S+\s+(\S+\s+\d+\s+\S+)\s+\S+ user (\S+) logged out from (\S+)</regex>
    <order>time,user,srcip</order>
</decoder>

<decoder name="mikrotik_winbox_login">
    <parent>mikrotik</parent>
    <regex>^\S+\s+(\S+\s+\d+\s+\S+)\s+\S+\s+user (\S+) logged in from (\S+)</regex>
    <order>time,user,srcip</order>
</decoder>

<decoder name="mikrotik_winbox_login_fail">
    <parent>mikrotik</parent>
    <regex>^\S+\s+(\S+\s+\d+\s+\S+)\s+\S+\s+login failure for user (\S+) from (\S+)</regex>
    <order>time,user,srcip</order>
</decoder>

Then first two child decoders works.

I can't understand, is there problem with my decoders or wazuh-logtest tool.

When I had only child decoders - mikrotik_winbox_logout and mikrotik_winbox_login - then both worked, to me now it looks like it's randomly working and randomly choosing which decoders work.

[Nauris Metlans]

And if I copy mikrotik_windox_logout child decoder to the end, like this:

<var name="hostname">NoName</var>

<decoder name="mikrotik">
    <prematch>$hostname</prematch>
</decoder>


<decoder name="mikrotik_winbox_login_fail">
    <parent>mikrotik</parent>
    <regex>^\S+\s+(\S+\s+\d+\s+\S+)\s+\S+\s+login failure for user (\S+) from (\S+)</regex>
    <order>time,user,srcip</order>
</decoder>

<decoder name="mikrotik_winbox_logout">
    <parent>mikrotik</parent>
    <regex>^\S+\s+(\S+\s+\d+\s+\S+)\s+\S+ user (\S+) logged out from (\S+)</regex>
    <order>time,user,srcip</order>
</decoder>

<decoder name="mikrotik_winbox_login">
    <parent>mikrotik</parent>
    <regex>^\S+\s+(\S+\s+\d+\s+\S+)\s+\S+\s+user (\S+) logged in from (\S+)</regex>
    <order>time,user,srcip</order>
</decoder>

<decoder name="mikrotik_winbox_login_fail">
    <parent>mikrotik</parent>
    <regex>^\S+\s+(\S+\s+\d+\s+\S+)\s+\S+\s+login failure for user (\S+) from (\S+)</regex>
    <order>time,user,srcip</order>
</decoder>

<decoder name="mikrotik_winbox_logout">
    <parent>mikrotik</parent>
    <regex>^\S+\s+(\S+\s+\d+\s+\S+)\s+\S+ user (\S+) logged out from (\S+)</regex>
    <order>time,user,srcip</order>
</decoder>

The all three decoders are working. Can anybody spot the problem?

[Nauris Metlans]

The logs that I check with  wazuh-logtest tool are:

192.168.0.1 Jun 10 14:51:14 NoName user admin logged out from 192.168.0.204 via winbox
192.168.0.1 Jun 10 14:51:45 NoName user admin logged in from 192.168.0.204 via winbox
192.168.0.1 Jun 10 15:51:05 NoName login failure for user admin from 192.168.0.204 via winbox 

[Luis González Romero]

Hello! Hope you are doing great.

Sorry for the late response. I am currently working on your issue.

[Luis González Romero]

Hello again, I found out that it is only using the first decoder because the prematch is equal between them.

Here you have three workarounds for avoid this:

• Using your decoders, you can add the prematch as follows:

  <decoder name="mikrotik_winbox_login_fail">
  <prematch>login failure for</prematch>
  <parent>mikrotik</parent>
  <regex>^\S+\s+(\S+\s+\d+\s+\S+)\s+\S+\s+login failure for user (\S+) from (\S+)</regex>
  <order>time,user,srcip</order>
  </decoder>
  
  <decoder name="mikrotik_winbox_logout">
  <prematch>logged out</prematch>
  <parent>mikrotik</parent>
  <regex>^\S+\s+(\S+\s+\d+\s+\S+)\s+\S+ user (\S+) logged out from (\S+)</regex>
  <order>time,user,srcip</order>
  </decoder>
  
  <decoder name="mikrotik_winbox_login">
  <prematch>logged in</prematch>
  <parent>mikrotik</parent>
  <regex>^\S+\s+(\S+\s+\d+\s+\S+)\s+\S+\s+user (\S+) logged in from (\S+)</regex>
  <order>time,user,srcip</order>
  </decoder>
  

• Fix the log format(syslog) by forwarding the log to a file and creating new decoders:

  • Store the logs in a plaintext file. I simulated this step but you can follow this guide about forwarding syslog events to a file.

  • Monitor that file using a localfile block in your ossec.conf file. I added the timestamp, so now it has the syslog timestamp, NoName as hostname, and mikrotik as program_name.

    <localfile>
    <log_format>syslog</log_format>
    <location>/var/log/mikrotik.log</location>
    <out_format>$(timestamp) NoName mikrotik: $(log)</out_format>
    </localfile>
    

  • Now, you will have logs like these within your alerts log files:

    Jun 13 09:16:43 NoName mikrotik: 192.168.0.1 Jun 10 15:51:05 NoName login failure for user admin from 192.168.0.204 via winbox
    Jun 13 09:16:43 NoName mikrotik: 192.168.0.1 Jun 10 15:51:05 NoName user admin logged out from 192.168.0.204 via winbox
    Jun 13 09:16:43 NoName mikrotik: 192.168.0.1 Jun 10 15:51:05 NoName user admin logged in from 192.168.0.204 via winbox
    

  • Then, you can add these decoders(similars to yours)

    <decoder name="mikrotik_winbox_logout">
      <program_name>mikrotik</program_name>
      <prematch>logged out</prematch>
      <regex>user (\S+) logged out from (\d+.\d+.\d+.\d+)</regex>
      <order>user, srcip</order>
    </decoder>
    
    <decoder name="mikrotik_winbox_login">
      <program_name>mikrotik</program_name>
      <prematch>logged in</prematch>
      <regex>user (\S+) logged in from (\d+.\d+.\d+.\d+)</regex>
      <order>user, srcip</order>
    </decoder>
    
    <decoder name="mikrotik_winbox_login_fail">
      <program_name>mikrotik</program_name>
      <prematch>login failure for</prematch>
      <regex offset="after_prematch">user (\S+) from (\d+.\d+.\d+.\d+)</regex>
      <order>user, srcip</order>
    </decoder>
    

• Same as previous one, but using a generic decoder(logic would be added with custom rules):

  You have to follow the previous workaround to have the correct syslog format and then you can add this decoder:

  <decoder name="mikrotik_winbox_generic">
      <program_name>mikrotik</program_name>
      <regex>user (\S+) \.*from (\d+.\d+.\d+.\d+)</regex>
      <order>user, srcip</order>
  </decoder>
  

Hope this helps you,
Luis.

​

[Luis González Romero]

Sorry, I messed it up with the forwarding logs guide.

Here you have the link.

Have a nice day!

​

[Nauris Metlans]

Ok, thank you for that, I understood my mistake. Found that if I look at archives.log then the log is like this:

192.168.0.1 Jun 10 15:51:05 NoName login failure for user nauris from 192.168.0.204 via winbox

But in wazuh UI, the log actually looks like this:

Jun 10 15:51:05 NoName login failure for user nauris from 192.168.0.204 via winbox

So I changed my decoder accordingly as you write and without IP address regex to:

<var name="hostname">NoName</var>

<decoder name="mikrotik">
    <prematch>$hostname</prematch>
</decoder>

<decoder name="mikrotik_winbox_failed_login">

    <prematch>login failure for</prematch>
    <parent>mikrotik</parent>

    <regex>^(\S+\s+\d+\s+\S+) \S+ login failure for user (\S+) from (\S+)</regex>
    <order>time,user,srcip</order>
</decoder>

And again decoder is not working and results are:

**Phase 1: Completed pre-decoding.
        full event: 'Jun 10 15:51:05 NoName login failure for user nauris from 192.168.0.204 via winbox'
        timestamp: 'Jun 10 15:51:05'
        hostname: 'NoName'

**Phase 2: Completed decoding.
        No decoder matched.

[Nauris Metlans]

Also interesting thing is if I add something to log without IP addres, it my be space or anything, then log is being decoded.

But problem is that in wazuh it comes without anything in front of it:

Jun 10 15:51:05 NoName login failure for user nauris from 192.168.0.204 via winbox

Looks like again I am missing something.

[Luis González Romero]

Okay, with this info we have a few things:

1. The log is already in a valid format:

1.  Jun 10 15:51:05 NoName login failure for user nauris from 192.168.0.204 via winbox
   

1. So timestamp and in your case hostname is obtained by default even with no decoder as you could see:

1.  **Phase 1: Completed pre-decoding.
        full event: 'Jun 10 15:51:05 NoName login failure for user nauris from 192.168.0.204 via winbox'
        timestamp: 'Jun 10 15:51:05'
        hostname: 'NoName'
   

1. There is no need to get the time as you are trying because with the valid format you have the timestamp as I said in the previous message.

2. It is failing because of the parent mikrotik decoder. Imagine this new log formatted as the out_format I provided before, <out_format>$(timestamp) NoName mikrotik: $(log)</out_format>. So, now you have the timestamp, the hostname(in your case the program name is missing because of your log), and the log. You can’t check for NoName.

   You can just use the prematch label, removing the parent label. Then, you have the same decoder as I provided you(replacing the IP format with "all characters before the next space" as you have):

    <decoder name="mikrotik_winbox_failed_login">
      <prematch>login failure for</prematch>
      <regex>user (\S+) from (\S+)</regex>
      <order>user, srcip</order>
    </decoder>
   

    Jun 10 15:51:05 NoName login failure for user nauris from 192.168.0.204 via winbox
   
    **Phase 1: Completed pre-decoding.
        full event: 'Jun 10 15:51:05 NoName login failure for user nauris from 192.168.0.204 via winbox'
        timestamp: 'Jun 10 15:51:05'
        hostname: 'NoName'
   
    **Phase 2: Completed decoding.
        name: 'mikrotik_winbox_failed_login'
        dstuser: 'nauris'
        srcip: '192.168.0.204'
   

Hope this helps you,
Luis.

​

[Nauris Metlans]

Ok, thank you, now it works. But I am a bit confused now about this parent decoder thing, what is a purpose of parent decoder and does it changes your incoming log.

In my case does it mean if I have parent like:

<decoder name="mikrotik">
    <prematch>NoName</prematch>
</decoder>

that it will remove from log:  Jun 10 15:51:05 NoName login failure for user nauris from 192.168.0.204 via winbox

Jun 10 15:51:05 NoName part?

And if I am making custom log decoder, then first I need to just paste the log in wazuh-logtest tool, see what it finds and removes in pre-decoding fase and then work with remnants of log, because pre-decoder removes the parts from log that it has decoded?

[Luis González Romero]

If you mean the log format, you can check more here.

For example, you could have this: HEADER (timestamp host) MSG (application: message),  where you can have the hostname(host) and the program_name(application). Or the one you provided the last, where the header is composed of the timestamp and header, and then you have the message. You can customize it using the out_format within the ossec.conf as I said in the first message.

If you do not know if it follows the default format expected, yes. You can paste it into the logtest tool and check if the timestamp is correctly decoded. If it is not the case, you can customize it with the out_format so you can start with a proper log to being decoded.

Do not hesitate if you have any doubts or anything else!

Luis.

​

[Nauris Metlans]

So does this means that if incoming log is not in syslog default format, I always need to  Monitor that file using a localfile and customize it using out_format?

[Luis González Romero]

If you want to have the timestamp, hostname, program_name, etc. by default without creating any regex, you should, yes. This is suggested because you do not need to waste time creating custom regexes for catching these fields in custom logs. Also, these fields are useful when you are creating a decoder. For example, this allows you to use the field program_name as I did in the first message: <program_name>mikrotik</program_name>.

The out_format provides you the timestamp in two ways:

• When you use timestamp -> Current timestamp (when the log is sent), in RFC3164 format.
• When you use timestamp <format> -> Custom timestamp, in strftime string format.

Here you have more about formatting the logs fields.

Hope this helps you,
Luis.

​

[Nauris Metlans]

So to clarify my knowledge about custom rules.

If I had a log like this: Jun 10 15:51:05 NoName login failure for user admin from 192.168.0.204 via winbox

1. Then first thing I need to do is input this log in wazuh-logtest.

The logtest shows:

**Phase 1: Completed pre-decoding.
        full event: 'Jun 10 15:51:05 NoName login failure for user admin from 192.168.0.204 via winbox'

        timestamp: 'Jun 10 15:51:05'
        hostname: 'NoName'

2. so I know that my custom decoder will only work with log part because pre-decoder already removed timestamp and hostname from starting log entry (is this correct):

login failure for user admin from 192.168.0.204 via winbox

and accordingly make my decoder like:

<decoder name="mikrotik_winbox_generic">
    <prematch>\.*from \.*</prematch>

    <regex>user (\S+) \.*from (\d+.\d+.\d+.\d+)</regex>
    <order>user, srcip</order>
</decoder>

and everything works?

[Luis González Romero]

Yes, that could be a workaround.

Whatever you need, do not hesitate and ask us within our community channels.

Have a nice day,

Luis.