Regex Not Matching for Decoder

[Stephen Hill]

Hi All,

I'm a bit new here but thought someone might be able to help.

I write custom rules a fair bit for OSSEC /Wazuh so I am used to using regex matching. However I've started to write a new decoder which seems to be causing me a lot of trouble.

The decoder is as below:

<decoder name="vshell">

  <prematch>^\d+-\d+-\d+\s\d+:\d+:\d+\sVShellSSH2\s\w+</prematch>

</decoder>

<decoder name="vshell-accepted">

  <parent>vshell</parent>

  <prematch offset="after_parent">^ \d+ (\d+.\d+.\d+.\d+) (\d+) \D+ \d+.\d+.\d+.\d+ \D+\d+\D \w+ </prematch>

  <regex offset="after_prematch">^accepted \w+ (\d+.\d+.\d+.\d+)\D(\d+)</regex>

  <order>srcip, srcport</order>

</decoder>

However when testing this I only match the parent decoder not the 'vshell-accepted' decoder afterwards:

2017-05-22 00:05:25 VShellSSH2 conn 35046 213.86.218.94 61461 - - - 0 0 0 0 - - "35046: Connection accepted from 213.86.218.94:61461"

**Phase 1: Completed pre-decoding.

       full event: '2017-05-22 00:05:25 VShellSSH2 conn 35046 213.86.218.94 61461 - - - 0 0 0 0 - - "35046: Connection accepted from 213.86.218.94:61461"'

       hostname: 'INF-MGTOSSEC02'

       program_name: '(null)'

       log: '2017-05-22 00:05:25 VShellSSH2 conn 35046 213.86.218.94 61461 - - - 0 0 0 0 - - "35046: Connection accepted from 213.86.218.94:61461"'

**Phase 2: Completed decoding.

       decoder: 'vshell'

I've tested the syntax in regex101.com using the python flavour and it matches correctly.

Any chance someone can spot what I'm doing wrong?

Cheers,

Steve

[Jesus Linares]

Hi Stephen,

using Wazuh ruleset, I had to create a child decoder of windows-date-format:

<decoder name="vshell-accepted">
  <parent>windows-date-format</parent>
  <prematch>VShellSSH2 </prematch>
  <regex offset="after_prematch">accepted from (\S+):(\S+)</regex>

  <order>srcip, srcport</order>
</decoder>

 ossec-logtest

**Phase 2: Completed decoding.
       decoder: 'windows-date-format'
       srcip: '213.86.218.94'
       srcport: '61461"'

Regarding to your decoder:

  <prematch offset="after_parent">^ \d+ (\d+.\d+.\d+.\d+) (\d+) \D+ \d+.\d+.\d+.\d+ \D+\d+\D \w+ </prematch>
  <regex offset="after_prematch">^accepted \w+ (\d+.\d+.\d+.\d+)\D(\d+)</regex>

prematch only allows sregex, so you can't use \d or other regular expression.

I hope it helps.

[Stephen Hill]

I just gave this a go and as you say works like a charm!

So, this looks like I just don't fully understand the decoder process, when I entered the log in before creating the custom parent 'vshell' I saw that it matched 'windows-date-format'.

Does this have to be a child of this decoder due to pre-decoding? Are there a hard set of decoders that can be matched by pre-decoding?

Thanks for the help on this, really appreciated! :) 

[Jesus Linares]

Hi,

the problem here is that the windows-date-format decoder is too generic:

https://github.com/wazuh/wazuh-ruleset/blob/master/decoders/0380-windows_decoders.xml#L16

<decoder name="windows-date-format">
  <prematch>^\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d </prematch>
</decoder>

So it is capturing your vshell log. That is the reason to create a child decoder of windows-date-format.

Regards.

[Stephen Hill]

Hi Jesus,

Thanks again for explaining it further, I've spent some time today making rules and adding to the decoder some other bits we needed for our own purposes.

I'm not sure if these would be useful to the ruleset:

Decoders:

<decoder name="vshell-connection">

  <parent>windows-date-format</parent>

  <prematch>VShellSSH2 dbg </prematch>

  <regex offset="after_prematch">(\d+.\d+.\d+.\d+) rejected by Deny Hosts file </regex>

  <order>srcip</order>

</decoder>

<decoder name="vshell-accepted">

  <parent>windows-date-format</parent>

  <prematch>VShellSSH2 conn </prematch>

  <regex offset="after_prematch">accepted from (\S+):(\S+)</regex>

  <order>srcip, srcport</order>

</decoder>

<decoder name="vshell-logon">

  <parent>windows-date-format</parent>

  <prematch>VShellSSH2 auth </prematch>

  <regex offset="after_prematch">Authentication for (\S+) failed|password for user (\S+) accepted</regex>

  <order>dstuser</order>

</decoder>

And the associated rules:

<group name="vshell,">

  <rule id="100313" level="0">

    <decoded_as>windows-date-format</decoded_as>

    <match>VShellSSH2</match>

    <description>VShell message grouped.</description>

  </rule>

  <rule id="100314" level="3">

    <if_sid>100313</if_sid>

    <match>Connection accepted from</match>

    <description>VShell connection attempt successful</description>

  </rule>

  <rule id="100315" level="5">

    <if_sid>100313</if_sid>

    <regex>Login failed|Authentication for (\w+) failed</regex>

    <description>VShell user failed to login or user does not exist</description>

  </rule>

  <rule id="100316" level="7">

    <if_sid>100315</if_sid>

    <regex>Maximum authentication retries for user (\w+) exceeded</regex>

    <description>VShell user used the maximum number of password attempts.</description>

  </rule>

  <rule id="100317" level="10">

    <if_sid>100313</if_sid>

    <regex>Connection from (\d+.\d+.\d+.\d+) rejected by Deny Hosts file</regex>

    <description>Host is trying to connect to VShell server but exists in the deny file.</description>

  </rule>

  <rule id="100318" level="3">

    <if_sid>100313</if_sid>

    <regex>password for user (\S+) accepted</regex>

    <description>VShell user successfully authenticated.</description>

  </rule>

  <rule id="100330" level="12" frequency="5" timeframe="120">

    <if_matched_sid>100317</if_matched_sid>

    <description>VShell multiple connection attempts within 2 minute by a host in the deny file, potential DOS or brute force attempt.</description>

  </rule>

  <rule id="100331" level="12" frequency="5" timeframe="60">

    <if_matched_sid>100315</if_matched_sid>

    <description>VShell host has exceeded the number of failed login attempts and has been added to the Hosts Deny file.</description>

  </rule>

</group>

Hope it's of some use and thanks again for all the help.

Steve

[Jesus Linares]

Hi Stephen,

you did a great job!. Could you send a pull request to wazuh-ruleset repository?. It will be very useful to the community.

Regards.

[Stephen Hill]

Hi Jesus,

Thanks! I've submitted the pull requests and added some comments to the decoders for example log entries. :)

Steve

[Jesus Linares]

It is accepted. We need more contributions like this one.

Thanks!