Decoder/Rule for MongoDB in Syslog format not working

[Phạm Q. Đạt]

Hi Wazuh team, 

I wrote a rule to generate alerts for the below type of MongoDB log:

2022-08-30T23:30:00.944+0700 I ACCESS   [conn3100] SASL SCRAM-SHA-1 authentication failed for user on admin from client 192.168.1.11:34212 ; UserNotFound: 

Could not find user user@admin

You can get more other logs in the attached file.

The problem is these logs aren't extracted properly in the Pre-Decoder phase. We can't change this phase behavior so we have to change the log type that will be transmitted to our Wazuh. Here is our solution for that: 

1. Edit the agent.conf file to change the format of the log transmitted to our Wazuh so that the Pre-decoder can extract it correctly.

 <agent_config> 

 <!-- Shared agent configuration here --> <localfile> 

     <log_format>syslog</log_format> 

     <location>/var/log/mongodb/mongo.log</location>

     <out_format>$(timestamp) $(hostname) mongosyslog: $(log)</out_format> </localfile> </agent_config>

The idea is to put all of the original logs into the last field in Syslog format so that we can write a custom decoder/rule for that.

We expected the log transmitted to our Wazuh will look like this:

Apr 30 23:30:00 192.168.1.1 mongosyslog: 2022-08-30T23:30:00.944+0700 I ACCESS   [conn3100] SASL SCRAM-SHA-1 authentication failed for user on admin from client 192.168.1.11:34212 ; UserNotFound: 

Could not find user user@admin

2. With the above log, we write decoder/rule for MongoDB named mongosyslog:

decoder: 00060-mongosyslog.xml

Rule: 00060-mongosyslog-rule.xml

I attached those files below.

Tested with Wazuh-logtest shows that it works without any issues. 

3. The wazuh-logtest runs well so we tested it in our environment but it seems not working.

As you can see in the above images, it seems like the logs ain't decoded properly with mongosyslog so the alert is a Syslog alert. 

The weird thing is that when I copied the full log of the above alert and then pasted it to logtest. It works properly. 

We debugged for days but can't find any clues so I posted the problem here and hope you got the answer for that. 

We will also so appreciate it if you could take you time to inspect and optimize our decoder/rule.

Thank you.

[Christian Borla]

Hi !
I hope you are doing fine!! 
I will try to run your example, it will take me a while, I will answer you as soon as possible.
Regards.

[Christian Borla]

Hi! 
I did some test and I found that yours custom rules and decoders works on local_decoder.xml and local_rules.xml files.
As first test I Included the deceoder and rule files in each respective paths,  then I restarted wazuh and found this WARNING message. Category was not found. Invalid 'category'. Rule '100100' will be ignored.

Warning message
# /var/ossec/bin/wazuh-control start
2022/12/26 14:22:29 wazuh-analysisd: WARNING: (7611): Category was not found. Invalid 'category'. Rule '100100' will be ignored.
2022/12/26 14:22:29 wazuh-analysisd: WARNING: (7617): Signature ID '100100' was not found and will be ignored in the 'if_sid' option of rule '100101'.
2022/12/26 14:22:29 wazuh-analysisd: WARNING: (7619): Empty 'if_sid' value. Rule '100101' will be ignored.
2022/12/26 14:22:29 wazuh-analysisd: WARNING: (7617): Signature ID '100100' was not found and will be ignored in the 'if_sid' option of rule '100102'.
2022/12/26 14:22:29 wazuh-analysisd: WARNING: (7619): Empty 'if_sid' value. Rule '100102' will be ignored.
2022/12/26 14:22:29 wazuh-analysisd: WARNING: (7617): Signature ID '100100' was not found and will be ignored in the 'if_sid' option of rule '100103'.
2022/12/26 14:22:29 wazuh-analysisd: WARNING: (7619): Empty 'if_sid' value. Rule '100103' will be ignored.
2022/12/26 14:22:29 wazuh-analysisd: WARNING: (7617): Signature ID '100100' was not found and will be ignored in the 'if_sid' option of rule '100104'.
2022/12/26 14:22:29 wazuh-analysisd: WARNING: (7619): Empty 'if_sid' value. Rule '100104' will be ignored.
2022/12/26 14:22:29 wazuh-analysisd: WARNING: (7617): Signature ID '100100' was not found and will be ignored in the 'if_sid' option of rule '100105'.
2022/12/26 14:22:29 wazuh-analysisd: WARNING: (7619): Empty 'if_sid' value. Rule '100105' will be ignored.
2022/12/26 14:22:29 wazuh-analysisd: WARNING: (7617): Signature ID '100105' was not found and will be ignored in the 'if_sid' option of rule '100106'.
2022/12/26 14:22:29 wazuh-analysisd: WARNING: (7619): Empty 'if_sid' value. Rule '100106' will be ignored.
2022/12/26 14:22:29 wazuh-analysisd: WARNING: (7617): Signature ID '100105' was not found and will be ignored in the 'if_sid' option of rule '100107'.
2022/12/26 14:22:29 wazuh-analysisd: WARNING: (7619): Empty 'if_sid' value. Rule '100107' will be ignored.
2022/12/26 14:22:29 wazuh-analysisd: WARNING: (7617): Signature ID '100105' was not found and will be ignored in the 'if_sid' option of rule '100108'.
2022/12/26 14:22:29 wazuh-analysisd: WARNING: (7619): Empty 'if_sid' value. Rule '100108' will be ignored.
2022/12/26 14:22:29 wazuh-analysisd: WARNING: (7617): Signature ID '100105' was not found and will be ignored in the 'if_sid' option of rule '100109'.
2022/12/26 14:22:29 wazuh-analysisd: WARNING: (7619): Empty 'if_sid' value. Rule '100109' will be ignored.
2022/12/26 14:22:29 wazuh-analysisd: WARNING: (7620): Signature ID '100109' was not found. Invalid 'if_matched_sid'.Rule '100110' will be ignored.
2022/12/26 14:22:29 wazuh-analysisd: WARNING: (7617): Signature ID '100105' was not found and will be ignored in the 'if_sid' option of rule '100111'.
2022/12/26 14:22:29 wazuh-analysisd: WARNING: (7619): Empty 'if_sid' value. Rule '100111' will be ignored.

I have not yet analyzed it in depth, but the file looks good, so I decided to include the decoders and rules in the local_decoder.xml and local_rules.xml files respectively. After that I tested on wazuh-logtest and works.

Running on wazuh-logtest tool

# /var/ossec/bin/wazuh-logtest
Type one log per line

Apr 30 23:30:00 192.168.1.1 mongosyslog: 2022-08-30T23:30:00.946+0700 I ACCESS   [conn3099] SASL SCRAM-SHA-1 authentication failed for user on admin from client 192.168.1.11:40128 ; UserNotFound: Could not find user user@admin

**Phase 1: Completed pre-decoding.
    full event: 'Apr 30 23:30:00 192.168.1.1 mongosyslog: 2022-08-30T23:30:00.946+0700 I ACCESS   [conn3099] SASL SCRAM-SHA-1 authentication failed for user on admin from client 192.168.1.11:40128 ; UserNotFound: Could not find user user@admin'
    timestamp: 'Apr 30 23:30:00'
    hostname: '192.168.1.1'
    program_name: 'mongosyslog'

**Phase 2: Completed decoding.
    name: 'mongosyslog'
    component: 'ACCESS'
    msg: '[conn3099] SASL SCRAM-SHA-1 authentication failed for user on admin from client 192.168.1.11:40128 ; UserNotFound: Could not find user user@admin'
    severity: 'I'
    srcip: '192.168.1.11'

**Phase 3: Completed filtering (rules).
    id: '100109'
    level: '4'
    description: 'MongoDB: Failed authentication'
    groups: '['local', 'syslog', 'sshd', 'authentication_failed']'
    firedtimes: '1'
    gdpr: '['IV_35.7.d', 'IV_32.2']'
    gpg13: '['7.1']'
    hipaa: '['164.312.b']'
    mail: 'False'
    nist_800_53: '['AU.14', 'AC.7']'
    pci_dss: '['10.2.4', '10.2.5']'
    tsc: '['CC6.1', 'CC6.8', 'CC7.2', 'CC7.3']'
**Alert to be generated.

Finally I decide to include the log in a sample file in an Agent, to simulate the data collection, and alerts were generated as expected.

Agent configuration|

  <localfile>
    <location>C:\Users\test.txt</location>
    <log_format>syslog</log_format>
  </localfile>

I pasted the example log in the test.txt file, including an enter space.

And look for the alert in alert.json file in manager side.

/var/ossec/logs/alerts/alerts.json

{"timestamp":"2022-12-26T14:09:35.772-0300","rule":{"level":4,"description":"MongoDB: Failed authentication","id":"100109","firedtimes":2,"mail":false,"groups":["local","syslog","sshd","authentication_failed"],"pci_dss":["10.2.4","10.2.5"],"gpg13":["7.1"],"gdpr":["IV_35.7.d","IV_32.2"],"hipaa":["164.312.b"],"nist_800_53":["AU.14","AC.7"],"tsc":["CC6.1","CC6.8","CC7.2","CC7.3"]},"agent":{"id":"001","name":"DESKTOP-U8OHD3A","ip":"2803:9800:9882:B7A5:5E87:5DE6:650F:1A7F"},"manager":{"name":"VBox"},"id":"167575.46967","full_log":"Apr 30 23:30:00 192.168.1.1 mongosyslog: 2022-08-30T23:30:00.946+0700 I ACCESS   [conn3099] SASL SCRAM-SHA-1 authentication failed for user on admin from client 192.168.1.11:40128 ; UserNotFound: Could not find user user@admin","predecoder":{"program_name":"mongosyslog","timestamp":"Apr 30 23:30:00","hostname":"192.168.1.1"},"decoder":{"name":"mongosyslog"},"data":{"srcip":"192.168.1.11","severity":"I","component":"ACCESS","msg":"[conn3099] SASL SCRAM-SHA-1 authentication failed for user on admin from client 192.168.1.11:40128 ; UserNotFound: Could not find user user@admin"},"location":"C:\\Users\\test.txt"}

Let me know if that works for you.
Regards.

[Christian Borla]

Hi, 
I renamed the rules and decoder files and it works.

# ls -l /var/ossec/etc/rules/
total 8
-rw-rw---- 1 wazuh wazuh  498 dic 26 14:53 local_rules.xml
-rwxrwxrwx 1 root  root  3563 dic 26 13:47 mongosyslog-rule.xml

# ls -l /var/ossec/etc/decoders/
-rw-rw---- 1 wazuh wazuh 1246 dic 26 14:05 local_decoder.xml
-rwxrwxrwx 1 root  root   428 dic 26 10:26 mongosyslog.xml

There a is a Default rule 0060-cisco-estreamer_decoders.xml maybe that is the problem.
It would be great if you can open an issue to our repo here https://github.com/wazuh/wazuh/issues to investigate and consider the issue.
Regards!

[Phạm Q. Đạt]

Hi,

The log that you pasted seems not the right one.

The original logs are attached above, could you try to paste all of these logs into your test.txt. 

That's the reason why we have to add the field <out_format> to the agent.conf.

<out_format>$(timestamp) $(hostname) mongosyslog: $(log)</out_format>

Hope to see your advice.

Thank you!

[Christian Borla]

Hi! 
You are right! I tested it without the out_format option but I simulate that configuration in the message.
bytheway I simulated your full configuration, I changed the localfile configuration to following:

  <localfile>
    <location>C:\Users\asus\test.txt</location>
    <log_format>syslog</log_format>

    <out_format>$(timestamp) $(hostname) mongosyslog: $(log)</out_format>
  </localfile>

Then copyed 2 original events in test file:
2022-08-30T23:31:00.912+0700 I NETWORK  [listener] connection accepted from 192.168.1.11:34944 #3102 (4 connections now open)
2022-08-30T23:31:00.912+0700 I NETWORK  [listener] connection accepted from 192.168.1.11:34944 #3102 (4 connections now open)

And found them on /var/ossec/logs/alerts/alerts.json

{"timestamp":"2022-12-28T08:07:02.594-0300","rule":{"level":3,"description":"MongoDB: Connection accepted","id":"100106","firedtimes":1,"mail":false,"groups":["mongodb"]},"agent":{"id":"001","name":"DESKTOP","ip":"2803:9800:9882:B7A5:5E66:5D66:650F:1A7F"},"manager":{"name":"VBox"},"id":"16225622.12544","full_log":"Dec 28 08:07:02 DESKTOP mongosyslog: 2022-08-30T23:31:00.912+0700 I NETWORK  [listen
er] connection accepted from 192.168.1.11:34944 #3102 (4 connections now open)","predecoder":{"program_name":"mongosyslog","timestamp":"Dec 28 08:07:02","hostname":"DESKTOP"},"decoder":{"name":"mongosysl
og"},"data":{"srcip":"192.168.1.11","severity":"I","component":"NETWORK","msg":"[listener] connection accepted from 192.168.1.11:34944 #3102 (4 connections now open)"},"location":"C:\\Users\\test.txt"}

{"timestamp":"2022-12-28T08:07:02.594-0300","rule":{"level":3,"description":"MongoDB: Connection accepted","id":"100106","firedtimes":2,"mail":false,"groups":["mongodb"]},"agent":{"id":"001","name":"DESKTOP","ip":"2803:9800:9882:B766:5E66:5D66:650F:1A7F"},"manager":{"name":"VBox"},"id":"16225622.13021","full_log":"Dec 28 08:07:02 DESKTOP mongosyslog: 2022-08-30T23:31:00.912+0700 I NETWORK  [listener] connection accepted from 192.168.1.11:34944 #3102 (4 connections now open)","predecoder":{"program_name":"mongosyslog","timestamp":"Dec 28 08:07:02","hostname":"DESKTOP"},"decoder":{"name":"mongosyslog"},"data":{"srcip":"192.168.1.11","severity":"I","component":"NETWORK","msg":"[listener] connection accepted from 192.168.1.11:34944 #3102 (4 connections now open)"},"location":"C:\\Users\\test.txt"}

Did you try renaming the rules and decoders files? or moving the rules to loca_rules  and decoders to loca_decoder files?
Regards.

[Phạm Q. Đạt]

Hi, 

It turns out that we have to restart our Wazuh to make it work.

We really appreciate your work. 

Thanks for supporting us.