Any new update for MongoDB Alert in JSON format

[Phạm Q. Đạt]

Hi Wazuh team,

While running Wazuh for SIEM, I've got a problem about monitoring MongoDB. 

I found that there is a pair of decoder and rule to generate alert for MongoDB log. But the problem is that the log input has to be syslog. 
Reference:

https://stackoverflow.com/questions/62406899/how-to-pull-mongodb-logs-with-wazuh-agent

Recently MongoDB new version was released which comes with JSON Log Format. It conflicts with our syslog one. I've done some research about it but I found nothing at all. 

What can I do about this issue? Or I have to write my own rules for that. 

Please let me know early. 

Thank you,

Regards.

[Abdullah Al Rafi Fahim]

Hello,

Thank you for using this community to share your query regarding Wazuh!

In Wazuh, we have some pre-built stock decoders and rules for MongoDB logs.

Decoders:  0405-mongodb_decoders.xml
Rules:  0450-mongodb_rules.xml

However, if you logs are in json format, you can collect the logs using a localfile configuration as below.

<localfile> 

      <log_format>json</log_format> 

      <location>YOUR_LOG_LOCATION</location>

</localfile> 

After that, enable the archives in the ossec.conf of the manager and verify in the archives.json if the logs are coming or not. If your logs are coming properly to the archives.log of the manager, you can verify using wazuh-logtest if the logs are being decoded properly with any of the present decoders and triggering any alerts or not. You can learn more about wazuh-logtest here: https://documentation.wazuh.com/current/user-manual/capabilities/wazuh-logtest/how-it-works.html

If these logs are not being decoded or not triggering any rules, you need to create custom decoders and rules for them to generate alerts. You can review the following documents to learn more about custom rules and decoders.

• Creating decoders and rules from scratch · Wazuh · The Open Source Security Platform
• Custom rules and decoders - Ruleset · Wazuh documentation
• Rules Syntax - Ruleset XML syntax · Wazuh documentation
• Decoders Syntax - Ruleset XML syntax · Wazuh documentation
• Regular Expression Syntax - Ruleset XML syntax · Wazuh documentation

I hope it helps. Please let us know how it goes.

[Phạm Q. Đạt]

Hi Fahim,
I did follow those steps but it doesn't seem to work.

When using wazuh-logtest with debug mode, I found that the mongodb log is decoded by JSON decoder then wazuh try to match syslog rule id 1, them trying all of the child of syslog rule but none of them matches then at final try, it turns out "Unknown problem somewhere in the system". It means that the mongodb rule isn't triggered at all.

I think I should custom my own rule for that. 

[Abdullah Al Rafi Fahim]

Hello,

Thank you for your response. 

If the logs are being decoded properly with the JSON decoder and fetch the fields as expected, you can only focus on creating rules following the documentations provided to trigger alerts for these logs then. 

Let us know if you need anything else. 

[Phạm Q. Đạt]

Hello,
I created another rules for the JSON log format. It works very well until I found out that the rule for Multiple Authenticate Failures needs to check same source IP to determine brute-force attack.
I used the <same_source_ip /> tag to do the job. But it seems not working. 

Here is my custom rule :

    <rule id="100110" level="10" frequency="8" timeframe="120" ignore="60">
      <if_matched_sid>100109</if_matched_sid>
      <same_source_ip />
      <description>MongoDB: Multiple authentication failures.</description>
      <mitre>
          <id>T1110</id>
      </mitre>
      <group>authentication_failures,pci_dss_10.2.4,pci_dss_10.2.5,pci_dss_11.4,gpg13_7.1,gdpr_IV_35.7.d,gdpr_IV_32.2,hipaa_164.312.b,nist_800_53_AU.14,nist_800_53_AC.7,nist_800_53_SI.4,tsc_CC6.1,tsc_CC6.8,tsc_CC7.2,tsc_CC7.3,</group>
    </rule>

After some checks, I found out that <same_source_ip> tag is the problem. It seems like the tag try to find "srcip" or "src_ip" from phase 2 decoding?
If so, I want to extract the field "attr.remote" to get srcip and srcport. What should I do in this case?

Here is phases of alerting for MongoDB Brute-force attack:
2022-11-17 10:10:15,446 wazuh-logtest[INFO] **Phase 1: Completed pre-decoding.
2022-11-17 10:10:15,446 wazuh-logtest[INFO] full event: '{"t":{"$date":"2022-11-17T05:05:05.509-05:00"},"s":"I", "c":"ACCESS", "id":20249, "ctx":"conn4","msg":"Authentication failed","attr":{"mechanism":"SCRAM-SHA-1","speculative":false,"principalName":"admin","authenticationDatabase":"admin","remote":"192.168.135.130:57432","extraInfo":{},"error":"UserNotFound: Could not find user \"admin\" for db \"admin\""}}'
2022-11-17 10:10:15,446 wazuh-logtest[INFO]
2022-11-17 10:10:15,447 wazuh-logtest[INFO] **Phase 2: Completed decoding.
2022-11-17 10:10:15,447 wazuh-logtest[INFO] name: 'json'
2022-11-17 10:10:15,447 wazuh-logtest[INFO] attr.authenticationDatabase: 'admin'
2022-11-17 10:10:15,447 wazuh-logtest[INFO] attr.error: 'UserNotFound: Could not find user "admin" for db "admin"'
2022-11-17 10:10:15,447 wazuh-logtest[INFO] attr.mechanism: 'SCRAM-SHA-1'
2022-11-17 10:10:15,447 wazuh-logtest[INFO] attr.principalName: 'admin'
2022-11-17 10:10:15,447 wazuh-logtest[INFO] attr.remote: '192.168.135.130:57432'
2022-11-17 10:10:15,447 wazuh-logtest[INFO] attr.speculative: 'false'
2022-11-17 10:10:15,447 wazuh-logtest[INFO] c: 'ACCESS'
2022-11-17 10:10:15,447 wazuh-logtest[INFO] ctx: 'conn4'
2022-11-17 10:10:15,447 wazuh-logtest[INFO] id: '20249'
2022-11-17 10:10:15,447 wazuh-logtest[INFO] msg: 'Authentication failed'
2022-11-17 10:10:15,447 wazuh-logtest[INFO] s: 'I'
2022-11-17 10:10:15,447 wazuh-logtest[INFO] t.$date: '2022-11-17T05:05:05.509-05:00'
2022-11-17 10:10:15,447 wazuh-logtest[INFO]
2022-11-17 10:10:15,447 wazuh-logtest[INFO] **Phase 3: Completed filtering (rules).
2022-11-17 10:10:15,447 wazuh-logtest[INFO] id: '100109'
2022-11-17 10:10:15,448 wazuh-logtest[INFO] level: '4'
2022-11-17 10:10:15,448 wazuh-logtest[INFO] description: 'MongoDB: Failed authentication'
2022-11-17 10:10:15,448 wazuh-logtest[INFO] groups: '['mongodb', 'authentication_failed']'
2022-11-17 10:10:15,448 wazuh-logtest[INFO] firedtimes: '8'
2022-11-17 10:10:15,448 wazuh-logtest[INFO] gdpr: '['IV_35.7.d', 'IV_32.2']'
2022-11-17 10:10:15,448 wazuh-logtest[INFO] gpg13: '['7.1']'
2022-11-17 10:10:15,448 wazuh-logtest[INFO] hipaa: '['164.312.b']'
2022-11-17 10:10:15,448 wazuh-logtest[INFO] mail: 'False'
2022-11-17 10:10:15,448 wazuh-logtest[INFO] nist_800_53: '['AU.14', 'AC.7']'
2022-11-17 10:10:15,448 wazuh-logtest[INFO] pci_dss: '['10.2.4', '10.2.5']'
2022-11-17 10:10:15,448 wazuh-logtest[INFO] tsc: '['CC6.1', 'CC6.8', 'CC7.2', 'CC7.3']'
2022-11-17 10:10:15,448 wazuh-logtest[INFO] **Alert to be generated.

[Abdullah Al Rafi Fahim]

Hello,

You should try using <same_srcip /> instead of <same_source_ip/>  as the rule syntax same_source_ip has been deprecated in version 4.2.x and later and replaced by same_srcip instead.

You can review this rule syntax here: https://documentation.wazuh.com/current/user-manual/ruleset/ruleset-xml-syntax/rules.html#same-srcip

I hope it helps. Please let us know how it goes.

[Phạm Q. Đạt]

Hi, 

I tried the tag <same_srcip> but seems like Wazuh Manager doesn't understand which part is the source IP. 

As you can see in the phase number 2 decoding, Wazuh successfully extracted the field "attr.remote" to "192.168.135.130:57432" but it can't understand which one is source IP. 

That's the problem.

[Abdullah Al Rafi Fahim]

Hello,

Can you please confirm that your log has a json field named srcip? Or is it the IP address in the  attr.remote field is what you are considering as the source IP here?

<same_srcip/> option will not work if you do not have a field decoded in the name "srcip". Other wise, if attr.remote is the source IP and port in your case, you can use the <same_field> option instead of <same_srcip/> here. It will be like:

<rule id="100110" level="10" frequency="8" timeframe="120" ignore="60">
      <if_matched_sid>100109</if_matched_sid>

       <same_field>attr.remote</same_field>

      <description>MongoDB: Multiple authentication failures.</description>
      <mitre>
          <id>T1110</id>
      </mitre>

   <group>authentication_failures,pci_dss_10.2.4,pci_dss_10.2.5,pci_dss_11.4,gpg13_7.1,gdpr_IV_35.7.d,gdpr_IV_32.2,hipaa_164.312.b,nist_800_53_AU.14,nist_800_53_AC.7,nist_800_53_SI.4,tsc_CC6.1,tsc_CC6.8,tsc_CC7.2,tsc_CC7.3,</group>
    </rule>

I hope it helps. Please let us know how it goes.

[Phạm Q. Đạt]

Hi, 

In my case, the attr.remote consists of 2 part ip_addr:some_random_ports  (for example 192.168.135.130:51234). 

The port field changed every time a client make a connection to mongodb server. That's the point. So I think <same_field /> tag still doesn't work as well.

Hope to see your advice. 

[Phạm Q. Đạt]

Hi, 

Do you have any clues for my question sir? I think it should have another way to get that.

Hope to see your advice. 

Thanks.

[Abdullah Al Rafi Fahim]

Hello, 

As the ports got changed, attr.remote can not be used to trigger alerts for <same_field>. Can you provide us some sample logs so that we can review and try to find a workaround here?

[Phạm Q. Đạt]

Hi, 

Here is the sample mongodb log  

{"t":{"$date":"2022-11-18T04:34:15.521-05:00"},"s":"I",  "c":"ACCESS",   "id":20249,   "ctx":"conn7","msg":"Authentication failed","attr":{"mechanism":"SCRAM-SHA-1","speculative":false,"principalName":"admin","authenticationDatabase":"admin","remote":"192.168.135.130:36202","extraInfo":{},"error":"UserNotFound: Could not find user \"admin\" for db \"admin\""}}

Hope to see your advice.

Thank you.

[Phạm Q. Đạt]

Hello,

Do you have any update for this type of log?

I try to overwrite JSON decoder for purpose of extracting srcip and srcport but it seems not work properly. 

JSON decoder plugin won't work if I exclude and rewrite it on local_decoder.xml file. 

I tried those following steps that I got from https://groups.google.com/g/wazuh/c/a-s3Rhi9qrY:

1. Copy 0006-jsondecoder content to a new file on /var/ossec/etc/decoders for later modification.

2. Exclude the original decoder file from the OSSEC loading list.

3. Modify the json decoder the file on /var/ossec/etc/decoders to extract srcip and srcport.

I run logtest but It just extracts srcip and srcport only, without any others information means JSON_Decoder plugin decoder didn't work.

Hope to see your update.

Thank you.

[Abdullah Al Rafi Fahim]

Hello! I am working on a workaround for your issue and will let you know the update as soon as possible.

[Abdullah Al Rafi Fahim]

Hello,

I am sorry for the late response. However, I have found an workaround for your issue. I have created a separate custom decoder for this log and set it to have precedence over the stock json decoder in case of these logs.

• First of all, I would like to let you know that the decoder and rule files in Wazuh are checked in alpha-numeric order during ruleset test. Therefore, if we want some decoders to have precedence over another one, we need to set the alpha-numeric order accordingly.

Therefore, I have created a new decoder file named 00059-mongojson_decoder.xml in the /var/ossec/etc/decoders/ directory and provided the file proper permission with the following commands:

chmod 660 00059-mongojson_decoder.xml
chown wazuh:wazuh 00059-mongojson_decoder.xml

• Added the following decoder script in the 00059-mongojson_decoder.xml file:

<decoder name="mongojson">
    <prematch>{"t":{"</prematch>
</decoder>

<decoder name="mongojson_child">
    <parent>mongojson</parent>
    <regex offset="after_parent">date":"(\d+-\d+-\d+)T(\d+:\d+:\d+.\d+)</regex>
    <order>date,time</order>
</decoder>

<decoder name="mongojson_child">
    <parent>mongojson</parent>
    <regex offset="after_regex">"c":"(\w+)"</regex>
    <order>type</order>
</decoder>

<decoder name="mongojson_child">
    <parent>mongojson</parent>
    <regex offset="after_regex">"id":(\d+)</regex>
    <order>id</order>
</decoder>

<decoder name="mongojson_child">
    <parent>mongojson</parent>
    <regex offset="after_regex">"ctx":"(\w+)"</regex>
    <order>ctx</order>
</decoder>

<decoder name="mongojson_child">
    <parent>mongojson</parent>
    <regex offset="after_regex">"msg":"(\.+)"</regex>
    <order>msg</order>
</decoder>

<decoder name="mongojson_child">
    <parent>mongojson</parent>
    <regex offset="after_regex">"mechanism":"(\S+)"</regex>
    <order>mechanism</order>
</decoder>

<decoder name="mongojson_child">
    <parent>mongojson</parent>
    <regex offset="after_regex">"speculative":(\w+)</regex>
    <order>speculative</order>
</decoder>

<decoder name="mongojson_child">
    <parent>mongojson</parent>
    <regex offset="after_regex">"principalName":"(\w+)"</regex>
    <order>principalName</order>
</decoder>

<decoder name="mongojson_child">
    <parent>mongojson</parent>
    <regex offset="after_regex">"authenticationDatabase":"(\w+)"</regex>
    <order>authenticationDatabase</order>
</decoder>

<decoder name="mongojson_child">
    <parent>mongojson</parent>
    <regex offset="after_regex">"remote":"(\d+.\d+.\d+.\d+):(\d+)"</regex>
    <order>srcIP, srcPort</order>
</decoder>

<decoder name="mongojson_child">
    <parent>mongojson</parent>
    <regex offset="after_regex">"error":"(\w+)"}</regex>
    <order>error</order>
</decoder>

• Saved the decoder file and restarted the wazuh-manager service.

Now, in the wazuh-logtest the output for this sample log is as following:

 

As the 00059-mongojson_decoder.xml is ahead in the alpha-numeric order than the 0006-json_decoders.xml file and the prematch of the decoder matches the sample log, it will take precedence for this type of log and decode the fields accordingly. 

As the srcIP and srcPort fields are separate here, you can use <same_field> option in your custom rule now for these types of log. You can also update the decoder I have provided according to you need.

I hope it helps. Please let us know how it goes. 

[Phạm Q. Đạt]

Hi,

Thank you for taking your time on it. I really appreciate that. 

For conclusion, I will upload the decoder and rule file for mongo json working my system here for people who struggle with this kind of mongo log.

Thanks again.

[Abdullah Al Rafi Fahim]

Hello,

I am happy to know that the custom decoders and rules are working fine now. I really appreciate your generosity of sharing the custom decoders and rules for the community to be benefited.

Please let us know anytime you have any query or issue regarding Wazuh in this community. Have a great day!

[Khul Sat]

Greetings!
Thank you for providing a starting point!
I was testing these custom decoders & ruleset files. I observed following:

1. <regex offset="after_regex">"s":"(\w+)"</regex> cannot be found in decoder
2. After adding above entry in decoder, none of the rules having field name parameter works. e.g. <field name="s">F|E|W|I|D</field>

Decoder works as expected.

Am I missing on anything? Can someone help?
Thanks, KS

​

[Abdullah Al Rafi Fahim]

Hello Khul Sat,

Can you please explain the issue you are facing a bit more here? Are you using the decoder and rule files Phạm Q. Đạt has shared above? I would like to let you know that the decoders Pham shared is different from the ones I shared before and the rules are also based on that. The decoder for field "s" is present in his decoder as below:

 <decoder name="mongojson_child">

    <parent>mongojson</parent>

    <regex offset="after_regex">"s":"(\w+)"</regex>
    <order>s</order>
</decoder>

And based on the value of this field, he prepared different custom rules to trigger. Therefore, if you are using those rules, I would suggest you to use the decoders as shared by Pham to make them compatible. 

I hope it helps. If you still face any issue, please share the decoders, rules and sample logs that you are testing with.

[Khul Sat]

Thank you so much for pointing that out!

I goofed up badly.  :(

I was trying to identify my mistake.. but couldn't find. Finally I started from scratch. It's working now.

Regards, KS

[Muhammad Farash P]

Hai everyone,

I tried using this decoders and rules. But it seems its not working for the file audit.logs

Here is a sample log

 { "atype" : "authCheck", "ts" : { "$date" : "2023-05-22T11:34:20.951+0530" }, "local" : { "ip" : "10.40.12.250", "port" : 29029 }, "remote" : { "ip" : "127.0.0.1", "port" : 58168 }, "users" : [ { "user" : "testuser", "db" : "admin" } ], "roles" : [ { "role" : "root", "db" : "admin" } ], "param" : { "command" : "replSetGetStatus", "ns" : "admin", "args" : { "replSetGetStatus" : 1, "forShell" : 1, "$db" : "admin" } }, "result" : 0 }

Thanks in advance

Muhammad farash P

[Khul Sat]

Per my understanding, these rulesets and decoders are for DB only. These won't work for audit logs as the log format is different.