Wazuh | MongoDB | Decoder Issue

[John Carry]

Dear Wazuh Team,

Despite of default Decoder and Rule-set for MongoDB, we are unable to observe alerts on wazuh, after some testing and manually log testing we observed below findings:

MongoDB Logs at (Monitored Endpoint End):

{"t":{"$date":"2023-04-26T12:10:12.184+05:00"},"s":"D3", "c":"STORAGE",  "id":22414,   "ctx":"TTLMonitor","msg":"WT begin_transaction","attr":{"snapshotId":66432,"readSource":"kNoTimestamp"}}
{"t":{"$date":"2023-04-26T12:10:12.184+05:00"},"s":"I",  "c":"INDEX",    "id":5479200, "ctx":"TTLMonitor","msg":"Deleted expired documents using index","attr":{"namespace":"config.system.sessions","index":"lsidTTLIndex","numDeleted":0,"durationMillis":0}}
{"t":{"$date":"2023-04-26T12:10:12.185+05:00"},"s":"D3", "c":"STORAGE",  "id":22413,   "ctx":"TTLMonitor","msg":"WT rollback_transaction","attr":{"snapshotId":66432}}
{"t":{"$date":"2023-04-26T12:10:14.974+05:00"},"s":"I",  "c":"STORAGE",  "id":22430,   "ctx":"Checkpointer","msg":"WiredTiger message","attr":{"message":"[1682493014:974802][4788:0x7fee5da99700], WT_SESSION.checkpoint: [WT_VERB_CHECKPOINT_PROGRESS] saving checkpoint snapshot min: 182, snapshot max: 182 snapshot count: 0, oldest timestamp: (0, 0) , meta checkpoint timestamp: (0, 0) base write gen: 550"}}

MongoDB Logs at Wazuh End After enabling Archives.log:

[root@localhost archives]# cat /var/ossec/logs/archives/archives.log| grep -w mongo
2023 Apr 26 11:40:58 (Linux-Testing-Machine) any->/var/log/mongodb/mongod.log {"t":{"$date":"2023-04-26T11:40:57.109+05:00"},"s":"D1", "c":"-",        "id":23074,   "ctx":"conn4","msg":"User assertion","attr":{"error":"HostUnreachable: Connection closed by peer","file":"src/mongo/transport/service_state_machine.cpp","line":394}}
2023 Apr 26 11:40:58 (Linux-Testing-Machine) any->/var/log/mongodb/mongod.log {"t":{"$date":"2023-04-26T11:40:57.109+05:00"},"s":"D1", "c":"-",        "id":23074,   "ctx":"conn4","msg":"User assertion","attr":{"error":"HostUnreachable: Connection closed by peer","file":"src/mongo/transport/service_state_machine.cpp","line":536}}
2023 Apr 26 11:40:59 (Linux-Testing-Machine) any->/var/log/mongodb/mongod.log {"t":{"$date":"2023-04-26T11:40:58.669+05:00"},"s":"D1", "c":"-",        "id":23074,   "ctx":"conn5","msg":"User assertion","attr":{"error":"NoReplicationEnabled: not running with --replSet","file":"src/mongo/db/repl/repl_set_get_status_cmd.cpp","line":56}}
2023 Apr 26 11:40:59 (Linux-Testing-Machine) any->/var/log/mongodb/mongod.log {"t":{"$date":"2023-04-26T11:40:58.669+05:00"},"s":"D1", "c":"-",        "id":23074,   "ctx":"conn5","msg":"User assertion","attr":{"error":"NoReplicationEnabled: not running with --replSet","file":"src/mongo/util/future_impl.h","line":1104}}
2023 Apr 26 11:40:59 (Linux-Testing-Machine) any->/var/log/mongodb/mongod.log {"t":{"$date":"2023-04-26T11:40:58.670+05:00"},"s":"D1", "c":"-",        "id":23074,   "ctx":"conn5","msg":"User assertion","attr":{"error":"NoReplicationEnabled: not running with --replSet","file":"src/mongo/util/future_impl.h","line":1087}}
2023 Apr 26 11:40:59 (Linux-Testing-Machine) any->/var/log/mongodb/mongod.log {"t":{"$date":"2023-04-26T11:40:58.670+05:00"},"s":"D1", "c":"-",        "id":23074,   "ctx":"conn5","msg":"User assertion","attr":{"error":"NoReplicationEnabled: not running with --replSet","file":"src/mongo/util/future_impl.h","line":1104}}
2023 Apr 26 11:40:59 (Linux-Testing-Machine) any->/var/log/mongodb/mongod.log {"t":{"$date":"2023-04-26T11:40:58.670+05:00"},"s":"D1", "c":"-",        "id":23074,   "ctx":"conn5","msg":"User assertion","attr":{"error":"NoReplicationEnabled: not running with --replSet","file":"src/mongo/util/future_impl.h","line":1087}}

As you have noticed That timestamp is added before the actual log and that is creating problem that we are not getting match the MongoDB decoder with the timestamp based logs because as per the LogTest pasted below, it seems it is working without timestamp.

LogTest Results:

Configuration at Agent End:

Please provide a way out that would enable us to successfully decode MongoDB logs.

[John Carry]

Anyone please ?

[victor....@wazuh.com]

Hello John,

First, it is important to notice that logs that appear in the archives file include a header not present in the real event. For example, consider this archive line:

2023 Apr 26 11:40:58 (Linux-Testing-Machine) any->/var/log/mongodb/mongod.log {"t":{"$date":"2023-04-26T11:40:57.109+05:00"},"s":"D1", "c":"-",    "id":23074, "ctx":"conn4","msg":"User assertion","attr":{"error":"HostUnreachable: Connection closed by peer","file":"src/mongo/transport/service_state_machine.cpp","line":394}}

In this case, the real event is {"t":{"$date":"2023-04-26T11:40:57.109+05:00"},"s":"D1", "c":"-",  "id":23074, "ctx":"conn4","msg":"User assertion","attr":{"error":"HostUnreachable: Connection closed by peer","file":"src/mongo/transport/service_state_machine.cpp","line":394}}.

This should be taken into account to create a custom ruleset or use the default one.

Regarding MongoDB default ruleset (decoders, rules) it seems that expects logs in Syslog format instead of JSON. In addition, currently, there is a bug in the mongodb decoders that prevent any Mongodb rule to trigger.

In this situation, the best approach would be to create a custom ruleset for your use case. Something like the following could be a good starting point:

Rules

<group name="mongodb">

  <rule id="100002" level="0">
    <description>MongoDB messages</description>
    <field name="t.$date">\.*</field>
  </rule>

  <rule id="100003" level="3">
    <if_sid>100002</if_sid>
    <description>MongoDB: Informational message</description>
    <field name="s">I</field>
  </rule>

  <rule id="100004" level="4">
    <if_sid>100002</if_sid>
    <description>MongoDB: Warning message</description>
    <field name="s">W</field>
  </rule>

  <rule id="100004" level="5">
    <if_sid>100002</if_sid>
    <description>MongoDB: Error message</description>
    <field name="s">E</field>
  </rule>

  <rule id="100005" level="9">
    <if_sid>100002</if_sid>
    <description>MongoDB: Fatal message</description>
    <field name="s">F</field>
  </rule>

  <rule id="100006" level="0">
    <if_sid>100002</if_sid>
    <description>MongoDB: Debug message</description>
    <field name="s">D</field>
  </rule>
</group>


Using these rules, your events will be supported by the ruleset:

# /var/ossec/bin/wazuh-logtest
Starting wazuh-logtest v4.3.11
Type one log per line

{"t":{"$date":"2023-04-26T11:40:57.109+05:00"},"s":"D1", "c":"-",        "id":23074,   "ctx":"conn4","msg":"User assertion","attr":{"error":"HostUnreachable: Connection closed by peer","file":"src/mongo/transport/service_state_machine.cpp","line":394}}

**Phase 1: Completed pre-decoding.
full event: '{"t":{"$date":"2023-04-26T11:40:57.109+05:00"},"s":"D1", "c":"-",        "id":23074,   "ctx":"conn4","msg":"User assertion","attr":{"error":"HostUnreachable: Connection closed by peer","file":"src/mongo/transport/service_state_machine.cpp","line":394}}'

**Phase 2: Completed decoding.
name: 'json'
attr.error: 'HostUnreachable: Connection closed by peer'
attr.file: 'src/mongo/transport/service_state_machine.cpp'
attr.line: '394'
c: '-'
ctx: 'conn4'
id: '23074'
msg: 'User assertion'
s: 'D1'
t.$date: '2023-04-26T11:40:57.109+05:00'

**Phase 3: Completed filtering (rules).
id: '100006'
level: '0'
description: 'MongoDB: Debug message'
groups: '['mongodb']'
firedtimes: '1'
mail: 'False'

Now it should be necessary to create the custom rules required for your use case.

If you have any doubt, do not hesitate to ask.

[Khul Sat]

Greetings!

Referring to this thread I tried to create custom rules & decoders for mongo json logs. But some how rules are not working.
Decoders decode well but field name parameter in rules is not working.
Could you please suggest?

Thanks, KS

​

[Khul Sat]

It worked for me. Thanks,

[Muhammad Farash P]

Hai Khul sat,

Can you please provide decoders and ruleset that worked for you

Thanks in advance

Muhammad Farash P