JSON output of command wodle is in alerts.json but not dashboard

[Arno van Wouwe]

Hi all.

I have written I little command wodle that uses the Google Admin SDK to get audit logs. I'm hoping to be able to contribute it once it works, but so far I am not getting it to show up events in the dashboard.

The command outputs each "activity" as one line, and I have added a new rule:

<group name="gworkspace,">
  <rule id="100010" level="6">
    <decoded_as>json</decoded_as>
    <field name="kind">^admin#</field>
    <description>Google Workspace event</description>
  </rule>
</group>

When I test the rule with one of the lines, it matches:

**Messages: INFO: (7202): Session initialized with token 'xxx00c' **Phase 1: Completed pre-decoding. full event: '{"kind": "admin#reports#activity", "id": {"time": "2024-02-13T22:47:01.655Z", "uniqueQualifier": "xxxxx", "applicationName": "drive", "customerId": "C03xxx"}, "etag": "\"xxxxx-x\"", "actor": {"email": "xxx...@domain.com", "profileId": "11389xxxx"}, "ipAddress": "xxx.xxx.xxx.xxx"}' **Phase 2: Completed decoding. name: 'json' actor.email: 'xxx...@domain.com' actor.profileId: '11389xxxx' id.applicationName: 'drive'
id.customerId: 'C03xxx' id.time: '2024-02-13T22:47:01.655Z' id.uniqueQualifier: 'xxxx' ipAddress: 'xxx.xxx.xxx.xxx' kind: 'admin#reports#activity' **Phase 3: Completed filtering (rules). id: '100010' level: '6' description: 'Google Workspace event' groups: '["gworkspace"]' firedtimes: '1' mail: 'false' **Alert to be generated.

.. and lines show up in alerts.json:

{"timestamp":"2024-02-15T09:28:44.129+0000","rule":{"level":6,"description":"Google Workspace event","id":"100010","firedtimes":1770,"mail":false,"groups":["gworkspace"]},"agent":{"id":"000","name":"wazuh.master","labels":{"dept":"it"}},"manager":{"name":"wazuh.master"},"id":"1707989324.162085392","cluster":{"name":"wazuh","node":"manager"},"full_log":"{\"kind\": \"admin#reports#activity\", \"id\": {\"time\": \"2024-02-13T09:35:59.733Z\", \"uniqueQualifier\": \"4950073755968662412\", \"applicationName\": \"drive\", \"customerId\": \"C03rm06r1\"}, \"etag\": \"\\\"sgN-xxxxxxx-awS03N_Ls/3tOhFWXcTYoXXA3rwKZ5Gx7hvrc\\\"\", \"actor\": {\"email\": \"xxx...@domain.com\", \"profileId\": \"xxxxxx\"}, \"ipAddress\": \"xxxx\"}","decoder":{"name":"json"},"data":{"kind":"admin#reports#activity","id":{"time":"2024-02-13T09:35:59.733Z","uniqueQualifier":"xxxx","applicationName":"drive","customerId":"C03xxxx"},"etag":"\"sgN-ZL0DA0oYN0xxxxHoXxxxxxxxx-awS03N_L","actor":{"email":"xxx...@domain.com","profileId":"1064747"},"ipAddress":"xxx.xxx.xxx.xxx"},"location":"command_gworkspace"}

But I do not get anything in the dashboard, even if the dashboard is working great otherwise (I have 80+ endpoints generating 400K events / day). Have tried to look around in the logstash logs, but can't even find the relevant logs that would indicate errors.

I am using the multi-node 4.7.1 docker-deployed version of Wazuh.

Any ideas?

   Arno van Wouwe

[Benjamin Nworah]

Hello Arno,

Thank you for using Wazuh!

Kindly perform the following steps:

1- Edit the file /var/ossec/etc/ossec.conf of the Wazuh manager. Change the value of <logall> from no to yes.

2- Run the command systemctl restart wazuh-manager

3- Run the command to retrieve the log you are testing your rule against. 

less /var/ossec/logs/archives/archives.log | grep -i  uniqueQualifier

Please share the log obtained in step 3.

I patiently await your response.

Regards,

[Benjamin Nworah]

Hello Arno,

After retrieving the log. Kindly change the value of <logall> back to no and restart the Wazuh manager (systemctl restart wazuh-manager)

This prevent your disk from getting filled up quickly.

I await the log so I can test this.

Regards,

[Arno van Wouwe]

Hi Benjamin,

Thank you for your prompt response.

Events show up in the archives (I already tested it but did not think it fit to mention):

2024 Feb 15 09:43:04 wazuh->command_gworkspace {"kind": "admin#reports#activity", "id": {"time": "2024-02-13T01:49:44.148Z", "uniqueQualifier": "8149258909545659838", "applicationName": "drive", "customerId": "C03xxxx"}, "etag": "\"sgN-xxxxx-awS03N_Ls/xxxxx\"", "actor": {"email": "xxx...@domain.com", "profileId": "1161184439xxxx1"}, "ipAddress": "xxx.xxx.xxxx.xxx"}

Events also show up in Slack (which I have integrated). 

Any idea?

    Arno

--
You received this message because you are subscribed to a topic in the Google Groups "Wazuh | Mailing List" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/wazuh/dkrQQLQKcoE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/d95b3f72-c6a1-47f8-865e-772db5e7fa3en%40googlegroups.com.

[Benjamin Nworah]

Hello Arno,

Thanks for the log.

Please give me some time to test and revert.

Regards,

You received this message because you are subscribed to the Google Groups "Wazuh | Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/CA%2BmdV4Vnh0CRc1te7qxjHAy0n%2Bb%2BvkNNN6RNxjHgTjJQoAJw6w%40mail.gmail.com.

[Benjamin Nworah]

Hello Arno,

Thank you for your patience on this thread.

I set up a Wazuh docker deployment to simulate the issue you are facing, and I can confirm that this is a known issue.
Dashboard does not displays alerts while `alert.log` file does in Docker deployments · Issue #21806 · wazuh/wazuh · GitHub

It is recommended to add the below parameters to the /etc/filebeat/filebeat.yml file. However I can confirm these parameters exist in the filebeat.yml on my Wazuh Docker deployment., and I still experience the same issue.

setup.template.json.path: '/etc/filebeat/wazuh-template.json'
setup.template.json.name: 'wazuh'

I will track this issue, and update you when this issue is resolved or a work around is available.

Regards,

You received this message because you are subscribed to the Google Groups "Wazuh | Mailing List" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/CA%2BmdV4Vnh0CRc1te7qxjHAy0n%2Bb%2BvkNNN6RNxjHgTjJQoAJw6w%40mail.gmail.com.

[Benjamin Nworah]

Hello Arno,

Thank you for your patience on this thread.

I deployed Wazuh docker 4.7.2 and I am experiencing the same issue. I can confirm the issue exists 4.8 Beta-1.

Dashboard does not displays alerts while `alert.log` file does in Docker deployments · Issue #21806 · wazuh/wazuh · GitHub

I am reviewing this internally to know why this is happening on version 4.7.2.
I will revert shortly.

Regards,

You received this message because you are subscribed to the Google Groups "Wazuh | Mailing List" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/CA%2BmdV4Vnh0CRc1te7qxjHAy0n%2Bb%2BvkNNN6RNxjHgTjJQoAJw6w%40mail.gmail.com.

[Benjamin Nworah]

Hello Arno,

I am still testing this. I will provide you with an update.

Regards,

[Benjamin Nworah]

Hello Arno,

Thank you for your patience on this thread. I have tested your log sample, and I observed that Wazuh is not generating alerts for your log with the field "id". It worked after changing this field from id to idf.

Test performed:

1- Created a file /root/google.log. This file was created inside the Wazuh manager node.

2- Added the below configuration in /var/ossec/etc/ossec.conf file 

<localfile>
  <log_format>json</log_format>
  <location>/root/google.log</location>
 </localfile>

3- Added the below log inside the file /root/google.log. I changed the field id to idf.

{"kind": "admin#reports#activity", "idf": {"time": "2024-02-13T22:47:01.655Z", "uniqueQualifier": "xxxxx", "applicationName": "drive", "customerId": "C03xxx"}, "etag": "\"xxxxx-x\"", "actor": {"email": "xxx...@domain.com", "profileId": "11389xxxx"}, "ipAddress": "xxx.xxx.xxx.xxx"}

4. Alert received.



Please give me time to discuss this internally.

Regards,

[Benjamin Nworah]

Hi Arno.

Please run this command on the Wazuh manager node, and share the output

less /var/log/filebeat/filebeat | grep -i "cannot index event"

Regards,

[Arno van Wouwe]

To sum up our exchange for the benefit of the group : my JSON object did not contain a "data.id" attribute, which was causing issues because of the filebeat template. As soon as I mapped a value to "data.id" the events showed up in the dashboard.

This seems to be a known issue as stated in this issue https://github.com/wazuh/wazuh/issues/9949.

Many thanks for your help.