office365 log duplication

[Rahul Manoj]

  Hello everyone, we are observing an intermittent issue with the Office 365 integration in our Wazuh environment where log duplication occurs once in a while, resulting in multiple identical events being ingested and processed. The duplication affects multiple Office 365 event types and is not limited to a single alert or rule ID. We have verified that the logs are not duplicated at the source and that the issue appears after ingestion by the Wazuh Manager. Since the Office 365 integration operates as a module rather than a built-in script, modification or custom handling from our side is not possible. When the duplication starts, it continues for subsequent events until the Wazuh Manager service is restarted, after which the issue temporarily stops. We are looking to understand what could cause this behavior in the Office 365 module, whether it could be related to state handling, offsets, or token management, and what recommended troubleshooting steps or diagnostics could help identify and permanently resolve the issue.

 wazuh-manager version -  4.3.8-1  
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.2 LTS
Release:        22.04
Codename:       jammy

office365 configuration in ossec.conf :

  <!-- Office 365 -->
  <office365>
    <enabled>yes</enabled>
    <interval>4m</interval>
    <curl_max_size>6M</curl_max_size>
    <only_future_events>yes</only_future_events>
    <api_auth>
        <tenant_id> REDACTED</tenant_id>
        <client_id>< REDACTED</client_id>
        <client_secret>REDACTED</client_secret>
    </api_auth>
    <subscriptions>
        <subscription>Audit.SharePoint</subscription>
        <subscription>Audit.AzureActiveDirectory</subscription>
        <subscription>Audit.Exchange</subscription>
        <subscription>Audit.General</subscription>
        <subscription>DLP.All</subscription>
    </subscriptions>
  </office365>

[Rahul Manoj]

i had forgotten to add a screenshot 

[Sandip Aryal]

Hi Rahul,
Your configuration appears to be correct, as outlined here: 

To check if log duplication is occurring, you can enable archives.json on the Wazuh Manager.
Update the ossec.conf file on the Wazuh Manager to allow JSON logging,
and then verify whether the logs are being received more than once.

<ossec_config>
  <global>
    <jsonout_output>yes</jsonout_output>
    <alerts_log>yes</alerts_log>
    <logall>no</logall>
    <logall_json>yes</logall_json>
    .... Truncuated Configuration
  </global>
 
After enabling this, please restart the Wazuh Manager, and check /var/ossec/logs/archives/archives.json
Note: Please disable the archives.json after the verification, as it will store all the logs in json may cause storage-related issues.

Additionally, you can enable debug logging for Wazuh modules by setting wazuh_modules.debug:2 in /var/ossec/etc/local_internal_options.conf. After enabling this, check /var/ossec/logs/ossec.log for more detailed information, which may help you to find out the issue.
Also, you are currently using an old version of Wazuh. You can follow the Wazuh upgrade guide https://documentation.wazuh.com/current/upgrade-guide/index.html
 to update to the latest version.
Furthermore, you can use a workaround to silence the duplicate alerts. Please refer to the following URL as referenced:   https://groups.google.com/g/wazuh/c/bMGsiDm-UIc 

References:
https://documentation.wazuh.com/current/user-manual/manager/event-logging.html
https://documentation.wazuh.com/4.3/user-manual/reference/internal-options.html

Hope it helps.