Windows Agent Crashes - syscollector.dll

[Kasey Linden]

I am having an issue with Windows agents randomly crashing for syscollector.dll. Has anyone ever seen this?

Windows Error

Faulting application name: wazuh-agent.exe, version: 4.7.3.0, time stamp: 0x65e0813c
Faulting module name: syscollector.dll_unloaded, version: 4.7.3.0, time stamp: 0x65e080cf
Exception code: 0xc0000005
Fault offset: 0x00001be0
Faulting process id: 0x1458
Faulting application start time: 0x01dacbc13c638643
Faulting application path: C:\Program Files (x86)\ossec-agent\wazuh-agent.exe
Faulting module path: syscollector.dll
Report Id: d6759bc4-917e-44c7-8068-f3b26a94d3cb
Faulting package full name:
Faulting package-relative application ID:

Wazuh Log

{"timestamp":"2024/07/01 15:16:29","tag":"wazuh-agent","level":"info","description":"FIM sync module started."}
{"timestamp":"2024/07/01 15:16:30","tag":"wazuh-agent","level":"info","description":"(6019): File integrity monitoring real-time Whodata engine started."}
{"timestamp":"2024/07/01 15:19:30","tag":"wazuh-agent","level":"critical","description":"(1103): Could not open file 'queue\\logcollector\\file_status.json' due to [(0)-(No error)]."}
{"timestamp":"2024/07/01 15:19:30","tag":"wazuh-modulesd:syscollector","level":"info","description":"Stop received for Syscollector."}
{"timestamp":"2024/07/01 15:19:30","tag":"wazuh-modulesd:syscollector","level":"info","description":"Module finished."}
{"timestamp":"2024/07/01 15:19:30","tag":"wazuh-agent","level":"info","description":"(1314): Shutdown received. Deleting responses."}

I also reported it on Github here:

https://github.com/wazuh/wazuh/issues/24376

[Felix Bocco]

Hello Kasey,

Regarding the Wazuh log, for your information, the file_status.json file stores information about the files that are being monitored and how far they have been read (in case the agent is turned off, when it is turned on again, the logs that have not been processed while it was turned off with the only_future_events option enabled are read). Do you have in your configuration any custom files that you are monitoring (the ones you have added with the localfile tag)? Does this agent have any integration that uses this option, like office365 or GitHub?

It would be helpful if you could paste the content of the /var/ossec/queue/logcollector/file_status.json file to check its content. It is recommended to remove any sensitive data.

Additionally, in order to have a better understanding of this issue, please run the debug for the Wazuh agent logs, by modifying the windows.debug setting in the internal_options file located at C:\Program Files (x86)\ossec-agent\internal_options. Change the value from 0 to 2 as follows:

windows.debug=2 

Then restart the Windows agent so changes take effect. From the command line:

net stop Wazuh && net start Wazuh

Check if the Windows agent is showing more logs related to this issue. Aside from this, please share the C:\Program Files (x86)\ossec-agent\ossec.log in order to check if something else could be missing: Windows shows a syscollector issue while Wazuh only shows a logcollector issue. Those are two different modules. 

Did someone make any changes in the C:\Program Files (x86)\ossec-agent\ossec.conf configuration file (or any other file) before this error appeared? Please share also this file to review (remember to remove any sensitive data before uploading it).

If that was not the case, could you recall when this error started to happen? Was it after upgrading Wazuh to version 4.7.3? Is your Wazuh app (manager, indexer, dashboard) version 4.7.3 or it has another version? If it does, what version is it?

We will be waiting for your findings.

[Felix Bocco]

One more thing, ensure that you don't have any localfile for the C:\Program Files (x86)\ossec-agent\queue\logcollector\file_status.json in order to see if we can isolate the issue.

[Kasey Linden]

We are using the VirusTotal integration.

The group config is set to the following:

<agent_config> <!-- Shared agent configuration here --> <localfile> <location>Security</location> <log_format>eventchannel</log_format> <query> Event/System[EventID != 5145 and EventID != 5156 and EventID != 5447 and EventID != 4656 and EventID != 4658 and EventID != 4663 and EventID != 4660 and EventID != 4670 and EventID != 4690 and EventID != 4703 and EventID != 4907 and EventID != 5152 and EventID != 5157 and EventID != 4634] </query> </localfile> <localfile> <location>Microsoft-Windows-Windows Defender/Operational</location> <log_format>eventchannel</log_format> </localfile> <syscheck> <directories whodata="yes">%PROGRAMDATA%\Microsoft\Windows\Start Menu\Programs\Startup</directories> </syscheck> <client_buffer> <disabled>no</disabled> <queue_size>5000</queue_size> <events_per_second>500</events_per_second> </client_buffer> </agent_config>

[Kasey Linden]

This issue happens randomly so putting debug mode on isn't very practical. Restarting the service doesn't have any issues.

[Felix Bocco]

Hello Kasey,

From your agent_config configuration block we don't see anything out of place. Please also share the requested information so we can help you further:

- The /var/ossec/queue/logcollector/file_status.json file.

- One of the agent's C:\Program Files (x86)\ossec-agent\ossec.log file after the issue happened.

- One of the agent's C:\Program Files (x86)\ossec-agent\ossec.conf file.

- Any further information to have a better context of the issue: could you recall when this error started to happen? Was it after upgrading Wazuh to version 4.7.3? Is your Wazuh app (manager, indexer, dashboard) version 4.7.3 or it has another version? If it does, what version is it?

- Additionally, does any of these agents have any particular pattern (example: same Windows version/product, some Wazuh configuration, etc.)?

We will wait for your findings.