Wazuh - Performance Impact

[Szymon Wyborny]

Hi!

I'm attempting to install Wazuh in a production setting.

We've encountered an issue during various deployments on the servers which had Wazuh running.

The servers in question are development Windows boxes that are running DFSR to sync files between each other.

It seems like when Wazuh agent is turned on, deployments start failing, causing files to get locked up, and some of them not to update.

I'm running the minimal config that looks something like this:

<agent_config>

 <syscheck>

   <disabled>no</disabled>

   <scan_on_start>yes</scan_on_start>

   <!-- Execute a scan every 60 minutes -->

   <frequency>3600</frequency>

   <directories check_all="yes" whodata="yes">C:\fl\directory</directories>

  

   <file_limit>

     <enabled>yes</enabled>

     <entries>5000000</entries>

   </file_limit>

 </syscheck>

</agent_config>

Everything else aside from the 'syscheck' module is disabled.

Do you know of any potential fixes for these types of performance issues that cause files to be locked up?

The frequency interval in this case is 60 minutes, but even if the changes are done in between the 'scan frequency times,' the files still get locked.

Thanks,

Szymon

[mauro.e...@wazuh.com]

Hi Szymon,

Could you tell me which agent version are you running? We identified a problem on version 3.12 were a similar condition happened while hashing files. We patched it in version 3.13, so I would like to know if this can be the source of the problem or if we might be facing a new issue similar to this.

Also, since you have whodata mode enabled, the frequency in itself could be irrelevant, as soon as a file inside C:\fl\directory is changed the agent will try to analyze it and trigger an event.

Best regards,

Mauro.

[Szymon Wyborny]

Hi Mauro,

I'm using version 3.13.

Is that really the case with whodata?

I was under the impression that i'd have to turn on realtime monitoring in order for the files to be scanned in 'realtime'...

Is there a way to keep the process and user data of when something runs and still run wazuh at a specific frequency?

thanks,

Szymon

[mauro.e...@wazuh.com]

Hi Szymon,

It sounds like we might have overlooked something when we changed the hashing algorithm for it to stop locking files then... Do the files that are being locked have any particular characteristic? Are they office files for instance? (.docx, .xlsx , etc.) Could you also check the agent log file for any error/warning message at around the time this problems surfaced? The log file is located at C:\program files (x86)\ossec-agent\ossec.log

Regarding your question, both realtime and whodata modes integrate with external systems that trigger events for the agent to process. In the case of whodata on Windows, the agent sets audit policies on the files that it's monitoring and the OS forwards security events to it whenever a criteria is met. Due to how we analyze files, if you had 2 changes on a given file and we were to wait until the following scan, we would not be able to know which change corresponds to which security event.

Best regards,

Mauro.

[Szymon Wyborny]

Hi Mauro,

Hmm, I wasn't aware of how 'whodata' would work, your answer is very helpful thank you!! 

No, it does not seem like we're "LOCKING" the files during the scan actually.. That's a mistake on my part, the confusion was due to the significantly affected processes on the host running Wazuh agent. 

Also, the tool is running on developer boxes with DFSR (basically file sync) across all of the code directories, and it was the file sync that was bugging out.

There was no 'specific' file types affected, basically affected the entire folder structure.

After removing the 'whodata' functionality, the tool became much more stable, thank you!

Regards,

Szymon

[mauro.e...@wazuh.com]

Hi Szymon,

Glad to hear switching to scheduled mode did the trick for you! It still sounds to me like whodata might be interfering with DFSR, I will look into it just to be safe.

Best regards,

Mauro.

[Szymon Wyborny]

Hi Mauro,

That would be interesting to know as it does seem like that's what was happenning!

If possible, let me know what you find.

Thank you!