Trouble getting real-time FIM alerts with whodata on Windows

[Dylan Buehler]

Hey Guys,

I've successfully setup Windows wazuh agents with real-time, whodata alerts in the past, but I'm having trouble with this particular deployment.

Manager Info:

version: 3.9.5

os: Ubuntu 16.04

Agent Info:

version: 3.9.5

os: Microsoft Windows Server 2012 R2 Standard [Ver: 6.3.9600]

My agent is a member of a custom group and ONLY this one custom group. There are no errors when I use the verify-agent-conf tool. I also made sure the agent got the shared config, etc. The agent.conf for the group is below:

<agent_config>

  <!-- Shared agent configuration here -->

  <syscheck>

    <disabled>no</disabled>

    <!-- Frequency that syscheck is executed default every 12 hours -->

    <frequency>43200</frequency>

    <scan_on_start>yes</scan_on_start>

    <!-- Generate alert when new file detected -->

    <alert_new_files>yes</alert_new_files>

    <!-- Don't ignore files that change more than 'frequency' times -->

    <auto_ignore>no</auto_ignore>

    <!-- Directories to check  (perform all possible verifications) -->

    <directories check_all="yes" whodata="yes" report_changes="yes">F:\.</directories>

    <directories check_all="yes" whodata="yes" report_changes="yes">E:\.</directories>

    <!-- Files/directories to ignore -->

    <ignore>f:\.\filedrop\._sync_9da36aea000d.db-shm</ignore>

    <!-- Check the file, but never compute the diff -->

    <skip_nfs>yes</skip_nfs>

    <!-- Remove not monitored files -->

    <remove_old_diff>yes</remove_old_diff>

    <!-- Allow the system to restart Auditd after installing the plugin -->

  </syscheck>

  <localfile>

    <location>Microsoft-Windows-Sysmon/Operational</location>

    <log_format>eventchannel</log_format>

  </localfile>

</agent_config>

I also made sure the necessary audit policies were set via group policy in accordance with these directions:

https://documentation.wazuh.com/3.4/user-manual/capabilities/auditing-whodata/who-windows.html

If I do not specify the "whodata" option, and instead configure the following:

<directories check_all="yes" realtime="yes" report_changes="yes">F:\.</directories>

then I have no issues, and real-time FIM alerts work as expected. Something is going wrong when I enable the whodata option.

I've attached screenshots of parts of the agent's ossec.log, showing the syscheck settings are enabled to monitor the specified directories and that the "whodata auditing engine started".

I don't see any related errors in my agent's ossec.log.

I originally had errors in my agent's log before upgraing to 3.9.5. I was seeing events like below when realtime alerts attempted to trigger:

ossec-agent: ERROR: The event could not be added to the ignored hash table because it is duplicated. Target: 'f:\filedrop\._sync_9da36aea000d.db-wal'.

From looking around on github, I found stuff suggesting to upgrade my agent, so I did and haven't seen these errors since.

Please let me know if you have any ideas or need more information, thanks!

[David Vidriales]

Hi Dylan,

A couple of questions:

- Does whodata work for you on a specific folder (not a drive; for example adding, modifying or deleting in an monitored empty folder)?

- Does this work for you without the report_changes option?

- What's your syscheck configuration in the agent's ossec.conf?

The report_changes option will make a text copy of the files inside the monitored folder (in your case you could have a lot of files inside the monitored drives and it will take a substantial space in your drive).

You could also activate debug messages and I could help you find what's wrong. In order to do that, edit the internal_options.conf inside the Wazuh installation folder (usually C:\Program Files(x86)\ossec-agent), search for syscheck.debug=0 and change it to syscheck.debug=2. Then restart the agent, wait for the centralized configuration to load (the agent will restart again) and once the scan has ended (a message like File integrity monitoring scan has ended should appear), try to add, modify or delete a file inside your monitored folders. Stop the agent and attach the log (ossec.log) to the answer.

You should revert the syscheck.debug option after that, as this option may flood your log.

Anyway, we're currently developing a rework of the syscheck module that will help doing these kind of scans. You can follow its development here: https://github.com/wazuh/wazuh/issues/3319

Best regards,

David

[Dylan Buehler]

Hey David,

Thanks for helping out.

I went ahead and enabled debug logging, disabled report_changes, and did some testing. Also, I included my agent's syscheck settings from ossec.conf at the bottom.

Unfortunately, I can't upload the entire ossec.log for security reasons, but after enabling debug and disabling the report changes setting, I restarted the agent and waited for the "whodata auditing engine started" message, then I created a test file in one of the monitored drives and saw this debug message:

2019/09/05 16:10:44 ossec-agent[12324] win_whodata.c:602 at whodata_callback(): DEBUG: (6239): 'f:\some\dir\

fim-test-9-5-2' is discarded because its monitoring is not activated.

I also saw some interesting messages that seemed to conflict related to creating SACL permissions, see screenshot.

I'll try monitoring just a specific folder next.

<syscheck>

    

    <disabled>no</disabled>

    <!-- Frequency that syscheck is executed default every 12 hours -->

    <frequency>43200</frequency>

    <!-- Default files to be monitored. -->

    <directories check_all="yes">%WINDIR%\regedit.exe</directories>

    <directories check_all="yes">%WINDIR%\system.ini</directories>

    <directories check_all="yes">%WINDIR%\win.ini</directories>

    <directories check_all="yes">%WINDIR%\SysNative\at.exe</directories>

    <directories check_all="yes">%WINDIR%\SysNative\attrib.exe</directories>

    <directories check_all="yes">%WINDIR%\SysNative\cacls.exe</directories>

    <directories check_all="yes">%WINDIR%\SysNative\cmd.exe</directories>

    <directories check_all="yes">%WINDIR%\SysNative\drivers\etc</directories>

    <directories check_all="yes">%WINDIR%\SysNative\eventcreate.exe</directories>

    <directories check_all="yes">%WINDIR%\SysNative\ftp.exe</directories>

    <directories check_all="yes">%WINDIR%\SysNative\lsass.exe</directories>

    <directories check_all="yes">%WINDIR%\SysNative\net.exe</directories>

    <directories check_all="yes">%WINDIR%\SysNative\net1.exe</directories>

    <directories check_all="yes">%WINDIR%\SysNative\netsh.exe</directories>

    <directories check_all="yes">%WINDIR%\SysNative\reg.exe</directories>

    <directories check_all="yes">%WINDIR%\SysNative\regedt32.exe</directories>

    <directories check_all="yes">%WINDIR%\SysNative\regsvr32.exe</directories>

    <directories check_all="yes">%WINDIR%\SysNative\runas.exe</directories>

    <directories check_all="yes">%WINDIR%\SysNative\sc.exe</directories>

    <directories check_all="yes">%WINDIR%\SysNative\schtasks.exe</directories>

    <directories check_all="yes">%WINDIR%\SysNative\sethc.exe</directories>

    <directories check_all="yes">%WINDIR%\SysNative\subst.exe</directories>

    <directories check_all="yes">%WINDIR%\SysNative\wbem\WMIC.exe</directories>

    <directories check_all="yes">%WINDIR%\SysNative\WindowsPowerShell\v1.0\powershell.exe</directories>

    <directories check_all="yes">%WINDIR%\SysNative\winrm.vbs</directories>

    <!-- 32-bit programs. -->

    <directories check_all="yes">%WINDIR%\System32\at.exe</directories>

    <directories check_all="yes">%WINDIR%\System32\attrib.exe</directories>

    <directories check_all="yes">%WINDIR%\System32\cacls.exe</directories>

    <directories check_all="yes">%WINDIR%\System32\cmd.exe</directories>

    <directories check_all="yes">%WINDIR%\System32\drivers\etc</directories>

    <directories check_all="yes">%WINDIR%\System32\eventcreate.exe</directories>

    <directories check_all="yes">%WINDIR%\System32\ftp.exe</directories>

    <directories check_all="yes">%WINDIR%\System32\net.exe</directories>

    <directories check_all="yes">%WINDIR%\System32\net1.exe</directories>

    <directories check_all="yes">%WINDIR%\System32\netsh.exe</directories>

    <directories check_all="yes">%WINDIR%\System32\reg.exe</directories>

    <directories check_all="yes">%WINDIR%\System32\regedit.exe</directories>

    <directories check_all="yes">%WINDIR%\System32\regedt32.exe</directories>

    <directories check_all="yes">%WINDIR%\System32\regsvr32.exe</directories>

    <directories check_all="yes">%WINDIR%\System32\runas.exe</directories>

    <directories check_all="yes">%WINDIR%\System32\sc.exe</directories>

    <directories check_all="yes">%WINDIR%\System32\schtasks.exe</directories>

    <directories check_all="yes">%WINDIR%\System32\sethc.exe</directories>

    <directories check_all="yes">%WINDIR%\System32\subst.exe</directories>

    <directories check_all="yes">%WINDIR%\System32\wbem\WMIC.exe</directories>

    <directories check_all="yes">%WINDIR%\System32\WindowsPowerShell\v1.0\powershell.exe</directories>

    <directories check_all="yes">%WINDIR%\System32\winrm.vbs</directories>

    <directories check_all="yes" realtime="yes">%PROGRAMDATA%\Microsoft\Windows\Start Menu\Programs\Startup</directories>

    <ignore type="sregex">.log$|.htm$|.jpg$|.png$|.chm$|.pnf$|.evtx$</ignore>

    <!-- Windows registry entries to monitor. -->

    <windows_registry>HKEY_LOCAL_MACHINE\Software\Classes\batfile</windows_registry>

    <windows_registry>HKEY_LOCAL_MACHINE\Software\Classes\cmdfile</windows_registry>

    <windows_registry>HKEY_LOCAL_MACHINE\Software\Classes\comfile</windows_registry>

    <windows_registry>HKEY_LOCAL_MACHINE\Software\Classes\exefile</windows_registry>

    <windows_registry>HKEY_LOCAL_MACHINE\Software\Classes\piffile</windows_registry>

    <windows_registry>HKEY_LOCAL_MACHINE\Software\Classes\AllFilesystemObjects</windows_registry>

    <windows_registry>HKEY_LOCAL_MACHINE\Software\Classes\Directory</windows_registry>

    <windows_registry>HKEY_LOCAL_MACHINE\Software\Classes\Folder</windows_registry>

    <windows_registry arch="both">HKEY_LOCAL_MACHINE\Software\Classes\Protocols</windows_registry>

    <windows_registry arch="both">HKEY_LOCAL_MACHINE\Software\Policies</windows_registry>

    <windows_registry>HKEY_LOCAL_MACHINE\Security</windows_registry>

    <windows_registry arch="both">HKEY_LOCAL_MACHINE\Software\Microsoft\Internet Explorer</windows_registry>

    <windows_registry>HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services</windows_registry>

    <windows_registry>HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\KnownDLLs</windows_registry>

    <windows_registry>HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\SecurePipeServers\winreg</windows_registry>

    <windows_registry arch="both">HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Run</windows_registry>

    <windows_registry arch="both">HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\RunOnce</windows_registry>

    <windows_registry>HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\RunOnceEx</windows_registry>

    <windows_registry arch="both">HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\URL</windows_registry>

    <windows_registry arch="both">HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Policies</windows_registry>

    <windows_registry arch="both">HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\Windows</windows_registry>

    <windows_registry arch="both">HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\Winlogon</windows_registry>

    <windows_registry arch="both">HKEY_LOCAL_MACHINE\Software\Microsoft\Active Setup\Installed Components</windows_registry>

    <!-- Windows registry entries to ignore. -->

    <registry_ignore>HKEY_LOCAL_MACHINE\Security\Policy\Secrets</registry_ignore>

    <registry_ignore>HKEY_LOCAL_MACHINE\Security\SAM\Domains\Account\Users</registry_ignore>

    <registry_ignore type="sregex">\Enum$</registry_ignore>

    <!-- Frequency for ACL checking (seconds) -->

    <windows_audit_interval>300</windows_audit_interval>

 </syscheck>

[David Vidriales]

Hi again Dylan,

I think I've figured out this problem. This configuration works:

   <directories check_all="yes" whodata="yes" report_changes="yes">F:\\</directories>

Instead of:

  <directories check_all="yes" whodata="yes" report_changes="yes">F:\.</directories>

With the former configuration whodata is configured to monitor a folder inside "F:\" called "."  which doesn't exists. The events that arrive from whodata in Windows convert the path we store (which is the one that is indicated in the configuration) while in realtime doesn't.  Anyway I've opened an issue about this in https://github.com/wazuh/wazuh/issues/3934 as both options (whodata and realtime) should have the same behavior regarding the configuration. Thank you so much for reporting this. I hope this works for you.

Best regards,

David

[Dylan Buehler]

David,

Thanks so much for investigating and opening the github issue, I'm going to try this today and see how it goes. Happy to help the project.

Thanks,

Dylan

[Dylan Buehler]

Confirmed, it's working! For all readers, the fix was pointed out by David above, but here it is again:

If trying to monitor an entire drive in Windows in realtime with whodata enabled, use the config below:

<directories check_all="yes" whodata="yes" report_changes="yes">F:\\</directories>

Do not use "F:\.", even though that is recommended in the wazuh documentation.

[David Vidriales]

Hi again Dylan,

Glad to hear it worked. This problem will be fixed in the aforementioned issue (documentation included). If you have any further problems regarding this topic, don't hesitate to contact us again.

Best regards,

David