FIM module is not working wazuh 4.8/ File integratiy not reporting as expected

[Shujaat Ali]

Hi Wazuh Team,

I thought Wazuh was set up and running properly. All Wazuh components are on one server (8 CPUs and 16 GB RAM) with about 35 agents. I was able to configure real-time event monitoring for a couple of additional directories, such as /home, on 2 agents only. However, I suddenly noticed that Wazuh is not reporting any new file changes to the /etc or /home directories. It does report modifications to /etc/resolv.conf, but it does not report file creation in /etc.

The health status appears to be fine and no errors are showing up. One point to note: weeks ago I noticed Wazuh was not working correctly, so I recreated the Wazuh server and the agents started to report to the new server. Initially, the Wazuh manager reported problems with the agent IDs and keys. However, it resolved itself. Then, I completely restarted the Wazuh server, and it worked for a few days without any issues. I have reverted the ossec configuration on the server and the agents to the default configuration with syscheck enabled, but it is not reporting any FIM-related changes.
One more thing: it shows  home/usrx/.bash_history integrity checksum changed but does not show actual changes teh way it used to show.

Please note that the server processor and memory appear to be in normal operation while there is plenty of free storage.
Any help would be greatly appreciated. I see the status of the indexer is green, and I do not see any warnings from Filebeat or the Wazuh manager. I have restarted the Wazuh server (the actual server). at the end of the email are the syscheck block that I currently have on the 2 agents and the wazuh server.

Thank you,  
Shujaat

wazuh server syscheck block below

<syscheck>
  <disabled>no</disabled>
  <frequency>43200</frequency> <!-- Run a full scan every 12 hours -->
  <scan_on_start>yes</scan_on_start>
  <alert_new_files>yes</alert_new_files>
  <auto_ignore frequency="10" timeframe="3600">no</auto_ignore>
 
  <!-- Directories to check with real-time monitoring -->
  <directories>/etc,/usr/bin,/usr/sbin</directories>
  <directories>/bin,/sbin,/boot</directories>
 
  <!-- Files/directories to ignore -->
  <ignore>/etc/mtab</ignore>
  <ignore>/etc/hosts.deny</ignore>
  <ignore>/etc/mail/statistics</ignore>
  <ignore>/etc/random-seed</ignore>
  <ignore>/etc/random.seed</ignore>
  <ignore>/etc/adjtime</ignore>
  <ignore>/etc/httpd/logs</ignore>
  <ignore>/etc/utmpx</ignore>
  <ignore>/etc/wtmpx</ignore>
  <ignore>/etc/cups/certs</ignore>
  <ignore>/etc/dumpdates</ignore>
  <ignore>/etc/svc/volatile</ignore>
 
  <!-- File types to ignore -->
  <ignore type="sregex">.log$|.swp$</ignore>
 
  <!-- Check the file, but never compute the diff -->
  <nodiff>/etc/ssl/private.key</nodiff>
 
  <skip_nfs>yes</skip_nfs>
  <skip_dev>yes</skip_dev>
  <skip_proc>yes</skip_proc>
  <skip_sys>yes</skip_sys>
 
  <!-- Nice value for Syscheck process -->
  <process_priority>10</process_priority>
 
  <!-- Maximum output throughput -->
  <max_eps>50</max_eps>
 
  <!-- Database synchronization settings -->
  <synchronization>
    <enabled>yes</enabled>
    <interval>5m</interval>
    <max_eps>10</max_eps>
  </synchronization>
</syscheck>

Wazuh agent syscheck block below

<syscheck>
    <disabled>no</disabled>

    <!-- Frequency that syscheck is executed default every 12 hours -->
    <frequency>3600</frequency>

    <scan_on_start>yes</scan_on_start>

    <!-- Directories to check  (perform all possible verifications) -->
    <directories>/etc,/usr/bin,/usr/sbin</directories>
    <directories>/bin,/sbin,/boot</directories>
    <directories>/home</directories>
    <!-- Files/directories to ignore -->
    <ignore>/etc/mtab</ignore>
    <ignore>/etc/hosts.deny</ignore>
    <ignore>/etc/mail/statistics</ignore>
    <ignore>/etc/random-seed</ignore>
    <ignore>/etc/random.seed</ignore>
    <ignore>/etc/adjtime</ignore>
    <ignore>/etc/httpd/logs</ignore>
    <ignore>/etc/utmpx</ignore>
    <ignore>/etc/wtmpx</ignore>
    <ignore>/etc/cups/certs</ignore>
    <ignore>/etc/dumpdates</ignore>
    <ignore>/etc/svc/volatile</ignore>

    <!-- File types to ignore -->
    <ignore type="sregex">.log$|.swp$</ignore>

    <!-- Check the file, but never compute the diff -->
    <nodiff>/etc/ssl/private.key</nodiff>

    <skip_nfs>yes</skip_nfs>
    <skip_dev>yes</skip_dev>
    <skip_proc>yes</skip_proc>
    <skip_sys>yes</skip_sys>

    <!-- Nice value for Syscheck process -->
    <process_priority>10</process_priority>

    <!-- Maximum output throughput -->
    <max_eps>50</max_eps>

    <!-- Database synchronization settings -->
    <synchronization>
      <enabled>yes</enabled>
      <interval>5m</interval>
      <max_eps>10</max_eps>
    </synchronization>
  </syscheck>

[Luis Daniel Avendaño Larios]

Hi,

Monitoring the entire /etc and /home directories can be inefficient and could lead to floodings due to the large number of files they contain. A more efficient approach is to selectively monitor specific subdirectories within these paths. By doing so, you can focus on relevant areas while avoiding unnecessary overhead and flooding.

Please try a more granular configuration to test that the module is working well, if it is working well everything would indicate that the problem you have presented with is a flood of events.

I will remain attentive to your feedback.

[Shujaat Ali]

Hi Luis,

Thank you for responding. Actually, I included the /home directory on the 2 test computers only.  Where I added a few files to check its functionality. Initially, it was showing file added as event, but now it only report ./bash history. ( detailed event at the end)

I am not monitoring the entire /etc , as there are other directories which are ignored and these are the default settings.  The status shows warning but it is operating.

"full_log": "File '/home/myuser-/.bash_history' modified\nMode: scheduled\nChanged attributes: size,mtime,md5,sha1,sha256\nSize changed from '558' to '624'\nOld modification time was: '1719604677', now it is '1719611310'\nOld md5sum was: '920493b3469a53e92d5efcbc0c3186f8'\nNew md5sum is : 'd03c3300318b4892d54ac2203fbcd600'\nOld sha1sum was: 'df6f3aceaa1bc276c187c6b195405d7af92f0584'\nNew sha1sum is : '31949e24438fe0e29155e7c2c75fc2f7209a7306'\nOld sha256sum was: '107707eaf6cfc52b5e795d328edc6184eff698294cccd53d0e36eb497fef080e'\nNew sha256sum is : 'cb6d600ef72188e25d09847898a91e496020c1c8cf6da4b5bb4ea7dce9822e04'\n","

Note: These are new files added. 

wazuh-indexer.service - Wazuh-indexer
     Loaded: loaded (/lib/systemd/system/wazuh-indexer.service; enabled; preset: enabled)
     Active: active (running) since Mon 2024-07-01 13:09:58 EDT; xxxx
       Docs: https://documentation.wazuh.com
   Main PID: 752 (java)
      Tasks: 130 (limit: 19134)
     Memory: 1.6G
        CPU: 1min 634ms
     CGroup: /system.slice/wazuh-indexer.service
             └─752 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m >

Jul 01 13:09:41 wazuh systemd[1]: Starting wazuh-indexer.service - Wazuh-indexer...
Jul 01 13:09:46 wazuh systemd-entrypoint[752]: WARNING: A terminally deprecated method in java.lang.System has been called
Jul 01 13:09:46 wazuh systemd-entrypoint[752]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/wazuh-indexer/lib/opensearch-2.10>
Jul 01 13:09:46 wazuh systemd-entrypoint[752]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
Jul 01 13:09:46 wazuh systemd-entrypoint[752]: WARNING: System::setSecurityManager will be removed in a future release
Jul 01 13:09:47 wazuh systemd-entrypoint[752]: WARNING: A terminally deprecated method in java.lang.System has been called
Jul 01 13:09:47 wazuh systemd-entrypoint[752]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/wazuh-indexer/lib/opensearch-2.10.0>
Jul 01 13:09:47 wazuh systemd-entrypoint[752]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
Jul 01 13:09:47 wazuh systemd-entrypoint[752]: WARNING: System::setSecurityManager will be removed in a future release
Jul 01 13:09:58 wazuh systemd[1]: Started wazuh-indexer.service - Wazuh-indexer.
lines 1-21/21 (END)

Some logs from indexer:

root@wazuh:/home/myuser-# cat  /var/log/wazuh-indexer/wazuh-cluster.log |grep 'WARN'
[2024-07-01T13:09:55,125][WARN ][o.o.s.c.Salt             ] [node-1] If you plan to use field masking pls configure compliance salt e1ukloTsQlOgPquJ to be a random string of 16 chars length identical on all nodes
[2024-07-01T13:09:55,173][WARN ][o.o.s.a.r.AuditMessageRouter] [node-1] No default storage available, audit log may not work properly. Please check configuration.
[2024-07-01T13:09:56,245][WARN ][o.o.s.p.SQLPlugin        ] [node-1] Master key is a required config for using create and update datasource APIs. Please set plugins.query.datasources.encryption.masterkey config in opensearch.yml in all the cluster nodes. More details can be found here: https://github.com/opensearch-project/sql/blob/main/docs/user/ppl/admin/datasources.rst#master-key-config-for-encrypting-credential-information
[2024-07-01T13:09:57,491][WARN ][o.o.g.DanglingIndicesState] [node-1] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually
[2024-07-01T13:09:58,674][WARN ][o.o.p.c.s.h.ConfigOverridesClusterSettingHandler] [node-1] Config override setting update called with empty string. Ignoring.
[2024-07-01T13:09:58,846][WARN ][o.o.o.i.ObservabilityIndex] [node-1] message: index [.opensearch-observability/Xw1ltJHrT7uCCj00Knnx8Q] already exists
[2024-07-01T13:09:58,880][WARN ][o.o.s.SecurityAnalyticsPlugin] [node-1] Failed to initialize LogType config index and builtin log types
root@wazuh:/home/myuser-# ^C

Note: I restarted the server half an hour ago, currently, it shows INFO events, Attached is a full log since it restarted in txt.

Is there a way I can reset the FIM module?  And also trace the issue to see what is causing it?

Regards

Shujaat

--
You received this message because you are subscribed to a topic in the Google Groups "Wazuh | Mailing List" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/wazuh/4bmsc6VClbU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/1cefbfdf-1e39-44cd-8522-62087ada64a4n%40googlegroups.com.

[Luis Daniel Avendaño Larios]

Hi 

In the configuration shared above the integration runs every 12 hours with the default scan, so if it performs a change, it needs to wait for that interval.

To check if the FIM module is working properly could you check the inventory of an agent in the file integrity monitoring dashboard, this is located in the ☰ menu > endpoint security> File integrity monitoring > inventory tab. You should see a table like the following:

If the table is populated the module should be working correctly. Are the agents you configured in version 4.8.0?

I will remain attentive to your response.

[Shujaat Ali]

Hi,
Thank you very much, yes I do see them under inventory.

But they are not showing on the dashboard.
Regards
Shujaat

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/cf9b28de-518f-4fbd-ad25-0b7ea61300fen%40googlegroups.com.

[Luis Daniel Avendaño Larios]

Hi,

Could you share with me the agent versions and OS of the workstations you are using?

Thanks,

Luis Avendaño.

IT Security Engineer - Wazuh, Inc.

WAZUH - The Open Source Security Platform

[Luis Daniel Avendaño Larios]

Hello,

Did you manage to solve this problem?

If not, could you confirm if It could be possible you have cleared the agents' DBs?

Additionally, can you confirm FIM is working properly by using realtime on a specific directory and adding some changes to it or by checking the agent's log file? Also, can you use restrict to specify which files or directories to monitor on a specified path?

Reference: https://documentation.wazuh.com/current/user-manual/reference/ossec-conf/syscheck.html#directories