No data on dashboard, but data present in log files

[Kamil Tańcula]

Hello,

I’m experiencing an issue with Wazuh.
A few days ago, around 2 a.m., all data stopped appearing on the dashboard — not only the alerts, but also other sections such as MITRE ATT&CK and Threat Hunting etc.

However, I can see data growth in the following files, which indicates that events are still being registered:

• /var/ossec/logs/alerts/alerts.json

• /var/ossec/logs/alerts/alerts.log

I upgraded Wazuh to version 4.13.1, but that didn’t solve the problem.
All services, including Filebeat, are running correctly.
Restarting the server also didn’t help.

Everything stopped being logged on the dashboard exactly at 2 a.m., and since then, no new data has appeared in any of the affected sections.

I’ve analyzed everything I could but haven’t been able to identify the cause.
Could you please assist me in troubleshooting this issue?

[Bony V John]

Hi,

From the shared input and screenshots, it appears that the issue might be related to indexing or a Wazuh indexer shard limit problem.
Based on your description, alerts are being generated and written in the alerts.json file, the Filebeat test output looks fine, and older alerts are visible on the dashboard.

Typically, when the Wazuh indexer shard limit (1000 shards per node) is reached, the indexer goes into read-only mode, preventing new alerts from being indexed—while older alerts remain viewable on the dashboard.

You can follow the steps below to troubleshoot the issue:

• Check the Wazuh indexer health

curl -k -u admin:<password> https://localhost:9200/_cluster/health?pretty

Replace <password> with your Wazuh dashboard admin credentials and localhost with your Wazuh indexer IP if running on a different server.

• In the output, review the active_shards count.
• If it’s 1000, the shard limit may be causing this issue.
• You can refer to the Wazuh Google Group discussion on shard limits for further guidance.

• If sufficient shards are available (i.e., fewer than 1000), check for errors in Filebeat logs:

cat /var/log/filebeat/filebeat | grep -iE "error|warn|crit|fatal"

Look for any indexing-related errors.

• Check the Wazuh indexer logs

cat /var/log/wazuh-indexer/wazuh-cluster.log | grep -iE "error|warn|crit|fatal"

Verify whether there are any connection or authentication-related issues. Validate your Wazuh manager ossec.conf file using this Wazuh documentation.

For further assistance, please share the outputs of the above commands so we can analyze the issue in more detail.

[Kamil Tańcula]

Yes, that's right, it's about the shard limit.

The filebeat log contains the following message:

{“type”:“validation_exception”,‘reason’:“Validation Failed: 1: this action would add [3] total shards, but this cluster currently has [998]/[1000] maximum shards open;”}

I set the wazuh-alert-retention-policy in Index Management, which deleted the old shards, and the data is now starting to be visible in the system.

Thank you for your help.