Not Showing Alerts or archives in wazuh dashboard

[Dhiren Chavda]

Hi Team,

I have a single node installation and from last two days i am not getting any alerts or even archives and last alert or archive is from two days before i have checked the ossec.log and also alerts.json and have received logs in that but not showing in dashboard.

I also tried restarting all the services and updated all the agents with the server version but still no good.

I have also attached the screenshot for your reference 

kindly look into this 

[hasitha.u...@wazuh.com]

Hi  Dhiren ,

I believe logs are reaching the Wazuh manager. However, we need to verify the latest logs received in alerts.json file. Please share the output of this.
tail /var/ossec/logs/alerts/alerts.json

I believe all Wazuh components are up and running.
However, Could you verify that all the services are up and running?
systemctl status wazuh-manager
systemctl status wazuh-indexer
systemctl status wazuh-dashboard
systemctl status filebeat

If yes,
Try restarting the services and checking again. If the issue is not resolved, can you share the following details to check further?

First, check cluster health.
If you can access to Wazuh dashboard, then try to navigate to Index Management > Dev Tools
Use this command:
GET _cluster/health

If you want to check in CLI try this command.
curl -XGET -k -u admin:pass "https://localhost:9200/_cluster/health"

Please share the cluster health command output to check further.

Also, share the output of these commands.
systemctl status filebeat
filebeat test output

Further, check the storage and memory usage while running all components.
free -h
top
df -h

Additionally, share the Indexer and filebeat logs to check further.
cat /var/log/wazuh-indexer/wazuh-cluster.log | grep -i -E "error|warn"
cat /var/log/filebeat/filebeat | grep -i -E "error|warn"

Also let me know the version of your Wazuh.
/var/ossec/bin/wazu-contrl info

Let me know the update on this so I can check further.

[Dhiren Chavda]

Hi team,

As you suggested i checked everything and everything is working good 

Cluster health is green 

Filebeat working 

sufficient storage

 I have attached the Indexer and filebeat logs 
 version 4.9.2

[hasitha.u...@wazuh.com]

Hi  Dhiren,

From indexer logs, It seems that your indexer has reached the maximum shard limit.
cluster currently has [1000]/[1000] maximum shards open

A single cluster can have 1000 shards at maximum. If you have one indexer cluster you need to add another indexer node or delete some old indices from your server to free up some space.

Adding Wazuh indexer nodes

To delete old indices go to
Index Management > Indices
Search with Wazuh-alerts
Select the indices you want to delete.
Click on Action and select Delete from the drop-down.
Check the screenshot for reference.



I will also suggest you check ILM and snapshot documents for better management of your indices to avoid facing issues in the future.
https://documentation.wazuh.com/current/user-manual/wazuh-indexer/index-life-management.html
https://documentation.wazuh.com/current/user-manual/wazuh-indexer/migrating-wazuh-indices.html

Let me know how it goes.

[Dhiren Chavda]

but why did this issue arrived its been working from last 2 years what can be the possible cause?

[hasitha.u...@wazuh.com]

Hi Dhiren,

This issue has arisen because your Wazuh indexer cluster has reached the maximum shard limit of 1000 shards. While it has been working fine for the last two years, several factors could have led to this situation:

By default, the shard limit per indexer is 1000.

Accumulation of Indices Over Time

• Over the years, Wazuh continuously creates new indices for alerts and events. If you haven’t been managing or deleting older indices, they keep accumulating, gradually filling up the shard capacity.

Retention Policy Not Applied or Changed

• If no index lifecycle policy is set to automatically delete old indices, they remain in the system indefinitely, consuming available shards.

I suggest you check ILM and snapshot documents for better management of your indices to avoid facing issues in the future.
https://documentation.wazuh.com/current/user-manual/wazuh-indexer/index-life-management.html
https://documentation.wazuh.com/current/user-manual/wazuh-indexer/migrating-wazuh-indices.html

Let me know if this helps.

Regards,
Hasitha Upekshitha