Indexer not indexing

[igor]

Hello everyone,

I'm fairly new to Wazuh - or rather, I've been using it for months, but's it an all-in-one installation (deb packages on Ubuntu) with very little customization so I never had to learn the insides. Until a few days ago when I noticed that events stopped appearing in the dashboard. Last events were recorded on 31st January, moments before midnight.

I searched the web, found it's a common symptom with many causes and did some troubleshooting:

- there were no recent changes on the server, nobody even logged in since January 22

- I updated packages, including Wazuh, on January 22 (maybe I should have done something during the upgrade, but if that's the root cause why it took 9 days to show up)

- my first thought was expired certificates, but no, they're valid for years

- I also reset the passwords (probably they were OK, but I needed it for testing)

- restarted all the components, rebooted the server

- I found commands for filebeat tests and curl for cluster health and indices - filebeat is OK, cluster health is green (but it shows 1000 active and primary shards, is that fine?),  indexes are not created, last ones are from January 31

- checked the flow of data through wazuh components: manager gets the events, sends them to filebeat, filebeat sends them to opensearch

- here's what I can see in filebeat logs:

2026-02-05T09:20:31.282Z        WARN    [elasticsearch] elasticsearch/client.go:408     Cannot index event [...] (status=400): {"type":"validation_exception","reason":"Validation Failed: 1: this action
would add [3] total shards, but this cluster currently has [1000]/[1000] maximum shards open;"}

At this point I'm stuck. I found many posts about similar problems, but in all of them people where able to find some kind of network or permission problem which isn't present in my case. What should I check next?

Igor

[Md. Nazmur Sakib]

Hi Igor,


If you look at this log.

: this action would add [3] total shards, but this cluster currently has [1000]/[1000] maximum shards open;"}

It seems that your indexer has reached the maximum shard limit.

A single-node indexer cluster can have up to 1000 shards by default. If you have one indexer cluster, you need to add another indexer node or delete some old indices

from your server to free up some space.


To add more Wazuh indexer nodes, follow this document:
https://documentation.wazuh.com/current/user-manual/wazuh-indexer-cluster.html#adding-wazuh-indexer-nodes>

To delete old indices, go to

Index Management > Indices

Search with Wazuh-alerts

Select the indices you want to delete

Click on Action and select Delete from the drop-down.

Check the screenshot for reference.

.


By default, Wazuh alerts have three shards per index. You can change the number of shards per index to one if you have a single-node index.
https://documentation.wazuh.com/current/user-manual/wazuh-indexer/wazuh-indexer-tuning.html#setting-the-number-of-shards

I will also suggest you check the ILM and snapshot documents for better
Management of your indices.
https://documentation.wazuh.com/current/user-manual/wazuh-indexer/index-life-management.html
https://documentation.wazuh.com/current/user-manual/wazuh-indexer/migrating-wazuh-indices.html
There is an option for increasing the shard limit of the index. But it is not advisable in the long run, as it can bring more problems in the future.
After that, you can recover your missing alerts using this document.
https://wazuh.com/blog/recover-your-data-using-wazuh-alert-backups/


I hope you find this information useful. 

[igor]

Thank you! The moment I deleted first few indexes, indexer immediately created one for February 5 alerts. I also created the policy and I can see it's slowly going through the indexes. I don't want to add another indexer - Wazuh is intimidating enough with a all-on-one setup and the current server has very little load. I can live with shorter retention.

Best wishes,

Igor

[Md. Nazmur Sakib]

I suggest you change the number of shards to 1 for each index. The default number is 3 shards for the alerts indices, but as you have a single indexer node. Shards are a kind of packet of data. When you have 3 shards, you are keeping the data in three small packets. If you make it one, you are keeping it one big packet.

Go to this index file /etc/filebeat/wazuh-template.json

Once you enter the file, then edit the index.number_of_shards to 1 and index.auto_expand_replicas to false

"settings": {

    "index.refresh_interval": "5s",

    "index.number_of_shards": "1",

    "index.number_of_replicas": "0",

    "index.auto_expand_replicas": "false",

Now load the configuration and restart the filebeat.

sudo filebeat setup -index-management
sudo systemctl restart filebeat

After configuring this, the new indices will have one primary shard instead of three.       


Ref:https://documentation.wazuh.com/current/user-manual/wazuh-indexer/wazuh-indexer-tuning.html#setting-the-number-of-shards


For the old indices, if you want to make changes in the number of shards, you will need to reindex every indices one by one.

Ex:

Go to Indexer Management > Dev Tools

Make a backup index.

POST _reindex

{

  "source": {

    "index": "wazuh-alerts-4.x-2026.02.05"

  },

  "dest": {

    "index": "wazuh-alerts-4.x-backup"

  }

}

Delete the main index

DELETE /wazuh-alerts-4.x-2026.02.05

Create the main index from back-up

POST _reindex

{

  "source": {

    "index": "wazuh-alerts-4.x-backup"

  },

  "dest": {

    "index": "wazuh-alerts-4.x-2026.02.05"

  }

}

Delete the backup index

DELETE /wazuh-alerts-4.x-backup

Ref:
https://documentation.wazuh.com/current/user-manual/wazuh-indexer/re-indexing.html

Let me know if you need any further help on this.