How to Reduce Index Size
Mithun Haridas
Hi Team,
I’m looking for guidance on reducing index size. I have already implemented custom rules to drop unnecessary or noisy logs; however, the index size remains significantly large.
Could you please recommend best practices or approaches to further reduce indexing size?
Additionally, I’d like to validate my understanding: I noticed that a large number of unused or unnecessary fields are being decoded by the JSON decoder, and these fields appear in high volume across events. I suspect that this may be contributing to the increased index size. If this assumption is correct, could you please advise on how to prevent such fields from being decoded or indexed, and share recommended methods to optimize index size effectively?
Any suggestions or guidance would be greatly appreciated.
Thank you in advance.
Bony V John
I understand your requirement. To reduce the index size, the main approach is to fine-tune the rules and avoid indexing unwanted or noisy alerts. You can create more specific custom rules to reduce unnecessary alerts. For that, you can refer to the Wazuh custom rules syntax documentation.
The second option you can consider is adding the following option in the rules:
This will stop including the full_log field in the alert. Since the full_log field contains the full raw log shown on the dashboard, removing it can reduce the size of each alert.
Regarding unused fields being indexed, it is possible to exclude specific fields from being indexed in the Wazuh Indexer after matching rules. To do this, you can modify the wazuh-template.json file used by Filebeat.
Example: disabling indexing for data.gcp.protoPayload.authorizationInfo
Back up the current template:
cp /etc/filebeat/wazuh-template.json /root/backup-wazuh-template.json
Stop the Filebeat service:
systemctl stop filebeat
Edit the /etc/filebeat/wazuh-template.json template file and add the mapping under mappings.properties.data.properties:
"gcp": {
"properties": {
"protoPayload": {
"properties": {
"authorizationInfo": {
"enabled": false
}
}
}
}
}
This configuration disables indexing for the data.gcp.protoPayload.authorizationInfo field.
Save the file and run the following commands to insert the new template:
filebeat setup --index-management
systemctl restart filebeat
After this, the specified field will no longer be indexed in Wazuh Indexer, and it will not be available for filtering in the Wazuh dashboard.
Note: This method is not officially recommended unless you are sure that the field is unnecessary for search, filtering, or visualization.
Disabling indexing for a field affects all logs that contain that field, not just a specific event source.
Once disabled, you cannot filter or query based on that field in the Wazuh dashboard.
Always evaluate the impact before disabling field indexing.
Mithun Haridas
Hi Bony,
I am currently testing the method you suggested to drop unwanted fields from indexing, fully understanding the potential impact. However, during testing, the approach did not work as expected, and I would appreciate your guidance on identifying the issue and resolving it.
Objective:
Exclude the field data.office365.PolicyId from being indexed.
Steps performed:
- Took a backup of wazuh-template.json.
- Stopped the Filebeat service.
- Opened the wazuh-template.json file.
- Added the below block toward the bottom of the corresponding section in the template file (only the highlighted portion was added):
- Executed filebeat setup --index-management.
- Restarted the Filebeat service.
After completing these steps, I still observe that the field data.office365.PolicyId is being decoded and remains visible in the Wazuh Dashboard.
Could you please help identify what may be incorrect in this implementation and advise on the correct way to exclude this field from indexing?
For reference, the Wazuh version currently in use is 4.10.3.
Thank you in advance for your support.
Bony V John
"type": "keyword",
"index": false
},
"properties": {
"index": false
},
Then save the file and apply the template:
filebeat setup --index-management
systemctl restart filebeat
After this, the field will still appear in the Wazuh dashboard, but it will not be searchable. This prevents the field from being indexed separately for search.
To verify it, try applying a filter using that field. It should show as an unindexed field, as shown in the screenshot.
If you want to remove the field completely from the dashboard, then you need to configure the ingest pipeline. Follow the steps below:
Back up the existing pipeline file:
cp /usr/share/filebeat/module/wazuh/alerts/ingest/pipeline.json /tmp/pipeline.json
Open the pipeline file:
vi /usr/share/filebeat/module/wazuh/alerts/ingest/pipeline.json
Go to the bottom of the configuration file and find the remove section. After the last remove section, add a comma and insert the following snippet:
{ "remove": { "field": "data.office365.PolicyId", "ignore_missing": true } }
Your configuration should look like this:
{ "remove": { "field": "event", "ignore_missing": true, "ignore_failure": true } },
{ "remove": { "field": "fileset", "ignore_missing": true, "ignore_failure": true } },
{ "remove": { "field": "service", "ignore_missing": true, "ignore_failure": true } },
{ "remove": { "field": "data.office365.PolicyId", "ignore_missing": true } }
Save the configuration and apply the pipeline:
filebeat setup --pipelines
systemctl restart filebeat
After this, the field will no longer be shown in the dashboard as a separate field.
Mithun Haridas
Hi,
The second method you suggested—removing the field completely using the Filebeat ingest pipeline—is working as expected for me. However, the first method (disabling indexing using the Filebeat index template) is still not working, and I would appreciate your guidance in resolving this.
When I opened /etc/filebeat/wazuh-template.json, I noticed that the following mapping for Office 365 already exists:
To disable indexing for the field data.office365.PolicyId, I updated the existing office365 block as suggested and added PolicyId with enabled: false, as shown below:
After applying the changes, I ran filebeat setup --index-management and restarted Filebeat successfully.
However, when validating the result in the Wazuh Dashboard, I still see the field data.office365.PolicyId being indexed and searchable.
Could you please help me understand why this field continues to be indexed despite the template change, and advise on what might be missing or incorrect in this approach?
Thank you in advance for your help.
Bony V John
After updating the template, have you checked the new alerts shown on the dashboard? The template change will only apply to new alerts ingested into Wazuh after the template was updated.
Also, was there any error after running the below commands?
systemctl restart filebeat
If there were any errors, please share the output with us.
Also, please share the modified /etc/filebeat/wazuh-template.json file with us.
Additionally, run the below command in Wazuh Dev Tools. On the dashboard, click the hamburger icon on the top left > Indexer Management > Dev Tools:
Replace <date> with today’s date, run the command, and share the full output with us. Also run the same command with yesterday’s date and share that output as well.
This will help us analyze the issue further.
Mithun Haridas
I did not observe any errors while running the following commands:
Both commands executed successfully.
As requested, I have attached the relevant files and outputs for your review:
- /etc/filebeat/wazuh-template.json
- Output of:
- GET /wazuh-alerts-4.x-2026.04.29
- GET /wazuh-alerts-4.x-2026.04.30
Please let me know if any additional information or logs are required for further analysis.
Thank you for your support.