Efficient methods for searching archived Wazuh logs with unknown attribute locations?

[Emr]

Hi everyone,

I need to search through 5 months of archived Wazuh alerts (gzipped JSON files), but I'm facing a challenge:

The Problem: I need to find logs containing specific values (like IP addresses, usernames, file paths, etc.), but I don't know in advance which JSON attributes/fields they might appear in. The values could be in various nested fields like data.srcip, data.dstip, data.win.eventdata.*, or other locations.

Current approach: Using grep + jq, but manually searching through each compressed archive month by month is extremely slow and inefficient for large datasets.

Question: What's the best practice for recursively searching archived Wazuh logs when you need to check all JSON attributes/fields without knowing the exact field names beforehand?

Has anyone solved this? Any recommended tools, scripts, or alternative archival strategies?

Thanks in advance!

[Md. Nazmur Sakib]

Hi Emir,

You can index old archive logs to your indexer and analyze them from the dashboard.

For this, follow these steps.


Download the script (recovery.py) from the blog page here.

1. Let's create a duplicate of the original script: 

• cp recovery.py recover-arch.py

3. We need to make one little adjustment to the recover-arch.py script:

• sed -i 's|logs/alerts/{1}/{2}/ossec-alerts-|logs/archives/{1}/{2}/ossec-archive-|1' recover-arch.py

4. Let's create a new empty file in the /tmp folder: 

• touch /tmp/recover-arch.json

5. Now, let's change the manifest for the archives module to monitor the new file (or edit the file manifest.yml in /usr/share/filebeat/module/wazuh/archives and add the path to the new file):  We will replace this - /var/ossec/logs/archives/archives.json with #      - /var/ossec/logs/archives/archives.json      - /tmp/recover-arch.json Run this command:

• sed -i 's|      - /var/ossec/logs/archives/archives.json|#      - /var/ossec/logs/archives/archives.json\n      - /tmp/recover-arch.json\n|1' /usr/share/filebeat/module/wazuh/archives/manifest.yml Edit /etc/filebeat/filebeat.yml and change enabled to true in the archives mapping. This enables events to be forwarded to the Wazuh indexer.

filebeat.modules:

 - module: wazuh

 alerts:

   enabled: true

 archives:

  enabled: true

Restart the Filebeat service to apply the change:

• systemctl restart filebeat

7. We are ready to execute the script to recover archives, here's an example, replace the parameters accordingly to your needs:

• nohup /var/ossec/framework/python/bin/python3 recover-arch.py -eps 1000 -min 2026-01-01T00:00:00 -max 2026-01-08T23:59:59 -o /tmp/recover-arch.json -log /tmp/recover-arch.log -sz 4 &

You can read more about the
Blog - Recover your data using Wazuh alerts backups

Now follow the defining-the-index-pattern section of this blog


I will suggest not restoring all the logs at once. Better restore a few days of logs analys them, and delete them before restoring more. The archive index can take up lots of disk space.


Let me know if this works for you or if you need any help.