recover deleted indices

[Bilal Al-saghier]

Hi Wazuh team!

I need your kind support with this issue, please. While I was testing index lifecycle policy (I created a policy to transfer wazuh-archives-* from Hot to delete (after one day) and I wanted to apply this to one index "for testing". What happened this policy has become active for all archived indices coming after its creation. Now I understand that I should stop this policy or modify it to prevent further unintentional deletion of new archives. 

My question is, since I have the raw logs in the local hard drive, is there a way to re-upload the deleted indices to the indexer again?

Noting the system info and version: Wazuh 4.3.9 (Indexer + Manager + Dashboard)

Thanks for your support.. 

Regards,

Bilal. 

[Julio Gasco]

Hi Bilal,

We have a recovery procedure to restore the alerts back into the wazuh-indexer so you can see them in your dashboard.

Below is a link to the procedure:

https://wazuh.com/blog/recover-your-data-using-wazuh-alert-backups/

This is an old post so there are a few things that will look different, here are some points to take into account

• The  recovery.py section will work with that same procedure you will end up executing something like this 

nohup ./recovery.py -eps 500 -min 2022-07-21T13:59:30 -max 2022-12-15T22:00:00 -o /tmp/recovery.json -log ./recovery.log -sz 2.5 &

With your min and max dates set to the time you want to recover, this will output the logs to be ingested into the /tmp/recovery.json file

• When editing the file  /usr/share/filebeat/module/wazuh/alerts/manifest.yml you will see a different format on your server than the one mentioned in the documentation. With the actual versions it will look something like this after editing it:

• Finally take into consideration the idea is to run the filebeat recovery while the recovery.json file is being generated, so once the nohup command is generating the file you can configure and restart filebeat so it starts ingesting logs while they are being gathered. 

Let me know if this helped!

Regards!

[Bilal Al-saghier]

Hi Julio,

Thanks a lot for your reply! Actually, the steps mentioned in the documents has become more clear now with your notes! So thanks a lot!

Just need to add something that may other people find helpful. The case mentioned in the document is about recovering lost alerts from 'wazuh-alerts-*' index. In my situation, I needed to recover lost alerts from 'wazuh-archives-*' index. So that, I had to modify two parameters:

1-The alert_file parameter within the recovery.py script from alert_file = "{0}logs/alerts/{1}/{2}/ossec-alerts-{3:02}.json.gz".format(wazuh_path,current_time.year,month_dict[current_time.month],current_time.day)  as in the document to alert_file = "{0}logs/archives/{1}/{2}/ossec-archive-{3:02}.json.gz".format(wazuh_path,current_time.year,month_dict[current_time.month],current_time.day). Notice the yellow-highleted changes.

2- for the yaml file to be edited for filebeat service to start alert indexing from recovery.json, I had to edit the file /usr/share/filebeat/module/wazuh/archives/manifest.yml, instead.

and in my wazuh server the file looked like this

____________________________________________________

module_version: 0.1

var:
  - name: paths
    default:
      - /var/ossec/logs/archives/archives.json
      - /tmp/recovery.json
  - name: index_prefix
    default: wazuh-archives-4.x-

input: config/archives.yml

ingest_pipeline: ingest/pipeline.json

_____________________________________________________

Now after applying the two modifications, to make it match my requirements, and taking into accounts the notes illustrated from your side and the shared documentation I was able to recover the lost alerts to the index wazuh-archives-4.x-2022.12.13 successfully as you can see below .

Thanks a lot Julio!! I really appreciate your support!

Regards,

Bilal