Re: Restoring a wazuh old instance to a new instance.
Md. Nazmur Sakib
Hi Bony John,
Hope you are doing well. Thank you for using Wazuh.
Can you share some details about your wazuh cluster? Are you using a multinode setup or a single node? The version of your Wazuh and the operating system of your environment?
Please share this information so that I can guide you further.
Regards
Md. Nazmur Sakib
Hi everyone,I'm trying to backup and restore the wazuh instance with the help of wazuh backup and restore document. The backup process is done but while restoring, the compressed alerts are not sending to the indexer. I added the file path of the decompressed file in the filebeat manifest file and restarted the filebeat. After that I checked the dashboard for the old logs but it is not showing.
Md. Nazmur Sakib
Hi Bony John,
Based on the provided information,
Perform the following actions on your Wazuh server to decompress logs and index them in the new Wazuh indexer:
- Create a Python script called recovery.py on your Wazuh server. This script decompresses all the old logs and stores them in the recovery.json file in the /tmp directory:
touch recovery.py
- Add the following content to the recovery.py script:
#!/usr/bin/env python
import gzip
import time
import json
import argparse
import re
import os
from datetime import datetime
from datetime import timedelta
def log(msg):
now_date = datetime.now().strftime('%Y-%m-%d %H:%M:%S')
final_msg = "{0} wazuh-reinjection: {1}".format(now_date, msg)
print(final_msg)
if log_file:
f_log.write(final_msg + "\n")
EPS_MAX = 400
wazuh_path = '/var/ossec/'
max_size=1
log_file = None
parser = argparse.ArgumentParser(description='Reinjection script')
parser.add_argument('-eps','--eps', metavar='eps', type=int, required = False, help='Events per second.')
parser.add_argument('-min', '--min_timestamp', metavar='min_timestamp', type=str, required = True, help='Min timestamp. Example: 2017-12-13T23:59:06')
parser.add_argument('-max', '--max_timestamp', metavar='max_timestamp', type=str, required = True, help='Max timestamp. Example: 2017-12-13T23:59:06')
parser.add_argument('-o', '--output_file', metavar='output_file', type=str, required = True, help='Output filename.')
parser.add_argument('-log', '--log_file', metavar='log_file', type=str, required = False, help='Logs output')
parser.add_argument('-w', '--wazuh_path', metavar='wazuh_path', type=str, required = False, help='Path to Wazuh. By default:/var/ossec/')
parser.add_argument('-sz', '--max_size', metavar='max_size', type=float, required = False, help='Max output file size in Gb. Default: 1Gb. Example: 2.5')
args = parser.parse_args()
if args.log_file:
log_file = args.log_file
f_log = open(log_file, 'a+')
if args.max_size:
max_size = args.max_size
if args.wazuh_path:
wazuh_path = args.wazuh_path
output_file = args.output_file
#Gb to bytes
max_bytes = int(max_size * 1024 * 1024 * 1024)
if (max_bytes <= 0):
log("Error: Incorrect max_size")
exit(1)
month_dict = ['Null','Jan','Feb','Mar','Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov','Dec']
if args.eps:
EPS_MAX = args.eps
if EPS_MAX < 0:
log("Error: incorrect EPS")
exit(1)
min_date = re.search('(\\d\\d\\d\\d)-(\\d\\d)-(\\d\\d)T\\d\\d:\\d\\d:\\d\\d', args.min_timestamp)
if min_date:
min_year = int(min_date.group(1))
min_month = int(min_date.group(2))
min_day = int(min_date.group(3))
else:
log("Error: Incorrect min timestamp")
exit(1)
max_date = re.search('(\\d\\d\\d\\d)-(\\d\\d)-(\\d\\d)T\\d\\d:\\d\\d:\\d\\d', args.max_timestamp)
if max_date:
max_year = int(max_date.group(1))
max_month = int(max_date.group(2))
max_day = int(max_date.group(3))
else:
log("Error: Incorrect max timestamp")
exit(1)
# Converting timestamp args to datetime
min_timestamp = datetime.strptime(args.min_timestamp, '%Y-%m-%dT%H:%M:%S')
max_timestamp = datetime.strptime(args.max_timestamp, '%Y-%m-%dT%H:%M:%S')
chunk = 0
written_alerts = 0
trimmed_alerts = open(output_file, 'w')
max_time=datetime(max_year, max_month, max_day)
current_time=datetime(min_year, min_month, min_day)
while current_time <= max_time:
alert_file = "{0}logs/alerts/{1}/{2}/ossec-alerts-{3:02}.json.gz".format(wazuh_path,current_time.year,month_dict[current_time.month],current_time.day)
if os.path.exists(alert_file):
daily_alerts = 0
compressed_alerts = gzip.open(alert_file, 'r')
log("Reading file: "+ alert_file)
for line in compressed_alerts:
# Transform line to json object
try:
line_json = json.loads(line.decode("utf-8", "replace"))
# Remove unnecessary part of the timestamp
string_timestamp = line_json['timestamp'][:19]
# Ensure timestamp integrity
while len(line_json['timestamp'].split("+")[0]) < 23:
line_json['timestamp'] = line_json['timestamp'][:20] + "0" + line_json['timestamp'][20:]
# Get the timestamp readable
event_date = datetime.strptime(string_timestamp, '%Y-%m-%dT%H:%M:%S')
# Check the timestamp belongs to the selected range
if (event_date <= max_timestamp and event_date >= min_timestamp):
chunk+=1
trimmed_alerts.write(json.dumps(line_json))
trimmed_alerts.write("\n")
trimmed_alerts.flush()
daily_alerts += 1
if chunk >= EPS_MAX:
chunk = 0
time.sleep(2)
if os.path.getsize(output_file) >= max_bytes:
trimmed_alerts.close()
log("Output file reached max size, setting it to zero and restarting")
time.sleep(EPS_MAX/100)
trimmed_alerts = open(output_file, 'w')
except ValueError as e:
print("Oops! Something went wrong reading: {}".format(line))
print("This is the error: {}".format(str(e)))
compressed_alerts.close()
log("Extracted {0} alerts from day {1}-{2}-{3}".format(daily_alerts,current_time.day,month_dict[current_time.month],current_time.year))
else:
log("Couldn't find file {}".format(alert_file))
#Move to next file
current_time += timedelta(days=1)
trimmed_alerts.close()
While you run the recovery.py script, you need to consider the following parameters:
usage: recovery.py [-h] [-eps eps] -min min_timestamp -max max_timestamp -o
output_file [-log log_file] [-w wazuh_path]
[-sz max_size]
-eps eps, --eps eps Events per second. Default: 400
-min min_timestamp, --min_timestamp min_timestamp
Min timestamp. Example: 2019-11-13T08:42:17
-max max_timestamp, --max_timestamp max_timestamp
Max timestamp. Example: 2019-11-13T23:59:06
-o output_file, --output_file output_file
Alerts output file.
-log log_file, --log_file log_file
Logs output.
-w wazuh_path, --wazuh_path wazuh_path
Path to Wazuh. By default:/var/ossec/
-sz max_size, --max_size max_size
Max output file size in Gb. Default: 1Gb. Example: 2.5
- Run the command below to make the recovery.py script executable:
chmod +x recovery.py
- Execute the script using nohup command in the background to keep it running after the session is closed. It may take time depending on the size of the old logs.
Usage example:
nohup ./recovery.py -eps 500 -min 2023-06-10T00:00:00 -max 2023-06-18T23:59:59 -o /tmp/recovery.json -log ./recovery.log -sz 2.5 &
- Add the /tmp/recovery.json path to the Wazuh Filebeat module /usr/share/filebeat/module/wazuh/alerts/manifest.yml so that Filebeat sends the old alerts to the Wazuh indexer for indexing:
module_version: 0.1
var:
- name: paths
default:
- /var/ossec/logs/alerts/alerts.json
- /tmp/recovery.json
- name: index_prefix
default: wazuh-alerts-4.x-
input: config/alerts.yml
ingest_pipeline: ingest/pipeline.json
- Restart Filebeat for the changes to take effect:
systemctl restart filebeat
Please go through this document to learn more:
Please let me know if this solves your issue or if you need any other information.
Hi Nazmur,I'm using single node. It is wazuh version 4.3 and it is running centos7 VM.
--
You received this message because you are subscribed to the Google Groups "Wazuh | Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/2881afcd-68d0-4176-a561-1c9ea3b2b63en%40googlegroups.com.
Jack Luther
--
Md. Nazmur Sakib
Hi Jack Luther,
The following steps should work.
Please check the parameter you are using while running recovery.py.
While you run the recovery.py script, you need to consider the following parameters:
usage: recovery.py [-h] [-eps eps] -min min_timestamp -max max_timestamp -o
output_file [-log log_file] [-w wazuh_path]
[-sz max_size]
-eps eps, --eps eps Events per second. Default: 400
-min min_timestamp, --min_timestamp min_timestamp
Min timestamp. Example: 2019-11-13T08:42:17
-max max_timestamp, --max_timestamp max_timestamp
Max timestamp. Example: 2019-11-13T23:59:06
-o output_file, --output_file output_file
Alerts output file.
-log log_file, --log_file log_file
Logs output.
-w wazuh_path, --wazuh_path wazuh_path
Path to Wazuh. By default:/var/ossec/
-sz max_size, --max_size max_size
Max output file size in Gb. Default: 1Gb. Example: 2.5
Example:
nohup ./recovery.py -eps 500 -min 2023-06-10T00:00:00 -max 2023-06-18T23:59:59 -o /tmp/recovery.json -log ./recovery.log -sz 2.5 &
Please make sure you have your log file after -log parameter and keep recovery.py and recovery.log in the same directory if you are following the above directory. Note that here recovery.log is the log file.
Also, keep in mind that execute the script using nohup command in the background to keep it running after the session is closed. It may take time depending on the size of the old logs
Check if you are getting any errors while running the command. Run the command without nohup to check what is happening. But keep in mind the process will discarded if the session is closed.
./recovery.py -eps 500 -min 2023-06-10T00:00:00 -max 2023-06-18T23:59:59 -o /tmp/recovery.json -log ./recovery.log -sz 2.5 &
I recommend you test it in a test environment before deploying it to the production server.
Also, make sure to Add the /tmp/recovery.json path to the Wazuh Filebeat module /usr/share/filebeat/module/wazuh/alerts/manifest.yml so that Filebeat sends the old alerts to the Wazuh indexer for indexing:
module_version: 0.1
var:
- name: paths
default:
- /var/ossec/logs/alerts/alerts.json
- /tmp/recovery.json
- name: index_prefix
default: wazuh-alerts-4.x-
input: config/alerts.yml
ingest_pipeline: ingest/pipeline.json
Finally, restart filebeat.
Check every step carefully to find out what you are missing.
I hope you find this helpful.
Regards
Md. Nazmur Sakib
Hi Sinu Soman,
This Restoring Wazuh from backup document was introduced for version 4.4 and above.Hi Nazmur,The compressed files are getting decompressed after running the python script using nohup command and tried without nohup it is also working. I'm able to view the old alerts as json format in /tmp/recovery.json file. After given the path to the filebeat and restarted but it is not showing in the dashboard. I'm using wazuh 4.3 and in wazuh document their backup document for 4.3 but there is restore document for version 4.3. why?