Problems Getting Wazuh to Process Existing Log Files

[Evair Silva]

Hello Comunity,

I have a question and would appreciate some guidance. Is it possible for Wazuh to read logs from pre existing files?

The logs I need Wazuh to consume are already generated and stored in .json files (formatted as NDJSON). In other words, I’ve extracted the logs from their original source and now need to feed them into Wazuh for monitoring. These files are located on the manager server, and I’ve configured log collection in the ossec configuration file. However, when I move the file into the monitored directory, nothing happens.

I ran some ingestion tests using the echo command on a file, and that worked the logs were processed, and I can see them in the dashboard.

I’d like to understand how to get Wazuh to read these existing files. Do I need to use a script? Should these logs be sent to an agent for monitoring?

This has been quite frustrating, as I haven’t been able to figure out how to properly monitor these logs.

My sincere apologies I’m new to Wazuh, so I’m still finding my way around.

Regards.

[Md. Nazmur Sakib]

Hi Evair,

If you need those old logs ingested to Wazuh, you may need to create a new file, add the new file in the localfile configurtaion, then write a script to parse the logs line by line to the newly created file from the old log file so the agent can start ingesting the logs. That is the only way you are able to get Wazuh to read from the old logs.

I am not very good at coding, but you can use a bash script like this.

Make a bash file.
incremental_log_copy.sh

Copy the script to the bash file added in the attachment.

Run:

  chmod +x incremental_log_copy.sh

  sudo ./incremental_log_copy.sh -src_log /var/log/old.json -dst_log /var/log/new.json -logs_per_run 20 -sleep_seconds 30

Parameters:

  -src_log                Source log file to read from (e.g., /var/log/old.json)

  -dst_log                Destination file to append logs to (e.g., /var/log/new.json)

  -logs_per_run      Number of log lines appended per run (e.g., 20)

  -sleep_seconds    Seconds to wait between runs (e.g., 30)


It's better not to push all the logs at once to avoid potential log flooding issues.

Make changes to the script based on your needs.


Make sure to configure the localfile configuration before running this script.

Also, keep in mind that these logs will be added to Wazuh with the current timestamp, not the actual timestamp of the original logs.

Let me know if you need any further information.