How to enable compression archives.log and alerts.log in WAZUH-DASHBOARD after 30 days of saving them?

[Adam Nowak]

How to enable compression (reduce file size) of archives.log and alerts.log in WAZUH-DASHBOARD after 30 days of saving them? And how to enable deletion of files archives.log and alerts.log files with a creation date older than 2 years in WAZUH-DASHBOARD?

[Jose Camargo]

Hello Adam,

Wazuh does not take care of these logs, you have to set up a script on your own that manages this; you can use the following one as an example:

#!/bin/bash

# Set locale to ensure month names are in English

export LC_ALL=C

# Define the directory paths

ALERTS_DIR="/var/ossec/logs/alerts"

ARCHIVES_DIR="/var/ossec/logs/archives"

# Calculate the date 30 days ago

THIRTY_DAYS_AGO=$(date -d "30 days ago" +"%Y/%b")

DAY_PART=$(date -d "30 days ago" +"%d")

# Function to clean up files older than 30 days in a given directory with a given pattern

cleanup_old_files() {

  local DIR=$1

  local PATTERN=$2

  local EXTENSIONS=$3

  # Check if the directory exists

  if [ ! -d "$DIR" ]; then

    TIMESTAMP=$(date +"%Y-%m-%d %H:%M:%S")

    echo "${TIMESTAMP} Error: Directory $DIR does not exist." >> /var/log/messages

    return 1

  fi

  # Iterate over each extension

  for EXT in $EXTENSIONS; do

    # Construct the file pattern to match

    FILE_PATTERN="${DIR}/${THIRTY_DAYS_AGO}/${PATTERN}-${DAY_PART}.${EXT}"

    # Check if any files match the pattern

    if compgen -G "$FILE_PATTERN" >/dev/null; then

      # Remove the files that match the pattern

      rm -f $FILE_PATTERN

      TIMESTAMP=$(date +"%Y-%m-%d %H:%M:%S")

      echo "${TIMESTAMP} Files with extension .$EXT older than 30 days have been deleted in $DIR." >> /var/log/messages

    else

      TIMESTAMP=$(date +"%Y-%m-%d %H:%M:%S")

      echo "${TIMESTAMP} No files found with extension .$EXT matching the pattern $FILE_PATTERN in $DIR." >> /var/log/messages

    fi

  done

}

# Clean up old alert files

cleanup_old_files "$ALERTS_DIR" "ossec-alerts" "log log.sum json sum log.gz json.gz json.sum"

# Clean up old archive files

cleanup_old_files "$ARCHIVES_DIR" "ossec-archives" "log log.sum json sum log.gz jon.gz json.sum"

And then set this cronjob:

0 2 * * * /root/logcleanup.sh >/dev/null 2>&1

This will delete all logs from the /var/ossec/logs/alerts and  /var/ossec/logs/archives directories older than 30 days. Please modify this as needed.

Regards,

Jose

[Adam Nowak]

Thanks for your help. 

By the way, I mean archiving important logs. 

What logs and where are they stored, if you are talking about logs important for WAZUH-manager/indexer (those logs that Wazuh takes care of) ?

[Jose Camargo]

Hi Adam,

Everything under /var/ossec/logs/alerts and  /var/ossec/logs/archives is important as they refer to the raw logs you've received in your environment. Then for other important log you have /var/ossec/logs/wazuh for wazuh application logs and /var/log/wazuh-indexer/ for Indexer logs.

Please let me know if there's anything else we can help you with.

Regards,

Jose

[Adam Nowak]

Hi Jose,

your script ( Deletion of archives.log and alerts.log files with a creation date older than 2 years) works perfect after little changes.

But I still don't know how to enable compression (reduction of file size) archives.log and alerts.log in WAZUH-DASHBOARD after 30 days from saving them (instead of the default 90 days)? 

#!/bin/bash

#during the test run, displays messages when each line of the script is run

set -x

#sends a script confirmation message to the /var/log/usuwanie_archives.log file

echo "Skrypt uruchomiony: $(date)" >> /var/log/usuwanie_archives.log

 # Set locale to ensure month names are in English

export LC_ALL=C

 # Define the directory paths

ALERTS_DIR="/var/ossec/logs/alerts"

ARCHIVES_DIR="/var/ossec/logs/archives"

 # Calculate the date 732 days ago

OLDER_THAN=$(date -d "732 days ago" +"%Y/%b")

DAY_PART=$(date -d "732 days ago" +"%d")

 # Function to clean up files older than a specified number of days in a given directory with a given pattern

cleanup_old_files() {

  local DIR=$1

  local PATTERN=$2

  local EXTENSIONS=$3

  # Check if the directory exists

  if [ ! -d "$DIR" ]; then

    TIMESTAMP=$(date +"%Y-%m-%d %H:%M:%S")

    echo "${TIMESTAMP} Error: Directory $DIR does not exist." >> /var/log/messages

    return 1

  fi

     # Iterate over each extension

  for EXT in $EXTENSIONS; do

    # Construct the file pattern to match

    FILE_PATTERN="${DIR}/${OLDER_THAN}/${PATTERN}-${DAY_PART}.${EXT}"

       # Check if any files match the pattern

    if compgen -G "$FILE_PATTERN" >/dev/null; then

      # Remove the files that match the pattern

      rm -f $FILE_PATTERN

      TIMESTAMP=$(date +"%Y-%m-%d %H:%M:%S")

      echo "${TIMESTAMP} Files with extension .$EXT older than 732 days have been deleted in $DIR." >> /var/log/messages

    else

      TIMESTAMP=$(date +"%Y-%m-%d %H:%M:%S")

      echo "${TIMESTAMP} No files found with extension .$EXT matching the pattern $FILE_PATTERN in $DIR." >> /var/log/messages

    fi

  done

}

 # Clean up old alert files

cleanup_old_files "$ALERTS_DIR" "ossec-alerts" "log log.sum json sum log.gz json.gz json.sum"

 # Clean up old archive filescleanup_old_files "$ARCHIVES_DIR" "ossec-archives" "log log.sum json sum log.gz jon.gz json.sum"