Question about log storage
69 views
Skip to first unread message
Matthias A
May 10, 2024, 2:37:47 AM5/10/24
to Wazuh | Mailing List
Hi all,
What is the difference between the /var/ossec/logs/archives/archives.log and /var/ossec/logs/archives/2024/May/ossec-alerts-10.log file?
I can clearly see that in the /var/ossec/logs/archives/2024/May/ossec-alerts-10.log the logs are decoded and in /var/ossec/logs/archives/archives.log the logs aren't decoded, besides from that, why is everything kept twice?
Stuti Gupta
May 10, 2024, 3:06:42 AM5/10/24
to Wazuh | Mailing List
Hi team!
Please allow me some time. I'm looking into this query and will update you with an appropriate answer.
Stuti Gupta
May 10, 2024, 6:47:45 AM5/10/24
to Wazuh | Mailing List
Hi
Matthias A
The distinction lies between the /var/ossec/logs/archives/archives.log and /var/ossec/logs/archives/2024/May/ossec-alerts-10.log files.
In /var/ossec/logs/archives/archives.log, raw, unprocessed logs received from Wazuh agents are stored, serving as a comprehensive archive of all agent-sent logs. These logs, often not easily readable in their original format, play essential roles in analysis or troubleshooting, necessitating examination of raw data from agents. However, they consume large storage sizes, which may lead to disk space issues. Additionally, the raw logs in archives.log provide a complete record of unprocessed data from both the agent and manager, regardless of whether they are decoded or matched to any rules.
The ossec-alerts-10.log file within /var/ossec/logs/archives/2024/May/ is specifically designated for containing decoded logs dated May 10, 2024. These logs have undergone thorough processing and decoding by Wazuh, resulting in a more organized and structured view of events. They primarily consist of alerts triggered by events or logs that match predefined or custom rules, enabling them to trigger alerts. These files, stored in month-specific folders, compress logs on a daily basis, with date-based names. By default, Wazuh archives are disabled because they store a large number of logs on the Wazuh server. When enabled, by the logall option in the Wazuh manager configuration file (/var/ossec/etc/ossec.conf). Wazuh archives allow organizations to store and retain security data for compliance and forensic purposes.
Q. I can clearly see that in the /var/ossec/logs/archives/2024/May/ossec-alerts-10.log the logs are decoded and in /var/ossec/logs/archives/archives.log the logs aren't decoded, besides from that, why is everything kept twice?
The Analysis module in the Wazuh server evaluates the decoded logs against rules and records all alerts in /var/ossec/logs/alerts/alerts.log and /var/ossec/logs/alerts/alerts.json files.
In addition to alert logs, Wazuh stores all collected logs in dedicated archive log files, specifically /var/ossec/logs/archives/archives.log and /var/ossec/logs/archives/archives.json. These archive log files comprehensively capture all logs, including those that do not trigger any alerts. This feature ensures a comprehensive record of all system activities for future reference and analysis.
Reference:
https://documentation.wazuh.com/current/user-manual/manager/wazuh-archives.html
https://documentation.wazuh.com/current/user-manual/capabilities/log-data-collection/how-it-works.html
Hope this helps
The distinction lies between the /var/ossec/logs/archives/archives.log and /var/ossec/logs/archives/2024/May/ossec-alerts-10.log files.
In /var/ossec/logs/archives/archives.log, raw, unprocessed logs received from Wazuh agents are stored, serving as a comprehensive archive of all agent-sent logs. These logs, often not easily readable in their original format, play essential roles in analysis or troubleshooting, necessitating examination of raw data from agents. However, they consume large storage sizes, which may lead to disk space issues. Additionally, the raw logs in archives.log provide a complete record of unprocessed data from both the agent and manager, regardless of whether they are decoded or matched to any rules.
The ossec-alerts-10.log file within /var/ossec/logs/archives/2024/May/ is specifically designated for containing decoded logs dated May 10, 2024. These logs have undergone thorough processing and decoding by Wazuh, resulting in a more organized and structured view of events. They primarily consist of alerts triggered by events or logs that match predefined or custom rules, enabling them to trigger alerts. These files, stored in month-specific folders, compress logs on a daily basis, with date-based names. By default, Wazuh archives are disabled because they store a large number of logs on the Wazuh server. When enabled, by the logall option in the Wazuh manager configuration file (/var/ossec/etc/ossec.conf). Wazuh archives allow organizations to store and retain security data for compliance and forensic purposes.
Q. I can clearly see that in the /var/ossec/logs/archives/2024/May/ossec-alerts-10.log the logs are decoded and in /var/ossec/logs/archives/archives.log the logs aren't decoded, besides from that, why is everything kept twice?
The Analysis module in the Wazuh server evaluates the decoded logs against rules and records all alerts in /var/ossec/logs/alerts/alerts.log and /var/ossec/logs/alerts/alerts.json files.
In addition to alert logs, Wazuh stores all collected logs in dedicated archive log files, specifically /var/ossec/logs/archives/archives.log and /var/ossec/logs/archives/archives.json. These archive log files comprehensively capture all logs, including those that do not trigger any alerts. This feature ensures a comprehensive record of all system activities for future reference and analysis.
Reference:
https://documentation.wazuh.com/current/user-manual/manager/wazuh-archives.html
https://documentation.wazuh.com/current/user-manual/capabilities/log-data-collection/how-it-works.html
Hope this helps
Matthias A
May 10, 2024, 7:07:51 AM5/10/24
to Wazuh | Mailing List
Yes, thank you! You're a hero.
Reply all
Reply to author
Forward
0 new messages