Difference between normalization and decoding

[Matthias A]

Hi all

An admin told me that this is the flow of a log in wazuh:

Log Collection: The process begins with log collection. Wazuh agents are responsible for collecting logs from various sources, such as system logs, application logs, network logs, etc.
Normalization/Decoding: Once the logs are collected, they are normalized or decoded. This step involves parsing the raw log data into a structured format that Wazuh can understand and work with. Decoders are used here to convert logs from their original format into a common format for analysis.
Analysis and Parsing: After normalization, the logs are analyzed. Wazuh parses the normalized logs and extracts relevant information, such as timestamps, source IP addresses, usernames, etc. This parsed data is then indexed for easier search and retrieval.
Rule Matching: Wazuh applies rules to the parsed logs to detect security incidents or anomalies. These rules define conditions or patterns that, when matched, indicate potential security threats or policy violations. If a log matches a rule, an alert is generated.
Alerting: When an alert is generated, Wazuh can take various actions based on configured policies. This may include sending notifications, logging the alert to a centralized location, executing custom scripts, or taking other automated response actions.
Storage and Indexing: The parsed and analyzed logs, along with any generated alerts, are stored and indexed for later retrieval and analysis. This allows security analysts to search and query the logs efficiently.

My question is: What is the difference between decoding and normalizing a log?i If I can take a guess, I'd say normalization is when there is no decoder available or something like that? 

[Roman Luna]

Hi,

The definition for normalization in this case is: Log normalization is the process of converting each log data field or entry to a standardized data representation and categorizing it consistently.

For each part of the log that is decoded, the information is set into a field that is then seen in the dashboard. Most likely, you will see data.<name> when using custom decoders. However, when a decoder is not available, you won't be able to create an alert because you need both a decoder and a rule to create one. After ingesting a log, it goes first to the decoding phase then into checking the rules.

Let me know if this answer your question.

Regards.