The full log time is different from the local time

[tsai kk]

  I noticed that the time of the alerts I received is different from the time shown in the full log. How can I make the log match the local time?  

[Antonio David Gutiérrez]

Hi, reviewing the screenshots:

Context: The Wazuh alerts define the timestamp field, not the @timestamp field. The @timestamp field is set through a processor in the ingest pipeline in the Wazuh indexer side for historical reasons. This means the Wazuh alerts contain the timestamp and @timestamp fields that should have the same value. See alerts ingest pipeline https://github.com/wazuh/wazuh/blob/v4.9.0/extensions/filebeat/7.x/wazuh-module/alerts/ingest/pipeline.json#L94-L102 and archives ingest pipeline https://github.com/wazuh/wazuh/blob/v4.9.0/extensions/filebeat/7.x/wazuh-module/archives/ingest/pipeline.json#L95-L102. The alerts and archives ingest pipelines are defined in the wazuh module for Filebeat, that indexes them to the Wazuh indexer.

I see the @timestamp has a difference of the date of the timestamp in the log. This is probably caused for the configuration of Wazuh dashboard to display the date fields. They are formatted to a specific format and they can take into account the timezone of your browser (or another one if this is specified in the setting). You can change the format or timezone to display the date fields on Dashboard management > Advanced settings. You can change the timezone through the dateFormat:tz setting.

For another hand, I see a difference in the second between the date in the full_log property (36) and the @timestamp (37). This is caused because the Wazuh server sets the timestamp field to the date when the alert was generated/processed. In the case you want the timestamp (and @timestamp) field has the equivalent value of the event, you could need to redefine the value of timestamp after this was generated by the Wazuh server. For example in the ingest pipeline of the Wazuh indexer, you could add a processor before to the processor that sets the @timestamp field. Depending on the log content, the date could be decoded in a specific field that you could use to redefine the timestamp and @timestamp fields to that value or you could need to enhance the log decoding to have a field with the date that can be copied to the timestamp or @timestamp fields.

For example, if you want to set the timestamp and @timestamp fields to the date value in the event log and taking into account the field for that log is decoded into the predecoder.timestamp field:
1. Edit the alerts (and archives if used) ingest pipeline definitions:

Warning: before editing files, you could consider take a backup of the original file.

Set the following ingest pipeline processor before the processor that sets the @timestamp field:
    {
      "date": {
        "field": "predecoder.timestamp",
        "target_field": "timestamp",
        "formats": ["MMM dd HH:mm:ss"],
        "ignore_failure": true
      }
    },

This ingest pipeline processor sets the timestamp field with the value of predecoder.timestamp field. The predecoder.timestamp field has the specified format, you could add more expected format here for other event logs. I added the ignore_failure option to true to ignore some failure of the processor avoiding the event can not be indexed due the processor can not be applied correctly.

2. Index the pipelines
filebeat setup --pipelines

3. Ensure the event has the expected values for timestamp and @timestamp fields.

Result:

Here you can see the date matches the second but it displays a difference related to the timeszone (dateFormat:tz is set as Browser):


Here you can see the date matches (dateFormat:tz is set as UTC):


Note: if you edited the ingest pipeline definition in the files of the wazuh module for Filebeat, ensure you apply the same change to all the Wazuh servers of the same cluster else some of them could redefine the ingest pipeline losing this change.
Note: if you edited the ingest pipeline definition, the configuration only applies to the new events that are generated after the change is applied.

Ingest pipeline processors documentation: https://www.elastic.co/guide/en/elasticsearch/reference/7.10/ingest-processors.html

[tsai kk]

  Hello, my requirement is to have the log time match my timestamp, as my timestamp is my local time and it is correct. However, the log time is missing +8 hours. How can I adjust this?

  

Antonio David Gutiérrez 在 2024年9月30日 星期一下午4:38:11 [UTC+8] 的信中寫道：

[Antonio David Gutiérrez]

The full_log field of the Wazuh alert represents the raw log that was analyzed by the Wazuh server and this should be considered as a reference. Depending on the log type and the decoders applied, it could have extracted into a specific field the date of the event that depending on the field this could be displayed taking into account your browser timezone through the Wazuh dashboard settings, so you could relate when the event ocurred in your timezone.

I understand you pretend the full_log field displays the date taking into account your timezone.

So, I guess the options are:
- the collected log represents the date with your timezome applied: depending on the log source, maybe it could be done through some setting or you could need to use some mechanism to transform the date before the log is collected. This could cause the event has duplicated the timezone difference if Wazuh dashboard is configured to use the browser timezone.
- edit the document data before it is indexed into the Wazuh indexer. You could use ingest pipeline processors: https://www.elastic.co/guide/en/elasticsearch/reference/7.10/ingest-processors.html. The partial edition of the full_log original field could be complicated.

Note the commented approach should be done for each type of log.

My recommendation is to treat the full_log field as a reference of the raw log and use the specific fields (@timestamp or timestamp) to know when the event ocurred.