Replace @timestamp with actual timestamp from log file

[Salam Salam]

Hello everyone,

 

How to force Wazuh  in custom decoder & rule to take  the timestamp from the time field within the log itself & rule, not, the time of loading of these log files.

Many thanks in advance.

Best Regards,

[Mauro Pedano]

Hi Salam! How are you?

I'm looking at this issue, can you give me a little more information about the decoders you are creating? and the system you are using?
for example Wazuh version, Kibana version, etc.

Also, could you validate how is your alert stored in `var/ossec/logs/alerts/alerts.json` file?
Because Kibana normally parse the timestamp to the configured timezone, you can find what is your configuration in Kibana menu / Stack Management / Advanced Settings -> Timezone for date formatting

Best!

[Salam Salam]

Dear,

 I'm so sorry for the late response.

I'm  using:

     OS:  CentOS 7
     Wazuh manager: 4.1.0
    Kibana: 7.10.0

    Open Distro for Elasticsearch: 7.10.0

......................................................................

For more explanation, In Wazuh manager I configured the collection of log data from files by editing ossec.conf  to

<localfile>
    <log_format>syslog</log_format>
    <location>/var/log/*</location>
</localfile>


 Logs files are put within the mentioned directory periodically, and parsed with Wazuh PAM decoder-Natively-

Lets say that we have the following log:

2021-07-27T22:00:01.669288+03:00 linux cron[235768]: pam_unix(crond:session): session opened for user root by (uid=0)

after loading this log into Wazuh, the following screenshot of var/ossec/logs/alerts/alerts.json displays:

Kibana screenshot:

We can see the timestamp of the actual log is (2021-07-27T22:00:01) And the timestamp (when the log is loaded) is Jul 28, 2021 @ 13:36:21.089

The conclusion: I need the "timestamp procedure" to be as the timestamp.

  Best Regards,

[Mauro Pedano]

Hi Salam, sorry for the late answer.

I will reproduce this behaviour and check with the team if we can configure it.

Best!

[Salam Salam]

Thank you dear.

Please be advised that my requirement "Timestamp of the actual log (predecoder.timestamp )to be as the timestamp." is to depend on the actual log date & time to be used on the related rules later.

for example : I need to create a simple rule triggers an alert if five failed login within two minutes. What I want really from Wazuh to check the actual  user login's time - not the time of read of these logs- in order to trigger the mentioned alert.

Many Thanks in advance.

Best Regards,

[Salam Salam]

Any  good  news  dear.

Best  Regards,

[Mauro Pedano]

Hi! Sorry for the time it took me to get back to you.

Here are some things you could try.

- Could you try adding a `<out_format>` to your log formatting file? More info here. With this, you could change the timestamp format (since I think the problem may be that it is not recognized as such).

- Another alternative would be to use the option `field name="timestamp1" type="pcre2"` to create a filter. but you will have to create a custom regex to handle the hour range.

Please, try these options and come back to me.
Best!

[Salam Salam]

Thank you for your helpful ideas.

I went to the second option and worked perfectly " the rule triggers when the timestamp1 is non-business hours (08:00 - 17:00) as follow:

Decoder test1:

Rule test1

I need also your kind similar solution to handle the  rule frequency within a specific timeframe based on timestamp1.

For example, I need a rule to trigger if we have three logs "login  during non-business hours" by the same user within three minutes (depending on timestamp1 within the log itself) as follow:

     Timestamp1              user          event  

"2021-08-17  01:47"       user1        login

"2021-08-17 01:48"        user1         login 

"2021-08-17 01:49"        user1         login

Many thanks in advance.

Best Regards,

[Mauro Pedano]

Hi Salam! sorry for the late response!

I think you already know how to create custom rules, so I will just point you to the Rules syntax document. There you will find every parameter you can use to create Wazuh rules.

Regarding your configuration, I suggest you take a look at frequency and timeframe parameters. The first one is used to specify the times the action needs to be triggered. The latter is used to specify a time period for the actions to be triggered. Also, I can recommend taking a look at ignore parameter. that is useful if you start to get a flood of these logs.

To better clarify this behavior I recommend you take a look at this tutorial from Wazuh learning, about Detecting an SSH brute-force attack. The rule used in that document (5712 | sshd: brute force trying to get access to the system.) is very similar to the one you need to create. So you can take a look at the rule in your manager and use it as an example.

I hope this information helps you! if you need any other assistance just let me know.
Best!

[Salam Salam]

Dear mauro,

Thank you for your reply and the provided info.

Actually, I  created successfully many custom rules in my wok  by using frequency and timeframe parameters. All the results of these rules are totally satisfied .

However, the rules I need to apply now is to use frequency and timeframe parameters depending on Timestamp of the actual log (predecoder.timestamp )not depending on timestamp.

So, Is there any way to achieve my needs by referring to the example rule  that was mentioned within my previous message?.

Many thanks in advance.

Regards,

[Mauro Pedano]

Hi Salam!

I have been talking about this with the development team and they told me that this is not possible to do right now. 

Thanks for your patience.

[Emre İnal]

Hello Wazuh Team,

Sorry for bumping an old topic, just in the second year anniversary of the last post :)

We've encountered the exact problem defined here, we need to have the timestamp from our parsed data, rather than the log creation time.

I was wondering in the last 2 years, has there been any development so that this is now achievable?

Thank you for your help and for the great tool!

Best regards,

Emre

8 Eylül 2021 Çarşamba tarihinde saat 19:23:14 UTC+3 itibarıyla Mauro Pedano şunları yazdı: