pre-decoding phase eats up to much of the full event
Andrija Radičević
I have a problem with the pre-decoding phase. I'm trying to handle wildfly logs, the event line looks like this:
2023-07-11 18:08:58,808 INFO [com.concludit.controller.SessionController] (default task-1) Accepted password for jo...@doe.net from 127.0.0.1
the predecoder extracts the timestamp from the beginning, howewer withod the milisecond digit:
**Phase 1: Completed pre-decoding.
full event: '2023-07-11 18:08:58,808 INFO [com.concludit.controller.SessionController] (default task-1) Accepted password for jo...@doe.net from 127.0.0.1'
timestamp: '2023-07-11 18:08:58,80'
The problem is that the pre-decoding phase eats up the event line, not only to the end of the timestamp, but until the spce character before the baracket, i.e. what's left of the full event is:
[com.concludit.controller.SessionController] (default task-1) Accepted password for jo...@doe.net from 127.0.0.1'
because of that i cannot get to the error lever, in the case abov it's 'INFO'.
Is there a way to get to this part of the event from the pre-decoding phase, or to the full event itself?
Gerardo David Caceres Fleitas
Thank you for using Wazuh!
I will perform some tests for this case and we will give you an answer as soon as possible.
Greetings.
Gerardo David Caceres Fleitas
Hello Andrija Radičević,
Thank you for your patience, I analyzed your queries and also did some tests, I tell you that as such we cannot directly edit the "pre-decoding" because this is part of the source code, what we can do to take the "INFO" and all the "timestamp" without cutting a decimal is to add an additional string to the original log via "out_format", eg the word "WILDFLY:" at the beginning of each log, thus creating a custom decoder and rule
Documentation:
Localfile – out_format:
· https://documentation.wazuh.com/current/user-manual/reference/ossec-conf/localfile.html
Custom rules and decoders:
· https://documentation.wazuh.com/current/user-manual/ruleset/custom.html
I did a test in the log-test doing the above mentioned and
the result was the following:
The steps I did were:
1. Edit the way in which the agent sends the logs received, that is, adding the additional text "WILDFLY" before the log, this is editing the configuration file /var/ossec/etc/ossec.conf
<localfile>
<log_format>syslog</log_format>
<location>/home/admin/Documents/test.log</location>
<out_format>WILDFLY $(log)</out_format>
</localfile>
Where <location> is the path of where we are taking the original log, in my test I took it from a file called test.log and added the WILDFLY field inside <out_format></out_format >
So the agent logs will be sent from the agent to the manager like this:
WILDFLY 2023-07-11 18:08:58,808 INFO [com.concluded.controller.SessionController] (default task-1) Accepted password for jo...@doe.net from 127.0.0.1
**make sure to restart the wazuh-agent service after some changes in ossec.conf
2. If we want to simulate sending logs to the /home/admin/Documents/test.log file, we can use the following command (from the agent)
echo "2023-07-11 18:08:58,808 INFO [com.concludit.controller.SessionController] (default task-1) Accepted password for jo...@doe.net from 127.0.0.1" >> /home/admin /Documents/test.log
What this will do is write the log to the test.log file and then the agent will send it to the manager, adding the previously mentioned "WILDFLY" header.
3. We could check (from the manager) the file /var/ossec/logs/archives/archives.json with, for example, a grep “WILDFLY” to know that the log was received correctly and that the desired string was added.
**Wazuh Archives is not enabled by default since it
accumulates all the logs whether they have matched with a rule or not, if it is
not going to be used often it is recommended to disable it in order to optimize
storage, in order to enable it you can follow the following documentation:
https://documentation.wazuh.com/current/user-manual/manager/wazuh-archives.html
3. So that Wazuh can process the received alert and that it can be stored in /var/ossec/logs/alerts/alerts.json, we must create a decoder and a custom rule like the following example:
Decoder:
<decoder name="wildfly-test">
<prematch>^WILDFLY </prematch>
</decoder>
<decoder name="wildfly-test-child">
<parent>wildfly-test</parent>
<regex>^WILDFLY (\d\d\d\d\p\d\d\p\d\d) (\d\d\p\d\d\p\d\d\p\d\d\d) (>
<order>wildfly.time, wildfly.hour, wildfly.state, wildfly.domain, w>
</decoder>
Rule:
<group name="wildflytest">
<rule id="100015" level="3">
<decoded_as>wildfly-test</decoded_as>
<description>wildfly event detected</description>
</rule>
</group>
4. We could test that the decoder and the rule work as desired, with var/ossec/bin/wazuh-logtest but taking into account that we must add the text "WILDFLY" at the beginning of our log
WILDFLY 2023-07-11 18:08:58,808 INFO [com.concluded.controller.SessionController] (default task-1) Accepted password for jo...@doe.net from 127.0.0.1
In this way we already have the different fields available for use in custom decoders and alerts, including "INFO" and "18:08:58,808"
I hope this helps you.
Greetings
Gerardo Caceres