Accumulate option in Decoders

[Nahid Hasan]

Hello there,

I needed values from another same type of logs. So I was exploring the <accumulate /> option in decoders. But It is not working as expected. 

Here is my agent side conf:

<localfile>
<log_format>syslog</log_format>
<location>/var/log/mailcow.log</location>
<out_format>$(timestamp) $(hostname) mailcow: $(log)</out_format>
</localfile>

And I am getting logs (sample logs given) like this in manager (archives.log):

Jul 29 10:27:33 mailHost mailcow: id=123, one=1, two=2
Jul 29 10:27:34 mailHost mailcow: id=123, three=3, four=4

Here is my decoders:

<decoder name="mailcow">
  <program_name>mailcow</program_name>
  <accumulate />
</decoder>

<decoder name="mailcow1">
<parent>mailcow</parent>
<regex type="pcre2">id=(\d*), one=(\d*), two=(\d*)</regex>
<order>id, mailcow.one, mailcow.two</order>
<accumulate />
</decoder>

<decoder name="mailcow2">
<parent>mailcow</parent>
<regex type="pcre2">id=(\d*), three=(\d*), four=(\d*)</regex>
<order>id, mailcow.three, mailcow.four</order>
<accumulate />
</decoder>

Here is my rules:

<group name="Mailcow,">

  <rule id="120000" level="0">
    <decoded_as>mailcow</decoded_as>
    <description>Mailcow event grouped.</description>
  </rule>
 
  <rule id="120001" level="3">
    <if_sid>120000</if_sid>
    <match>three</match>
    <description>Mailcow id=$(id), one=$(mailcow.one), three=$(mailcow.three)</description>
  </rule>
 
</group>

Here is my logtest result:

# /var/ossec/bin/wazuh-logtest
Starting wazuh-logtest v4.8.0
Type one log per line

Jul 29 10:27:33 mailHost mailcow: id=123, one=1, two=2

**Phase 1: Completed pre-decoding.
full event: 'Jul 29 10:27:33 mailHost mailcow: id=123, one=1, two=2'
timestamp: 'Jul 29 10:27:33'
hostname: 'mailHost'
program_name: 'mailcow'

**Phase 2: Completed decoding.
name: 'mailcow'
accumulate: '1'
id: '123'
mailcow.one: '1'
mailcow.two: '2'

**Phase 3: Completed filtering (rules).
id: '120000'
level: '0'
description: 'Mailcow event grouped.'
groups: '['Mailcow']'
firedtimes: '1'
mail: 'False'

Jul 29 10:27:34 mailHost mailcow: id=123, three=3, four=4

**Phase 1: Completed pre-decoding.
full event: 'Jul 29 10:27:34 mailHost mailcow: id=123, three=3, four=4'
timestamp: 'Jul 29 10:27:34'
hostname: 'mailHost'
program_name: 'mailcow'

**Phase 2: Completed decoding.
name: 'mailcow'
accumulate: '1'
id: '123'
mailcow.four: '4'
mailcow.three: '3'

**Phase 3: Completed filtering (rules).
id: '120006'
level: '3'
description: 'Mailcow id=123, one=, three=3'
groups: '['Mailcow']'
firedtimes: '1'
mail: 'False'
**Alert to be generated.

I was expecting those mailcow.one, mailcow.two values will also appear in the second log test. But it is not accumulating previous values. What are the things I am missing?

Or is there any other ways to do so rather than the <accumulate /> option?
FYI, using Wazuh v4.8.0.

Regards.

[Marcel Kemp]

Hi Nahid,

The <accumulate/> option checks if the processed log gets an ID field decoded. In this case, it brings from previous logs using the same ID value the following fields to populate the alerts for the log (static fields), if these fields were previously decoded.

• dstuser / srcuser
• dstip / srcip
• dstport / srcport
• More info: https://documentation.wazuh.com/current/user-manual/ruleset/decoders/dynamic-fields.html#traditional-decoders

It means it accumulates what is known for these specific fields from previous logs. You need to decode your ID field using the ID field name. Similarly, the fields that get accumulated are those specific fields mentioned above and not any field decoded with generic names. I understand it works for syslog-like logs that share same information like the same hostname. It also has an expiration time of 120 seconds.

• Issue: https://github.com/wazuh/wazuh-ruleset/issues/123#issuecomment-383992331

That is why the use case you have tried did not work correctly, because it does not accumulate the dynamic fields that have been patched.

Maybe you can expand your use case. Wazuh has the multi-line log format option as well as the multiline_regex.
I hope this helps.

[Nahid Hasan]

Hello Marcel,
Hope you are doing great. Thanks for your reply.

After your explanation I was looking at the source file src/analysisd/accumulator.c and found this snippet:

/**
* @brief Struct to save data from events sharing the same ID
*/
typedef struct _OS_ACM_Store {
    time_t timestamp;
    char *dstuser;
    char *srcuser;
    char *dstip;
    char *srcip;
    char *dstport;
    char *srcport;
    char *data;
} OS_ACM_Store;

I think I got the things. Thank you so much.

Regards.