Create custom decoder from 2 logs but with same regex info

[riiky devils]

Hi Teams,

I have problem to create custom decoder from 2 logs but same regex info so if using logtest not all field return value.

This is logs example

Feb 18 18:55:22 amlfmp.manage.trendmicro.com CEF:0|Trend Micro|Apex Central|2019|700211|Attack Discovery Detections|3|deviceExternalId=5 rt=Jan 17 2019 03:38:06 GMT+00:00 dhost=VCAC-Window-331 dst=10.201.86.150 customerExternalID=8c1e2d8f-a03b-47ea-aef8-5aeab99ea697 cn1Label=SLF_RiskLevel cn1=0 cn2Label=SLF_PatternNumber cn2=30.1012.00 cs1Label=SLF_RuleID cs1=powershell invoke expression cat=point of entry cs2Label=SLF_ADEObjectGroup_Info_1 cs2=process - powershell.exe - {#012   "META_FILE_MD5" : "7353f60b1739074eb17c5f4dddefe239",#012   "META_FILE_NAME" : "powershell.exe",#012   "META_FILE_SHA1" : "6cbce4a295c163791b60fc23d285e6d84f28ee4c",#012   "META_FILE_SHA2" : "de96a6e69944335375dc1ac238336066889d9ffc7d73628ef4fe1b1b160ab32c",#012   "META_PATH" : "c:\\\\windows\\\\system32\\\\windowspowershell\\\\v1.0\\\\",#012   "META_PROCESS_CMD" : [ "powershell  iex test2" ],#012   "META_PROCESS_PID" : 10924,#012   "META_SIGNER" : "microsoft windows",#012   "META_SIGNER_VALIDATION" : true,#012   "META_USER_USER_NAME" : "Administrator",#012   "META_USER_USER_SERVERNAME" : "VCAC-WINDOW-331",#012   "OID" : 1#012}#012

 Feb 18 18:55:22 amlfmp.manage.trendmicro.com CEF:0|Trend Micro|Apex Central|2019|BM:1000|Behavior Monitoring|3|rt=Aug 16 2017 05:00:40 GMT+00:00 dvchost=localhost cn1Label=Risk_Level cn1=1 cs2Label=Policy cs2=1000 sproc=C:\\Windows\\SysWOW64\\rundll32.exe cn2Label=Event_Type cn2=4 cs1Label=Target cs1=HKCU\\Software\\Microsoft\\Windows\\CurrentVersion\\Run\\COM+ act=3 cn3Label=Operation cn3=302 shost=shost1 src=10.0.76.40 deviceFacility=Apex One

from that logs using same regex attribute like cs1label, cs1, cs2label, cs2 etc

I'm already create decoder and success to return value except cs1label, cs1, cs2label, cs2 etc (using same attribute)

so how to create decoder from that logs provided to return all value separately based on module (ex : behavior monitoring and attack discovery detections)?

Thank You,

[victor....@wazuh.com]

Hello riiky,

Maybe something like this will do the job:

<decoder name="trend_micro">
 <program_name>CEF</program_name>
</decoder>

<decoder name="trend_micro_add">
 <parent>trend_micro</parent>
 <prematch>Attack Discovery</prematch>
 <regex>cn1Label=(\S+) cn1=(\S+)</regex>
 <order>add_cn1Label, add_cn1 </order>
</decoder>

<decoder name="trend_micro_bm">
 <parent>trend_micro</parent>
 <prematch>Behavior Monitoring</prematch>
 <regex>cn2Label=(\S+) cn2=(\S+)</regex>
 <order>bm_cn2Label, bm_cn2 </order>
</decoder>

• trend_micro: Generic decoder for all Trend Micro events

• trend_micro_add: Decoder for Attack Discovery Detections (add)
  For this kind of event, we are going to expect add_cn1Label and add_cn1 ( include the add header to differentiate fields for different kinds of TrendMicro events )

• trend_micro_bm: Decoder for Behavior Monitoring (bm).
  Similarly, we expect bm_cn2Label, bm_cn2.

Using this hierarchy we can decode different fields for each type of event.
This is only a PoC, the trend_micro generic decoder should be improved in order to include the rest of the desired fields.
Using logtest we get the following:

• Attack Discovery Detections event

    timestamp: 'Feb 18 18:55:22'
    hostname: 'amlfmp.manage.trendmicro.com'
    program_name: 'CEF'

**Phase 2: Completed decoding.
    name: 'trend_micro'
    parent: 'trend_micro'
    add_cn1: '0'
    add_cn1Label: 'SLF_RiskLevel'

• Behavior Monitoring

    timestamp: 'Feb 18 18:55:22'
    hostname: 'amlfmp.manage.trendmicro.com'
    program_name: 'CEF'

**Phase 2: Completed decoding.
    name: 'trend_micro'
    parent: 'trend_micro'
    bm_cn2: '4'
    bm_cn2Label: 'Event_Type'

If you have any doubt, please share your current decoders and I will help you with them.

​

[riiky devils]

Hi Victor,

I'm already try your decoder and its success to return just cslabel and cs value.

<decoder name="trend-micro">

    <program_name>CEF</program_name>
</decoder>

<decoder name="trend_micro_add">

    <parent>trend-micro</parent>
    <prematch>Attack Discovery</prematch>
    <regex>cn1Label=(\S+) cn1=(\S+) cn2Label=(\S+) cn2=(\S+) cs1Label=(\S+) cs1=(\.+) cat=</regex>
    <order>add_cn1Label, add_cn1, add_cn2Label, add_cn2, add_cs1Label, add_cs1 </order>
</decoder>

<decoder name="trend_micro_bm">
 <parent>trend-micro</parent>

 <prematch>Behavior Monitoring</prematch>
 <regex>cn2Label=(\S+) cn2=(\S+)</regex>
 <order>bm_cn2Label, bm_cn2 </order>
</decoder>

**Phase 2: Completed decoding.
        name: 'trend-micro'
        parent: 'trend-micro'
        add_cn1: '0'
        add_cn1Label: 'SLF_RiskLevel'
        add_cn2: '30.1012.00'
        add_cn2Label: 'SLF_PatternNumber'
        add_cs1: 'powershell invoke expression'
        add_cs1Label: 'SLF_RuleID'

**Phase 3: Completed filtering (rules).
        id: '400002'
        level: '10'
        description: 'Trend Micro Apex One - Attack Discovery Detected.'
        groups: '['apex_one', 'trend-micro']'
        firedtimes: '1'
        mail: 'True'
**Alert to be generated.

But if i'm try to parse other field like dhost and dst its always state that duplicated decoder with prematch

<decoder name="trend-micro">

    <program_name>CEF</program_name>
</decoder>

<decoder name="trend_micro_add">

    <parent>trend-micro</parent>
    <prematch>Attack Discovery</prematch>
    <regex>dhost=(\S+)</regex>
    <order>dhost</order>
</decoder>

<decoder name="trend_micro_add">
    <parent>trend-micro</parent>
    <prematch>Attack Discovery</prematch>
    <regex>dst=(\d+.\d+.\d+.\d+)</regex>
    <order>dst</order>
</decoder>

<decoder name="trend_micro_add">
    <parent>trend-micro</parent>
    <prematch>Attack Discovery</prematch>
    <regex>cn1Label=(\S+) cn1=(\S+) cn2Label=(\S+) cn2=(\S+) cs1Label=(\S+) cs1=(\.+) cat=</regex>
    <order>add_cn1Label, add_cn1, add_cn2Label, add_cn2, add_cs1Label, add_cs1 </order>
</decoder>

<decoder name="trend_micro_bm">
 <parent>trend-micro</parent>

 <prematch>Behavior Monitoring</prematch>
 <regex>cn2Label=(\S+) cn2=(\S+)</regex>
 <order>bm_cn2Label, bm_cn2 </order>
</decoder>

ERROR: (2102): Duplicated decoder with prematch: 'trend_micro_add'.
        ERROR: (2105): Error loading decoder options.
        ERROR: (2106): Error adding decoder plugin.
        ERROR: (7311): Failure to initializing session

so how to decode another desired fields?

Thank You,

[victor....@wazuh.com]

You can not duplicate decoder prematch.
The best approach is to create a consistent decoder/rule hierarchy that matches your use case. In this case, there are a few options:

1.- Desired fields always appear in Attack Discovery event type.
If those values are always generated in the Attack Discovery Detections event type, you can use something like the following

<decoder name="trend_micro_add">
 <parent>trend_micro</parent>
 <prematch>Attack Discovery</prematch>
 <regex>dhost=(\S+) dst=(\S+) \.* cn1Label=(\S+) cn1=(\S+)</regex>
 <order>dhost,dst,add_cn1Label, add_cn1 </order>
</decoder>

wazuh-logtest

**Phase 2: Completed decoding.
    name: 'trend_micro'
    parent: 'trend_micro'
    add_cn1: '0'
    add_cn1Label: 'SLF_RiskLevel'
    dhost: 'VCAC-Window-331'
    dst: '10.201.86.150'

2.- Only a few events include desired fields

In this use case, this hierarchy is more practical:

<decoder name="trend_micro_add">
 <program_name>CEF</program_name>
 <prematch>Attack Discovery</prematch>
</decoder>

<decoder name="trend_micro_add_complete">
 <parent>trend_micro_add</parent>
 <regex>dhost=(\S+) dst=(\S+) \.* cn1Label=(\S+) cn1=(\S+)</regex>
 <order>dhost,dst,add_cn1Label, add_cn1 </order>
</decoder>

<decoder name="trend_micro_add_basic">
 <parent>trend_micro_add</parent>
 <regex>dhost=cn1Label=(\S+) cn1=(\S+)</regex>
 <order>add_cn1Label, add_cn1 </order>
</decoder>

This can be customized for your use case.

If you have any doubt do not hesitate to ask.

​

[riiky devils]

Hi Victor,

Thank you for your response. I'm already try to create decoder for attack discovery and behavior monitoring and success to decode all event from BM but from attack discovery i'm still a little bit confuse.
How to decode field cat from my below decoder because its always failed and i'm cannopt decode event after field cs1?
I mean how to decode field with some of word with space between

<decoder name="trend_micro_attd">

 <program_name>CEF</program_name>
 <prematch>Attack Discovery</prematch>
</decoder>

<decoder name="trend_micro_bm">
 <program_name>CEF</program_name>
 <prematch>Behavior Monitoring</prematch>
</decoder>

<decoder name="trend_micro_add_complete">
 <parent>trend_micro_attd</parent>
 <regex>dhost=(\S+) dst=(\S+) \.* cn1Label=(\S+) cn1=(\S+) cn2Label=(\S+) cn2=(\S+) cs1Label=(\S+) cs1=(\.+) cat=</regex>
 <order>dhost,dst,add_cn1Label, add_cn1, add_cn2Label, add_cn2, add_cs1Label, add_cs1 </order>
</decoder>

<decoder name="trend_micro_add_basic">
 <parent>trend_micro_attd</parent>
 <regex>cat=(\.+) cs2Label=</regex>
 <order>cat </order>
</decoder>

<decoder name="trend_micro_bm_complete">
 <parent>trend_micro_bm</parent>
 <regex>dvchost=(\S+) cn1Label=(\S+) cn1=(\S+) cs2Label=(\S+) cs2=(\S+) sproc=(\S+) cn2Label=(\S+) cn2=(\S+) cs1Label=(\S+) cs1=(\S+) act=(\S+) cn3Label=(\S+) cn3=(\S+) shost=(\S+) src=(\d+.\d+.\d+.\d+) </regex>
 <order>dvchost, bm_cn1Label, bm_cn1, bm_cs2Label, bm_cs2, sproc, bm_cn2Label, bm_cn2, bm_cs1Label, bm_cs1, act, bm_cn3Label, bm_cn3, shost, src </order>
</decoder>

the result :

**Phase 2: Completed decoding.
        name: 'trend_micro_attd'

        add_cn1: '0'
        add_cn1Label: 'SLF_RiskLevel'
        add_cn2: '30.1012.00'
        add_cn2Label: 'SLF_PatternNumber'
        add_cs1: 'powershell invoke expression'
        add_cs1Label: 'SLF_RuleID'

        dhost: 'VCAC-Window-331'
        dst: '10.201.86.150'

**Phase 3: Completed filtering (rules).
        id: '400002'
        level: '10'
        description: 'Trend Micro Apex One - Attack Discovery Detected.'

        groups: '['apex_one_attd', 'trend-micro']'
        firedtimes: '1'
        mail: 'True'

Thank You,

[victor....@wazuh.com]

The best approach is to use \. regex and a delimiter string.
In your use case, the cs2Label label.

This decoder should do the job:

<decoder name="trend_micro_add_complete">
 <parent>trend_micro_attd</parent>
 <regex>dhost=(\S+) dst=(\S+) \.* cn1Label=(\S+) cn1=(\S+) cn2Label=(\S+) cn2=(\S+) cs1Label=(\S+) cs1=(\.+) cat=(\.+) cs2Label=(\S+)</regex>
 <order>dhost,dst,add_cn1Label, add_cn1, add_cn2Label, add_cn2, add_cs1Label, add_cs1, cat, add_c2Label </order>
</decoder>

**Phase 2: Completed decoding.
    name: 'trend_micro_attd'
    add_c2Label: 'SLF_ADEObjectGroup_Info_1'
    add_cn1: '0'
    add_cn1Label: 'SLF_RiskLevel'
    add_cn2: '30.1012.00'
    add_cn2Label: 'SLF_PatternNumber'
    add_cs1: 'powershell invoke expression'
    add_cs1Label: 'SLF_RuleID'
    cat: 'point of entry'
    dhost: 'VCAC-Window-331'
    dst: '10.201.86.150'

​