Kindly construct decoder

[ekta dhussa]

Hi,

Please construct decode for below log

2024 Dec 24 10:42:31 netskopece->10.1.100.6 Dec 24 05:11:43 netskopece CEF:0|Netskope|test|NULL|connection|NULL|Unknown|IncidentID=0 appcategory=Technology browser=null cci=0 ccl=unknown clientBytes=0 device=null dst=20.163.45.188 os=null page=fe2cr.update.microsoft.com requestClientApplication=null serverBytes=0 sourceServiceName=microsoft src=14.141.44.133 suser=a...@test.com timestamp=1735017132 url=fe2cr.update.microsoft.com

[hasitha.u...@wazuh.com]

Hi ekta,

I have created the decoders for your log, You can apply these decoders to the /var/ossec/etc/rules/local_rules.xml file or you can create new XML file.

1. <decoder name="Netskope-decoder">
2.   <prematch>netskopece</prematch>
3. </decoder>
4.  
5. <decoder name="Netskope-decoder-child">
6.   <parent>Netskope-decoder</parent>
7.   <regex>\.+IncidentID=(\d+)</regex>
8.   <order>IncidentID</order>
9. </decoder>
10.  
11. <decoder name="Netskope-decoder-child">
12.   <parent>Netskope-decoder</parent>
13.   <regex>appcategory=(\S+)</regex>
14.   <order>appcategory</order>
15. </decoder>
16.  
17. <decoder name="Netskope-decoder-child">
18.   <parent>Netskope-decoder</parent>
19.   <regex>browser=(\S+)</regex>
20.   <order>browser</order>
21. </decoder>
22.  
23. <decoder name="Netskope-decoder-child">
24.   <parent>Netskope-decoder</parent>
25.   <regex>cci=(\d+)</regex>
26.   <order>cci</order>
27. </decoder>
28.  
29. <decoder name="Netskope-decoder-child">
30.   <parent>Netskope-decoder</parent>
31.   <regex>ccl=(\S+)</regex>
32.   <order>ccl</order>
33. </decoder>
34.  
35. <decoder name="Netskope-decoder-child">
36.   <parent>Netskope-decoder</parent>
37.   <regex>clientBytes=(\d+)</regex>
38.   <order>clientBytes</order>
39. </decoder>
40.  
41. <decoder name="Netskope-decoder-child">
42.   <parent>Netskope-decoder</parent>
43.   <regex>device=(\S+)</regex>
44.   <order>device</order>
45. </decoder>
46.  
47. <decoder name="Netskope-decoder-child">
48.   <parent>Netskope-decoder</parent>
49.   <regex>dst=(\d+.\d+.\d+.\d+)</regex>
50.   <order>destinationIP</order>
51. </decoder>
52.  
53. <decoder name="Netskope-decoder-child">
54.   <parent>Netskope-decoder</parent>
55.   <regex>os=(\S+)</regex>
56.   <order>os</order>
57. </decoder>
58.  
59. <decoder name="Netskope-decoder-child">
60.   <parent>Netskope-decoder</parent>
61.   <regex>page=(\S+)</regex>
62.   <order>page</order>
63. </decoder>
64.  
65. <decoder name="Netskope-decoder-child">
66.   <parent>Netskope-decoder</parent>
67.   <regex>requestClientApplication=(\S+)</regex>
68.   <order>requestClientApplication</order>
69. </decoder>
70.  
71. <decoder name="Netskope-decoder-child">
72.   <parent>Netskope-decoder</parent>
73.   <regex>serverBytes=(\d+)</regex>
74.   <order>serverBytes=</order>
75. </decoder>
76.  
77. <decoder name="Netskope-decoder-child">
78.   <parent>Netskope-decoder</parent>
79.   <regex>sourceServiceName=(\S+)</regex>
80.   <order>sourceServiceName</order>
81. </decoder>
82.  
83. <decoder name="Netskope-decoder-child">
84.   <parent>Netskope-decoder</parent>
85.   <regex>src=(\d+.\d+.\d+.\d+)</regex>
86.   <order>sourceIP</order>
87. </decoder>
88.  
89. <decoder name="Netskope-decoder-child">
90.   <parent>Netskope-decoder</parent>
91.   <regex>suser=(\S+)</regex>
92.   <order>suser</order>
93. </decoder>
94.  
95. <decoder name="Netskope-decoder-child">
96.   <parent>Netskope-decoder</parent>
97.   <regex>url=(\S+)</regex>
98.   <order>url</order>
99. </decoder>

You can test these decoders by using wazuh-logtest
/var/ossec/bin/wazuh-logtest

You can learn more about custom decoders and rules by following documents.
https://documentation.wazuh.com/current/user-manual/ruleset/decoders/custom.html
https://documentation.wazuh.com/current/user-manual/ruleset/ruleset-xml-syntax/decoders.html
https://documentation.wazuh.com/current/user-manual/ruleset/ruleset-xml-syntax/rules.html
https://documentation.wazuh.com/current/user-manual/ruleset/rules/custom.html#custom-rules
https://wazuh.com/blog/creating-decoders-and-rules-from-scratch/

Let me know if this works for you. 

Regards,
Hasitha Upekshitha

[ekta dhussa]

Hi Hasitha,

Thanks for the help. Kindly assist in creating rules for this decoder.

Regards,

Ekta

[hasitha.u...@wazuh.com]

Hi ekta,

I have created the basic custom rule for your log to capture the events. You can apply these rules to the /var/ossec/etc/rules/local_rules.xml file or you can create new XML file.

1. <group name="Netskope">
2.  
3.   <rule id="100400" level="3">
4.     <decoded_as>Netskope-decoder</decoded_as>
5.     <description>Netskope messages grouped.</description>
6.   </rule>
7.  
8. </group>

You can learn more about how to create custom rules by following documents.

https://documentation.wazuh.com/current/user-manual/ruleset/ruleset-xml-syntax/rules.html
https://documentation.wazuh.com/current/user-manual/ruleset/rules/custom.html#custom-rules
https://wazuh.com/blog/creating-decoders-and-rules-from-scratch/

Let me know if this helps.

Regards,
Hasitha Upekshitha

[ekta dhussa]

Hi Hasitha,

Thanks a bunch for the quick help.

Could you please tell what changes required on decoder if log format is changed to this 

Dec 24 05:11:43 netskopece CEF:0|Netskope|test|NULL|connection|NULL|Unknown|IncidentID=0 appcategory=Technology browser=null cci=0 ccl=unknown clientBytes=0 device=null dst=20.163.45.188 os=null page=fe2cr.update.microsoft.com requestClientApplication=null serverBytes=0 sourceServiceName=microsoft src=14.141.44.133 suser=a...@test.com timestamp=1735017132 url=fe2cr.update.microsoft.com

As in the original log which is this

2024 Dec 24 10:42:31 netskopece->10.1.100.6 Dec 24 05:11:43 netskopece CEF:0|Netskope|test|NULL|connection|NULL|Unknown|IncidentID=0 appcategory=Technology browser=null cci=0 ccl=unknown clientBytes=0 device=null dst=20.163.45.188 os=null page=fe2cr.update.microsoft.com requestClientApplication=null serverBytes=0 sourceServiceName=microsoft src=14.141.44.133 suser=a...@test.com timestamp=1735017132 url=fe2cr.update.microsoft.com

But when logs are processed 2024 Dec 24 10:42:31 netskopece->10.1.100.6  this will be trimmed so please let me know how to handle this in the provided decoder.

Regards,

Ekta

[ekta dhussa]

As while processing first timestamp will trim but the second timestamp will bether in log how to handle it.

[hasitha.u...@wazuh.com]

Hi ekta,

If you log format is this.

Dec 24 05:11:43 netskopece CEF:0|Netskope|test|NULL|connection|NULL|Unknown|IncidentID=0 appcategory=Technology browser=null cci=0 ccl=unknown clientBytes=0 device=null dst=20.163.45.188 os=null page=fe2cr.update.microsoft.com requestClientApplication=null serverBytes=0 sourceServiceName=microsoft src=14.141.44.133 suser=a...@test.com timestamp=1735017132 url=fe2cr.update.microsoft.com

Then you need modify the first decoder(parent decoder) like this.

1. <decoder name="Netskope-decoder">
2. <program_name type="osregex">\.+</program_name>
3. <prematch>Netskope</prematch>
4. </decoder>

Because, In the pre-decoder phase, this log decodes program_name as CEF, ,timestamp and the hostname.
Therefore, you need to match program_name with regex, otherwise if you have any other log source which is taken program_name as CEF it will conflict. Therefore I have created the parent decoder for specific to Netskope.

Let me know if this helps.

Regards,
Hasitha Upekshitha

[ekta dhussa]

1. 
   
2. <decoder name="Netskope-decoder-child">
3.   <parent>Netskope-decoder</parent>
4.   <regex>appcategory=(\S+)</regex>
5.   <order>appcategory</order>
6. </decoder>

Thanks for the help.

 This takes only single string but appcategory is Search Engine but it extracts only Search.

[hasitha.u...@wazuh.com]

Hi ekta,

You can apply this regex to capture the exact value of appcategory.
 <regex>appcategory=(\.+)\s\S+=</regex>

Let me know if this works for you.

Regards,
Hasitha Upekshitha

[ekta dhussa]

Thanks Hasitha.

In the above log please tell how to decode this CEF:0|Netskope|test|NULL|connection|NULL|Unknown|IncidentID=0  each "|" separated value is unique.

Kindly assist.

Regards,

Ekta

[hasitha.u...@wazuh.com]

Hi ekta,

I have created new child decoders to capture the mentioned parts.

I have attached the decoder XML file, you can replace your Netskope decoder with this. 

Let me know if this issue is resolved.

Regards,
Hasitha Upekshitha

[ekta dhussa]

Hi Hasitha,

Thanks for the help but this decoder will work only if one character matches and also for few use cases it is failing as well.

e.g. 

Jan 06 09:26:03 netskopece CEF:0|Netskope|Test|NULL|uba|Bulk Download|Unknown|IncidentID=2585164467939514765 accessMethod=Client act=Download action=anomaly_detection appcategory=Chat, IM & other communication browser=Chrome cci=44 ccl=poor device=Windows Device deviceClassification=managed dst=157.240.23.53 event_type=sequence hostname=W10-S0QjdcqxodV managementId=null object=null os=Windows 11 policy_actions=['Download'] policy_name=null requestClientApplication=WhatsApp sourceServiceName=WhatsApp src=14.141.44.133 suser=te...@TEST.com timestamp=1736153213 uba_ap1=null uba_ap2=null uba_inst1=null uba_inst2=null url=media-maa2-1.cdn.whatsapp.net/o1/v/t62.7118-24/f2/m233/AQOHhkZVzqIJVcXRFQwdMPrGk2-1fFor this decoded value is coming like in attachment.

You can check the decoded values where severity should come Unknown it is coming as 24/f2/m233/AQOHhkZVzqIJVcXRFQwdMPrGk2 and event description should come as Bulk Download  but coming as 7118.

Could you please assist.

[hasitha.u...@wazuh.com]

Hi ekta,

I have remodified the decoders as you expected. Please have a look at the attached decoders list and replace it with your file.

If you have further fields in other logs of Netskope, I suggest you create child decoders by referring to the attached decoders. You can add it in last for new decoders.

Let me know if this helps.

Regards,
Hasitha Upekshitha

[hasitha.u...@wazuh.com]

Hi ekta,

I believe your issue was resolved. if not let me know the update on this.

Regards,

Hasitha Upekshitha

[ekta dhussa]

Hi Hasitha,

Yes issue is sorted decoders are working fine.

Could you please help me creating a rule that will trigger if same suser value is seen for multiple times in a minute.

Regards,

Ekta