Checkpoint logs not maches with wazuh decoder

[Kazim Koybasi]

Hello,

We are trying to decode our checkpoint logs and we aim to produce alerts. When we use ossec-logtest to catch logs with Wazuh we fails always. How can we change decoder to properly decode our logs?

Our logs are Checkpoint IPS logs which is given at the end of the post, so decoder component is like below.

<decoder name="checkpoint-syslog"> 
 <program_name>^Checkpoint</program_name> 
 <prematch>^\s+\S+ \d\d:\d\d:\d\d </prematch> 
</decoder>

<decoder name="checkpoint-syslog-ids"> 
 <parent>checkpoint-syslog</parent> 
 <type>ids</type> 
 <prematch offset="after_parent">^monitor|^drop</prematch> 
 <regex offset="after_prematch">attack: (\S.+)</regex> 
 <regex>src: (\S+); dst: (\S+); </regex> 
 <regex>proto: (\S+); </regex> 
 <order>extra_data, srcip, dstip, protocol</order> 
 <fts>name, extra_data, srcip, dstip</fts> 
 <ftscomment>First time Checkpoint rule fired.</ftscomment> 
</decoder>

 CheckPoint_Logs is given below.

Apr 10 20:43:34 2018 ngman Checkpoint: 10Apr2018 20:43:33 monitor 10.10.1.12  <bond0 Protection Name:Header Rejection;Severity:4;Confidence Level:4;protection_id:HttpHeaderRejection;SmartDefense Profile:SU2_Protection;Performance Impact:2;Industry Reference:CVE-2002-0032, CAN-2003-0237, CAN-2002-0254, CVE-2002-0155, CAN-2003-0397, CAN-2002-0314;Protection Type:protection;Signature Info:^User-Agent[^I ]*:[^I ]*.*esb|ESB;Update Version:634182243;rule:26;rule_uid:{405CB782-3274-4D7F-8AAA-4FB24CE726A0};resource:http://dnl-02.geo.kaspersky.com/bases/av/kdb/i386/kdb-i386-1211g.xml.klz;reject_id:5accf7c4-10053-c00080a-c0000003;web_client_type:Other: *BcfBAAAAgCCAAEFBAAwQfKXVzrzGvyfPESboPxow0mHhxRLAXAQAAIAAKAA=;Attack Info:WSE0100001 header rejection pattern found in request;attack:Header Rejection;src:10.36.52.125;dst:94.75.236.122;proto:6;proxy_src_ip:10.36.52.125;product:SmartDefense;service:80;s_port:51642;FollowUp:Not Followed;product_family:Network

Best Regards.

[Ricardo Muhammad]

Hi Kazim,

maybe the decoder doesn't recognize the <program_ name> properly.

In ossec-logtest the predecoding phase already fails?

If so, you should try to use only <prematch>...</prematch>

With regards
Ricardo

[Kazim Koybasi]

Hello Ricardo,

I disabled <program_ name> section but result is still same that No decoder matched.

Regards,

[Ricardo Muhammad]

Hi Kazim,

as far as I know Ossec/Wazuh doesn't recognize line breaks.
Your log is one line, or indeed separated over several lines?

It checks his decoder-patterns at one line, and checks with the same patterns in the next line.

With regards
Ricardo

[Kazim Koybasi]

Hello Ricardo,

I tried as one line but still fails. I think I must change something in decoder regex.

Regards,

On Tuesday, 24 April 2018 14:07:45 UTC+3, Kazim Koybasi wrote:

[Ricardo Muhammad]

Hi Kazim,

yes, I think so too. My decoders often matched nothing, cause there was a fail in the regex structure. Sometimes just little things.

With regards
Ricardo

[francisco...@wazuh.com]

Hi Kazim and Ricardo,

as you both guessed correctly, the decoder regex doesn't fit your log, so it's not being properly decoded.

We've opened an issue to fix these decoders (https://github.com/wazuh/wazuh-ruleset/issues/126) but, until then, you can try to change the regex to better fix your log format. Here you can find some information about the regex syntax to help you to do so: https://documentation.wazuh.com/current/user-manual/ruleset/ruleset-xml-syntax/regex.html?highlight=regex%20syntax

Thank you very much for pointing out the problem, because community feedback really helps us to improve our product.

Best regards,

Fran G.

[alfonso.r...@wazuh.com]

Hello Kazim and Ricardo, 

We regret the delay. We have been reviewing the Checkpoint decoders cited and there are indeed things to fix.

The first decoder did not work on certain occasions, because in the pre-decoding phase, it was forced to read a blank space before reading the date. This blank space was not always present at the event. For example:

Feb 17 00:01:02 HostWazuh Checkpoint:  3Apr2008 ...

This event had two blank spaces, one of them was suppressed in the pre-decoding phase and the decoder acted without any problem.

Feb 17 00:01:02 HostWazuh Checkpoint: 23Apr2008 ...

In this case, we only have a blank space, suppressed in the pre-decoding phase, causing the decoder to have no effect. 

In the case of the second decoder, we read a series of blank spaces that do not appear in your log.  

The resulting decoders are: 

<decoder name="checkpoint-syslog">
  <program_name>^Checkpoint</program_name>

  <prematch>^\s*\S+ \d\d:\d\d:\d\d </prematch>

</decoder>

<decoder name="checkpoint-syslog-ids">
  <parent>checkpoint-syslog</parent>
  <type>ids</type>
  <prematch offset="after_parent">^monitor|^drop</prematch>

  <regex offset="after_prematch">attack:\s*(\.+);\s*</regex>
  <regex>src:\s*(\S+);\s*dst:\s*(\S+);\s*</regex>
  <regex>proto:\s*(\S+);</regex>

  <order>extra_data, srcip, dstip, protocol</order>
  <fts>name, extra_data, srcip, dstip</fts>
  <ftscomment>First time Checkpoint rule fired.</ftscomment>
</decoder>

<decoder name="checkpoint-syslog">
  <program_name>^Checkpoint</program_name>

  <prematch>^\s*\S+ \d\d:\d\d:\d\d </prematch>

</decoder>

<decoder name="checkpoint-syslog-ids">
  <parent>checkpoint-syslog</parent>
  <type>ids</type>
  <prematch offset="after_parent">^monitor|^drop</prematch>

  <regex offset="after_prematch">attack:\s*(\.+);\s*</regex>
  <regex>src:\s*(\S+);\s*dst:\s*(\S+);\s*</regex>
  <regex>proto:\s*(\S+);</regex>

  <order>extra_data, srcip, dstip, protocol</order>
  <fts>name, extra_data, srcip, dstip</fts>
  <ftscomment>First time Checkpoint rule fired.</ftscomment>
</decoder>

Now, the Syslog format that our decoders are prepared to receive does not match yours: 

Our:

Apr 10 20:43:34 ngman Checkpoint: 

Yours:

Apr 10 20:43:34 2018 ngman Checkpoint: 

If you could modify Checkpoint's output or use a template for Syslog to get our format, we would have the following event: 

Apr 10 20:43:34 ngman Checkpoint: 10Apr2018 20:43:33 monitor 10.10.1.12  <bond0 Protection Name:Header Rejection;Severity:4;Confidence Level:4;protection_id:HttpHeaderRejection;SmartDefense Profile:SU2_Protection;Performance Impact:2;Industry Reference:CVE-2002-0032, CAN-2003-0237, CAN-2002-0254, CVE-2002-0155, CAN-2003-0397, CAN-2002-0314;Protection Type:protection;Signature Info:^User-Agent[^I ]*:[^I ]*.*esb|ESB;Update Version:634182243;rule:26;rule_uid:{405CB782-3274-4D7F-8AAA-4FB24CE726A0};resource:http://dnl-02.geo.kaspersky.com/bases/av/kdb/i386/kdb-i386-1211g.xml.klz;reject_id:5accf7c4-10053-c00080a-c0000003;web_client_type:Other: *BcfBAAAAgCCAAEFBAAwQfKXVzrzGvyfPESboPxow0mHhxRLAXAQAAIAAKAA=;Attack Info:WSE0100001 header rejection pattern found in request;attack:Header Rejection;src:10.36.52.125;dst:94.75.236.122;proto:6;proxy_src_ip:10.36.52.125;product:SmartDefense;service:80;s_port:51642;FollowUp:Not Followed;product_family:Network


**Phase 1: Completed pre-decoding.
       full event: 'Apr 10 20:43:34 ngman Checkpoint: 10Apr2018 20:43:33 monitor 10.10.1.12  <bond0 Protection Name:Header Rejection;Severity:4;Confidence Level:4;protection_id:HttpHeaderRejection;SmartDefense Profile:SU2_Protection;Performance Impact:2;Industry Reference:CVE-2002-0032, CAN-2003-0237, CAN-2002-0254, CVE-2002-0155, CAN-2003-0397, CAN-2002-0314;Protection Type:protection;Signature Info:^User-Agent[^I ]*:[^I ]*.*esb|ESB;Update Version:634182243;rule:26;rule_uid:{405CB782-3274-4D7F-8AAA-4FB24CE726A0};resource:http://dnl-02.geo.kaspersky.com/bases/av/kdb/i386/kdb-i386-1211g.xml.klz;reject_id:5accf7c4-10053-c00080a-c0000003;web_client_type:Other: *BcfBAAAAgCCAAEFBAAwQfKXVzrzGvyfPESboPxow0mHhxRLAXAQAAIAAKAA=;Attack Info:WSE0100001 header rejection pattern found in request;attack:Header Rejection;src:10.36.52.125;dst:94.75.236.122;proto:6;proxy_src_ip:10.36.52.125;product:SmartDefense;service:80;s_port:51642;FollowUp:Not Followed;product_family:Network'
       timestamp: 'Apr 10 20:43:34'
       hostname: 'ngman'
       program_name: 'Checkpoint'
       log: '10Apr2018 20:43:33 monitor 10.10.1.12  <bond0 Protection Name:Header Rejection;Severity:4;Confidence Level:4;protection_id:HttpHeaderRejection;SmartDefense Profile:SU2_Protection;Performance Impact:2;Industry Reference:CVE-2002-0032, CAN-2003-0237, CAN-2002-0254, CVE-2002-0155, CAN-2003-0397, CAN-2002-0314;Protection Type:protection;Signature Info:^User-Agent[^I ]*:[^I ]*.*esb|ESB;Update Version:634182243;rule:26;rule_uid:{405CB782-3274-4D7F-8AAA-4FB24CE726A0};resource:http://dnl-02.geo.kaspersky.com/bases/av/kdb/i386/kdb-i386-1211g.xml.klz;reject_id:5accf7c4-10053-c00080a-c0000003;web_client_type:Other: *BcfBAAAAgCCAAEFBAAwQfKXVzrzGvyfPESboPxow0mHhxRLAXAQAAIAAKAA=;Attack Info:WSE0100001 header rejection pattern found in request;attack:Header Rejection;src:10.36.52.125;dst:94.75.236.122;proto:6;proxy_src_ip:10.36.52.125;product:SmartDefense;service:80;s_port:51642;FollowUp:Not Followed;product_family:Network'

**Phase 2: Completed decoding.
       decoder: 'checkpoint-syslog'
       extra_data: 'Header Rejection'
       srcip: '10.36.52.125'
       dstip: '94.75.236.122'
       protocol: '6'

**Phase 3: Completed filtering (rules).
       Rule id: '20101'
       Level: '6'
       Description: 'IDS event.'
**Alert to be generated.

If it is not possible to change the Syslog format, you can change the first decoder for the next one: 

<decoder name="checkpoint-syslog">
  <prematch>\S+ Checkpoint: \s*\S+ \d\d:\d\d:\d\d </prematch>
</decoder>

Apr 10 20:43:34 2018 ngman Checkpoint: 10Apr2018 20:43:33 monitor 10.10.1.12  <bond0 Protection Name:Header Rejection;Severity:4;Confidence Level:4;protection_id:HttpHeaderRejection;SmartDefense Profile:SU2_Protection;Performance Impact:2;Industry Reference:CVE-2002-0032, CAN-2003-0237, CAN-2002-0254, CVE-2002-0155, CAN-2003-0397, CAN-2002-0314;Protection Type:protection;Signature Info:^User-Agent[^I ]*:[^I ]*.*esb|ESB;Update Version:634182243;rule:26;rule_uid:{405CB782-3274-4D7F-8AAA-4FB24CE726A0};resource:http://dnl-02.geo.kaspersky.com/bases/av/kdb/i386/kdb-i386-1211g.xml.klz;reject_id:5accf7c4-10053-c00080a-c0000003;web_client_type:Other: *BcfBAAAAgCCAAEFBAAwQfKXVzrzGvyfPESboPxow0mHhxRLAXAQAAIAAKAA=;Attack Info:WSE0100001 header rejection pattern found in request;attack:Header Rejection;src:10.36.52.125;dst:94.75.236.122;proto:6;proxy_src_ip:10.36.52.125;product:SmartDefense;service:80;s_port:51642;FollowUp:Not Followed;product_family:Network

**Phase 1: Completed pre-decoding.
       full event: 'Apr 10 20:43:34 2018 ngman Checkpoint: 10Apr2018 20:43:33 monitor 10.10.1.12  <bond0 Protection Name:Header Rejection;Severity:4;Confidence Level:4;protection_id:HttpHeaderRejection;SmartDefense Profile:SU2_Protection;Performance Impact:2;Industry Reference:CVE-2002-0032, CAN-2003-0237, CAN-2002-0254, CVE-2002-0155, CAN-2003-0397, CAN-2002-0314;Protection Type:protection;Signature Info:^User-Agent[^I ]*:[^I ]*.*esb|ESB;Update Version:634182243;rule:26;rule_uid:{405CB782-3274-4D7F-8AAA-4FB24CE726A0};resource:http://dnl-02.geo.kaspersky.com/bases/av/kdb/i386/kdb-i386-1211g.xml.klz;reject_id:5accf7c4-10053-c00080a-c0000003;web_client_type:Other: *BcfBAAAAgCCAAEFBAAwQfKXVzrzGvyfPESboPxow0mHhxRLAXAQAAIAAKAA=;Attack Info:WSE0100001 header rejection pattern found in request;attack:Header Rejection;src:10.36.52.125;dst:94.75.236.122;proto:6;proxy_src_ip:10.36.52.125;product:SmartDefense;service:80;s_port:51642;FollowUp:Not Followed;product_family:Network'
       timestamp: 'Apr 10 20:43:34'
       hostname: '2018'
       program_name: '(null)'
       log: 'ngman Checkpoint: 10Apr2018 20:43:33 monitor 10.10.1.12  <bond0 Protection Name:Header Rejection;Severity:4;Confidence Level:4;protection_id:HttpHeaderRejection;SmartDefense Profile:SU2_Protection;Performance Impact:2;Industry Reference:CVE-2002-0032, CAN-2003-0237, CAN-2002-0254, CVE-2002-0155, CAN-2003-0397, CAN-2002-0314;Protection Type:protection;Signature Info:^User-Agent[^I ]*:[^I ]*.*esb|ESB;Update Version:634182243;rule:26;rule_uid:{405CB782-3274-4D7F-8AAA-4FB24CE726A0};resource:http://dnl-02.geo.kaspersky.com/bases/av/kdb/i386/kdb-i386-1211g.xml.klz;reject_id:5accf7c4-10053-c00080a-c0000003;web_client_type:Other: *BcfBAAAAgCCAAEFBAAwQfKXVzrzGvyfPESboPxow0mHhxRLAXAQAAIAAKAA=;Attack Info:WSE0100001 header rejection pattern found in request;attack:Header Rejection;src:10.36.52.125;dst:94.75.236.122;proto:6;proxy_src_ip:10.36.52.125;product:SmartDefense;service:80;s_port:51642;FollowUp:Not Followed;product_family:Network'

**Phase 2: Completed decoding.
       decoder: 'checkpoint-syslog'
       extra_data: 'Header Rejection'
       srcip: '10.36.52.125'
       dstip: '94.75.236.122'
       protocol: '6'

**Phase 3: Completed filtering (rules).
       Rule id: '20101'
       Level: '6'
       Description: 'IDS event.'
**Alert to be generated.

In this way, we could solve the problems that have occurred, but we are less in favour of this option. 

These fixes will soon be added to our repository. We hope it has been helpful and thank you very much for your collaboration. 

Kind regards,

Alfonso Ruiz-Bravo

[alfonso.r...@wazuh.com]

We regret that the section on modified decoders has been repeated. I should have only shown up once. 

<decoder name="checkpoint-syslog">
  <program_name>^Checkpoint</program_name>

  <prematch>^\s*\S+ \d\d:\d\d:\d\d </prematch>

</decoder>

<decoder name="checkpoint-syslog-ids">
  <parent>checkpoint-syslog</parent>
  <type>ids</type>
  <prematch offset="after_parent">^monitor|^drop</prematch>

<regex offset="after_prematch">attack:\s*(\.+);\s*</regex>
  <regex>src:\s*(\S+);\s*dst:\s*(\S+);\s*</regex>

  <regex>proto:\s*(\S+);</regex>

  <order>extra_data, srcip, dstip, protocol</order>
  <fts>name, extra_data, srcip, dstip</fts>
  <ftscomment>First time Checkpoint rule fired.</ftscomment>
</decoder>

(Repeated code)

Regards,

Alfonso Ruiz-Bravo 

On Tuesday, April 24, 2018 at 1:07:45 PM UTC+2, Kazim Koybasi wrote: