Decoding long messages - multiple regex statements

[Fredrik]

Hi All,

Came across this where I think I would be helped by extracting fields both in forward (from beginning) and in reverse (from end) order of messages!? Is this possible, if so, is it stupid given that there are other (better) ways to accomplish the same thing :/ ? 

In addition to the fields my current decoder extracts, I was hoping to extract the resource (http://www.aliveproxy.com/) and also the product (Application Control;). My idea was to add a secondary statement, before the <order> statement, something in the lines of:

<regex>$/p\w+\s    [...] and work my way backward so that I can extract Application Control and resource . How would you suggest I do this?! 

Thanks again for all the great help - hope my threads (and questions) can be useful for other newstarters outhere trying to get there feet off the ground ;) 

Best regards,

Fredrik 

LOG-MESSAGE

Jan 27 09:41:01 127.0.0.1 Jan 27 9:32:28 st4600fw01n1 allow <eth1 mail src: 192.168.1.15; dst: 89.208.212.2; proto: tcp; appi_name: ******; app_desc: ******; app_id: 10063753; app_category: ******; matched_category: ******; app_properties: ******; app_risk: ******; app_rule_id: ******; app_rule_name: ******; web_client_type: Chrome; web_server_type: Microsoft-IIS; app_sig_id: 10063753:5; resource: http://www.aliveproxy.com/; proxy_src_ip: 192.168.1.15 product: Application Control; service: http; s_port: 58579; product_family: Network;

MY CURRENT DECODER

<decoder name="Checkpoint">

  <prematch>^\w+ \d+ \d+:\d+:\d+ st4600fw01n\d</prematch>

  <type>firewall</type>

</decoder>

<decoder name="Checkpoint-alert">

  <parent>Checkpoint</parent>

  <regex offset="after_parent">(\w+) \p\w+ \w+ src:\s(\d+.\d+.\d+.\d+);\sdst:\s(\d+.\d+.\d+.\d+)</regex>

  <order>action,srcip,dstip</order>

</decoder>

LOGTEST OUTPUT

**Phase 1: Completed pre-decoding.

       full event: 'Jan 27 09:41:01 127.0.0.1 Jan 27 9:32:28 st4600fw01n1 allow <eth1 mail src: 192.168.1.15 dst: 89.208.212.2; proto: tcp; appi_name: ******; app_desc: ******; app_id: 10063753; app_category: ******; matched_category: ******; app_properties: ******; app_risk: ******; app_rule_id: ******; app_rule_name: ******; web_client_type: Chrome; web_server_type: Microsoft-IIS; app_sig_id: 10063753:5; resource: http://www.aliveproxy.com/; proxy_src_ip: 192.168.1.15; product: Application Control; service: http; s_port: 58579; product_family: Network;'

       hostname: '127.0.0.1'

       program_name: '(null)'

       log: 'Jan 27 9:32:28 st4600fw01n1 allow <eth1 mail src: 192.168.1.15; dst: 89.208.212.2; proto: tcp; appi_name: ******; app_desc: ******; app_id: 10063753; app_category: ******; matched_category: ******; app_properties: ******; app_risk: ******; app_rule_id: ******; app_rule_name: ******; web_client_type: Chrome; web_server_type: Microsoft-IIS; app_sig_id: 10063753:5; resource: http://www.aliveproxy.com/; proxy_src_ip: 192.168.1.15; product: Application Control; service: http; s_port: 58579; product_family: Network;'

**Phase 2: Completed decoding.

       decoder: 'Checkpoint'

       action: 'allow'

       srcip: '192.168.1.15'

       dstip: '89.208.212.2'

**Phase 3: Completed filtering (rules).

       Rule id: '4100'

       Level: '0'

       Description: 'Firewall rules grouped.'

[Pedro S]

Hi Fredrik,

I don't think OSSEC allow regex to work backwards, from end to beginning, I know that can be specify on other languages with some flags, but I am not sure if we can do that here. 

Regarding to your decoder, we have two options, include the extraction of "resource" and product" fields on the same decoder:

FULL DECODER

<decoder name="Checkpoint-alert">
  <parent>Checkpoint</parent>

  <regex offset="after_parent">(\w+) \p\w+ \w+ src:\s(\d+.\d+.\d+.\d+);\sdst:\s(\d+.\d+.\d+.\d+)\.*resource: (\.*);\.*product: (\.*);</regex>
  <order>action,srcip,dstip,url,extra_data</order>
</decoder>

Or in a better way, separate the extraction in two different decoders, so we can be sure that in case of "resource" and "product" fields do not exists, our decoder still will parse and work.

SPLIT DECODERS:

<decoder name="Checkpoint-alert">
  <parent>Checkpoint</parent>
  <regex offset="after_parent">(\w+) \p\w+ \w+ src:\s(\d+.\d+.\d+.\d+);\sdst:\s(\d+.\d+.\d+.\d+)</regex>
  <order>action,srcip,dstip</order>
</decoder>


<decoder name="Checkpoint-alert">
  <parent>Checkpoint</parent>

  <regex offset="after_regex">\.*resource: (\.*);\.*product: (\.*);</regex>
  <order>url,extra_data</order>
</decoder>

LOGTEST OUTPUT

**Phase 1: Completed pre-decoding.
       full event: 'Jan 27 09:41:01 127.0.0.1 Jan 27 9:32:28 st4600fw01n1 allow <eth1 mail src: 192.168.1.15; dst: 89.208.212.2; proto: tcp; appi_name: ******; app_desc: ******; app_id: 10063753; app_category: ******; matched_category: ******; app_properties: ******; app_risk: ******; app_rule_id: ******; app_rule_name: ******; web_client_type: Chrome; web_server_type: Microsoft-IIS; app_sig_id: 10063753:5; resource: http://www.aliveproxy.com/; proxy_src_ip: 192.168.1.15 product: Application Control; service: http; s_port: 58579; product_family: Network;'
       hostname: '127.0.0.1'
       program_name: '(null)'
       log: 'Jan 27 9:32:28 st4600fw01n1 allow <eth1 mail src: 192.168.1.15; dst: 89.208.212.2; proto: tcp; appi_name: ******; app_desc: ******; app_id: 10063753; app_category: ******; matched_category: ******; app_properties: ******; app_risk: ******; app_rule_id: ******; app_rule_name: ******; web_client_type: Chrome; web_server_type: Microsoft-IIS; app_sig_id: 10063753:5; resource: http://www.aliveproxy.com/; proxy_src_ip: 192.168.1.15 product: Application Control; service: http; s_port: 58579; product_family: Network;'


**Phase 2: Completed decoding.
       decoder: 'Checkpoint'
       action: 'allow'
       srcip: '192.168.1.15'
       dstip: '89.208.212.2'

       url: 'http://www.aliveproxy.com/'
       extra_data: 'Application Control'

**Phase 3: Completed filtering (rules).
       Rule id: '4100'
       Level: '0'
       Description: 'Firewall rules grouped.'

In both decoders, I am using wildcards .* and expecting always "resource" before "product", either way won't work. 

You asked about using another "regex" line in the same decoder, it will work too, like this:

<decoder name="Checkpoint-alert">
  <parent>Checkpoint</parent>
  <regex offset="after_parent">(\w+) \p\w+ \w+ src:\s(\d+.\d+.\d+.\d+);\sdst:\s(\d+.\d+.\d+.\d+)</regex>

  <regex>\.*resource: (\.*);\.*product: (\.*);</regex>
  <order>action,srcip,dstip, url, extra_data</order>
</decoder>

Best regards,

Pedro S.

[Jesus Linares]

Hi,

I would add a prematch tag:

<decoder name="Checkpoint-alert">
  <parent>Checkpoint</parent>

  <prematch>XXXXXXXX</prematch>

  <regex offset="after_parent">(\w+) \p\w+ \w+ src:\s(\d+.\d+.\d+.\d+);\sdst:\s(\d+.\d+.\d+.\d+)</regex>
  <order>action,srcip,dstip</order>
</decoder>

<decoder name="Checkpoint-alert">
  <parent>Checkpoint</parent>
  <regex offset="after_regex">\.*resource: (\.*);\.*product: (\.*);</regex>
  <order>url,extra_data</order>
</decoder>

Each decoder must have a prematch tag. Try this example without prematch and see what happens.

<!--
Mar  3 12:15:24 LinMV TestDecoder[1963]: TypeA value1: hi; value2: bye; value3: seeyou
Mar  3 12:15:24 LinMV TestDecoder[1963]: TypeB field1: hi; value2: bye; value3: seeyou
-->
<decoder name="TestDecoder">
    <program_name>TestDecoder</program_name>
</decoder>


<decoder name="TestDecoder-1">
    <parent>TestDecoder</parent>
    <prematch>TypeA</prematch>
    <regex offset="after_parent">value1: hi; value2: (\S+)</regex>
    <order>id</order>
</decoder>


<decoder name="TestDecoder-1">
    <parent>TestDecoder</parent>
    <regex offset="after_regex">value3: (\S+)</regex>
    <order>extra_data</order>
</decoder>


<decoder name="TestDecoder-2">
    <parent>TestDecoder</parent>
    <prematch>TypeB</prematch>
    <regex offset="after_parent">field1: hi; value2: (\S+)</regex>
    <order>id</order>
</decoder>

Also, when it is possible, try to don't use the character "\.". Maybe you can do it whit \S+.

Regards,

Jesus Linares.

[Fredrik]

Fantastic! Thank you so much! Makes sense what you wrote about my suggested (mis)use of regex to try and get it start reading backwards. I should have realised that wildcard (*) was the way to go, without having to match the full string with the expressions available. I guess Jesus' recommendation not to use \. where alternatives are viable is due to performance? (Feel free to chime in Jesus :)) . Anyway, thanks alot. I learned a great deal from your suggestions, that should come in handy as write more decoders :)

/f

On Thursday, March 3, 2016 at 10:05:16 AM UTC+1, Pedro S wrote:

[Fredrik]

Hi (again) Jesus!

Big thanks for your time and effort! Greatly appreciated! 

I tried your example below, to better understand why each decoder should have a prematch. I haven't quite figured out yet, but I see (obviously) that different results are produced. What would be a good prematch in my example? Just so I got that part right. Giving two sections the same <decoder-name>Checkpoint-alert</decoder> in essence means that it is one decoder, but defined in two sections? 

Your recommendation not to use \. where alternatives are viable is this due to performance, or implications coming with being too wide (as opposed to narrow) in matching a string? 

Best regards,

Fredrik 

[Fredrik]

Hi All,

In this context and with your great response. What would you PROs suggest I do when decoding another type of message from the same firewall - but a different blade (i.e. module). Turns out that the messages look somewhat different. This is a sample from the other module and it won't match with the current decoder. Should I add an OR to the parent decoder to do the first match and then use different subdecoders to extract the useful information from the other type of message? How do you deal with these type of scenarios?

MESSAGE:

1Mar2016 15:17:09 redirect st4600fw01n1 <eth1 alert web_client_type: Chrome; resource: http://sc1.checkpoint.com/za/images/threatwiki/pages/TestAntiBotBlade.html; src: 192.168.1.15; dst: 23.8.4.103; proto: tcp; session_id: {0x56d5a2de,0x4,0xc50d2e0a,0xc0000001}; Protection name: Check Point - Testing Bot; malware_family: Check Point; Source OS: Windows; Confidence Level: 5; severity: 2; malware_action: Communication with C&C site; rule_uid: {9AF67731-0D35-4117-AF2B-9A47F9396D26}; Protection Type: URL reputation; malware_rule_id: {000000CE-00A4-0046-9658-621EA5468654}; protection_id: 00233CFEE; log_id: 2; proxy_src_ip: 192.168.1.15; scope: 10.46.5.133; product: Anti Malware; service: http; s_port: 61834;

Best regards,

Fredrik 

On Wednesday, March 2, 2016 at 11:03:08 AM UTC+1, Fredrik wrote:

[Jesus Linares]

Hi Fredrik,

The expression "\.+" matches for anything. Usually, it is not a good idea because is slow and maybe you capture something that you don't want. So, when it is possible, it is better to use something specific.

When you have different decoders (different name) with the same parent, you should use a prematch. If you don't use prematch, it is fired the first rule. In the previous example:

Log:

Mar  3 12:15:24 LinMV TestDecoder[1963]: TypeB field1: hi; value2: bye; value3: seeyou

Without prematch:

**Phase 2: Completed decoding.
       decoder: 'TestDecoder'
       extra_data: 'seeyou'

With prematch:

**Phase 2: Completed decoding.
       decoder: 'TestDecoder'
       id: 'bye;'

Without prematch, the decoder is TestDecoder-1, but it should be TestDecoder2 (because it has the string "field1". In my view, it is a good practice use prematch, but sometimes it is no necessary.

Regarding your last question, could you use the same log format in your firewall and in the blade?. Paste here two logs of each one (firewall and blade) and your decoders, and we will take a look ;)

Regards.

Jesus Linares

[Fredrik]

Hi Jesus,

Got sidetracked with other projects, and finally getting back to my questions about handling different messages from the same device (firewall). Also, Jesus your suggestion about placing a prematch in the suggested decoder in this thread - what would be a good prematch here? 

Should I add an OR to the parent decoder to do the first match and then use different subdecoders to extract the useful information from the other type of message? How do you deal with these type of scenarios?

Just so I got that part right. Giving two sections the same <decoder-name>Checkpoint-alert</decoder> in essence means that it is one decoder, but defined in two sections? 

Please find the two message-types below for reference.

MESSAGE1:

1Mar2016 15:17:09 redirect st4600fw01n1 <eth1 alert web_client_type: Chrome; resource: http://sc1.checkpoint.com/za/images/threatwiki/pages/TestAntiBotBlade.html; src: 192.168.1.15; dst: 23.8.4.103; proto: tcp; session_id: {0x56d5a2de,0x4,0xc50d2e0a,0xc0000001}; Protection name: Check Point - Testing Bot; malware_family: Check Point; Source OS: Windows; Confidence Level: 5; severity: 2; malware_action: Communication with C&C site; rule_uid: {9AF67731-0D35-4117-AF2B-9A47F9396D26}; Protection Type: URL reputation; malware_rule_id: {000000CE-00A4-0046-9658-621EA5468654}; protection_id: 00233CFEE; log_id: 2; proxy_src_ip: 192.168.1.15; scope: 10.46.5.133; product: Anti Malware; service: http; s_port: 61834;

MESSAGE2:

Jan 27 09:41:01 127.0.0.1 Jan 27 9:32:28 st4600fw01n1 allow <eth1 mail src: 192.168.1.15; dst: 89.208.212.2; proto: tcp; appi_name: ******; app_desc: ******; app_id: 10063753; app_category: ******; matched_category: ******; app_properties: ******; app_risk: ******; app_rule_id: ******; app_rule_name: ******; web_client_type: Chrome; web_server_type: Microsoft-IIS; app_sig_id: 10063753:5; resource: http://www.aliveproxy.com/; proxy_src_ip: 192.168.1.15 product: Application Control; service: http; s_port: 58579; product_family: Network;

[Jesus Linares]

Hi,

first, I would use the same format for both messages. Two options:

• Change log format in each device. 
• Choose one:

• 1Mar2016 15:17:09 redirect st4600fw01n1

• Jan 27 09:41:01 127.0.0.1 Jan 27 9:32:28 st4600fw01n1

• This part could be your parent decoder (using regular expressions)
• Change the log received with rsyslog, for example, add a string:
• MyFirewall 1Mar2016 15:17:09 redirect st4600fw01n1
• So, the parent decoder will be <prematch>^MyFirewall </prematch>

The prematch of each sub-decoder (child decoder) could be the type of log, maybe "web_client_type" or "mail".

What firewall are you using? Version?.

Paste here more logs.

Regards,

Jesus Linares

[Fredrik]

Hi Jose,

I got some help to sort out the different timestamps (format) and all log types now use "Jan 27 09:41:01".  You asked about the firewall, this particular one is a Checkpoint currently running version R77.20.

The remaining question, that might be of interest to others on the path to OSSEC mastery ;) ;) is how to handle messages with different "format" coming from the same host. I have collected a bunch of messages that I would like to be able to decode, but I'm not sure about the most efficient way to build the parent/child decoder tree for this. 

With the help received previously in this thread, I currently have the following in my local_decoder and I'm experimenting with different addition - none of which is working so far ;) 

<decoder name="Checkpoint">

  <prematch>^\w+ \d+ \d+:\d+:\d+ st4600fw01n\d</prematch>

  <type>firewall</type>

</decoder>

<decoder name="Checkpoint-alert">

  <parent>Checkpoint</parent>

  <regex offset="after_parent">(\w+) \p\w+ \w+ src:\s(\d+.\d+.\d+.\d+);\sdst:\s(\d+.\d+.\d+.\d+)</regex>

  <order>action,srcip,dstip</order>

</decoder>

<decoder name="Checkpoint-alert">

  <parent>Checkpoint</parent>

  <regex offset="after_regex">\.*resource: (\.*);\.*product: (\.*);</regex>

  <order>url,extra_data</order>

</decoder>

Below is a collection of syslog messages recieved from the firewall where the first section is currently decoded using the local_decoder above:

Mar 29 10:09:40 127.0.0.1 Mar 29 9:57:49 st4600fw01n1 block <eth6 mail src: 192.168.7.206; dst: 54.72.9.51; proto: tcp; bytes: 4962; sent_bytes: 530; received_bytes: 4432; app_id: 3404393449; browse_time: ******; Suppressed logs: 1; Referrer_self_uid: ******; product: URL Filtering; service: http; s_port: 54693; product_family: Network;

Mar 29 20:57:00, st4600fw01n1, allow <eth1 mail src: 192.168.5.133; dst: 89.208.212.2; proto: tcp; appi_name: ******; app_desc: ******; app_id: 10063753; app_category: ******; matched_category: ******; app_properties: ******; app_risk: ******; app_rule_id: ******; app_rule_name: ******; web_client_type: Microsoft IE; web_server_type: Microsoft-IIS; app_sig_id: 10063753:5; resource: http://aliveproxy.com/; proxy_src_ip: 192.168.5.133; product: Application Control; service: http; s_port: 63867; product_family: Network;

Mar 30 09:04:14 127.0.0.1 Mar 30 8:52:22 st4600fw01n1 allow <eth6 mail src: 192.168.5.133; dst: 23.67.132.180; proto: tcp; appi_name: ******; app_desc: ******; app_id: 10003219; app_category: ******; matched_category: ******; app_properties: ******; app_risk: ******; app_rule_id: ******; app_rule_name: ******; app_sig_id: 10003219:4; proxy_src_ip: 192.168.5.133; product: Application Control; service: https; s_port: 64166; product_family: Network;

Mar 30 09:03:56 127.0.0.1 Mar 30 8:52:05 st4600fw01n1 allow <eth6 mail src: 192.168.5.133; dst: 89.208.212.2; proto: tcp; bytes: 2396; sent_bytes: 1180; received_bytes: 1216; app_id: 10063753; browse_time: ******; Suppressed logs: 19; Referrer_self_uid: ******; product: Application Control; service: http; s_port: 64136; product_family: Network;

Mar 30 09:03:35 127.0.0.1 Mar 30 8:51:43 st4600fw01n1 allow <eth6 mail src: 192.168.5.133; dst: 89.208.212.2; proto: tcp; appi_name: ******; app_desc: ******; app_id: 10063753; app_category: ******; matched_category: ******; app_properties: ******; app_risk: ******; app_rule_id: ******; app_rule_name: ******; web_client_type: Chrome; web_server_type: Microsoft-IIS; app_sig_id: 10063753:5; resource: http://www.aliveproxy.com/; proxy_src_ip: 192.168.5.133; product: Application Control; service: http; s_port: 64136; product_family: Network;

Mar 29 09:04:21 127.0.0.1 Mar 29 8:52:29 st4600fw01n1 block <eth6 mail src: 192.168.5.136; dst: 37.157.4.15; proto: tcp; appi_name: ******; app_desc: ******; app_id: 1875144601; app_category: ******; matched_category: ******; app_properties: ******; app_risk: ******; app_rule_id: ******; app_rule_name: ******; web_client_type: Other: Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; Touch; rv:11.0) like Gecko; web_server_type: Other: nginx; resource: http://adx.adform.net/adx/?rp=3&pv=1&bWlkPTE1NzU3NiZybmQ9ODU1M2JjNGMtMmY2MC00YzVjLWFhMzAtYmY5NzZlNDllZDNk&callback=_adform_cb_1459234242747_7491414591872548; proxy_src_ip: 192.168.5.136; product: URL Filtering; service: http; s_port: 54051; product_family: Network;

Mar 29 09:06:12 127.0.0.1 Mar 29 8:54:21 st4600fw01n1 block <eth6 mail src: 192.168.6.157; dst: 37.157.4.15; proto: tcp; appi_name: ******; app_desc: ******; app_id: 1875144601; app_category: ******; matched_category: ******; app_properties: ******; app_risk: ******; app_rule_id: ******; app_rule_name: ******; web_client_type: Other: Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; Touch; rv:11.0) like Gecko; web_server_type: Other: nginx; resource: http://adx.adform.net/adx/?rp=3&pv=1&bWlkPTk1NzI4&callback=_adform_cb_1459234355177_008705563130792681; proxy_src_ip: 192.168.6.157; product: URL Filtering; service: http; s_port: 51746; product_family: Network;

Mar 29 08:37:54 127.0.0.1 Mar 29 8:26:03 st4600fw01n1 block <eth6 mail src: 192.168.6.157; dst: 152.115.75.210; proto: tcp; appi_name: ******; app_desc: ******; app_id: 1875144601; app_category: ******; matched_category: ******; app_properties: ******; app_risk: ******; app_rule_id: ******; app_rule_name: ******; web_client_type: Other: Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; Touch; rv:11.0) like Gecko; web_server_type: Other: nginx; resource: http://adx.adform.net/adx/?rp=3&pv=1&bWlkPTk3ODMyJmN0dXJsPWh0dHA6Ly9mdXNpb24uYm9ubmllcnRpZHNrcmlmdGVyLnNlL2V2ZW50L3VlZXVqL2J0LmVrb25vbWkucGFmLnJvcy8xNDc3OTkyNzcvY2xpY2s_dXJsPQ&bWlkPTk3ODMxJmN0dXJsPWh0dHA6Ly9mdXNpb24uYm9ubmllcnRpZHNrcmlmdGVyLnNlL2V2ZW50L3lydmd2L2J0LmVrb25vbWkucGFmLnJvcy8xNDc3OTkyNzAvY2xpY2s_dXJsPQ&callback=_adform_cb_1459232656248_7681010355476278; proxy_src_ip: 192.168.6.157; product: URL Filtering; service: http; s_port: 51104; product_family: Network;

Mar 29 08:35:17 127.0.0.1 Mar 29 8:23:24 st4600fw01n1 block <eth6 mail src: 192.168.6.157; dst: 152.115.75.199; proto: tcp; appi_name: ******; app_desc: ******; app_id: 1875144601; app_category: ******; matched_category: ******; app_properties: ******; app_risk: ******; app_rule_id: ******; app_rule_name: ******; web_client_type: Other: Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; Touch; rv:11.0) like Gecko; web_server_type: Other: nginx; resource: http://adx.adform.net/adx/?rp=3&pv=1&bWlkPTk1NzMw&callback=_adform_cb_1459232497393_6046677836175733; proxy_src_ip: 192.168.6.157; product: URL Filtering; service: http; s_port: 50904; product_family: Network;

Mar 30 10:04:39 127.0.0.1 redirect <eth1 alert web_client_type: Chrome; resource: http://sc1.checkpoint.com/za/images/threatwiki/pages/TestAntiBotBlade.html; src: 192.168.5.133; dst: 172.226.217.148; proto: tcp; session_id: {0x56fb8896,0x10009,0xc50d2e0a,0xc0000001}; Protection name: Check Point - Testing Bot; malware_family: Check Point; Source OS: Windows; Confidence Level: 5; severity: 2; malware_action: Communication with C&C site; rule_uid: {9AF67731-0D35-4117-AF2B-9A47F9396D26}; Protection Type: URL reputation; malware_rule_id: {000000CE-00A4-0046-9658-621EA5468654}; protection_id: 00233CFEE; log_id: 2; proxy_src_ip: 192.168.5.133; scope: 192.168.5.133; product: Anti Malware; service: http; s_port: 49244;

Mar 29 20:13:14 127.0.0.1 prevent <eth1 alert web_client_type: Microsoft IE; resource: http://www.bing.com/fd/ls/GLinkPing.aspx?IG=9A0044152B65437D93F87086B9E730D9&&ID=SERP,5118.1&url=http://sc1.checkpoint.com/z...; src: 192.168.5.133; dst: 204.79.197.200; proto: tcp; session_id: {0x56fac5ba,0x10004,0xc50d2e0a,0xc0000000}; Protection name: Check Point - Testing Bot; malware_family: Check Point; Source OS: Windows; Confidence Level: 5; severity: 2; malware_action: Communication with C&C site; rule_uid: {9AF67731-0D35-4117-AF2B-9A47F9396D26}; Protection Type: URL reputation; malware_rule_id: {000000CE-00A4-0046-9658-621EA5468654}; protection_id: 00233CFEE; log_id: 2; proxy_src_ip: 192.168.5.133; scope: 192.168.5.133; product: Anti Malware; service: http; s_port: 63119;

Mar 30 9:04:59, st4600fw01n1, allow <eth6 mail src: 192.168.5.133; dst: 207.244.85.73; proto: tcp; appi_name: ******; app_desc: ******; app_id

: 10064017; app_category: ******; matched_category: ******; app_properties: ******; app_risk: ******; app_rule_id: ******; app_rule_name: ******; web_client_type: Chrome; web_server_type: Apache; app_sig_id: 10064017:2; resource: http://www.bypassthat.com/; proxy_src_ip: 192.168.5.133; product: Application Control; service: http; s_port: 64499; product_family: Network;

Mar 30 08:55:41 127.0.0.1 Mar 30 8:49:25 < sto-fwm03 mail System Alert message: A Firewall Policy has been successfully installed on st4600fw01n2; Object: st4600fw01n2; Event: Change; Parameter: policy_time; Condition: changes Tue Mar 22 11:07:17 2016; Current value: Wed Mar 30 08:39:57 2016; product: System Monitor; product_family: Network;

Mar 30 08:56:02 127.0.0.1 Mar 30 8:49:47 < sto-fwm03 mail System Alert message: A Firewall Policy has been successfully installed on st4600fw01n1; Object: st4600fw01n1; Event: Change; Parameter: policy_time; Condition: changes Tue Mar 22 11:09:21 2016; Current value: Wed Mar 30 08:43:12 2016; product: System Monitor; product_family: Network;

 

[Jesus Linares]

Hi Fredrik,

here an example of decoding allow/block events (with the option after_regex):

<!--
pattern:
Mar 29 09:04:21 127.0.0.1 Mar 29 8:52:29 st4600fw01n1 Text
-->
<decoder name="Checkpoint-test">
  <prematch>^\w\w\w \d+ \d+:\d+:\d+ \S+ </prematch>
  <type>firewall</type>
</decoder>


<!--

Mar 30 09:03:56 127.0.0.1 Mar 30 8:52:05 st4600fw01n1 allow <eth6 mail src: 192.168.5.133; dst: 89.208.212.2; proto: tcp; bytes: 2396; sent_bytes: 1180; received_bytes: 1216; app_id: 10063753; browse_time: ******; Suppressed logs: 19; Referrer_self_uid: ******; product: Application Control; service: http; s_port: 64136; product_family: Network;

-->
<decoder name="Checkpoint-block-allow">
  <parent>Checkpoint-test</parent>
  <prematch offset="after_parent">^block|^allow</prematch>
  <regex offset="after_parent">(\w+) \S+ \S+ src: (\d+.\d+.\d+.\d+); dst: (\d+.\d+.\d+.\d+)</regex>
  <order>action,srcip,dstip</order>
</decoder>


<!--
Checkpoint-block-allow: extra fields: resource and product

Mar 30 09:03:35 127.0.0.1 Mar 30 8:51:43 st4600fw01n1 allow <eth6 mail src: 192.168.5.133; dst: 89.208.212.2; proto: tcp; appi_name: ******; app_desc: ******; app_id: 10063753; app_category: ******; matched_category: ******; app_properties: ******; app_risk: ******; app_rule_id: ******; app_rule_name: ******; web_client_type: Chrome; web_server_type: Microsoft-IIS; app_sig_id: 10063753:5; resource: http://www.aliveproxy.com/; proxy_src_ip: 192.168.5.133; product: Application Control; service: http; s_port: 64136; product_family: Network;

-->
<decoder name="Checkpoint-block-allow">
  <parent>Checkpoint-test</parent>
  <regex offset="after_regex">resource: (\S+); proxy_src_ip: \S+; product: (\.+); </regex>
  <order>url, extra_data</order>
</decoder>

I recommend you configure all your checkpoint devices with the same log format. If you can't you could use several parents:

<!--
pattern:
Mar 29 09:04:21 127.0.0.1 Mar 29 8:52:29 st4600fw01n1 Text
-->
<decoder name="Checkpoint-test">
  <prematch>^\w\w\w \d+ \d+:\d+:\d+ \S+ </prematch>
  <type>firewall</type>
</decoder>


<!--
pattern:

Mar 30 10:04:39 127.0.0.1 redirect <eth1 alert web_client_type: Chrome; resource: http://sc1.checkpoint.com/za/images/threatwiki/pages/TestAntiBotBlade.html; src: 192.168.5.133; dst: 172.226.217.148; proto: tcp; session_id: {0x56fb8896,0x10009,0xc50d2e0a,0xc0000001}; Protection name: Check Point - Testing Bot; malware_family: Check Point; Source OS: Windows; Confidence Level: 5; severity: 2; malware_action: Communication with C&C site; rule_uid: {9AF67731-0D35-4117-AF2B-9A47F9396D26}; Protection Type: URL reputation; malware_rule_id: {000000CE-00A4-0046-9658-621EA5468654}; protection_id: 00233CFEE; log_id: 2; proxy_src_ip: 192.168.5.133; scope: 192.168.5.133; product: Anti Malware; service: http; s_port: 49244;


Mar 29 20:13:14 127.0.0.1 prevent <eth1 alert web_client_type: Microsoft IE; resource: http://www.bing.com/fd/ls/GLinkPing.aspx?IG=9A0044152B65437D93F87086B9E730D9&&ID=SERP,5118.1&url=http://sc1.checkpoint.com/z...; src: 192.168.5.133; dst: 204.79.197.200; proto: tcp; session_id: {0x56fac5ba,0x10004,0xc50d2e0a,0xc0000000}; Protection name: Check Point - Testing Bot; malware_family: Check Point; Source OS: Windows; Confidence Level: 5; severity: 2; malware_action: Communication with C&C site; rule_uid: {9AF67731-0D35-4117-AF2B-9A47F9396D26}; Protection Type: URL reputation; malware_rule_id: {000000CE-00A4-0046-9658-621EA5468654}; protection_id: 00233CFEE; log_id: 2; proxy_src_ip: 192.168.5.133; scope: 192.168.5.133; product: Anti Malware; service: http; s_port: 63119;

-->
<decoder name="Checkpoint-test">
  <prematch>^redirect \p|^prevent \p</prematch>
  <type>firewall</type>
</decoder>


<!--

Mar 30 09:03:56 127.0.0.1 Mar 30 8:52:05 st4600fw01n1 allow <eth6 mail src: 192.168.5.133; dst: 89.208.212.2; proto: tcp; bytes: 2396; sent_bytes: 1180; received_bytes: 1216; app_id: 10063753; browse_time: ******; Suppressed logs: 19; Referrer_self_uid: ******; product: Application Control; service: http; s_port: 64136; product_family: Network;

-->
<decoder name="Checkpoint-block-allow">
  <parent>Checkpoint-test</parent>
  <prematch offset="after_parent">^block|^allow</prematch>
  <regex offset="after_parent">(\w+) \S+ \S+ src: (\d+.\d+.\d+.\d+); dst: (\d+.\d+.\d+.\d+)</regex>
  <order>action,srcip,dstip</order>
</decoder>


<!--
Checkpoint-block-allow: extra fields: resource and product

Mar 30 09:03:35 127.0.0.1 Mar 30 8:51:43 st4600fw01n1 allow <eth6 mail src: 192.168.5.133; dst: 89.208.212.2; proto: tcp; appi_name: ******; app_desc: ******; app_id: 10063753; app_category: ******; matched_category: ******; app_properties: ******; app_risk: ******; app_rule_id: ******; app_rule_name: ******; web_client_type: Chrome; web_server_type: Microsoft-IIS; app_sig_id: 10063753:5; resource: http://www.aliveproxy.com/; proxy_src_ip: 192.168.5.133; product: Application Control; service: http; s_port: 64136; product_family: Network;

-->
<decoder name="Checkpoint-block-allow">
  <parent>Checkpoint-test</parent>
  <regex offset="after_regex">resource: (\S+); proxy_src_ip: \S+; product: (\.+); </regex>
  <order>url, extra_data</order>
</decoder>

P.S. My name is Jesus, not Jose ;).

Regards,

Jesus Linares.

...

[Fredrik]

Very sorry about the mistake with your name - hope I haven't done it before!? 

Will try out your much much appreciated suggestions for decoders over the weekend! Very excited! :)

Thanks,

Fredrik 

...

[Fredrik]

Hello Jesus!

Story continues. Just wanted to let you know that I have been able, with help, to unify ALL the messages for easier handling in OSSEC. Thing is now that the hostname is extracted automagically (by OSSEC) from the message and I guess can't be used for my prematch, or? Ossec-logtest will treat the hostname as part of the header and start the 'Log:' section with e.g. 

block <eth6 mail src: 10.46.7.196; dst: 37.157.4.16; protocol     ...

How would you tackle this? Right a prematch with all operative words (actions) that is used with the messages I'm interested in (e.g. ^allow|^block|^prevent|^redirect)? In my scenario this shouldn't conflict with other type of messages. I'm guessing that you Ossec-pros will have options and better alternative though ;) I would also like to match the decoder regardless of which node in the firewall cluster is the source of the event? I The two possibilities are st4600fw01n1 and st4600fw01n2 .

Here are more message samples:

pr 15 14:41:53 st4600fw01n1 allow <eth6 mail src: 192.168.5.133; dst: 216.131.91.92; proto: tcp; appi_name: ******; app_desc: ******; app_id: 60461422; app_category: ******; matched_category: ******; app_properties: ******; app_risk: ******; app_rule_id: ******; app_rule_name: ******; web_client_type: Chrome; web_server_type: Apache; app_sig_id: 60461422:1; resource: http://strongvpn.com/difference_between_proxy_and_vpn.html?utm_source=adwords&utm_medium=sem&gclid=Cj0KEQjwosK4BRCYhsngx4_SybcBEiQAowaCJTFp6qNVmL7E-BhfeTkQouJTwpHN5v1wslK79jD62k4aAqBB8P8HAQ; proxy_src_ip: 192.168.5.133; product: Application Control; service: http; s_port: 59319; product_family: Network;

Apr 15 14:21:37 st4600fw01n1 redirect <eth1 alert web_client_type: Chrome; resource: http://sc1.checkpoint.com/za/images/threatwiki/pages/TestAntiBotBlade.html; src: 192.168.5.133; dst: 184.31.90.152; proto: tcp; session_id: {0x5710dcd1,0x10002,0xc50d2e0a,0xc0000000}; Protection name: Check Point - Testing Bot; malware_family: Check Point; Source OS: Windows; Confidence Level: 5; severity: 2; malware_action: Communication with C&C site; rule_uid: {9AF67731-0D35-4117-AF2B-9A47F9396D26}; Protection Type: URL reputation; malware_rule_id: {000000CE-00A4-0046-9658-621EA5468654}; protection_id: 00233CFEE; log_id: 2; proxy_src_ip: 192.168.5.133; scope: 192.168.5.133; product: Anti Malware; service: http; s_port: 57878;

Apr 15 05:35:51 st4600fw01n1 prevent <eth6 alert src: 82.221.102.34; dst: 192.168.99.4; proto: tcp; session_id: {0x57106197,0x10003,0xc50d2e0a,0xc0000001}; Protection name: Trojan.Win32.HackerDefender.C; malware_family: HackerDefender; Source OS: Solaris; Confidence Level: 5; severity: 4; malware_action: Malicious network activity; rule_uid: {25157EEE-C09C-4FE0-A872-E0A1486526B8}; rule_name: #extweb; Protection Type: protection; malware_rule_id: {000000CE-00A4-0046-9658-621EA5468654}; protection_id: 000043FBC; log_id: 2; scope: 192.168.99.4; product: Anti Malware; service: http; s_port: 49228;

Apr 15 14:13:17 st4600fw01n1 block <eth6 mail src: 192.168.7.196; dst: 37.157.2.24; proto: tcp; appi_name: ******; app_desc: ******; app_id: 1875144601; app_category: ******; matched_category: ******; app_properties: ******; app_risk: ******; app_rule_id: ******; app_rule_name: ******; web_client_type: Chrome; web_server_type: Other: nginx; resource: http://adx.adform.net/adx/?rp=3&pv=1&bWlkPTk3ODMyJmN0dXJsPWh0dHA6Ly9mdXNpb24uYm9ubmllcnRpZHNrcmlmdGVyLnNlL2V2ZW50L2ZsaXlmL2J0LmVrb25vbWkucGFmLnJvcy8xNDc3OTkyNzcvY2xpY2s_dXJsPQ&bWlkPTk3ODMxJmN0dXJsPWh0dHA6Ly9mdXNpb24uYm9ubmllcnRpZHNrcmlmdGVyLnNlL2V2ZW50L2V0YnN3L2J0LmVrb25vbWkucGFmLnJvcy8xNDc3OTkyNzAvY2xpY2s_dXJsPQ&bWlkPTk3ODI5JmN0dXJsPWh0dHA6Ly9mdXNpb24uYm9ubmllcnRpZHNrcmlmdGVyLnNlL2V2ZW50L3FmdWh5L2J0LmVrb25vbWkucGFmLnJvcy8xNDc3OTkyNTYvY2xpY2s_dXJsPQ&callback=_adform_cb_1460722287088_3438587873323349; proxy_src_ip: 192.168.7.196; product: URL Filtering; service: http; s_port: 51190; product_family: Network;

Apr 15 11:16:05 st4600fw01n1 block <eth6 mail src: 192.168.8.67; dst: 64.207.139.185; proto: tcp; appi_name: ******; app_desc: ******; app_id: 3723664659; app_category: ******; matched_category: ******; app_properties: ******; app_risk: ******; app_rule_id: ******; app_rule_name: ******; web_client_type: Chrome; web_server_type: Apache; resource: http://cdn.wibiya.com/Toolbars/dir_0650/Toolbar_650079/Loader_650079.js; proxy_src_ip: 192.168.8.67; product: URL Filtering; service: http; s_port: 61907; product_family: Network;

The two outliers now are the messages below. Not quite sure how to handle them, but two additional decoders seem required, At least I'm down to two outliers and not a whole bunch of exceptions as previously :) :) What would be your take on how to treat these two?

Mar  7 13:07:53 sto-fwm03 mail System Alert message: A Firewall Policy has been successfully installed on st4600fw01n1; Object: st4600fw01n1; Event: Change; Parameter: policy_time; Condition: changes Mon Mar  7 13:03:42 2016; Current value: Mon Mar  7 13:08:48 2016; product: Test Monitor; product_family: Network;

Apr 15 12:23:16 > st4600fw01n1 alert Protection Name: TCP Urgent Data Enforcement; Severity: 0; Confidence Level: 0; protection_id: tcp_block_urg_bit_enable; SmartDefense Profile: Recommended_Protection; Performance Impact: 0; Protection Type: settings; rule: 20; rule_uid: {3F67BCCB-8087-4974-95FA-F4A4FF466D49}; rule_name: #sample; Attack Info: TCP segment with urgent pointer (no data). Urgent data indication was stripped. Please refer to sk36869.; attack: Streaming Engine: TCP Urgent Data Enforcement; Total logs: 3; Suppressed logs: 2; proto: tcp; dst: 104.16.65.50; src: 192.168.10.204; product: SmartDefense; service: https; s_port: 56814; FollowUp: Not Followed; product_family: Network;

Best regards,

Fredrik 

...

[Jesus Linares]

Hi Fredrik,

It is good progress. You can capture all events with:

<decoder name="Checkpoint-test">
  <prematch>^redirect \p|^prevent \p|^allow \p|^block \p|^mail System Alert|\S+ alert Protection Name:</prematch>
  <type>firewall</type>
</decoder>

I know... It is not very elegant, but it controls all your events. Also, you can add a tag in the beginning of the log (by the firewall settings or with rsyslog) and the decoder will be vey easy:

Logs:

FredikFirewall Apr 15 12:23:16 > st4600fw01n1 alert Protection Name: TCP Urgent Data Enforcement; Severity: 0; Confidence Level: 0; protection_id: tcp_block_urg_bit_enable; SmartDefense Profile: Recommended_Protection; Performance Impact: 0; Protection Type: settings; rule: 20; rule_uid: {3F67BCCB-8087-4974-95FA-F4A4FF466D49}; rule_name: #sample; Attack Info: TCP segment with urgent pointer (no data). Urgent data indication was stripped. Please refer to sk36869.; attack: Streaming Engine: TCP Urgent Data Enforcement; Total logs: 3; Suppressed logs: 2; proto: tcp; dst: 104.16.65.50; src: 192.168.10.204; product: SmartDefense; service: https; s_port: 56814; FollowUp: Not Followed; product_family: Network;

Decoder:

<decoder name="Checkpoint-test">
  <prematch>^FredikFirewall </prematch>
  <type>firewall</type>
</decoder>

Regards,

Jesus Linares.

...

[Fredrik]

Hello Jesus!

Thanks for getting back to me. Again :) :) I'm trying out your enhancement to the first decoder and trying to combine it with child-decoders from our previous posts. I currently have this (which obviously doesn't work), but how do I best create the parent-child tree to handle the slight variations in messages? One per action (e.g. block, prevent) and where needed use two child-decoders with same name to capture all field of interest - as you have instructed previously (i.e. one for fields action,srcip,dstip , the second for url, extra_data fields?

Best regards,

Fredrik 

<decoder name="Checkpoint-test">

  <prematch>^redirect \p|^prevent \p|^allow \p|^block \p|^mail System Alert|\S+ alert Protection Name:</prematch>

  <type>firewall</type>

</decoder>

<decoder name="Checkpoint-block-allow">

  <parent>Checkpoint-test</parent>

  <regex>(\w+) \S+ \S+ src: (\d+.\d+.\d+.\d+); dst: (\d+.\d+.\d+.\d+)</regex>

  <order>action,srcip,dstip</order>

</decoder>

...

[Antonio Querubin]

On Fri, 15 Apr 2016, Fredrik wrote:

> Thanks for getting back to me. Again :) :) I'm trying out your enhancement
> to the first decoder and trying to combine it with child-decoders from our
> previous posts. I currently have this (which obviously doesn't work), but
> how do I best create the parent-child tree to handle the slight variations
> in messages? One per action (e.g. block, prevent) and where needed use two
> child-decoders with same name to capture all field of interest - as you
> have instructed previously (i.e. one for fields action,srcip,dstip , the
> second for url, extra_data fields?

> <decoder name="Checkpoint-block-allow">
> <parent>Checkpoint-test</parent>
> <regex>(\w+) \S+ \S+ src: (\d+.\d+.\d+.\d+); dst:
> (\d+.\d+.\d+.\d+)</regex>
> <order>action,srcip,dstip</order>
> </decoder>

If you plan on detecting IPv6 activity on your network you may want to
change \d+.\d+.\d+.\d+ to \S+


Antonio Querubin
e-mail: to...@lavanauts.org
xmpp: antonio...@gmail.com

[Fredrik]

Thanks Antonio! Noted! 

Best regards,

Fredrik

[Fredrik]

Hi Jesus!

Hope you have had a nice summer so far :) I'm revisiting this decoder with, what I hoped would be, a fresh (rested) pair of eyes ;) Unfortunately, I realize I still have trouble sorting this one out in an efficient manner. I was hoping I could ask you for a few additional pointers especially regarding how to best extract the relevant information from all five types of messages (outlined below). My current challenge is that the 'catch all' parent decoder and allow-block-child, result in events of types redirect/alert/system alert not be parsed completely i.e. information like action/src/dst/message not being extracted. I have attacked the problem from the few angles I can think of, but have come up short :(

Note: All messages except "System Alert" now share a commonality in the hostname which is the active node in the cluster, possible values being st4600fw01n1 or st4600fw01n2 . Previously (in thread) you showed me how the actions could be used in parent decoder <prematch>^redirect \p|^prevent \p|^allow \p|^block \p|^mail System Alert|\S+ alert Protection Name:</prematch>.

- Should I use the hostname as the matching criteria for my parent decoder and group these four events together whilst creating a separate decoder for System Alert? 

- How would you then suggest I go about extracting the information action/srcip/dstip/url/extra data for the for types of messages in this 'group' using child decoders?

BLOCK

Jun  2 13:24:13 st4600fw01n1 block <eth6 mail src: 192.168.71.151; dst: 54.164.78.72; proto: tcp; bytes: 1845; sent_bytes: 749; received_bytes: 1096; app_id: 1347922162; browse_time: ******; Suppressed logs: 7; Referrer_self_uid: ******; Referrer_Parent_uid: ******; product: URL Filtering; service: http; s_port: 51096; product_family: Network;

Jun  2 13:24:54 st4600fw01n1 block <eth6 mail src: 192.168.71.151; dst: 54.164.78.72; proto: tcp; bytes: 11122; sent_bytes: 4494; received_bytes: 6628; app_id: 1347922162; browse_time: ******; Referrer_self_uid: ******; product: URL Filtering; service: http; s_port: 51096; product_family: Network;

Jun  2 13:31:57 st4600fw01n1 block <eth6 mail src: 192.168.71.3; dst: 152.115.75.210; proto: tcp; appi_name: ******; app_desc: ******; app_id: 1875144601; app_category: ******; matched_category: ******; app_properties: ******; app_risk: ******; app_rule_id: ******; app_rule_name: ******; web_client_type: Other: Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; Touch; rv:11.0) like Gecko; web_server_type: Other: nginx; resource: http://adx.adform.net/adx/?rp=3&pv=1&bWNyPWhiX2FkaWQ6MmIwZmQyMDc0OTllYTEmZHByPTQuMjA2NjQmbWt2PWhiX2JpZGRlcjpydWJpY29uJm1rdz12ZWN0dXJhJTJDZmFzdGlnaGV0ZXIlMkNhYiUyQzU1NjkwMzA1ODcmbWlkPTExODY4NQ&bWNyPWhiX2FkaWQ6MWEwNDNmY2MyNjk2MTk4JmRwcj04LjMwODk2OSZta3Y9aGJfYmlkZGVyOnJ1Ymljb24mbWt3PXZlY3R1cmElMkNmYXN0aWdoZXRlciUyQ2FiJTJDNTU2OTAzMDU4NyZtaWQ9MTE4Njg0&callback=_adform_cb_1464866991419_10159400672628005; proxy_src_ip: 192.168.71.3; product: URL Filtering; service: http; s_port: 51311; product_family: Network;

ALLOW

Jun  2 13:50:15 st4600fw01n1 allow <eth6 mail src: 192.168.99.11; dst: 107.170.204.55; proto: tcp; appi_name: ******; app_desc: ******; app_id: 60520086; app_category: ******; matched_category: ******; app_properties: ******; app_risk: ******; app_rule_id: ******; app_rule_name: ******; app_sig_id: 60520086:4; proxy_src_ip: 192.168.99.11; product: Application Control; service: https; s_port: 54159; product_family: Network;

Jun  2 13:59:05 st4600fw01n1 allow <eth1 mail src: 192.168.99.11; dst: 89.208.212.2; proto: tcp; appi_name: ******; app_desc: ******; app_id: 10063753; app_category: ******; matched_category: ******; app_properties: ******; app_risk: ******; app_rule_id: ******; app_rule_name: ******; web_client_type: Chrome; web_server_type: Microsoft-IIS; app_sig_id: 10063753:5; resource: http://www.aliveproxy.com/; proxy_src_ip: 192.168.99.11; product: Application Control; service: http; s_port: 54473; product_family: Network;

ALERT

May 31 08:05:18 > st4600fw01n1 alert Protection Name: TCP Urgent Data Enforcement; Severity: 0; Confidence Level: 0; protection_id: tcp_block_urg_bit_enable; SmartDefense Profile: Recommended_Protection; Performance Impact: 0; Protection Type: settings; Attack Info: TCP segment with urgent pointer (no data). Urgent data indication was stripped. Please refer to sk36869.; attack: Streaming Engine: TCP Urgent Data Enforcement; Total logs: 8; Suppressed logs: 7; proto: tcp; dst: 52.22.193.162; src: 192.168.10.204; product: SmartDefense; service: http; s_port: 50869; FollowUp: Not Followed; product_family: Network;

May 31 17:51:04 > st4600fw01n1 alert Protection Name: TCP Urgent Data Enforcement; Severity: 0; Confidence Level: 0; protection_id: tcp_block_urg_bit_enable; SmartDefense Profile: Recommended_Protection; Performance Impact: 0; Protection Type: settings; rule: 21; rule_uid: {3F67BCCB-8087-4974-95FA-F4A4FF466D49}; rule_name: #name; Attack Info: TCP segment with urgent pointer (no data). Urgent data indication was stripped. Please refer to sk36869.; attack: Streaming Engine: TCP Urgent Data Enforcement; Total logs: 24; Suppressed logs: 23; proto: tcp; dst: 54.239.168.11; src: 192.168.10.204; product: SmartDefense; service: http; s_port: 60324; FollowUp: Not Followed; product_family: Network;

Aug 17 12:37:14 > st4600fw01n1 alert Protection Name: TCP Urgent Data Enforcement; Severity: 0; Confidence Level: 0; protection_id: tcp_block_urg_bit_enable; SmartDefense Profile: Recommended_Protection; Performance Impact: 0; Protection Type: settings; rule: 23; rule_uid: {3F67BCCB-8087-4974-95FA-F4A4FF466D49}; rule_name: #name; Attack Info: TCP segment with urgent pointer (no data). Urgent data indication was stripped. Please refer to sk36869.; attack: Streaming Engine: TCP Urgent Data Enforcement; Total logs: 11; Suppressed logs: 10; proto: tcp; dst: 80.251.201.102; src: 172.18.46.230; product: SmartDefense; service: https; s_port: 57991; FollowUp: Not Followed; product_family: Network;

Aug 17 04:33:21 > st4600fw01n1 alert Protection Name: Packet Sanity; Severity: 2; Confidence Level: 5; protection_id: PacketSanity; SmartDefense Profile: Recommended_Protection; Performance Impact: 1; Industry Reference: CAN-2002-1071; Protection Type: anomaly; Attack Info: Invalid TCP packet - source / destination port 0; attack: Malformed Packet; Total logs: 3; Suppressed logs: 2; proto: tcp; dst: 80.169.184.240; src: 185.65.132.121; product: SmartDefense; FollowUp: Not Followed; product_family: Network;

REDIRECT

Jun  2 13:54:09 st4600fw01n1 redirect <eth1 alert web_client_type: Chrome; resource: http://sc1.checkpoint.com/za/images/threatwiki/pages/TestAntiBotBlade.html; src: 192.168.99.11; dst: 172.226.217.148; proto: tcp; session_id: {0x57501e61,0x1001b,0xc50d2e0a,0xc0000000}; Protection name: Check Point - Testing Bot; malware_family: Check Point; Source OS: Windows; Confidence Level: 5; severity: 2; malware_action: Communication with C&C site; rule_uid: {9AF67731-0D35-4117-AF2B-9A47F9396D26}; Protection Type: URL reputation; malware_rule_id: {000000CE-00A4-0046-9658-621EA5468654}; protection_id: 00233CFEE; log_id: 2; proxy_src_ip: 192.168.99.11; scope: 192.168.99.11; product: Anti Malware; service: http; s_port: 54402;

SYSTEM ALERT

May 31 21:31:58 sto-fwm03 mail System Alert message: A Firewall Policy has been successfully installed on st4600fw01n1; Object: st4600fw01n1; Event: Change; Parameter: policy_time; Condition: changes Tue May 31 14:44:13 2016; Current value: Tue May 31 21:04:34 2016; product: System Monitor; product_family: Network;

I realize I'm pushing my luck here! You have done more than enough to set me off in the right direction, and it might just be that I should work up my skills on simpler messages. However, hoping that the concepts you share on creating parent/child decoders is useful to the rest of the community. 

Best regards,

Fredrik 

[Jesus Linares]

Hi Fredik,

Long time no see!. It is a hot summer here and, as always, playing with OSSEC ;).

I don't have time to create all the decoders, but here a template to help you:

<decoder name="checkpoint">

  <prematch>^redirect \p|^prevent \p|^allow \p|^block \p|^mail System Alert|\S+ alert Protection Name:</prematch>
  <type>firewall</type>
</decoder>

<!--

BLOCK
Jun  2 13:24:13 st4600fw01n1 block <eth6 mail src: 192.168.71.151; dst: 54.164.78.72; proto: tcp; bytes: 1845; sent_bytes: 749; received_bytes: 1096; app_id: 1347922162; browse_time: ******; Suppressed logs: 7; Referrer_self_uid: ******; Referrer_Parent_uid: ******; product: URL Filtering; service: http; s_port: 51096; product_family: Network;

-->
<decoder name="checkpoint-block">
  <parent>checkpoint</parent>
  <prematch>^block \p</prematch>
  <regex offset="after_prematch">src: (\S+); dst: (\S+); proto: (\S+);</regex>
  <order>srcip,dstip,protocol</order>
</decoder>


<!--
ALLOW
...
-->


<!--

ALERT
May 31 08:05:18 > st4600fw01n1 alert Protection Name: TCP Urgent Data Enforcement; Severity: 0; Confidence Level: 0; protection_id: tcp_block_urg_bit_enable; SmartDefense Profile: Recommended_Protection; Performance Impact: 0; Protection Type: settings; Attack Info: TCP segment with urgent pointer (no data). Urgent data indication was stripped. Please refer to sk36869.; attack: Streaming Engine: TCP Urgent Data Enforcement; Total logs: 8; Suppressed logs: 7; proto: tcp; dst: 52.22.193.162; src: 192.168.10.204; product: SmartDefense; service: http; s_port: 50869; FollowUp: Not Followed; product_family: Network;

-->
<decoder name="checkpoint-alert">
  <parent>checkpoint</parent>
  <prematch>alert Protection Name: </prematch>
  <regex offset="after_prematch">(\.+); Severity: (\d+);</regex>
  <order>url,status</order>
</decoder>


<!--
REDIRECT
...
-->


<!--
SYSTEM ALERT
...
-->

If you have questions with a specific decoder, just post it here.

To make useful decoders for the community, you should:

• Add the version of your checkpoint at the begging of the file.
• Use a standard log format: are you using the default log format of checkpoint?. If not, you should use it or at least, use the syslog standard. In this way, the decoders will work for other users.
• Provide log samples for each decoder: I usually paste the log as comments in the decoders.

If you don't mind, when you have the decoders working, you could send them to our ruleset repositoy and to ossec-hids.

Also, you could use plugin decoders instead of standard decoders. Plugin decoders ara coded in c and it is useful with complex logs (like firewalls).

I hope it helps.

Regards.

[Fredrik]

Thanks again Jesus!

I will definitely share what I come up with and thanks for all your suggestions and bearing with me through this (long) thread :)

Fredrik