Configure palo alto firewall logs in wazuh

7,335 views
Skip to first unread message

Piyush Chhabra

unread,
Apr 16, 2019, 3:42:35 AM4/16/19
to Wazuh mailing list
Hi guys

I want to configure palo alto firewall logs in wazuh. I have added the firewall ips in <remote> as same as for switches and also the in ossec logs remote syslog allowed from firewall IP. 
Even when i ping from firewall to my wazuh manager i can see that ICMP packet from tcpdump in wazuh server but there are no logs coming in my wazuh server. Please help me on this.


Thanks
Message has been deleted

francisc...@wazuh.com

unread,
Apr 17, 2019, 5:54:41 AM4/17/19
to Wazuh mailing list

Hello Piyush!

Currently, Wazuh doesn’t have decoders and rules for Palo Alto firewall logs, so the manager won’t analyze them.

However, you can define your own decoders and rules for certain program and allow Wazuh to process the logs and generate alerts if you want. See custom rules and decoders for more information.

We will be glad to help you to write some decoder and rules if you share with us some examples of logs that you are interested in analyzing.

What is necessary to do:

  • Find what kind of log is triggered by PAN firewall and how they are presented on syslog, for example, on a centOS7 machine, in /var/log/secure appear the following line related to sshd when there is a failed attempt of connection with an invalid user:
Apr 17 08:22:49 centos7 sshd[3936]: Invalid user user-example from 172.16.1.1 port 45298

As an example of how Wazuh decodes logs, you can check /var/ossec/ruleset/decoders directory. For this example with sshd I suggest you checking 0310-ssh_decoders.xml file. There, you have this rule:

<decoder name="ssh-invalid-user">
  <parent>sshd</parent>
  <prematch>^Invalid user|^Illegal user</prematch>
  <regex offset="after_prematch">(\S+) from (\S+)</regex>
  <order>srcuser,srcip</order>
</decoder>

Which is used to detect this kind of messages and extract the invalid user and the source IP.

  • Once we understand this and know the format of PAN firewall logs you need to write a custom decoder on /var/ossec/etc/decoders/local_decoder.xml with the desired messages types to analyze.
  • Once the logs are “detected”, it is necessary to have a rule that triggers the alert. In the example with sshd, you have the file /var/ossec/ruleset/rules/0095-sshd_rules.xml you have the following rule:
  <rule id="5710" level="5">
    <if_sid>5700</if_sid>
    <match>illegal user|invalid user</match>
    <description>sshd: Attempt to login using a non-existent user</description>
    <group>invalid_login,authentication_failed,pci_dss_10.2.4,pci_dss_10.2.5,pci_dss_10.6.1,gpg13_7.1,gdpr_IV_35.7.d,gdpr_IV_32.2,</group>
  </rule>

Here we define, for example, the level of the alert generated, an id, the alert group…

We need to write the customs rules for the custom decoder on /var/ossec/etc/rules/local_rules.xml

  • Once we have write your decoder and rules we can test if everything works with /var/ossec/bin/ossec-logtest

Let us know if you need anything else!

Best regards.


    donetz errasti

    unread,
    May 3, 2019, 7:38:58 AM5/3/19
    to Wazuh mailing list

    Hi, 


    I'm working in a custom wazuh decoder for Palo Alto log but I'm not doing well and It would be great if anyone could help me. This is Palo ALto log format:


    FUTURE_USE, Receive Time, Serial Number, Type, Threat/Content Type, FUTURE_USE, Generated Time, Source IP, Destination IP, NAT Source IP, NAT Destination IP, Rule Name, Source User, Destination User, Application, Virtual System, Source Zone, Destination Zone, Inbound Interface, Outbound Interface, Log Action, FUTURE_USE, Session ID, Repeat Count, Source Port, Destination Port, NAT Source Port, NAT Destination Port, Flags, Protocol, Action, Bytes, Bytes Sent, Bytes Received, Packets, Start Time, Elapsed Time, Category, FUTURE_USE, Sequence Number, Action Flags, Source Location, Destination Location, FUTURE_USE, Packets Sent, Packets Received, Session End Reason, Device Group Hierarchy Level 1, Device Group Hierarchy Level 2, Device Group Hierarchy Level 3, Device Group Hierarchy Level 4, Virtual System Name, Device Name, Action Source, Source VM UUID, Destination VM UUID, Tunnel ID/IMSI, Monitor Tag/IMEI, Parent Session ID, Parent Start Time, Tunnel Type


    And there is a log examples I'm receiving in my wazuh manager:



    2019 May 03 08:43:08 pamgmt->10.10.10.10 May  3 10:43:08 pamgmt 1,2019/05/03 10:43:08,XXXXXXXXXXX,THREAT,url,2049,2019/05/03 10:43:08,172.17.2.4,X.X.X.X,10.0.1.4,X.X.X.X,navegacion,domain\username,,ssl,vsys1,VPN,UNTRUST,tunnel.2,ethernet1/1,LogstashSys,2019/05/03 10:43:08,188036,1,57132,443,32403,443,0x40b000,tcp,alert,"c.urs.microsoft.com/",(9999),computer-and-internet-info,informational,client-to-server,356197,0x2000000000000000,172.16.0.0-172.31.255.255,Netherlands,0,,0,,,0,,,,,,,,0,0,0,0,0,,pamgmt,,,,,0,,0,,N/A,unknown,AppThreat-0-0,0x0,0,4294967295,

    francisc...@wazuh.com

    unread,
    May 6, 2019, 7:50:31 AM5/6/19
    to Wazuh mailing list

    Hello Donetz,

    I am working on generating some decoders and example rules for the log that you have shared, tomorrow I will give you an answer.

    Thank you for your patience.

    Best regards
    - Francisco Navarro

    donetz errasti

    unread,
    May 6, 2019, 10:02:57 AM5/6/19
    to Wazuh mailing list
    Hi Francisco,

    Thank you for your help, I appreciate it so much. 

    Donetz

    francisc...@wazuh.com

    unread,
    May 7, 2019, 7:08:34 AM5/7/19
    to Wazuh mailing list

    Hello Donetz! Let’s do this.

    First of all, let me introduce you some documents from Wazuh documentation that will be helpful:

    Now, we should start by writing a custom decoder on /var/ossec/etc/decoders/local_decoder.xml as is indicated on [1].

    We should start by writing a simple decoder which recognizes the entire log string as a PaloAlto event by identifying the Serial Number of the log, which seems to be a combination of eleven numbers (so we will use \d\d\d\d\d\d\d\d\d\d to identify it, see [3] )

    <!-- Custom decoder for PaloAlto Firewalls Events -->  
    <decoder name="paloalto">  
    <prematch>,\d\d\d\d\d\d\d\d\d\d\d,</prematch>  
    </decoder>
    

    What we are saying here to the decoder is “find eleven numbers between two commas and anything else”.

    Now, we could save the decoder and test it by executing /var/ossec/bin/ossec-logtest as specified in [1]. I will introduce the log you give us but changing the XXXXXXXXXXX by numbers (12345678987) in order to allow the decoder to recognize the log as a PaloAlto one.`

    The output of the command is:

    **Phase 1: Completed pre-decoding.  
    full event: '2019 May 03 08:43:08 pamgmt->10.10.10.10 May 3 10:43:08 pamgmt 1,2019/05/03 10:43:08,12345678987,THREAT,url,2049,2019/05/03 10:43:08,172.17.2.4,X.X.X.X,10.0.1.4,X.X.X.X,navegacion,domain\username,,ssl,vsys1,VPN,UNTRUST,tunnel.2,ethernet1/1,LogstashSys,2019/05/03 10:43:08,188036,1,57132,443,32403,443,0x40b000,tcp,alert,"c.urs.microsoft.com/",(9999),computer-and-internet-info,informational,client-to-server,356197,0x2000000000000000,172.16.0.0-172.31.255.255,Netherlands,0,,0,,,0,,,,,,,,0,0,0,0,0,,pamgmt,,,,,0,,0,,N/A,unknown,AppThreat-0-0,0x0,0,4294967295,'  
    timestamp: '2019 May 03 08:43:08'  
    hostname: 'centos7'  
    program_name: '(null)'  
    log: 'pamgmt->10.10.10.10 May 3 10:43:08 pamgmt 1,2019/05/03 10:43:08,12345678987,THREAT,url,2049,2019/05/03 10:43:08,172.17.2.4,X.X.X.X,10.0.1.4,X.X.X.X,navegacion,domain\username,,ssl,vsys1,VPN,UNTRUST,tunnel.2,ethernet1/1,LogstashSys,2019/05/03 10:43:08,188036,1,57132,443,32403,443,0x40b000,tcp,alert,"c.urs.microsoft.com/",(9999),computer-and-internet-info,informational,client-to-server,356197,0x2000000000000000,172.16.0.0-172.31.255.255,Netherlands,0,,0,,,0,,,,,,,,0,0,0,0,0,,pamgmt,,,,,0,,0,,N/A,unknown,AppThreat-0-0,0x0,0,4294967295,'  
    
    **Phase 2: Completed decoding.  
    decoder: 'paloalto'
    

    That means that the decoder is recognizing our log. Now we can start working with it.

    We will create a new, more specific, decoder which will get parameters from the log and will be used to generate custom alerts, it will be:

    <!-- Custom decoder for PaloAlto Firewalls Events -->  
    <decoder name="paloalto-child">  
    <parent>paloalto</parent>  
    <regex offset="after_parent">^(\w+),(\w+),(\S+),(\d\d\d\d/\d\d/\d\d \d\d:\d\d:\d\d),(\d+.\d+.\d+.\d+),(\d+.\d+.\d+.\d+)</regex>  
    <order>pa.type, pa.ctype, extra_data, pa.gtime, pa.sip, pa.dip</order>  
    </decoder>
    

    This decoder has “paloalto” decoder as parent, is called “paloalto-child” and uses a regex for extract information from the log. The “after_parent” value for offset mean that this regex will start just after the part of the log where the parent ends, in our case, it would be just after the Serial Number. We will be using dynamic fields for the order values (which are “variables” we define for store our information and will content the regex inside parenthesis in the order they appear).

    Now, If we execute the testing program with a log with numbers intead of X.X.X.X we got:

    **Phase 1: Completed pre-decoding.  
    full event: '2019 May 03 08:43:08 pamgmt->10.10.10.10 May 3 10:43:08 pamgmt 1,2019/05/03 10:43:08,12345678987,THREAT,url,2049,2019/05/03 10:43:08,172.17.2.4,1.1.1.1,10.0.1.4,2.2.2.2,navegacion,domain\username,,ssl,vsys1,VPN,UNTRUST,tunnel.2,ethernet1/1,LogstashSys,2019/05/03 10:43:08,188036,1,57132,443,32403,443,0x40b000,tcp,alert,"c.urs.microsoft.com/",(9999),computer-and-internet-info,informational,client-to-server,356197,0x2000000000000000,172.16.0.0-172.31.255.255,Netherlands,0,,0,,,0,,,,,,,,0,0,0,0,0,,pamgmt,,,,,0,,0,,N/A,unknown,AppThreat-0-0,0x0,0,4294967295,'  
    timestamp: '2019 May 03 08:43:08'  
    hostname: 'centos7'  
    program_name: '(null)'  
    log: 'pamgmt->10.10.10.10 May 3 10:43:08 pamgmt 1,2019/05/03 10:43:08,12345678987,THREAT,url,2049,2019/05/03 10:43:08,172.17.2.4,1.1.1.1,10.0.1.4,2.2.2.2,navegacion,domain\username,,ssl,vsys1,VPN,UNTRUST,tunnel.2,ethernet1/1,LogstashSys,2019/05/03 10:43:08,188036,1,57132,443,32403,443,0x40b000,tcp,alert,"c.urs.microsoft.com/",(9999),computer-and-internet-info,informational,client-to-server,356197,0x2000000000000000,172.16.0.0-172.31.255.255,Netherlands,0,,0,,,0,,,,,,,,0,0,0,0,0,,pamgmt,,,,,0,,0,,N/A,unknown,AppThreat-0-0,0x0,0,4294967295,'  
    
    **Phase 2: Completed decoding.  
    decoder: 'paloalto'  
    pa.type: 'THREAT'  
    pa.ctype: 'url'  
    extra_data: '2049'  
    pa.gtime: '2019/05/03 10:43:08'  
    pa.sip: '172.17.2.4'  
    pa.dip: '1.1.1.1'
    

    So… we’re getting information from our log. Now we can start writing the rules that will be triggering alerts on the manager.

    In /var/ossec/etc/rules/local_rules.xml we create the following example rules:

    <group name="paloalto">  
    <rule id="123451" level="0">  
    <decoded_as>paloalto</decoded_as>  
    <description>Paloalto firewall event</description>  
    </rule>  
    
    <rule id="123452" level="3">  
    <if_sid>123451</if_sid>  
    <field name="pa.type">THREAT</field>  
    <description>Paloalto THREAT event.</description>  
    </rule>  
    
    <rule id="123453" level="5">  
    <if_sid>123451</if_sid>  
    <field name="pa.type">THREAT</field>  
    <field name="pa.ctype">url</field>  
    <description>Paloalto THREAT url event!! src IP: $(pa.sip), dest IP: $(pa.dip) </description>  
    </rule>  
    </group>
    

    The first one trigger each time a log is detected as a PaloAlto one, even if no correct information is given to it (it will appear each time a log contains a valid PaloAlto Serial number between commas).

    Second one, with alert level=3, trigger if the type of the PaloAlto log is “THREAT” (as in the example) and inform of that situation.

    Third one, with alert level=5, trigger if the log type is “THREAD” and the content type is “url”… also in this example I’ve used the extracted IPs from the log to inform in the alert description of which source and destination IPs are found.

    Now, test our configuration:

    **Phase 1: Completed pre-decoding.
           full event: '2019 May 03 08:43:08 pamgmt->10.10.10.10 May  3 10:43:08 pamgmt 1,2019/05/03 10:43:08,12345678987,THREAT,url,2049,2019/05/03 10:43:08,172.17.2.4,1.1.1.1,10.0.1.4,2.2.2.2,navegacion,domain\username,,ssl,vsys1,VPN,UNTRUST,tunnel.2,ethernet1/1,LogstashSys,2019/05/03 10:43:08,188036,1,57132,443,32403,443,0x40b000,tcp,alert,"c.urs.microsoft.com/",(9999),computer-and-internet-info,informational,client-to-server,356197,0x2000000000000000,172.16.0.0-172.31.255.255,Netherlands,0,,0,,,0,,,,,,,,0,0,0,0,0,,pamgmt,,,,,0,,0,,N/A,unknown,AppThreat-0-0,0x0,0,4294967295,'
           timestamp: '2019 May 03 08:43:08'
           hostname: 'centos7'
           program_name: '(null)'
           log: 'pamgmt->10.10.10.10 May  3 10:43:08 pamgmt 1,2019/05/03 10:43:08,12345678987,THREAT,url,2049,2019/05/03 10:43:08,172.17.2.4,1.1.1.1,10.0.1.4,2.2.2.2,navegacion,domain\username,,ssl,vsys1,VPN,UNTRUST,tunnel.2,ethernet1/1,LogstashSys,2019/05/03 10:43:08,188036,1,57132,443,32403,443,0x40b000,tcp,alert,"c.urs.microsoft.com/",(9999),computer-and-internet-info,informational,client-to-server,356197,0x2000000000000000,172.16.0.0-172.31.255.255,Netherlands,0,,0,,,0,,,,,,,,0,0,0,0,0,,pamgmt,,,,,0,,0,,N/A,unknown,AppThreat-0-0,0x0,0,4294967295,'
    
    **Phase 2: Completed decoding.
           decoder: 'paloalto'
           pa.type: 'THREAT'
           pa.ctype: 'url'
           extra_data: '2049'
           pa.gtime: '2019/05/03 10:43:08'
           pa.sip: '172.17.2.4'
           pa.dip: '1.1.1.1'
    
    **Phase 3: Completed filtering (rules).
           Rule id: '123453'
           Level: '5'
           Description: 'Paloalto THREAT url event!! src IP: 172.17.2.4, dest IP: 1.1.1.1 '
    **Alert to be generated.
    

    Everything is working correctly!

    The following steps would be:

    • Decide if you need more child decoders (if you want to analyze the logs in different ways depending, for example, on the type of log) or you can use just one.
    • Add more wildcard to the child decoder in order to get all the data you would need for your rules and store it in “order” variables.
    • Decide what kind of alerts you want to generate and how important is each one. Then, write rules for all of them and set the desired level.

    I hope my answer will be helpful, if you need more help with that, do not hesitate to ask.

    Best regards
    -Francisco Navarro



    Hello Donetz! Let's do this.

    First of all, let me introduce you some documents from Wazuh documentation that will be helpful:

    - [1] Custom rules and decoders: https://documentation.wazuh.com/current/user-manual/ruleset/custom.html  

    - [2] Decoder syntax: https://documentation.wazuh.com/current/user-manual/ruleset/ruleset-xml-syntax/decoders.html  

    - [3] Regular expression syntax: https://documentation.wazuh.com/current/user-manual/ruleset/ruleset-xml-syntax/regex.html#os-regex-or-regex-syntax

     
    Now, we should start by writing a custom decoder on  `/var/ossec/etc/decoders/local_decoder.xml` as is indicated on [1].  

     

    We should start by writing a simple decoder which recognizes the entire log string as a PaloAlto event by identifying the Serial Number of the log, which seems to be a combination of eleven numbers (so we will use `\d\d\d\d\d\d\d\d\d\d`  to identify it, see [3] )  

    ```xml
    <!-- Custom decoder for PaloAlto Firewalls Events -->  
    <decoder name="paloalto">  
    <prematch>,\d\d\d\d\d\d\d\d\d\d\d,</prematch>  
    </decoder>  
    ```

    What we are saying here to the decoder is "find eleven numbers between two commas and anything else".  


    Now, we could save the decoder and test it by executing `/var/ossec/bin/ossec-logtest`  as specified in [1]. I will introduce the log you give us but changing the XXXXXXXXXXX by numbers (12345678987) in order to allow the decoder to recognize the log as a PaloAlto one.`

    The output of the command is:

    ```bash
    **Phase 1: Completed pre-decoding.  
    full event: '2019 May 03 08:43:08 pamgmt->10.10.10.10 May 3 10:43:08 pamgmt 1,2019/05/03 10:43:08,12345678987,THREAT,url,2049,2019/05/03 10:43:08,172.17.2.4,X.X.X.X,10.0.1.4,X.X.X.X,navegacion,domain\username,,ssl,vsys1,VPN,UNTRUST,tunnel.2,ethernet1/1,LogstashSys,2019/05/03 10:43:08,188036,1,57132,443,32403,443,0x40b000,tcp,alert,"c.urs.microsoft.com/",(9999),computer-and-internet-info,informational,client-to-server,356197,0x2000000000000000,172.16.0.0-172.31.255.255,Netherlands,0,,0,,,0,,,,,,,,0,0,0,0,0,,pamgmt,,,,,0,,0,,N/A,unknown,AppThreat-0-0,0x0,0,4294967295,'  
    timestamp: '2019 May 03 08:43:08'  
    hostname: 'centos7'  
    program_name: '(null)'  
    log: 'pamgmt->10.10.10.10 May 3 10:43:08 pamgmt 1,2019/05/03 10:43:08,12345678987,THREAT,url,2049,2019/05/03 10:43:08,172.17.2.4,X.X.X.X,10.0.1.4,X.X.X.X,navegacion,domain\username,,ssl,vsys1,VPN,UNTRUST,tunnel.2,ethernet1/1,LogstashSys,2019/05/03 10:43:08,188036,1,57132,443,32403,443,0x40b000,tcp,alert,"c.urs.microsoft.com/",(9999),computer-and-internet-info,informational,client-to-server,356197,0x2000000000000000,172.16.0.0-172.31.255.255,Netherlands,0,,0,,,0,,,,,,,,0,0,0,0,0,,pamgmt,,,,,0,,0,,N/A,unknown,AppThreat-0-0,0x0,0,4294967295,'  
     
    **Phase 2: Completed decoding.  
    decoder: 'paloalto'  
    ```

    That means that the decoder is recognizing our log. Now we can start working with it.  


    We will create a new, more specific, decoder which will get parameters from the log and will be used to generate custom alerts, it will be:
    ```xml
    <!-- Custom decoder for PaloAlto Firewalls Events -->  
    <decoder name="paloalto-child">  
    <parent>paloalto</parent>  
    <regex offset="after_parent">^(\w+),(\w+),(\S+),(\d\d\d\d/\d\d/\d\d \d\d:\d\d:\d\d),(\d+.\d+.\d+.\d+),(\d+.\d+.\d+.\d+)</regex>  
    <order>pa.type, pa.ctype, extra_data, pa.gtime, pa.sip, pa.dip</order>  
    </decoder>  
    ```

    This decoder has "paloalto" decoder as parent, is called "paloalto-child" and uses a regex for extract information from the log. The "after_parent" value for offset mean that this regex will start just after the part of the log where the parent ends, in our case, it would be just after the Serial Number. We will be using dynamic fields for the order values (which are "variables" we define for store our information and will content the regex inside parenthesis in the order they appear).  


    Now, If we execute the testing program with a log with numbers intead of X.X.X.X we got:

    ```bash
    **Phase 1: Completed pre-decoding.  
    full event: '2019 May 03 08:43:08 pamgmt->10.10.10.10 May 3 10:43:08 pamgmt 1,2019/05/03 10:43:08,12345678987,THREAT,url,2049,2019/05/03 10:43:08,172.17.2.4,1.1.1.1,10.0.1.4,2.2.2.2,navegacion,domain\username,,ssl,vsys1,VPN,UNTRUST,tunnel.2,ethernet1/1,LogstashSys,2019/05/03 10:43:08,188036,1,57132,443,32403,443,0x40b000,tcp,alert,"c.urs.microsoft.com/",(9999),computer-and-internet-info,informational,client-to-server,356197,0x2000000000000000,172.16.0.0-172.31.255.255,Netherlands,0,,0,,,0,,,,,,,,0,0,0,0,0,,pamgmt,,,,,0,,0,,N/A,unknown,AppThreat-0-0,0x0,0,4294967295,'  
    timestamp: '2019 May 03 08:43:08'  
    hostname: 'centos7'  
    program_name: '(null)'  
    log: 'pamgmt->10.10.10.10 May 3 10:43:08 pamgmt 1,2019/05/03 10:43:08,12345678987,THREAT,url,2049,2019/05/03 10:43:08,172.17.2.4,1.1.1.1,10.0.1.4,2.2.2.2,navegacion,domain\username,,ssl,vsys1,VPN,UNTRUST,tunnel.2,ethernet1/1,LogstashSys,2019/05/03 10:43:08,188036,1,57132,443,32403,443,0x40b000,tcp,alert,"c.urs.microsoft.com/",(9999),computer-and-internet-info,informational,client-to-server,356197,0x2000000000000000,172.16.0.0-172.31.255.255,Netherlands,0,,0,,,0,,,,,,,,0,0,0,0,0,,pamgmt,,,,,0,,0,,N/A,unknown,AppThreat-0-0,0x0,0,4294967295,'  
     
    **Phase 2: Completed decoding.  
    decoder: 'paloalto'  
    pa.type: 'THREAT'  
    pa.ctype: 'url'  
    extra_data: '2049'  
    pa.gtime: '2019/05/03 10:43:08'  
    pa.sip: '172.17.2.4'  
    pa.dip: '1.1.1.1'  
    ```

    So... we're getting information from our log. Now we can start writing the rules that will be triggering alerts on the manager.  


    In `/var/ossec/etc/rules/local_rules.xml`  we create the following example rules:



    ```xml
    <group name="paloalto">  
    <rule id="123451" level="0">  
    <decoded_as>paloalto</decoded_as>  
    <description>Paloalto firewall event</description>  
    </rule>  
     
    <rule id="123452" level="3">  
    <if_sid>123451</if_sid>  
    <field name="pa.type">THREAT</field>  
    <description>Paloalto THREAT event.</description>  
    </rule>  
     
    <rule id="123453" level="5">  
    <if_sid>123451</if_sid>  
    <field name="pa.type">THREAT</field>  
    <field name="pa.ctype">url</field>  
    <description>Paloalto THREAT url event!! src IP: $(pa.sip), dest IP: $(pa.dip) </description>  
    </rule>  
    </group>  
    ```

    The first one trigger each time a log is detected as a PaloAlto one, even if no correct information is given to it (it will appear each time a log contains a valid PaloAlto Serial number between commas).

    Second one, with alert level=3, trigger if the type of the PaloAlto log is "THREAT" (as in the example) and inform of that situation.

    Third one, with alert level=5, trigger if the log type is "THREAD" and the content type is "url"... also in this example I've used the extracted IPs from the log to inform in the alert description of which source and destination IPs are found.  

    Now, test our configuration:

    ```bash
    **Phase 1: Completed pre-decoding.
           full event: '2019 May 03 08:43:08 pamgmt->10.10.10.10 May  3 10:43:08 pamgmt 1,2019/05/03 10:43:08,12345678987,THREAT,url,2049,2019/05/03 10:43:08,172.17.2.4,1.1.1.1,10.0.1.4,2.2.2.2,navegacion,domain\username,,ssl,vsys1,VPN,UNTRUST,tunnel.2,ethernet1/1,LogstashSys,2019/05/03 10:43:08,188036,1,57132,443,32403,443,0x40b000,tcp,alert,"c.urs.microsoft.com/",(9999),computer-and-internet-info,informational,client-to-server,356197,0x2000000000000000,172.16.0.0-172.31.255.255,Netherlands,0,,0,,,0,,,,,,,,0,0,0,0,0,,pamgmt,,,,,0,,0,,N/A,unknown,AppThreat-0-0,0x0,0,4294967295,'
           timestamp: '2019 May 03 08:43:08'
           hostname: 'centos7'
           program_name: '(null)'
           log: 'pamgmt->10.10.10.10 May  3 10:43:08 pamgmt 1,2019/05/03 10:43:08,12345678987,THREAT,url,2049,2019/05/03 10:43:08,172.17.2.4,1.1.1.1,10.0.1.4,2.2.2.2,navegacion,domain\username,,ssl,vsys1,VPN,UNTRUST,tunnel.2,ethernet1/1,LogstashSys,2019/05/03 10:43:08,188036,1,57132,443,32403,443,0x40b000,tcp,alert,"c.urs.microsoft.com/",(9999),computer-and-internet-info,informational,client-to-server,356197,0x2000000000000000,172.16.0.0-172.31.255.255,Netherlands,0,,0,,,0,,,,,,,,0,0,0,0,0,,pamgmt,,,,,0,,0,,N/A,unknown,AppThreat-0-0,0x0,0,4294967295,'

    **Phase 2: Completed decoding.
           decoder: 'paloalto'
           pa.type: 'THREAT'
           pa.ctype: 'url'
           extra_data: '2049'
           pa.gtime: '2019/05/03 10:43:08'
           pa.sip: '172.17.2.4'
           pa.dip: '1.1.1.1'

    **Phase 3: Completed filtering (rules).
           Rule id: '123453'
           Level: '5'
           Description: 'Paloalto THREAT url event!! src IP: 172.17.2.4, dest IP: 1.1.1.1 '
    **Alert to be generated.

    ```

    Everything is working correctly!

    The following steps would be:

    - Decide if you need more child decoders (if you want to analyze the logs in different ways depending, for example, on the type of log) or you can use just one.
    - Add more wildcard to the child decoder in order to get all the data you would need for your rules and store it in "order" variables.
    -  Decide what kind of alerts you want to generate and how important is each one. Then, write rules for all of them and set the desired level.

    I hope my answer will be helpful, if you need more help with that, do not hesitate to ask.

    Best regards
    -Francisco Navarro

    Hello Donetz! Let's do this.

    First of all, let me introduce you some documents from Wazuh documentation that will be helpful:

    - [1] Custom rules and decoders: https://documentation.wazuh.com/current/user-manual/ruleset/custom.html 

    - [2] Decoder syntax: https://documentation.wazuh.com/current/user-manual/ruleset/ruleset-xml-syntax/decoders.html 

    - [3] Regular expression syntax: https://documentation.wazuh.com/current/user-manual/ruleset/ruleset-xml-syntax/regex.html#os-regex-or-regex-syntax

     
    Now, we should start by writing a custom decoder on  `/var/ossec/etc/decoders/local_decoder.xml` as is indicated on [1]. 

     

    We should start by writing a simple decoder which recognizes the entire log string as a PaloAlto event by identifying the Serial Number of the log, which seems to be a combination of eleven numbers (so we will use `\d\d\d\d\d\d\d\d\d\d`  to identify it, see [3] ) 

    ```xml
    <!-- Custom decoder for PaloAlto Firewalls Events --> 
    <decoder name="paloalto"> 
    <prematch>,\d\d\d\d\d\d\d\d\d\d\d,</prematch> 
    </decoder> 
    ```

    What we are saying here to the decoder is "find eleven numbers between two commas and anything else". 


    Now, we could save the decoder and test it by executing `/var/ossec/bin/ossec-logtest`  as specified in [1]. I will introduce the log you give us but changing the XXXXXXXXXXX by numbers (12345678987) in order to allow the decoder to recognize the log as a PaloAlto one.`

    The output of the command is:

    ```bash
    **Phase 1: Completed pre-decoding. 
    full event: '2019 May 03 08:43:08 pamgmt->10.10.10.10 May 3 10:43:08 pamgmt 1,2019/05/03 10:43:08,12345678987,THREAT,url,2049,2019/05/03 10:43:08,172.17.2.4,X.X.X.X,10.0.1.4,X.X.X.X,navegacion,domain\username,,ssl,vsys1,VPN,UNTRUST,tunnel.2,ethernet1/1,LogstashSys,2019/05/03 10:43:08,188036,1,57132,443,32403,443,0x40b000,tcp,alert,"c.urs.microsoft.com/",(9999),computer-and-internet-info,informational,client-to-server,356197,0x2000000000000000,172.16.0.0-172.31.255.255,Netherlands,0,,0,,,0,,,,,,,,0,0,0,0,0,,pamgmt,,,,,0,,0,,N/A,unknown,AppThreat-0-0,0x0,0,4294967295,' 
    timestamp: '2019 May 03 08:43:08' 
    hostname: 'centos7' 
    program_name: '(null)' 
    log: 'pamgmt->10.10.10.10 May 3 10:43:08 pamgmt 1,2019/05/03 10:43:08,12345678987,THREAT,url,2049,2019/05/03 10:43:08,172.17.2.4,X.X.X.X,10.0.1.4,X.X.X.X,navegacion,domain\username,,ssl,vsys1,VPN,UNTRUST,tunnel.2,ethernet1/1,LogstashSys,2019/05/03 10:43:08,188036,1,57132,443,32403,443,0x40b000,tcp,alert,"c.urs.microsoft.com/",(9999),computer-and-internet-info,informational,client-to-server,356197,0x2000000000000000,172.16.0.0-172.31.255.255,Netherlands,0,,0,,,0,,,,,,,,0,0,0,0,0,,pamgmt,,,,,0,,0,,N/A,unknown,AppThreat-0-0,0x0,0,4294967295,' 
     
    **Phase 2: Completed decoding. 
    decoder: 'paloalto' 
    ```

    That means that the decoder is recognizing our log. Now we can start working with it. 


    We will create a new, more specific, decoder which will get parameters from the log and will be used to generate custom alerts, it will be:
    ```xml
    <!-- Custom decoder for PaloAlto Firewalls Events --> 
    <decoder name="paloalto-child"> 
    <parent>paloalto</parent> 
    <regex offset="after_parent">^(\w+),(\w+),(\S+),(\d\d\d\d/\d\d/\d\d \d\d:\d\d:\d\d),(\d+.\d+.\d+.\d+),(\d+.\d+.\d+.\d+)</regex> 
    <order>pa.type, pa.ctype, extra_data, pa.gtime, pa.sip, pa.dip</order> 
    </decoder> 
    ```

    This decoder has "paloalto" decoder as parent, is called "paloalto-child" and uses a regex for extract information from the log. The "after_parent" value for offset mean that this regex will start just after the part of the log where the parent ends, in our case, it would be just after the Serial Number. We will be using dynamic fields for the order values (which are "variables" we define for store our information and will content the regex inside parenthesis in the order they appear). 


    Now, If we execute the testing program with a log with numbers intead of X.X.X.X we got:

    ```bash
    **Phase 1: Completed pre-decoding. 
    full event: '2019 May 03 08:43:08 pamgmt->10.10.10.10 May 3 10:43:08 pamgmt 1,2019/05/03 10:43:08,12345678987,THREAT,url,2049,2019/05/03 10:43:08,172.17.2.4,1.1.1.1,10.0.1.4,2.2.2.2,navegacion,domain\username,,ssl,vsys1,VPN,UNTRUST,tunnel.2,ethernet1/1,LogstashSys,2019/05/03 10:43:08,188036,1,57132,443,32403,443,0x40b000,tcp,alert,"c.urs.microsoft.com/",(9999),computer-and-internet-info,informational,client-to-server,356197,0x2000000000000000,172.16.0.0-172.31.255.255,Netherlands,0,,0,,,0,,,,,,,,0,0,0,0,0,,pamgmt,,,,,0,,0,,N/A,unknown,AppThreat-0-0,0x0,0,4294967295,' 
    timestamp: '2019 May 03 08:43:08' 
    hostname: 'centos7' 
    program_name: '(null)' 
    log: 'pamgmt->10.10.10.10 May 3 10:43:08 pamgmt 1,2019/05/03 10:43:08,12345678987,THREAT,url,2049,2019/05/03 10:43:08,172.17.2.4,1.1.1.1,10.0.1.4,2.2.2.2,navegacion,domain\username,,ssl,vsys1,VPN,UNTRUST,tunnel.2,ethernet1/1,LogstashSys,2019/05/03 10:43:08,188036,1,57132,443,32403,443,0x40b000,tcp,alert,"c.urs.microsoft.com/",(9999),computer-and-internet-info,informational,client-to-server,356197,0x2000000000000000,172.16.0.0-172.31.255.255,Netherlands,0,,0,,,0,,,,,,,,0,0,0,0,0,,pamgmt,,,,,0,,0,,N/A,unknown,AppThreat-0-0,0x0,0,4294967295,' 
     
    **Phase 2: Completed decoding. 
    decoder: 'paloalto' 
    pa.type: 'THREAT' 
    pa.ctype: 'url' 
    extra_data: '2049' 
    pa.gtime: '2019/05/03 10:43:08' 
    pa.sip: '172.17.2.4' 
    pa.dip: '1.1.1.1' 
    ```

    So... we're getting information from our log. Now we can start writing the rules that will be triggering alerts on the manager. 


    In `/var/ossec/etc/rules/local_rules.xml`  we create the following example rules:



    ```xml
    <group name="paloalto"> 
    <rule id="123451" level="0"> 
    <decoded_as>paloalto</decoded_as> 
    <description>Paloalto firewall event</description> 
    </rule> 
     
    <rule id="123452" level="3"> 
    <if_sid>123451</if_sid> 
    <field name="pa.type">THREAT</field> 
    <description>Paloalto THREAT event.</description> 
    </rule> 
     
    <rule id="123453" level="5"> 
    <if_sid>123451</if_sid> 
    <field name="pa.type">THREAT</field> 
    <field name="pa.ctype">url</field> 
    <description>Paloalto THREAT url event!! src IP: $(pa.sip), dest IP: $(pa.dip) </description> 
    </rule> 
    </group> 
    ```

    The first one trigger each time a log is detected as a PaloAlto one, even if no correct information is given to it (it will appear each time a log contains a valid PaloAlto Serial number between commas).

    Second one, with alert level=3, trigger if the type of the PaloAlto log is "THREAT" (as in the example) and inform of that situation.

    Third one, with alert level=5, trigger if the log type is "THREAD" and the content type is "url"... also in this example I've used the extracted IPs from the log to inform in the alert description of which source and destination IPs are found. 

    Now, test our configuration:

    ```bash
    **Phase 1: Completed pre-decoding.
           full event: '2019 May 03 08:43:08 pamgmt->10.10.10.10 May  3 10:43:08 pamgmt 1,2019/05/03 10:43:08,12345678987,THREAT,url,2049,2019/05/03 10:43:08,172.17.2.4,1.1.1.1,10.0.1.4,2.2.2.2,navegacion,domain\username,,ssl,vsys1,VPN,UNTRUST,tunnel.2,ethernet1/1,LogstashSys,2019/05/03 10:43:08,188036,1,57132,443,32403,443,0x40b000,tcp,alert,"c.urs.microsoft.com/",(9999),computer-and-internet-info,informational,client-to-server,356197,0x2000000000000000,172.16.0.0-172.31.255.255,Netherlands,0,,0,,,0,,,,,,,,0,0,0,0,0,,pamgmt,,,,,0,,0,,N/A,unknown,AppThreat-0-0,0x0,0,4294967295,'
           timestamp: '2019 May 03 08:43:08'
           hostname: 'centos7'
           program_name: '(null)'
           log: 'pamgmt->10.10.10.10 May  3 10:43:08 pamgmt 1,2019/05/03 10:43:08,12345678987,THREAT,url,2049,2019/05/03 10:43:08,172.17.2.4,1.1.1.1,10.0.1.4,2.2.2.2,navegacion,domain\username,,ssl,vsys1,VPN,UNTRUST,tunnel.2,ethernet1/1,LogstashSys,2019/05/03 10:43:08,188036,1,57132,443,32403,443,0x40b000,tcp,alert,"c.urs.microsoft.com/",(9999),computer-and-internet-info,informational,client-to-server,356197,0x2000000000000000,172.16.0.0-172.31.255.255,Netherlands,0,,0,,,0,,,,,,,,0,0,0,0,0,,pamgmt,,,,,0,,0,,N/A,unknown,AppThreat-0-0,0x0,0,4294967295,'

    **Phase 2: Completed decoding.
           decoder: 'paloalto'
           pa.type: 'THREAT'
           pa.ctype: 'url'
           extra_data: '2049'
           pa.gtime: '2019/05/03 10:43:08'
           pa.sip: '172.17.2.4'
           pa.dip: '1.1.1.1'

    **Phase 3: Completed filtering (rules).
           Rule id: '123453'
           Level: '5'
           Description: 'Paloalto THREAT url event!! src IP: 172.17.2.4, dest IP: 1.1.1.1 '
    **Alert to be generated.

    ```

    Everything is working correctly!

    The following steps would be:

    - Decide if you need more child decoders (if you want to analyze the logs in different ways depending, for example, on the type of log) or you can use just one.
    - Add more wildcard to the child decoder in order to get all the data you would need for your rules and store it in "order" variables.
    -  Decide what kind of alerts you want to generate and how important is each one. Then, write rules for all of them and set the desired level.

    I hope my answer will be helpful, if you need more help with that, do not hesitate to ask.

    Best regards
    -Francisco Navarro

    donetz errasti

    unread,
    May 8, 2019, 6:29:38 AM5/8/19
    to Wazuh mailing list
    Hi Francisco, 

    It's been incredibly helpful!!! Thank you so much. Now I'm expanding the decoder in order to get all data I can. 
    On the other hand, I've tried your decoder and rules and the test is okay, but when I get the events via syslog, the rules are not triggered because I can't see the results in Alerts.log
    file. Do you know what could be the issue??

    Thanks again, I really appreciate your help. 

    Donetz

    Francisco Navarro

    unread,
    May 9, 2019, 5:01:41 AM5/9/19
    to Wazuh mailing list

    I am glad to heard that my answer was helpful to you.

    Regarding your question, I have been testing the rules, and they generated alerts as would be expected. Nevertheless, I can come up with that maybe you are testing with the first rule example, which is from level 0. As you can see here in the docs, there is an alert level threshold which, by default, allow only events from level 1 or bigger to generate alerts. That means the first example won’t generate an alert even when it is recognized in the test.

    Maybe that is not what you mean but… just for be sure, what alert are you trying to generate? Please try to write to a log file the last message I wrote:

    '2019 May 03 08:43:08 pamgmt->10.10.10.10 May 3 10:43:08 pamgmt 1,2019/05/03 10:43:08,12345678987,THREAT,url,2049,2019/05/03 10:43:08,172.17.2.4,1.1.1.1,10.0.1.4,2.2.2.2,navegacion,domain\username,,ssl,vsys1,VPN,UNTRUST,tunnel.2,ethernet1/1,LogstashSys,2019/05/03 10:43:08,188036,1,57132,443,32403,443,0x40b000,tcp,alert,"c.urs.microsoft.com/",(9999),computer-and-internet-info,informational,client-to-server,356197,0x2000000000000000,172.16.0.0-172.31.255.255,Netherlands,0,,0,,,0,,,,,,,,0,0,0,0,0,,pamgmt,,,,,0,,0,,N/A,unknown,AppThreat-0-0,0x0,0,4294967295,'

    In the test, it should return this in the phase 3:

    **Phase 3: Completed filtering (rules).
           Rule id: '123453'
           Level: '5'
           Description: 'Paloalto THREAT url event!! src IP: 172.17.2.4, dest IP: 1.1.1.1 '
    **Alert to be generated.
    

    And it should generate an alert on the manager as the event generated is level 5.

    Check this and don’t hesitate to ask again if that is not the problem.

    Best regards.
    Francisco Navarro.

    Message has been deleted

    donetz errasti

    unread,
    Jun 10, 2019, 4:37:01 AM6/10/19
    to Wazuh mailing list
    hi Francisco, 

    I'm getting some errors to match the first part of the log of Palo Alto. I mean, I would like to detect the Palo Alto log matching "pamgmt" string and then distinguish between THREAT logs and SYSTEM log for example. It is what I've tried:


    Thanks for your help, 

    Donetz

    Francisco Navarro

    unread,
    Jun 11, 2019, 7:45:10 AM6/11/19
    to Wazuh mailing list

    Hello, Donetz

    Let’s analyze the work you’ve done

    
    <prematch>Jun 10 09:15:20 pamgmt 1,2019/06/10 09:15:20,007057000058155,</prematch>              
    
    
    <type>syslog</type>              
    
    
    </decoder>
    
    <decoder name="paloalto-firewall-v9">
    

    First of all, that is not a good way to implement a decoder, even for testing purpose I would recommend you to use regular expressions… Nevertheless, I assume you are just testing that everything works correctly.

    If you add only this first decoder to you local decoders and test the log you will receive this output:

    [root@centos7 ossec]# /var/ossec/bin/ossec-logtest 
    2019/06/11 10:11:15 ossec-testrule: INFO: Started (pid: 24702).
    ossec-testrule: Type one log per line.
    
    Jun 10 09:15:20 pamgmt 1,2019/06/10 09:15:20,007057000058155,THREAT,url,2049,2019/06/10 09:15:20,1.1.1.1,1.2.2.2,3.3.3.3,4.4.4.4,navegacion,domain\user,,ssl,vsys1,VPN,UNTRUST,tunnel.2,ethernet1/1,LogstashSys,2019/06/10 09:15:20,63472,1,56605,443,52860,443,0x40                        $informational,client-to-server,700830,0x2000000000000000,,172.16.0.0-172.31.255.255,United States,0,,0,,,0,,,,,,,,0,0,0,0,0,,pamgmt,,,,,0,,0,,N/A,unknown,AppThreat-0-0,0x,0,4294967295
    
    **Phase 1: Completed pre-decoding.
           full event: 'Jun 10 09:15:20 pamgmt 1,2019/06/10 09:15:20,007057000058155,THREAT,url,2049,2019/06/10 09:15:20,1.1.1.1,1.2.2.2,3.3.3.3,4.4.4.4,navegacion,domain\user,,ssl,vsys1,VPN,UNTRUST,tunnel.2,ethernet1/1,LogstashSys,2019/06/10 09:15:20,63472,1,56605,443,52860,443,0x40                        $informational,client-to-server,700830,0x2000000000000000,,172.16.0.0-172.31.255.255,United States,0,,0,,,0,,,,,,,,0,0,0,0,0,,pamgmt,,,,,0,,0,,N/A,unknown,AppThreat-0-0,0x,0,4294967295'
           timestamp: 'Jun 10 09:15:20'
           hostname: 'pamgmt'
           program_name: '(null)'
           log: '1,2019/06/10 09:15:20,007057000058155,THREAT,url,2049,2019/06/10 09:15:20,1.1.1.1,1.2.2.2,3.3.3.3,4.4.4.4,navegacion,domain\user,,ssl,vsys1,VPN,UNTRUST,tunnel.2,ethernet1/1,LogstashSys,2019/06/10 09:15:20,63472,1,56605,443,52860,443,0x40                        $informational,client-to-server,700830,0x2000000000000000,,172.16.0.0-172.31.255.255,United States,0,,0,,,0,,,,,,,,0,0,0,0,0,,pamgmt,,,,,0,,0,,N/A,unknown,AppThreat-0-0,0x,0,4294967295'
    
    **Phase 2: Completed decoding.
           No decoder matched.
    

    This is an interesting issue in how to do analysisd works. If you specify the ‘syslog’ type for the decoder it will look for a syslog header in the given log and, if it is found, will crop it from the log. see syslog page on Wikipedia for more info about syslog header

    In your case, Jun 10 09:15:20 pamgmt 1 is not actually a syslog header but a parameter from your log and analysisd is interpreting it as the header (see it returns you the value of hostname as pamgmt: hostname: 'pamgmt').

    The problem is that the actual log you’re analyzing is

    log: '1,2019/06/10 09:15:20,007057000058155,THREAT,url,2049,2019/06/10 09:15:20,1.1.1.1,1.2.2.2,3.3.3.3,4.4.4.4,navegacion,domain\user,,ssl,vsys1,VPN,UNTRUST,tunnel.2,ethernet1/1,LogstashSys,2019/06/10 09:15:20,63472,1,56605,443,52860,443,0x40 $informational,client-to-server,700830,0x2000000000000000,,172.16.0.0-172.31.255.255,United States,0,,0,,,0,,,,,,,,0,0,0,0,0,,pamgmt,,,,,0,,0,,N/A,unknown,AppThreat-0-0,0x,0,4294967295'

    So your prematch won’t be matched.

    My recommendation would be to use the following decoder:

    <decoder name="paloalto-firewall-v9">
            <prematch>\d+,\d\d\d\d/\d\d/\d\d \d\d:\d\d:\d\d,\d+,</prematch>
            <type>syslog</type>
    </decoder>
    

    One important detail is that I’ve write \d+ for the Serial Number. In my last post, I was using because\d\d\d\d\d\d\d\d\d\d\d in your first post you specified that the format of the paloalto log serial numbers was a 12 digits number, but in your last post, you have written a 15 digits serial numbers. Is that correct? If the serial number can have a variable length, then you should use but\d+, if you know the exact number of digits I would suggest to use something like.\d\d\d\d\d\d\d\d\d\d\d

    Now, with this simple change, you could continue developing your decoders as you prefer but I will suggest you not to write a different decoder for something you could do with just one of them.

    In this case, if you want to differentiate between THREAT and SYSTEM logs, I would write a second decoder with that code (similar to the one I propose to you in my last post)

    <decoder name="paloalto-child">
    <parent>paloalto-firewall-v9</parent>
    
    <regex offset="after_parent">^(\w+),(\w+),(\S+),(\d\d\d\d/\d\d/\d\d \d\d:\d\d:\d\d),(\d+.\d+.\d+.\d+),(\d+.\d+.\d+.\d+)</regex>
    <order>pa.type, pa.ctype, extra_data, pa.gtime, pa.sip, pa.dip</order>
    </decoder>
    

    It will allow you to access the values:

           pa.type: 'THREAT'
           pa.ctype: 'url'
           extra_data: '2049'
           pa.gtime: '2019/06/10 09:15:20'
           pa.sip: '1.1.1.1'
           pa.dip: '1.2.2.2'
    

    And, modifying the rules we use in our last conversation:

    <group name="paloalto">
    <rule id="123451" level="0">  
    
    <decoded_as>paloalto-firewall-v9</decoded_as>
    
    <description>Paloalto firewall event</description>  
    </rule>  
    
    <rule id="123452" level="3">  
    <if_sid>123451</if_sid>  
    <field name="pa.type">THREAT</field>  
    <description>Paloalto THREAT event.</description>  
    </rule>  
    
    
    <rule id="123453" level="3">
    <if_sid>123451</if_sid>
    <field name="pa.type">SYSTEM</field>
    <description>Paloalto SYSTEM event.</description> 
    </rule>
    
    </group>
    

    We could differentiate between threat & system logs without having various decoders. You must think that a decoder serves to obtain a certain type of log and a rule to distinguish different events within those logs that are analyzed. It doesn’t make sense to have several decoders for logs that have the same format. Instead, you should use different rules.

    I hope this could help you to continue developing your local decoders & rules, I hope to hear from you with the results obtained and if you have achieved what you intended. If you have any questions don’t hesitate to ask again.

    Best regards.

    F. N.

    donetz errasti

    unread,
    Jun 11, 2019, 11:31:58 AM6/11/19
    to Wazuh mailing list
    Hi Francisco, 

    Thanks for your time. I didn't know how exactly analysisd works, so I hope after this explanation and the blog entry (Sibling decoders: flexible extraction of information) my acknowledge about decoders will be better. 

    Like you said, I tried with all literally string to check if it had matched, but it hadn't. 
    I've developing different decoders because Palo Alto syslog format is not the same between its logs. I mean, THREAT log hasn't the same fields comparing with SYSTEM logs. So, Would it be a good way to keep developing different decoders?

    SYSTEM FORMAT: 
    FUTURE_USE, Receive Time, Serial Number, Type, Subtype, FUTURE_USE, Generated Time, Virtual System, Event ID, Object, FUTURE_USE, FUTURE_USE, Module, Severity, Description, Sequence Number, Action Flags, Device Group Hierarchy Level 1, Device Group Hierarchy Level 2, Device Group Hierarchy Level 3, Device Group Hierarchy Level 4, Virtual System Name, Device Name
    THREAT format: 
    FUTURE_USE, Receive Time, Serial Number, Type, Subtype, FUTURE_USE, Generated Time, Source IP, Destination IP, NAT Source IP, NAT Destination IP, Rule Name, Source User, Destination User, Application, Virtual System, Source Zone, Destination Zone, Ingress Interface, Egress Interface, Log Forwarding Profile, FUTURE_USE, Session ID, Repeat Count, Source Port, Destination Port, NAT Source Port, NAT Destination Port, Flags, Protocol, Action, Bytes, Bytes Sent, Bytes Received, Packets, Start Time, Elapsed Time, Category, FUTURE_USE, Sequence Number, Action Flags, Source Location, Destination Location, FUTURE_USE, Packets Sent, Packets Received, Session End Reason, Device Group Hierarchy Level 1, Device Group Hierarchy Level 2, Device Group Hierarchy Level 3, Device Group Hierarchy Level 4, Virtual System Name, Device Name, Action Source

    Finally, I've tried like you've taught me and it works. 

    Thanks and let me know about decoders format. 

    Donetz

    Francisco Navarro

    unread,
    Jun 13, 2019, 6:43:01 AM6/13/19
    to Wazuh mailing list

    Hello Donetz,

    I’m really glad it was helpful. You’ve done well reading the blog article you mentioned, is really illustrative.

    About the fact of keep developing different decoders: I have been checking the Syslog field descriptions of paloalto logs and I have detected a common pattern that repeats in all the logs types: Traffic, Threat, HIP, Config, and System.

    That pattern is: FUTURE_USE,Receive Time,Serial Number,Type,Subtype,FUTURE_USE,Generated Time

    Nevertheless, as we know there are different log format depending on the type value, I recommend writing different decoders for each king of the log.

    My proposal:

    Have the main decoder which will detect paloalto decoders, this decoder could use all these fields I describe but, for simplicity and easier development I would suggest using only the fields FUTURE_USE, Receive Time and Serial Number (remember that the first field will be cropped by analysisd, and also that I’m not sure about the length of the Serial Number, I haven’t found it in Palo Alto docs). This could be the decoder:

    <decoder name="paloalto-firewall">
     <prematch>\d+,(\d\d\d\d/\d\d/\d\d \d\d:\d\d:\d\d),(\d+),</prematch>
    </decoder>
    

    Now you can write special decoders for each type (as they will have different fields)

    For example, for threat events:

    
    <decoder name="paloalto-threat">
     <parent>paloalto-firewall</parent>
     <prematch offset="after_parent">^THREAT,</prematch>
     <regex offset="after_parent">^THREAT,(\w+),\.*,\.*,(\d+.\d+.\d+.\d+),(\d+.\d+.\d+.\d+),(\d+.\d+.\d+.\d+),(\d+.\d+.\d+.\d+),(\w+),(\w*\\*\w*),(\w*\\*\w*)</regex>
     <order>subtype,srcip,dstip,natsrc,natdts,rulename,srcuser,dstuser</order>
    </decoder>
    

    An example of completed decoding:

    **Phase 2: Completed decoding.
           decoder: 'paloalto-firewall'
           subtype: 'url'
           srcip: '1.1.1.1'
           dstip: '2.2.2.2'
           natsrc: '3.3.3.3'
           natdts: '4.4.4.4'
           rulename: 'navegacion'
           srcuser: 'domain\user'
           dstuser: ''
    

    I encourage you to finish adding the missing fields (those that you consider necessary) and also to write yourself a decoder for the events of type “SYSTEM”.

    Also, I would like to ask you to share with us those decoders you write. I would like to open a pull request to adding support for Palo Alto logs on Wazuh and I think we could work on it together :) let me know if you are interested in that or at least if you plan to write those decoders soon so I can use them for that purpose.

    One last thing, I don’t like the way in which analysisd treat the first field, cropping it and confusing it with the syslog header. The only way to fix it would be to make Palo alto log generator to write the logs with syslog format, that means: ensure each log line is written with the syslog header: log-date hostname program-name something like Jun 10 09:15:20 ubuntu-example paloalto-firewall. If you can archive that, we could modify the first log and replace the first \d+ by \.*, or even (\.*), if you were interested in getting such information in a new order.

    If you need anything else, please don’t hesitate to ask.

    Best regards, F.N

    Reply all
    Reply to author
    Forward
    0 new messages