Creating a decoder for pfsense syslogs

[JP Dubé]

Hello,

I've started forwarding my pfsense syslog to Wazuh, but I'm struggling to decode them in Wazuh (default decoders are not working).

My pfsense is set to  send log in syslog format (RFC 5424, with RFC 3339 microsecond-precision timestamp). I have tested turning the logall option for a bit, I can positively confirm that the wazuh manager is receiving the syslog.

Here is a few lines that i found in my /var/ossec/logs/archives/archives.log after turning logall to on:

2023 Jul 03 04:53:45 wazuh->192.168.100.1 1 2023-07-03T04:53:45.034138+00:00 fw1.network.arpa filterlog 73381 - - 230,,,1630278040,lagg1.60,match,block,in,4,0x0,,255,1974,0,none,17,udp,125,192.168.60.53,224.0.0.251,5353,5353,105
2023 Jul 03 04:53:45 wazuh->192.168.100.1 1 2023-07-03T04:53:45.034173+00:00 fw1.network.arpa filterlog 73381 - - 231,,,1630278040,lagg1.60,match,block,in,6,0x00,0xe0000,255,UDP,17,105,fe80::14c4:d75e:c885:f081,ff02::fb,5353,5353,105
2023 Jul 03 04:53:45 wazuh->192.168.100.1 1 2023-07-03T04:53:45.034158+00:00 fw1.network.arpa filterlog 73381 - - 231,,,1630278040,lagg1.60,match,block,in,6,0x00,0xe0000,255,UDP,17,105,fe80::14c4:d75e:c885:f081,ff02::fb,5353,5353,105
2023 Jul 03 04:53:46 wazuh->192.168.100.1 1 2023-07-03T04:53:46.035108+00:00 fw1.network.arpa filterlog 73381 - - 327,,,1657929018,lagg1.120,match,block,in,4,0x0,,128,48607,0,DF,6,tcp,52,192.168.120.52,192.168.60.170,26384,9100,0,S,2886259890,,64240,,mss;nop;wscale;nop;nop;sackOK

In this snippet, we can see that the ordering of certain fields is different depending on ipv4 is used vs ipv6. I was thinking of approaching this by creating 2 decoder, one for ipv4, and one for ipv6.

I started working on the ipv4 decoder, but I cannot get it to work as intended. This is what my decoder is looking like so far:


<decoder name="pf-fw">
    <prematch>filterlog</prematch>
</decoder>

<decoder name="pf-fw-ipv4">
  <parent>pf-fw</parent>
  <regex>\S*?,\S*?,\S*?,(\S*?),(\S*?),\S*?,(\S*?),\S*?,4,\S*?,\S*?,\S*?,\S*?,\S*?,\S*?,\S*?,(\S*?),\S*?,(\S*?),(\S*?),(\S*?),(\S*?),</regex>
  <order>ruleid,interface,action,protocol,srcip,dstip,srcport,dstport</order>
</decoder>

When testing the rule, it doesn't decode the fields I want to. Here's the output of the test:

**Messages: INFO: (7202): Session initialized with token 'XXXX'

**Phase 1: Completed pre-decoding. full event: '2023 Jul 03 04:53:45 wazuh->192.168.100.1 1 2023-07-03T04:53:45.034138+00:00 fw1.network.arpa filterlog 73381 - - 230,,,1630278040,lagg1.60,match,block,in,4,0x0,,255,1974,0,none,17,udp,125,192.168.60.53,224.0.0.251,5353,5353,105' timestamp: '2023 Jul 03 04:53:45'

**Phase 2: Completed decoding. name: 'pf-fw'

Can someone help me understand what I'm doing wrong? I am still fairly new to Wazuh, any help would be appreciated.

Thank you,

/JP

[Nicolas Zapata]

Hi JP,

thanks for using wazuh!

Although Wazuh includes rules for PFsense, it is very possible that your events are not matching our decoders or rules. Thus, you might need to create some customer decoders and rules for your events.

You can check samples of supported PFsense events in PFsense decoders file (/var/ossec/ruleset/decoders/0455-pfsense_decoders.xml): https://github.com/wazuh/wazuh/blob/master/ruleset/decoders/0455-pfsense_decoders.xml

To address this issue, you have started working on creating two decoders, one for IPv4 and one for IPv6. However, you are encountering difficulties in getting the IPv4 decoder to work as intended.

From the decoder configuration you shared, it appears that you have defined the necessary regex pattern to extract the desired fields. However, when testing the rule, the fields are not being decoded as expected.

To troubleshoot this issue, we recommend the following steps:

- Verify that the decoder is properly loaded by checking the Wazuh logs for any error messages related to the decoder configuration.
- Double-check the regex pattern to ensure it accurately matches the log format and captures the desired fields. You can use online regex testers to validate the pattern against sample log messages.
- Check if there are any conflicting decoders or rules that may be interfering with the decoding process.
- Test the decoder with different log messages to see if the issue is specific to certain messages or a general problem.

Regards!

[JP Dubé]

Hello,

Thank you for your reply, I'm still experiencing issues. See my reply to some of your suggestion.

- Verify that the decoder is properly loaded by checking the Wazuh logs for any error messages related to the decoder configuration.

I've checked /var/ossec/logs/osssec.log and nothing suggests that there is any issues. Is there any other logs that I should check?

- Double-check the regex pattern to ensure it accurately matches the log format and captures the desired fields. You can use online regex testers to validate the pattern against sample log messages.

I've checked my regex using regexr.com and based on this, I believe that it should be working. See screenshot attached. Is there something I'm doing wrong with the XML?

- Check if there are any conflicting decoders or rules that may be interfering with the decoding process.

How do check if it's caused by a conflict? I still have the rules that are built-in with Wazuh, including the pfsense decoders that are not working. Could this be the root cause of this issue?

- Test the decoder with different log messages to see if the issue is specific to certain messages or a general problem

I've tried other logs samples, I'm experiencing the same issue with all of them.

One point to note, I've put my decoder in the local_decoder.xml file. Since this is the 1st decoder that I'm creating, there isn't any other decoders in there.

[JP Dubé]

Hello,

Apologies for bumping this thread, I am still in need of assistance with this.

[Nicolas Zapata]

Hi JP, sorry for the delay!

-  I've checked /var/ossec/logs/osssec.log and nothing suggests that there is any issues. Is there any other logs that I should check?
No, if there have been any errors they will appear in that file. additionally debugging can be enabled but in this case it is not necessary.

-  I've checked my regex using regexr.com and based on this, I believe that it should be working. See screenshot attached. Is there something I'm doing wrong with the XML?

The configuration looks good! However, I will test it in a local environment to see if anything is missing.

-  How do check if it's caused by a conflict? I still have the rules that are built-in with Wazuh, including the pfsense decoders that are not working. Could this be the root cause of this issue?

This was in case you had already created other decoder/rules, if it is the first one you create there should be no problem.

I will run some tests in my test environment and get back to you!

Regards 

[JP Dubé]

Hi,

I'll be waiting to hear back. FYI, In case this is relevant, I am running v 4.4.4 and I have very little customization so far (only suppression of some noisy alerts, none of them having to do with things related to pfsense). I haven't written any rules for pfsense, I am trying to get the decoder working first.

Thank you for your assistance.

[Nicolas Zapata]

Hi again JP!

I have been testing this decoder and it seems to work with the provided alert. You can modify it to your liking to add or remove fields.

<decoder name="pf-fw">
    <prematch>filterlog</prematch>
</decoder>

<decoder name="pf-fw-ipv4">
    <parent>pf-fw</parent>

    <regex offset="after_parent" type="pcre2"> - - [^,]*,[^,]*,[^,]*,([^,]*),[^,]*,[^,]*,([^,]*),[^,]*,[^,]*,[^,]*,[^,]*,[^,]*,[^,]*,[^,]*,[^,]*,[^,]*,([^,]*),[^,]*,([^,]*),([^,]*),(\d*),(\d*),[^,]*</regex>
    <order>id,action,protocol,srcip,dstip,srcport,dstport</order>
</decoder>

wazuh-logstest:

Starting wazuh-logtest v4.4.3
Type one log per line

2023 Jul 03 04:53:45 wazuh->192.168.100.1 1 2023-07-03T04:53:45.034138+00:00 fw1.network.arpa filterlog 73381 - - 230,,,1630278040,lagg1.60,match,block,in,4,0x0,,255,1974,0,none,17,udp,125,192.168.60.53,224.0.0.251,5353,5353,105

**Phase 1: Completed pre-decoding.
        full event: '2023 Jul 03 04:53:45 wazuh->192.168.100.1 1 2023-07-03T04:53:45.034138+00:00 fw1.network.arpa filterlog 73381 - - 230,,,1630278040,lagg1.60,match,block,in,4,0x0,,255,1974,0,none,17,udp,125,192.168.60.53,224.0.0.251,5353,5353,105'
        timestamp: '2023 Jul 03 04:53:45'

**Phase 2: Completed decoding.
        name: 'pf-fw'

        action: 'block'
        dstip: '224.0.0.251'
        dstport: '5353'
        id: '1630278040'
        protocol: 'udp'
        srcip: '192.168.60.53'
        srcport: '5353'

I hope this help!

Regards

[wal Bz]

i want to parse dnsbl.log in wazuh so i create a custom decoder but it seems not working

# /var/ossec/bin/wazuh-logtest

Starting wazuh-logtest v4.5.2

Type one log per line

Sep 29 09:39:53 pfSense - DNSBL-Full,Sep 29 09:39:53,cocaine.com,192.168.210.101,-|GET / HTTP/1.1|Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:109.0) Gecko/20100101 Firefox/115.0,DNSBL,DNSBL_test,cocaine.com,test_custom,+

**Phase 1: Completed pre-decoding.
full event: 'Sep 29 09:39:53 pfSense - DNSBL-Full,Sep 29 09:39:53,cocaine.com,192.168.210.101,-|GET / HTTP/1.1|Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:109.0) Gecko/20100101 Firefox/115.0,DNSBL,DNSBL_test,cocaine.com,test_custom,+'
timestamp: 'Sep 29 09:39:53'
hostname: 'pfSense'

**Phase 2: Completed decoding.
No decoder matched.

i want to extarct the principal fields like timestamp, domain(here is cocaine.com which i considered malicious), src_ip...
anyone can help me !
thank you.

[Hammad Ansari]

hi all 
I have tried too many Decoder for my Opnsense log I am blocking facebook.com 
here is my full log  Aug 30 23:25:20 OPNsense.internal filterlog[92551] 82,,,e9ab5533f94b9fbbfae0793d5c8048c5,em1,match,block,in,4,0x0,,64,65015,0,DF,6,tcp,60,192.168.10.135,157.240.227.35,50288,443,0,S,3746117633,,64240,,mss;sackOK;TS;nop;wscale
can anyone help to give me Decoder and rule on behave of this full log 
it would be highly appreciated