suricata pfsense -> syslog-ng -> wazuh -- events not showing up in Kibana and rule issues

[Geoff Nordli]

Hi.

I am trying to get my suricata logs into wazuh from a pfsense
firewall.   I configured syslog-ng and the logs are are showing up in
the archive.json file.

I am running Wazuh 4.2.6 with opendistroforelasticsearch 1.13.2

Some log entries are not getting recognized/decoded properly.

For example this is one entry in the archive.json file.

{"timestamp":"2022-03-30T19:28:24.337-0700","rule":{"level":2,"description":"Unknown
problem somewhere in the
system.","id":"1002","firedtimes":1455,"mail":false,"groups":["syslog","errors"],"gpg13":["4.3"]},"agent":{"id":"000","name":"monitor2"},"manager":{"name":"monitor2"},"id":"1648693704.152921423","full_log":"Mar
30 19:27:46 fw1 suricata:
{\"timestamp\":\"2022-03-30T19:27:46.385986-0700\",\"flow_id\":158011075148661,\"in_iface\":\"ix2\",\"event_type\":\"sip\",\"src_ip\":\"172.16.1.XX\",\"src_port\":5060,\"dest_ip\":\"10.0.8.10\",\"dest_port\":64112,\"proto\":\"UDP\",\"sip\":{\"version\":\"SIP/2.0\",\"code\":\"401\",\"reason\":\"Unauthorized\",\"response_line\":\"SIP/2.0
401
Unauthorized\"}}","predecoder":{"program_name":"suricata","timestamp":"Mar
30 19:27:46","hostname":"fw1"},"decoder":{},"location":"172.16.1.X"}

This is the same log entry on the pfsense machine

{"timestamp":"2022-03-30T19:27:46.385986-0700","flow_id":158011075148661,"in_iface":"ix2","event_type":"sip","src_ip":"172.16.1.40","src_port":5060,"dest_ip":"10.0.8.10","dest_port":64112,"proto":"UDP","sip":{"version":"SIP/2.0","code":"401","reason":"Unauthorized","response_line":"SIP/2.0
401 Unauthorized"}}

This is what I get when I do a log test.   It can properly decode the entry.


**Phase 1: Completed pre-decoding.

**Phase 2: Completed decoding.
        name: 'json'
        dest_ip: '10.0.8.10'
        dest_port: '64112'
        event_type: 'sip'
        flow_id: '158011075148661.000000'
        in_iface: 'ix2'
        proto: 'UDP'
        sip.code: '401'
        sip.reason: 'Unauthorized'
        sip.response_line: 'SIP/2.0 401 Unauthorized'
        sip.version: 'SIP/2.0'
        src_ip: '172.16.1.40'
        src_port: '5060'
        timestamp: '2022-03-30T19:27:46.385986-0700'

**Phase 3: Completed filtering (rules).
        id: '86600'
        level: '0'
        description: 'Suricata messages.'
        groups: '['ids', 'suricata']'
        firedtimes: '2'
        mail: 'False'

Here is another log entry from the archives.json.

{"timestamp":"2022-03-30T14:56:42.812-0700","agent":{"id":"000","name":"monitor2"},"manager":{"name":"monitor2"},"id":"1648677402.152835367","full_log":"Mar
30 14:56:42 fw1 suricata:
{\"timestamp\":\"2022-03-30T14:56:38.898563-0700\",\"flow_id\":1133909250595693,\"in_iface\":\"ix2\",\"event_type\":\"tls\",\"src_ip\":\"172.16.1.X\",\"src_port\":50226,\"dest_ip\":\"185.167.164.42\",\"dest_port\":443,\"proto\":\"TCP\",\"tls\":{\"session_resumed\":true,\"sni\":\"c1.adform.net\",\"version\":\"TLS
1.2\",\"ja3\":{},\"ja3s\":{}}}","predecoder":{"program_name":"suricata","timestamp":"Mar
30 14:56:42","hostname":"fw1"},"decoder":{},"location":"172.16.1.xx"}

when I run it through the log test I see:

**Phase 1: Completed pre-decoding.
        full event:
'{"timestamp":"2022-03-30T14:56:42.812-0700","agent":{"id":"000","name":"monitor2"},"manager":{"name":"monitor2"},"id":"1648677402.152835367","full_log":"Mar
30 14:56:42 fw1 suricata:
{\"timestamp\":\"2022-03-30T14:56:38.898563-0700\",\"flow_id\":1133909250595693,\"in_iface\":\"ix2\",\"event_type\":\"tls\",\"src_ip\":\"172.16.1.X\",\"src_port\":50226,\"dest_ip\":\"185.167.164.42\",\"dest_port\":443,\"proto\":\"TCP\",\"tls\":{\"session_resumed\":true,\"sni\":\"c1.adform.net\",\"version\":\"TLS
1.2\",\"ja3\":{},\"ja3s\":{}}}","predecoder":{"program_name":"suricata","timestamp":"Mar
30 14:56:42","hostname":"fw1"},"decoder":{},"location":"172.16.1.X"}'

**Phase 2: Completed decoding.
        name: 'json'
        agent.id: '000'
        agent.name: 'monitor2'
        full_log: 'Mar 30 14:56:42 fw1 suricata:
{"timestamp":"2022-03-30T14:56:38.898563-0700","flow_id":1133909250595693,"in_iface":"ix2","event_type":"tls","src_ip":"172.16.1.X","src_port":50226,"dest_ip":"185.167.164.42","dest_port":443,"proto":"TCP","tls":{"session_resumed":true,"sni":"c1.adform.net","version":"TLS
1.2","ja3":{},"ja3s":{}}}'
        id: '1648677402.152835367'
        location: '172.16.1.X'
        manager.name: 'monitor2'
        predecoder.hostname: 'fw1'
        predecoder.program_name: 'suricata'
        predecoder.timestamp: 'Mar 30 14:56:42'
        timestamp: '2022-03-30T14:56:42.812-0700'


I am also not seeing any of these events in the Kibana interface.

Any thoughts?


thanks,

Geoff

[Jose Antonio Izquierdo]

Hi Geoff,

Wazuh rules for Suricata are configured by default to alert only on event_type - alert

This is the Suricata rules file - https://github.com/wazuh/wazuh/blob/master/ruleset/rules/0475-suricata_rules.xml

Other event_type event rules have 0 as level or they do not exist as SIP in your examples.

Those are current 0 level rules 

<rule id="86602" level="0">

  <if_sid>86600</if_sid>
  <field name="event_type">^http$</field>
  <description>Suricata: HTTP.</description>
  <options>no_full_log</options>
</rule>

<rule id="86603" level="0">
  <if_sid>86600</if_sid>
  <field name="event_type">^dns$</field>
  <description>Suricata: DNS.</description>
  <options>no_full_log</options>
</rule>

<rule id="86604" level="0">
  <if_sid>86600</if_sid>
  <field name="event_type">^tls$</field>
  <description>Suricata: TLS.</description>
  <options>no_full_log</options>
</rule>

You should modify the Suricata rules file to add the needed rules and modify the rule level as needed to see them in Kibana. 

Follow our directives to create custom rules/decoders here to avoid problems with the default ruleset as it will be overwritten between updates.

Ping me if you need further assistance

Thanks 

Jose.

[Geoff Nordli]

Hi Jose.

OK, that makes sense.

If the level = 0 then it won't show up in Elastic?   Is that how it works?

thanks,

Geoff

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/08f397a1-1fe3-4e6b-806a-076c987c7b7en%40googlegroups.com.

[Jose Antonio Izquierdo]

Hi Geoff, 

You define the minimum level an event must have to become an alert. 

https://documentation.wazuh.com/current/user-manual/manager/alert-threshold.html?highlight=level

Here you will find detailed information about how this level works. By default, the minimum level to generate alerts is 3, so rules with 0, 1, or 2 level values won't create any alert. 

I hope this helps.

Jose.

[Geoff Nordli]

Hi Jose.

So only "alerts" will get fed into the kibana/elastic interface?   

thanks,

Geoff

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/3710b4cf-a137-4efb-ac5f-20bc922125dfn%40googlegroups.com.

[Jose Antonio Izquierdo]

Hi Geoff. 

Yep, but not. By default, only alerts get fed to elastic. all of them get stored in the wazuh-alerts-* index, and you can access them from Wazuh dashboard/kibana

But, you have the option to send every single collected event to elastic to a wazuh-archives-* index. Then, you must enable the logall_json option in your manager and enable filebeat->wazuh module -> archives in your /etc/filebeat/filebeat.yml configuration file, and restart the manager and filebeat. After you complete the steps, you may be able to access all collected events in the discover section of Kibana. But, of course, take care; the logall option means tons of events to be collected. 

Thanks 

BR, 

Jose.