Learning to create decoders and rules

224 views
Skip to first unread message

IBM Qradar

unread,
Dec 10, 2020, 6:07:23 AM12/10/20
to Wazuh mailing list
Hello, 
I am trying to create custom decoders and rules but they seem to be ignored by the manager.

This is the decoder I have created for Adobe cloud web services:

<decoder name="Adobe_cloud">
  <program_name>^Adobe_cloud</program_name>
</decoder>

<decoder name="Adobe_cloud">
  <prematch>^20\d\d\s\w\w\w\s\d\d\s\d\d:\d\d:\d\d\s\(xxxxxxxxxx\d</prematch>
</decoder>

<decoder name="Adobe_cloud_custom1">
  <parent>Adobe_cloud</parent>
  <regex>^20\d\d\s\w\w\w\s\d\d\s\d\d:\d\d:\d\d\s\(xxxxxxxxxx\d\)\s+\S+\s+\S+\s+\S+\s+\S+\s+\S+\s+access.log\s+(\S+)\s+\S+\s+\S+\s+\S+\s+\S+\s+\S+\s+\S+\s+\S+\s+(\d+)</regex>
  <order>srcip,status</order>
</decoder>

*********************************************************************
And this is a sample rule just to check if it triggers, it is supposed to detect http response code (in var "status") if 200:

<rule id="100001" level="3">
  <decoded_as>Adobe_cloud</decoded_as>
  <description>Pruebas AEM</description>
</rule>

<rule id="90303" level="0">
  <if_sid>100001</if_sid>
  <status>200</status>
  <description>Prueba en reglas</description>
</rule>

*********************************************************************

Testing both regular expressions works fine, both are detected by the regex engine, but when using logtest utility to see if the rule triggers, I get the following result:

echo "2020 Dec 07 15:40:16 (xxxxxxxxxx1) any->\aem\logpath\logs\access.log 07.12.2020 15:40:32.058 *INFO* [qtp305148339-1116906] access.log 11.11.111.11 - anonymous 07/Dec/2020:15:40:32 +0100 GET /url/random HTTP/1.1 200 2 " | /var/ossec/bin/ossec-logtest

**Phase 1: Completed pre-decoding.
       full event: '2020 Dec 07 15:40:16 (xxxxxxxxxx1) any->\aem\logpath\logs\access.log 07.12.2020 15:40:32.058 *INFO* [qtp305148339-1116906] access.log 11.11.111.11 - anonymous 07/Dec/2020:15:40:32 +0100 GET /url/random HTTP/1.1 200 2 '
       timestamp: '2020 Dec 07 15:40:16'
       hostname: '(xxxxxxxxxx1)'
       program_name: '(null)'
       log: 'any->\aem\logpath\logs\access.log 07.12.2020 15:40:32.058 *INFO* [qtp305148339-1116906] access.log 11.11.111.11 - anonymous 07/Dec/2020:15:40:32 +0100 GET /url/random HTTP/1.1 200 2 '

**Phase 2: Completed decoding.
       No decoder matched.
 
It seems that our decoder is not matching the log. But the decoder path is correct in the ossec config file, as well as the rule, and the regex is detecting that log. Would be possible that the log gets detected by another decoder and thus it stops comparing with the following ones? 

Can anyone help me understand what is happening?

Thank you all!

Jonathan Martín Valera

unread,
Dec 10, 2020, 7:26:18 AM12/10/20
to Wazuh mailing list

Hi IBM Qradar,

When you are testing decoders and rules in ossec-logtest tool, you have to enter the raw log without any event header. For example, I see that you are entering the following:

"2020 Dec 07 15:40:16 (xxxxxxxxxx1) any->\aem\logpath\logs\access.log 07.12.2020 15:40:32.058 *INFO* [qtp305148339-1116906] access.log 11.11.111.11 - anonymous 07/Dec/2020:15:40:32 +0100 GET /url/random HTTP/1.1 200 2"

and that log corresponds to the entire event (you probably got it from the archives.log file). As you can see, this log contains some extra headers added by wazuh, so it does not correspond to the original log sent by the agent.

The log that you have to enter in ossec-logtest tool would be as follows:

\aem\logpath\logs\access.log 07.12.2020 15:40:32.058 *INFO* [qtp305148339-1116906] access.log 11.11.111.11 - anonymous 07/Dec/2020:15:40:32 +0100 GET /url/random HTTP/1.1 200 2"

I also see that the log that you are entering does not correspond to the format specified in the decoder since you have added the event headers in your decoders regex. Please, adapt the decoders to the log that I mentioned above.

If you need help with a particular case, tell me exactly which log you want to decode, what fields you want to decode, and any particular case to generate an alert.

Best regards.

​

csoc qs

unread,
Dec 10, 2020, 10:15:45 AM12/10/20
to Wazuh mailing list
Hello Jonathan, 
Thank you for your help, this is the complete log I want to decode, including header:

2020 Dec 07 15:40:16 (xxxxxxxxxx1) any->\aem\publish\crx-quickstart\logs\access.log 07.12.2020 15:40:32.058 *INFO* [qtp305148339-1116906] access.log 213.8.114.1 - anonymous 07/Dec/2020:15:40:32 +0100 "GET /random/urlquery/token.json HTTP/1.1" 200 2 "https://referrer/random.html" "Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"

But echo command gets confused with the double quotes, that is why I tried only a piece of log.

I would like to decode source IP, username, URL, http response code and user-agent

Actually, is not exactly that I want to decode this log, but learn how to create decoders and rules, since I believe this is just the first of many.

So, the regex used in the decoder... Should not count the header?

Best Regards, 
Jorge

Jonathan Martín Valera

unread,
Dec 11, 2020, 3:08:13 AM12/11/20
to Wazuh mailing list

Hi IBM Qradar,

You have to write the decoder and rule regex from the original log stored and sent by the agent to the manager. Where did you get the log that you have shared with me? Surely from archives.log, but this file contains all received events, including the headers like I commented to you in the last message.

I suppose that the original log is the following:

\aem\publish\crx-quickstart\logs\access.log 07.12.2020 15:40:32.058 *INFO* [qtp305148339-1116906] access.log 213.8.114.1 - anonymous 07/Dec/2020:15:40:32 +0100 "GET /random/urlquery/token.json HTTP/1.1" 200 2 "https://referrer/random.html" "Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"

From this log, I am going to create some custom decoders to decode your desired fields. I have added them in /var/ossec/etc/decoders/local_decoder.xml file.

<decoder name="adobe_cloud">
   <prematch>\S+ \d+.\d+.\d+ \d+:\d+:\d+.\d+ \S+ [\.+]</prematch>
</decoder>

<decoder name="adobe_cloud_custom">
   <parent>adobe_cloud</parent>
   <regex offset="after_parent">\S+ (\d+.\d+.\d+.\d+) - (\w+) \S+ \S+ "\S+ (\S+) \S+" (\d+) \d+ "(\S+)" "(\.+)"</regex>
   <order>source_ip, username, uri, status_code, url, user_agent</order>
</decoder>

If we run again the ossec-logtest tool, we can see now that your desired fields are decoded:

\aem\publish\crx-quickstart\logs\access.log 07.12.2020 15:40:32.058 *INFO* [qtp305148339-1116906] access.log 213.8.114.1 - anonymous 07/Dec/2020:15:40:32 +0100 "GET /random/urlquery/token.json HTTP/1.1" 500 2 "https://referrer/random.html" "Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"

**Phase 1: Completed pre-decoding.
       full event: '\aem\publish\crx-quickstart\logs\access.log 07.12.2020 15:40:32.058 *INFO* [qtp305148339-1116906] access.log 213.8.114.1 - anonymous 07/Dec/2020:15:40:32 +0100 "GET /random/urlquery/token.json HTTP/1.1" 200 2 "https://referrer/random.html" "Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"'
       timestamp: '(null)'
       hostname: 'manager'
       program_name: '(null)'
       log: '\aem\publish\crx-quickstart\logs\access.log 07.12.2020 15:40:32.058 *INFO* [qtp305148339-1116906] access.log 213.8.114.1 - anonymous 07/Dec/2020:15:40:32 +0100 "GET /random/urlquery/token.json HTTP/1.1" 200 2 "https://referrer/random.html" "Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"'

**Phase 2: Completed decoding.
       decoder: 'adobe_cloud'
       source_ip: '213.8.114.1'
       username: 'anonymous'
       uri: '/random/urlquery/token.json'
       status_code: '200'
       url: 'https://referrer/random.html'
       user_agent: 'Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36'

Important: Note that this log only works when the log format matches with the decoder regex. In case a log can have multiple formats, you will have to write more decoders to cover all possible cases.

Regarding the rules, for example, if we want to generate an alert when the status code is 500, we could do something like this in /var/ossec/etc/rules/local_rules.xml:

<rule id="100120" level="5">
  <decoded_as>adobe_cloud</decoded_as>
  <field name="status_code">500</field>
  <description>Server error</description>
</rule>

Now, if we test a log with status code 500, we can check that the alert would be generated:

  \aem\publish\crx-quickstart\logs\access.log 07.12.2020 15:40:32.058 *INFO* [qtp305148339-1116906] access.log 213.8.114.1 - anonymous 07/Dec/2020:15:40:32 +0100 "GET /random/urlquery/token.json HTTP/1.1" 500 2 "https://referrer/random.html" "Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"

**Phase 1: Completed pre-decoding.
       full event: '\aem\publish\crx-quickstart\logs\access.log 07.12.2020 15:40:32.058 *INFO* [qtp305148339-1116906] access.log 213.8.114.1 - anonymous 07/Dec/2020:15:40:32 +0100 "GET /random/urlquery/token.json HTTP/1.1" 500 2 "https://referrer/random.html" "Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"'
       timestamp: '(null)'
       hostname: 'manager'
       program_name: '(null)'
       log: '\aem\publish\crx-quickstart\logs\access.log 07.12.2020 15:40:32.058 *INFO* [qtp305148339-1116906] access.log 213.8.114.1 - anonymous 07/Dec/2020:15:40:32 +0100 "GET /random/urlquery/token.json HTTP/1.1" 500 2 "https://referrer/random.html" "Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"'

**Phase 2: Completed decoding.
       decoder: 'adobe_cloud'
       source_ip: '213.8.114.1'
       username: 'anonymous'
       uri: '/random/urlquery/token.json'
       status_code: '500'
       url: 'https://referrer/random.html'
       user_agent: 'Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36'

**Phase 3: Completed filtering (rules).
       Rule id: '100120'
       Level: '5'
       Description: 'Server error'
**Alert to be generated.

You can find more useful information about how to create custom decoders and rules in the following links:

• Creating decoders and rules from scratch: https://wazuh.com/blog/creating-decoders-and-rules-from-scratch/
• Sibling decoders: flexible extraction of information: https://wazuh.com/blog/sibling-decoders-flexible-extraction-of-information/
• Custom rules and decoders: https://documentation.wazuh.com/4.0/user-manual/ruleset/custom.html
• Testing decoders and rules: https://documentation.wazuh.com/4.0/user-manual/ruleset/testing.html

I hope this helps you :)

Best regards.

​

csoc qs

unread,
Dec 11, 2020, 4:54:06 AM12/11/20
to Wazuh mailing list
Thank you very much Jonathan, your decoder and rule works fine and I believe I can write new ones following your example.
Enjoy your weekend!

csoc qs

unread,
Dec 11, 2020, 1:36:36 PM12/11/20
to Wazuh mailing list
Hi Jonathan.

I have created exactly the decoder and the rule that it indicates on the web page that you have sent me and it does not generate the rule even if you follow all the steps. 

The version I have is 4.0, do you know if it has any problem that does not allow to create rules? 

All the tests work but then they do not generate and display the message in Wazuh, nor in the alerts.log file. 

I need downgrade the version? 

Many thanks

csoc qs

unread,
Dec 11, 2020, 4:32:26 PM12/11/20
to Wazuh mailing list
Hi. 

I have installed a new clean OS ( Ubuntu 20.10 ) and have reinstalled the wazuh with the installation script unattended.

I have configured as the steps in this blog say. ( https://wazuh.com/blog/creating-decoders-and-rules-from-scratch/ )
I have checked that ossec-logtest works correctly and indicates that it generates an alert. 
I have configured the monitoring of a temporary file and I come back with an echo, the content I tried before. 

But it doesn't work and still doesn't show the alerts in the wazuh panel or in the alerts.log file.

I have generated two files inside the rules and decoder folder with the name test.xml and I have changed the permissions and owners so everything is the same. 

I don't know what I can try anymore. 

Any help is appreciated. 

Jonathan Martín Valera

unread,
Dec 14, 2020, 2:58:04 AM12/14/20
to Wazuh mailing list

Hi,

No, there is nothing reported about it for 4.0. A few notes about your use case:

  • Have you restarted the wazuh-manager after setting the new decoders and rules? When you are going to test it in ossec-logtest tool, it is not necessary, but when you want to apply the new changes, you have to restart the wazuh-manager to load the new configuration.

  • As you tell me, it works in ossec-logtest tool, but it does not work when you are adding the log in test_file.log. Maybe something strange is happening with the echo.

    • Can you try to use single quotes instead of double quotes?
    echo '<test_log>' >> /var/log/test_file.log
    • Can you try to add the log manually? For example, using vi or nano commands.

Check it and tell me. If it does not work, then you have to share with me all the related information: decoders, rules, localfile block in ossec.conf … in order to help you better.

Best regards.

​

csoc qs

unread,
Dec 14, 2020, 5:43:36 PM12/14/20
to Wazuh mailing list
Hi Jonathan.

Sorry for not answering you earlier. I have been doing other tests ( 3.9 ) with other versions and so far with the same result. 

I've discovered that by listening to the blog mentioned above and running the log-test first and then pasting the log line it performs all the steps correctly. 

/var/ossec/bin/ossec-logtest

/var/ossec/bin/ossec-logtest -U 222000:12:fortigate-custom
2020/12/14 22:34:30 ossec-testrule: INFO: Started (pid: 53302).
ossec-testrule: Type one log per line.

date=2019-10-10 time=17:01:31 devname="FG111E-INFT2" devid="FG201E4Q17901611" logid="0000000020" type="traffic" subtype="forward" level="notice" vd="root" eventtime=1573570891 srcip=192.168.56.105 srcname="wazuh.test.local" srcport=63874 srcintf="port1" srcintfrole="lan" dstip=54.97.146.111 dstport=443 dstintf="wan1" dstintfrole="wan" poluuid="3e421d8c-0210-51ea-2e5e-6dd151c37590" sessionid=261713795 proto=6 action="accept" user="WAZUH" authserver="FSSO_TEST_LOCAL" policyid=131 policytype="policy" service="HTTPS" dstcountry="United Kingdom" srccountry="Reserved" trandisp="snat" transip=195.46.111.2 transport=63874 appid=45553 app="Microsoft.Outlook.Office.365" appcat="Email" apprisk="medium" applist="INF-APP-MONITOR" appact="detected" duration=815 sentbyte=13941 rcvdbyte=13429 sentpkt=58 rcvdpkt=63 sentdelta=360 rcvddelta=2189 devtype="Windows PC" devcategory="Windows Device" osname="Windows" osversion="8.1" mastersrcmac="fc:45:96:44:79:c9" srcmac="fc:45:96:44:79:c9" srcserver=1 dstdevtype="Router/NAT Device" dstdevcategory="None" masterdstmac="28:8b:1c:db:7c:48" dstmac="28:8b:1c:db:7c:48" dstserver=0


**Phase 1: Completed pre-decoding.
       full event: 'date=2019-10-10 time=17:01:31 devname="FG111E-INFT2" devid="FG201E4Q17901611" logid="0000000020" type="traffic" subtype="forward" level="notice" vd="root" eventtime=1573570891 srcip=192.168.56.105 srcname="wazuh.test.local" srcport=63874 srcintf="port1" srcintfrole="lan" dstip=54.97.146.111 dstport=443 dstintf="wan1" dstintfrole="wan" poluuid="3e421d8c-0210-51ea-2e5e-6dd151c37590" sessionid=261713795 proto=6 action="accept" user="WAZUH" authserver="FSSO_TEST_LOCAL" policyid=131 policytype="policy" service="HTTPS" dstcountry="United Kingdom" srccountry="Reserved" trandisp="snat" transip=195.46.111.2 transport=63874 appid=45553 app="Microsoft.Outlook.Office.365" appcat="Email" apprisk="medium" applist="INF-APP-MONITOR" appact="detected" duration=815 sentbyte=13941 rcvdbyte=13429 sentpkt=58 rcvdpkt=63 sentdelta=360 rcvddelta=2189 devtype="Windows PC" devcategory="Windows Device" osname="Windows" osversion="8.1" mastersrcmac="fc:45:96:44:79:c9" srcmac="fc:45:96:44:79:c9" srcserver=1 dstdevtype="Router/NAT Device" dstdevcategory="None" masterdstmac="28:8b:1c:db:7c:48" dstmac="28:8b:1c:db:7c:48" dstserver=0'
       timestamp: '(null)'
       hostname: 'minis'
       program_name: '(null)'
       log: 'date=2019-10-10 time=17:01:31 devname="FG111E-INFT2" devid="FG201E4Q17901611" logid="0000000020" type="traffic" subtype="forward" level="notice" vd="root" eventtime=1573570891 srcip=192.168.56.105 srcname="wazuh.test.local" srcport=63874 srcintf="port1" srcintfrole="lan" dstip=54.97.146.111 dstport=443 dstintf="wan1" dstintfrole="wan" poluuid="3e421d8c-0210-51ea-2e5e-6dd151c37590" sessionid=261713795 proto=6 action="accept" user="WAZUH" authserver="FSSO_TEST_LOCAL" policyid=131 policytype="policy" service="HTTPS" dstcountry="United Kingdom" srccountry="Reserved" trandisp="snat" transip=195.46.111.2 transport=63874 appid=45553 app="Microsoft.Outlook.Office.365" appcat="Email" apprisk="medium" applist="INF-APP-MONITOR" appact="detected" duration=815 sentbyte=13941 rcvdbyte=13429 sentpkt=58 rcvdpkt=63 sentdelta=360 rcvddelta=2189 devtype="Windows PC" devcategory="Windows Device" osname="Windows" osversion="8.1" mastersrcmac="fc:45:96:44:79:c9" srcmac="fc:45:96:44:79:c9" srcserver=1 dstdevtype="Router/NAT Device" dstdevcategory="None" masterdstmac="28:8b:1c:db:7c:48" dstmac="28:8b:1c:db:7c:48" dstserver=0'

**Phase 2: Completed decoding.
       decoder: 'fortigate-custom'
       devid: 'FG201E4Q17901611'
       logid: '0000000020'
       type: 'traffic"'
       subtype: 'forward'

**Phase 3: Completed filtering (rules).
       Rule id: '222000'
       Level: '12'
       Description: 'Fortigate messages grouped.'
**Alert to be generated.


lf->decoder_info->name: 'fortigate-custom'
ut_decoder_name       : 'fortigate-custom'

But if I execute the following command:

cat /var/log/test_file.log | /var/ossec/bin/ossec-logtest -U 222000:12:fortigate-custom

cat /var/log/test_file.log | /var/ossec/bin/ossec-logtest -U 222000:12:fortigate-custom
2020/12/14 22:35:37 ossec-testrule: INFO: Started (pid: 53322).
ossec-testrule: Type one log per line.

**Phase 1: Completed pre-decoding.
       full event: 'date=2019-10-10 time=17:01:31 devname=FG111E-INFT2 devid=FG201E4Q17901611 logid=0000000020 type=traffic subtype=forward level=notice vd=root eventtime=1573570891 srcip=192.168.56.105 srcname=wazuh.test.local srcport=63874 srcintf=port1 srcintfrole=lan dstip=54.97.146.111 dstport=443 dstintf=wan1 dstintfrole=wan poluuid=3e421d8c-0210-51ea-2e5e-6dd151c37590 sessionid=261713795 proto=6 action=accept user=WAZUH authserver=FSSO_TEST_LOCAL policyid=131 policytype=policy service=HTTPS dstcountry=United Kingdom srccountry=Reserved trandisp=snat transip=195.46.111.2 transport=63874 appid=45553 app=Microsoft.Outlook.Office.365 appcat=Email apprisk=medium applist=INF-APP-MONITOR appact=detected duration=815 sentbyte=13941 rcvdbyte=13429 sentpkt=58 rcvdpkt=63 sentdelta=360 rcvddelta=2189 devtype=Windows PC devcategory=Windows Device osname=Windows osversion=8.1 mastersrcmac=fc:45:96:44:79:c9 srcmac=fc:45:96:44:79:c9 srcserver=1 dstdevtype=Router/NAT Device dstdevcategory=None masterdstmac=28:8b:1c:db:7c:48 dstmac=28:8b:1c:db:7c:48 dstserver=0'
       timestamp: '(null)'
       hostname: 'minis'
       program_name: '(null)'
       log: 'date=2019-10-10 time=17:01:31 devname=FG111E-INFT2 devid=FG201E4Q17901611 logid=0000000020 type=traffic subtype=forward level=notice vd=root eventtime=1573570891 srcip=192.168.56.105 srcname=wazuh.test.local srcport=63874 srcintf=port1 srcintfrole=lan dstip=54.97.146.111 dstport=443 dstintf=wan1 dstintfrole=wan poluuid=3e421d8c-0210-51ea-2e5e-6dd151c37590 sessionid=261713795 proto=6 action=accept user=WAZUH authserver=FSSO_TEST_LOCAL policyid=131 policytype=policy service=HTTPS dstcountry=United Kingdom srccountry=Reserved trandisp=snat transip=195.46.111.2 transport=63874 appid=45553 app=Microsoft.Outlook.Office.365 appcat=Email apprisk=medium applist=INF-APP-MONITOR appact=detected duration=815 sentbyte=13941 rcvdbyte=13429 sentpkt=58 rcvdpkt=63 sentdelta=360 rcvddelta=2189 devtype=Windows PC devcategory=Windows Device osname=Windows osversion=8.1 mastersrcmac=fc:45:96:44:79:c9 srcmac=fc:45:96:44:79:c9 srcserver=1 dstdevtype=Router/NAT Device dstdevcategory=None masterdstmac=28:8b:1c:db:7c:48 dstmac=28:8b:1c:db:7c:48 dstserver=0'

**Phase 2: Completed decoding.
       No decoder matched.
lf->decoder_info->name: '(null)'
ut_decoder_name       : 'fortigate-custom'
decoder matched : 'NULL' 

I have attached the configuration of my files. 
ossec.conf , test_file.log, decoder and rule custom. My installation as I said is from scratch and has no more customizations. The alerts.log file doesn't have anything either. 

I don't known if you have possible test this configuration in your environment.  

Many thanks. 

ossec.conf
rule.txt
decoder.txt
test_file.log

Jonathan Martín Valera

unread,
Dec 15, 2020, 3:46:10 AM12/15/20
to Wazuh mailing list

Hi,

As I see it, you are not using the same log to check both cases. If you notice, in the first case, the string fields of the log are between double quotes, while in the second case (which is the one that doesn’t work) they are not.

Note that the regex of the decoders does specify the use of these quotes, so the reason is that.

<decoder name="fortigate-custom">
  <prematch>^date=\d\d\d\d-\d\d-\d\d time=\d\d:\d\d:\d\d devname="\S+"</prematch>
</decoder>

<decoder name="fortigate-custom1">
  <parent>fortigate-custom</parent>
  <regex>devid="(\S+)" logid="(\S+)" type="(\S+) subtype="(\S+)"</regex>
  <order>devid, logid, type, subtype</order>
</decoder>

Replace test_file.log with the following one and try again:

date=2019-10-10 time=17:01:31 devname="FG111E-INFT2" devid="FG201E4Q17901611" logid="0000000020" type="traffic" subtype="forward" level="notice" vd="root" eventtime=1573570891 srcip=192.168.56.105 srcname="wazuh.test.local" srcport=63874 srcintf="port1" srcintfrole="lan" dstip=54.97.146.111 dstport=443 dstintf="wan1" dstintfrole="wan" poluuid="3e421d8c-0210-51ea-2e5e-6dd151c37590" sessionid=261713795 proto=6 action="accept" user="WAZUH" authserver="FSSO_TEST_LOCAL" policyid=131 policytype="policy" service="HTTPS" dstcountry="United Kingdom" srccountry="Reserved" trandisp="snat" transip=195.46.111.2 transport=63874 appid=45553 app="Microsoft.Outlook.Office.365" appcat="Email" apprisk="medium" applist="INF-APP-MONITOR" appact="detected" duration=815 sentbyte=13941 rcvdbyte=13429 sentpkt=58 rcvdpkt=63 sentdelta=360 rcvddelta=2189 devtype="Windows PC" devcategory="Windows Device" osname="Windows" osversion="8.1" mastersrcmac="fc:45:96:44:79:c9" srcmac="fc:45:96:44:79:c9" srcserver=1 dstdevtype="Router/NAT Device" dstdevcategory="None" masterdstmac="28:8b:1c:db:7c:48" dstmac="28:8b:1c:db:7c:48" dstserver=0

You will see the following expected output:

**Phase 1: Completed pre-decoding.
       full event: 'date=2019-10-10 time=17:01:31 devname="FG111E-INFT2" devid="FG201E4Q17901611" logid="0000000020" type="traffic" subtype="forward" level="notice" vd="root" eventtime=1573570891 srcip=192.168.56.105 srcname="wazuh.test.local" srcport=63874 srcintf="port1" srcintfrole="lan" dstip=54.97.146.111 dstport=443 dstintf="wan1" dstintfrole="wan" poluuid="3e421d8c-0210-51ea-2e5e-6dd151c37590" sessionid=261713795 proto=6 action="accept" user="WAZUH" authserver="FSSO_TEST_LOCAL" policyid=131 policytype="policy" service="HTTPS" dstcountry="United Kingdom" srccountry="Reserved" trandisp="snat" transip=195.46.111.2 transport=63874 appid=45553 app="Microsoft.Outlook.Office.365" appcat="Email" apprisk="medium" applist="INF-APP-MONITOR" appact="detected" duration=815 sentbyte=13941 rcvdbyte=13429 sentpkt=58 rcvdpkt=63 sentdelta=360 rcvddelta=2189 devtype="Windows PC" devcategory="Windows Device" osname="Windows" osversion="8.1" mastersrcmac="fc:45:96:44:79:c9" srcmac="fc:45:96:44:79:c9" srcserver=1 dstdevtype="Router/NAT Device" dstdevcategory="None" masterdstmac="28:8b:1c:db:7c:48" dstmac="28:8b:1c:db:7c:48" dstserver=0'
       timestamp: '(null)'
       hostname: 'manager2'
       program_name: '(null)'
       log: 'date=2019-10-10 time=17:01:31 devname="FG111E-INFT2" devid="FG201E4Q17901611" logid="0000000020" type="traffic" subtype="forward" level="notice" vd="root" eventtime=1573570891 srcip=192.168.56.105 srcname="wazuh.test.local" srcport=63874 srcintf="port1" srcintfrole="lan" dstip=54.97.146.111 dstport=443 dstintf="wan1" dstintfrole="wan" poluuid="3e421d8c-0210-51ea-2e5e-6dd151c37590" sessionid=261713795 proto=6 action="accept" user="WAZUH" authserver="FSSO_TEST_LOCAL" policyid=131 policytype="policy" service="HTTPS" dstcountry="United Kingdom" srccountry="Reserved" trandisp="snat" transip=195.46.111.2 transport=63874 appid=45553 app="Microsoft.Outlook.Office.365" appcat="Email" apprisk="medium" applist="INF-APP-MONITOR" appact="detected" duration=815 sentbyte=13941 rcvdbyte=13429 sentpkt=58 rcvdpkt=63 sentdelta=360 rcvddelta=2189 devtype="Windows PC" devcategory="Windows Device" osname="Windows" osversion="8.1" mastersrcmac="fc:45:96:44:79:c9" srcmac="fc:45:96:44:79:c9" srcserver=1 dstdevtype="Router/NAT Device" dstdevcategory="None" masterdstmac="28:8b:1c:db:7c:48" dstmac="28:8b:1c:db:7c:48" dstserver=0'

**Phase 2: Completed decoding.
       decoder: 'fortigate-custom'
       devid: 'FG201E4Q17901611'
       logid: '0000000020'
       type: 'traffic"'
       subtype: 'forward'

**Phase 3: Completed filtering (rules).
       Rule id: '222000'
       Level: '12'
       Description: 'Fortigate messages grouped.'
**Alert to be generated.

lf->decoder_info->name: 'fortigate-custom'
ut_decoder_name       : 'fortigate-custom'

Best regards.​

csoc qs

unread,
Dec 15, 2020, 6:18:37 AM12/15/20
to Wazuh mailing list
Hi Jonathan.

I see the difference, when in the page it is indicated that the echo is used to inject the line of the event into the log file it removes the quotes and inside the file everything appears without any kind of delimiter. 

To make it work well as in the page you have to change the " , by ', this way it will respect the internal quotes of the message. 

We will return to the starting point with the original log decoder, now that we have discovered how to debug and what points we have to revise. 

Thank you very much for your help. 

Reply all
Reply to author
Forward
0 new messages