Wazuh archive log decoder not matched

Arsalan Shahid

unread,

Aug 2, 2021, 3:42:31 AM8/2/21

to Wazuh mailing list

Hello,

i have created a decoder for my oracle web logic server, the decoders are matching and generating alert when tested on wazuh-logtest.

i can see logs of remote host(weblogic server) in my archives.log in wazuh manager, but its not generating alerts.

-----------------sample log---------------start----

2021-07-30 09:46:27 192.168.17.40:9003 GET 192.168.32.6 200 11553 /eClaim_WebApp/main.jsp?null,null

-----------------sample log---------------end----

-----------------Logs test on wazuh-logtest---------------start----

**Phase 1: Completed pre-decoding.

full event: '2021-07-30 09:46:27 192.168.17.40:9003 GET 192.168.32.6 200 11553 /eClaim_WebApp/main.jsp?null,null'

**Phase 2: Completed decoding.

Bytes: '11553'

date: '2021-07-30'

dstip: '192.168.17.40:9003'

method: 'GET'

srcip: '192.168.32.6'

status: '200'

time: '09:46:27'

url: '/eClaim_WebApp/main.jsp?null,null'

**Phase 3: Completed filtering (rules).

id: '31103'

level: '7'

description: 'SQL injection attempt.'

groups: '['web', 'accesslog', 'attack', 'sql_injection']'

firedtimes: '1'

gdpr: '['IV_35.7.d']'

mail: 'False'

mitre.id: '['T1190']'

mitre.tactic: '['Initial Access']'

mitre.technique: '['Exploit Public-Facing Application']'

nist_800_53: '['SA.11', 'SI.4']'

pci_dss: '['6.5', '11.4', '6.5.1']'

tsc: '['CC6.6', 'CC7.1', 'CC8.1', 'CC6.1', 'CC6.8', 'CC7.2', 'CC7.3']'

**Alert to be generated.

-----------------Logs test on wazuh-logtest---------------end----

-----------------decoder---------------start----

</decoder>

</decoder>

<order>dstip</order>

</decoder>

<order>srcip</order>

</decoder>

<order>status</order>

</decoder>

<order>Bytes</order>

</decoder>

</decoder>

<order>method</order>

</decoder>

-----------------decoder---------------end----

-----------------/var/ossec/logs/archives/archives.log---------------start----

2021 Aug 02 06:51:31 (adf.efuinsurance.com) any->/u01/MW/Oracle_Home/user_projects/domains/WLS/servers/ADF_APPS1/logs/access.log 2021-08-02 11:51:29 192.168.17.40:9003 GET 172.16.7.186 200 87834 /policy_verify_pages/index2.jsp?null,null

--------sample---------/var/ossec/logs/archives/archives.log---------------start----

Yana Zaeva

unread,

Aug 2, 2021, 7:00:30 AM8/2/21

to Wazuh mailing list

Hi,

In order to test if your decoders are useful for certain events, we always advise enabling the archives.json, instead of the archives.log. You can do so by going to the ossec.conf file, and enabling the <logall_json> option. Then, restart the Wazuh manager to apply the changes. Enabling this file, and not the archives.log one, is useful because in these logs we can see the field full_log, which is the one being parsed by analysisd. Probably, your events are not matching the decoders because the log for which your decoders are written differs from the full_log field log.

Here is an example of one of the archives.json events (you can find the field we are interested in in bold):

{"timestamp":"2021-08-02T10:43:16.435+0000","agent":{"id":"000","name":"default"},"manager":{"name":"default"},"id":"1627900996.1751","cluster":{"name":"wazuh","node":"master-node"},"full_log":"Aug 2 10:43:15 default env: Started ossec-logcollector...","predecoder":{"program_name":"env","timestamp":"Aug 2 10:43:15","hostname":"default"},"decoder":{},"location":"/var/log/messages"}

If your full_log field looks like this (these are the logs you have in archives.log), you should write a decoder for this log, and not the first one you sent:

"full_log":"2021 Aug 02 06:51:31 (adf.efuinsurance.com) any->/u01/MW/Oracle_Home/user_projects/domains/WLS/servers/ADF_APPS1/logs/access.log 2021-08-02 11:51:29 192.168.17.40:9003 GET 172.16.7.186 200 87834 /policy_verify_pages/index2.jsp?null,null",

If you want, you can send me some of the full_log values, and I can help you with writing decoders for them.

Hope I was helpful. Let me know if you need anything else.

Regards,

Yana.

Arsalan Shahid

unread,

Aug 2, 2021, 7:17:47 AM8/2/21

to Wazuh mailing list

Thank You Yana,

I will try to make the decoders because i wanna learn.

if i got stuck then sure i will disturb you with this :)

Best Regards,

Arsalan

Arsalan Shahid

unread,

Aug 3, 2021, 3:41:53 AM8/3/21

to Wazuh mailing list

2021 Aug 03 05:23:24 (adf.efuinsurance.com) any->/u01/MW/Oracle_Home/user_projects/domains/WLS/servers/ADF_APPS1/logs/access.log 2021-08-03 10:23:13 192.168.17.40:9003 GET 10.0.50.42 200 2554000 /eClaim_WebApp/imgdisplay.jsp

this is the sample log, please help me with the decoders

Yana Zaeva

unread,

Aug 3, 2021, 4:06:25 AM8/3/21

to Wazuh mailing list

Hi Arslan,

Is this the value of the full_log field, from the archives.json file?

Waiting for your reply,

Yana.

Arsalan Shahid

unread,

Aug 3, 2021, 7:57:07 AM8/3/21

to Wazuh mailing list

{"timestamp":"2021-08-03T04:40:37.416+0000","agent":{"id":"009","name":"adf.efuinsurance.com","ip":"192.168.17.40"},"manager":{"name":"soc-test"},"id":"1627965637.66043474","full_log":"2021-08-03\t09:40:36\t192.168.17.40:9003\tGET\t192.168.52.22\t404\t1164\t/eClaim_WebApp/eclaim/surveyorbill_scale.php?lossno=4252000534/2021&bill=Professianal%20Fee%20including%20expenses","decoder":{},"location":"/u01/MW/Oracle_Home/user_projects/domains/WLS/servers/ADF_APPS1/logs/access.log"}

are you asking for this?

Arsalan Shahid

unread,

Aug 4, 2021, 2:02:43 AM8/4/21

to Wazuh mailing list

I have made these decoders, also i have checked these on ossec-regex, still not working

</decoder>

<parent>weblogic</parent>

<order>timestamp</order>

</decoder>

<parent>weblogic</parent>

</decoder>

<parent>weblogic</parent>

</decoder>

<parent>weblogic</parent>

</decoder>

<parent>weblogic</parent>

</decoder>

<parent>weblogic</parent>

</decoder>

<parent>weblogic</parent>

<order>dstip</order>

</decoder>

<parent>weblogic</parent>

<order>method</order>

</decoder>

<parent>weblogic</parent>

<order>srcip</order>

</decoder>

<parent>weblogic</parent>

<order>status</order>

</decoder>

<parent>weblogic</parent>

<order>bytes</order>

</decoder>

<parent>weblogic</parent>

</decoder>

Yana Zaeva

unread,

Aug 5, 2021, 11:22:58 AM8/5/21

to Wazuh mailing list

Hi Arslan,

I have modified a little bit the begging of your decoder. You can check the result below:

<decoder name="weblogic">

<prematch>^\d\d\d\d-\d\d-\d\d\s+\d\d:\d\d:\d\d\s+\d+.\d+.\d+.\d+:\d+</prematch>

</decoder>

<decoder name="weblogic1">

<parent>weblogic</parent>

<regex>(\d\d\d\d-\d\d-\d\d\s+\d\d:\d\d:\d\d)</regex>

<order>timestamp</order>

</decoder>

<!-- The section below is commented

-->

You can use ossec-logtest to check if your decoders are working. This is the output of that decoder:

2021/08/05 15:06:59 ossec-testrule: INFO: Started (pid: 1579).

Since Wazuh v4.1.0 this binary is deprecated. Use wazuh-logtest instead

ossec-testrule: Type one log per line.

2021-08-03 10:23:13 192.168.17.40:9003 GET 10.0.50.42 200 2554000 /eClaim_WebApp/imgdisplay.jsp

**Phase 1: Completed pre-decoding.

full event: '2021-08-03 10:23:13 192.168.17.40:9003 GET 10.0.50.42 200 2554000 /eClaim_WebApp/imgdisplay.jsp'

timestamp: '(null)'

hostname: 'default'

program_name: '(null)'

log: '2021-08-03 10:23:13 192.168.17.40:9003 GET 10.0.50.42 200 2554000 /eClaim_WebApp/imgdisplay.jsp'

**Phase 2: Completed decoding.

decoder: 'weblogic'

timestamp: '2021-08-03 10:23:13'

The prematch for the logs was not right so I modified it and it started working. Try modifying the rest of the decoders and let me know if you need further help. I will be happy to keep modifying them for you!

Regards,

Yana.

Arsalan Shahid

unread,

Aug 6, 2021, 12:54:22 AM8/6/21

to Wazuh mailing list

actually you got it wrong.
The log in archives.log is:
2021 Aug 06 04:45:48 (adf.efuinsurance.com) any->/u01/MW/Oracle_Home/user_projects/domains/WLS/servers/ADF_APPS1/logs/access.log 2021-08-06 09:45:35 192.168.17.40:9003 GET 202.47.32.51 200 6605986 /eClaim_WebApp/imgdisplay.jsp

i need decoder for this.

Yana Zaeva

unread,

Aug 6, 2021, 9:33:52 AM8/6/21

to Wazuh mailing list

Hi Arslan,

You have to pay attention to the full_log field:

"full_log":"2021-08-03\t09:40:36\t192.168.17.40:9003\tGET\t192.168.52.22\t404\t1164\t/eClaim_WebApp/eclaim/surveyorbill_scale.php?lossno=4252000534/2021&bill=Professianal%20Fee%20including%20expenses"

The information that is being decoded starts with 2021-08-03\t09:40:36\t192.168.17.40:9003\tGET\, meaning that this is the information being decoded. Taking the log from the archives.log, you will have to write a decoder for only the bold part:

2021 Aug 06 04:45:48 (adf.efuinsurance.com) any->/u01/MW/Oracle_Home/user_projects/domains/WLS/servers/ADF_APPS1/logs/access.log 2021-08-06 09:45:35 192.168.17.40:9003 GET 202.47.32.51 200 6605986 /eClaim_WebApp/imgdisplay.jsp

Enabling the archives.json file is much more useful, as it shows exactly the information that is being parsed (all the data in the full_log field), meanwhile, the archives.log can be confusing, as it does not show only the log.

Hope this helps. Let me know if you need anything else.

Regards,

Yana.

Arsalan Shahid

unread,

Aug 7, 2021, 1:24:17 AM8/7/21

to Wazuh mailing list

Thank You Yana for you help,
if i check on wazuh-logtest i can see that **Alert to be Generated**. but i cannot see logs in kibana

============================

2021-08-06 09:45:35 192.168.17.40:9003 GET 202.47.32.51 200 6605986 /eClaim_WebApp/imgdisplay.jsp?../../

**Phase 1: Completed pre-decoding.

full event: '2021-08-06 09:45:35 192.168.17.40:9003 GET 202.47.32.51 200 6605986 /eClaim_WebApp/imgdisplay.jsp?../../'

**Phase 2: Completed decoding.

bytes: '6605986'

date: '2021-08-06'

dstip: '192.168.17.40:9003'

method: 'GET'

srcip: '202.47.32.51'

status: '200'

time: '09:45:35'

url: '/eClaim_WebApp/imgdisplay.jsp?../../'

**Phase 3: Completed filtering (rules).

id: '31104'

level: '6'

description: 'Common web attack.'

groups: '['web', 'accesslog', 'attack']'

firedtimes: '1'

gdpr: '['IV_35.7.d']'

mail: 'False'

mitre.id: '['T1055', 'T1083', 'T1190']'

mitre.tactic: '['Defense Evasion', 'Privilege Escalation', 'Discovery', 'Initial Access']'

mitre.technique: '['Process Injection', 'File and Directory Discovery', 'Exploit Public-Facing Application']'

nist_800_53: '['SA.11', 'SI.4']'

pci_dss: '['6.5', '11.4', '6.5.1']'

tsc: '['CC6.6', 'CC7.1', 'CC8.1', 'CC6.1', 'CC6.8', 'CC7.2', 'CC7.3']'

**Alert to be generated.

===========================

Arsalan Shahid

unread,

Aug 9, 2021, 3:20:58 AM8/9/21

to Wazuh mailing list

I have made the rules and still i cannot see in KIbana UI

========RULES=========

<decoded_as>weblogic</decoded_as>

<description>Web Simple HTTPS Request</description>

</rule>

<if_sid>999991</if_sid>

<description>SQL injection attempt.</description>

<mitre>

</mitre>

<group>attack,sqlinjection,attack,pci_dss_6.5,pci_dss_11.4,pci_dss_6.5.1,gdpr_IV_35.7.d,nist_800_53_SA.11,nist_800_53_SI.4,tsc_CC6.6,tsc_CC7.1,tsc_CC8.1,tsc_CC6.1,tsc_CC6.8,tsc_CC7.2,tsc_CC7.3,</group>

</rule>

<if_sid>999991</if_sid>

<url>union+|where+|null,null|xp_cmdshell</url>

<description>SQL injection attempt.</description>

<mitre>

</mitre>

<group>attack,sql_injection,pci_dss_6.5,pci_dss_11.4,pci_dss_6.5.1,gdpr_IV_35.7.d,nist_800_53_SA.11,nist_800_53_SI.4,tsc_CC6.6,tsc_CC7.1,tsc_CC8.1,tsc_CC6.1,tsc_CC6.8,tsc_CC7.2,tsc_CC7.3,</group>

</rule>

</group>

============wazuh-logtest=============

New session was created with token "023f3f24"

**Phase 1: Completed pre-decoding.

full event: '2021-08-09 12:02:54 192.168.17.40:9003 GET 39.57.24.19 404 1164 /eClaim_WebApp/assets/images/images/bck.gif?null,null'

**Phase 2: Completed decoding.

bytes: '1164'

date: '2021-08-09'

dstip: '192.168.17.40:9003'

method: 'GET'

srcip: '39.57.24.19'

status: '404'

time: '12:02:54'

url: '/eClaim_WebApp/assets/images/images/bck.gif?null,null'

**Phase 3: Completed filtering (rules).

id: '31103'

level: '12'

description: 'SQL injection attempt.'

groups: '['syslog', 'web-logattack', 'sql_injection']'

firedtimes: '1'

gdpr: '['IV_35.7.d']'

mail: 'True'

mitre.id: '['T1190']'

mitre.tactic: '['Initial Access']'

mitre.technique: '['Exploit Public-Facing Application']'

nist_800_53: '['SA.11', 'SI.4']'

pci_dss: '['6.5', '11.4', '6.5.1']'

tsc: '['CC6.6', 'CC7.1', 'CC8.1', 'CC6.1', 'CC6.8', 'CC7.2', 'CC7.3']'

**Alert to be generated.

what should i do now.

Yana Zaeva

unread,

Aug 11, 2021, 12:54:08 PM8/11/21

to Wazuh mailing list

Hi,

Sorry for the late response. Have you restarted the manager after adding these rules and decoders? Just to be sure, restart it again. Once you have restarted, wait for a while for one of these events to happen, and check if any of these events are present in the archives.json. If you can find them in the archives.json, take the full_log field and test it in the log-test tool. Check the rule that is matching and look it up in the alerts.json file. You can use grep for major clarity.

Let me know if you were able to find these events in the alerts.json after they were generated.

Waiting for your reply,

Yana.

Arsalan Shahid

unread,

Aug 12, 2021, 2:06:20 AM8/12/21

to Wazuh mailing list

no result for full lo gin archive.json

Yana Zaeva

unread,

Aug 13, 2021, 2:51:03 PM8/13/21

to Wazuh mailing list

Hi Arsalan,

If you do not have any results in the archives.json, than means that this event did not happen, so this is why you are not receiving any alerts. We can wait for these events to happened and see if they trigger your rules correctly.

Let me know when you notice these events in the archives.json.

Waiting for your reply,

Yana.

Arsalan Shahid

unread,

Aug 16, 2021, 3:41:27 AM8/16/21

to Wazuh mailing list

Here are the files, you can check if I have made any mistake,

decoders.txt

archives-json.txt

archives-log.txt

orignal-access-log.txt

Yana Zaeva

unread,

Aug 18, 2021, 6:21:52 AM8/18/21

to Wazuh mailing list

Hi Arsalan,

I have tested your decoders and they seem to be working great. Apparently, what is not working properly are your rules. I only have these ones, which you sent in a previous message:

Let me know if you have more rules besides these ones. Your rules are working but only for this type of event: 2021-08-09 12:02:54 192.168.17.40:9003 GET 39.57.24.19 404 1164 /eClaim_WebApp/assets/images/images/bck.gif?null,null. Probably, you haven't received any alert because this event did not happen yet. Other types of events, like for example the ones you sent me above, are matching the rule 31100, which level is 0, so that's why you are not seeing them in Kibana.

What we can do to receive every event, is to overwrite every rule of this rule file, increasing the level to at least 3, so this way we should start receiving all the events. Let me know if after this change you are still not receiving them. Also, do not forget to remove your custom rules while trying this way. After overwriting the rules, restart the manager to apply the changes.

Waiting for your reply,

Yana.

Arsalan Shahid

unread,

Aug 18, 2021, 10:19:45 AM8/18/21

to Wazuh mailing list

hello yana,
tried your steps, still cannot see alert,

Yana Zaeva

unread,

Aug 20, 2021, 6:19:21 AM8/20/21

to Wazuh mailing list

Hi Arsalan,

Which version of Wazuh are you using? Could you send me your decoders again? I see that in the picture above your decoder is working with the log: 2021-08-12\t10:10:32\t192.168.17.40:9003\tGET\t39.32.116.235\t200\t3399\t/eClaim_WebApp/changepwd.jsp, but when I paste this type of log (meaning a log that contains \t), I do not get a matching log, meanwhile if I paste the log with spaces (2021-08-11 13:02:22 192.168.17.40:9003 GET 192.140.150.49 404 1164 /favicon.ico), this log does get parsed. You can check it in the attached image.

Also, I have tested completely your scenario. I created a test directory (test.json), which is monitored by Wazuh. I sent one of the raw logs to this file, then I checked that this file was in the archives.json one and also in the alerts.json one. I was able to find the event in both of them. Then, I checked Kibana and the event was displayed in there two. I will attach two screenshots so you can see the followed process.

After all this, my thought is that maybe you are not receiving the logs in Kibana because of two reasons:

- You have modified the decoders (thus that's why I am not getting the decoder parsed with the full_log value, but with the raw value)

- Because of the Wazuh version, due to the differences between my archives.json full_log field and yours:

My output:

{"timestamp":"2021-08-20T10:08:55.839+0000","rule":{"level":5,"description":"Access log messages grouped.","id":"31100","firedtimes":1,"mail":false,"groups":["web","accesslog"]},"agent":{"id":"000","name":"default"},"manager":{"name":"default"},"id":"1629454135.2633047","cluster":{"name":"wazuh","node":"master-node"},"full_log":"2021-08-11 13:02:22 192.168.17.40:9003 GET 192.140.150.49 404 1164 /favicon.ico","decoder":{"name":"weblogic"},"data":{"srcip":"192.140.150.49","dstip":"192.168.17.40:9003","status":"404","url":"/favicon.ico","date":"2021-08-11","time":"13:02:22","method":"GET","bytes":"1164"},"location":"/test/test.json"}

Your output:

{"timestamp":"2021-08-12T05:10:37.233+0000","agent":{"id":"009","name":"adf.efuinsurance.com","ip":"192.168.17.40"},"manager":{"name":"soc-test"},"id":"1628745037.46094303","full_log":"2021-08-12\t10:10:32\t192.168.17.40:9003\tGET\t39.32.116.235\t200\t3399\t/eClaim_WebApp/changepwd.jsp","decoder":{},"location":"/u01/MW/Oracle_Home/user_projects/domains/WLS/servers/ADF_APPS1/logs/access.log"}

Let me know if you had modified your decoders.

Regards,

Yana.

process2.png

process1.png

ossec-logtest_output.png

Danish Ibrar

unread,

Dec 22, 2023, 5:59:11 AM12/22/23

to Wazuh | Mailing List

Hi Arsalan Shahid,
Hope you're doing well, I'm looking to integrate weblogic server can you plz share the steps how you configured and forwarded logs to wazuh server.
Would be really helpful.

Reply all

Reply to author

Forward