sibling decoders and extracting JSON out of a custom log

[Dunter]

Hi Reddit - I need some help with sibling decoders and extracting JSON out of a custom log. Here's the log i'd like to decode:

[2023-03-16 09:48:33] production.INFO: API=validate-device UUID=0DC81B93-D79B-45Q9-B72F-45CBD05F6137 USERAGENT="Company/2 CFNetwork/1504.0.5 Darwin/22.3.0" AUTH=BLOCKED REQUEST={"os":"iOS","device_id":"0DC81B93-D79B-45Q9-B72F-45CBD05F6137","app_version":"9.3.0"} RESPONSE={"status":401,"message":"This device is not registered.Please register using the passcode.","auth":"BLOCKED"}

These are my decoders.

 <decoder name="app-server">
<prematch type="pcre2">\[202\d-\d\d\-\d\d\s\d\d\:\d\d\:\d\d\]\sproduction.</prematch> </decoder> <decoder name="app-server-child1"> <parent>app-server</parent> <regex type="pcre2">\[(202\d-\d\d\-\d\d)\s(\d\d\:\d\d\:\d\d)\]\sproduction.(\b(?:EMERGENCY|ALERT|CRITICAL|ERROR|WARNING|NOTICE|INFO|DEBUG)\b):\s\bAPI\b\=(\S+)\s\bUUID\b\=(\S+)\s\bUSERAGENT\b\="(.*?)"\s\bAUTH\b=(.*?)\s</regex> <order>date,time,level,api,uuid,user-agent,auth-status</order> </decoder> <decoder name="app-server-child2"> <parent>app-server</parent> <prematch type="pcre2">REQUEST=</prematch> <plugin_decoder offset="after_prematch">JSON_Decoder</plugin_decoder> </decoder> <decoder name="app-server-child3"> <parent>app-server</parent> <prematch type="pcre2">RESPONSE=</prematch> <plugin_decoder offset="after_prematch">JSON_Decoder</plugin_decoder> </decoder>

I want the parent decoder to catch the log, then app-server-child1 to extract everything up until the REQUEST field and then the app-server-child2 & 3 to extract the JSON from the REQUEST & REPONSE fields. However it stops at app-server-child1. Can anyone advise please?

**Messages: INFO: (7202): Session initialized with token 'a4b0b8be' **Phase 1: Completed pre-decoding. full event: '[2023-03-16 09:48:33] production.INFO: API=validate-device UUID=0DC81B93-D79B-45Q9-B72F-45CBD05F6137 USERAGENT="Company/2 CFNetwork/1504.0.5 Darwin/22.3.0" AUTH=BLOCKED REQUEST={"os":"iOS","device_id":"0DC81B93-D79B-45Q9-B72F-45CBD05F6137","app_version":"9.3.0"} RESPONSE={"status":401,"message":"This device is not registered.Please register using the passcode.","auth":"BLOCKED"}' **Phase 2: Completed decoding. name: 'app-server' api: 'validate-device' auth-status: 'BLOCKED' date: '2023-03-16' level: 'INFO' time: '09:48:33' user-agent: 'Company/2 CFNetwork/1504.0.5 Darwin/22.3.0' uuid: '0DC81B93-D79B-45Q9-B72F-45CBD05F6137' **Phase 3: Completed filtering (rules). id: '100102' level: '2' description: 'Informational app-server log' groups: '["app-server"]' firedtimes: '1' mail: 'false' **Alert to be generated.

[Nicolas Agustin Guevara Pihen]

Hello Dunter, thank you for using Wazuh!

This approach is not working because Wazuh decoders cannot have "grandchildren". This means that once a decoder without a parent is matched, Wazuh will sequentially search through the decoders with the first decoder as a parent, and once one of them is matched, it will stop searching. You can read more about that in this documentation.

This is when we can create sibling decoders, that are parents of themselves and should have the same name. The problem with this is that unfortunately, it is not possible to have several <prematch>, and thus it is not possible to use <plugin_decoder>. However, a possible workaround is to create sibling decoders for each of the request and response fields. For example:

<decoder name="app-server">
  <prematch type="pcre2">\[202\d-\d\d\-\d\d\s\d\d\:\d\d\:\d\d\]\sproduction.</prematch>
</decoder>

<decoder name="app-server">

  <parent>app-server</parent>
  <regex type="pcre2">\[(202\d-\d\d\-\d\d)\s(\d\d\:\d\d\:\d\d)\]\sproduction.(\b(?:EMERGENCY|ALERT|CRITICAL|ERROR|WARNING|NOTICE|INFO|DEBUG)\b):\s\bAPI\b\=(\S+)\s\bUUID\b\=(\S+)\s\bUSERAGENT\b\="(.*?)"\s\bAUTH\b=(.*?)\sREQUEST=</regex>

  <order>date,time,level,api,uuid,user-agent,auth-status</order>
</decoder>

<decoder name="app-server">

  <parent>app-server</parent>
  <regex type="pcre2">"os":"(.*?)"</regex>
  <order>request_os</order>
</decoder>
<decoder name="app-server">
  <parent>app-server</parent>
  <regex type="pcre2">"device_id":"(.*?)"</regex>
  <order>request_device_id</order>
</decoder>
<decoder name="app-server">
  <parent>app-server</parent>
  <regex type="pcre2">"app_version":"(.*?)"</regex>
  <order>request_app_version</order>
</decoder>
<decoder name="app-server">
  <parent>app-server</parent>
  <regex type="pcre2">"status":"(.*?)"</regex>
  <order>response_status</order>
</decoder>
<decoder name="app-server">
  <parent>app-server</parent>
  <regex type="pcre2">"message":"(.*?)"</regex>
  <order>response_message</order>
</decoder>
<decoder name="app-server">
  <parent>app-server</parent>
  <regex type="pcre2">"auth":"(.*?)"</regex>
  <order>response_auth</order>
</decoder>

With this approach, you can add all the possible fields, which will work even if some of them are not in some logs. For example, if you add the next decoder to the previous example, it will decode all the fields and simply ignore the next one as there is not "another_field"

<decoder name="app-server">
  <parent>app-server</parent>
  <regex type="pcre2">"another_field":"(.*?)"</regex>
  <order>response_another_field</order>
</decoder>

I hope you find this information helpful, let me know if you have any questions.

Kind regards,

[Hunter Goncalves]

Thank you so much! This is exactly what I needed and helped to understand parent/sibling decoders. :)

--
You received this message because you are subscribed to a topic in the Google Groups "Wazuh mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/wazuh/WoycCK3tpy4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/77703f02-5ce6-4f67-bd3c-e691b74ab001n%40googlegroups.com.