syslog message incapsulated in json

[Павел Покровский]

Hi.

I have a task to implement processing for log messages, which are json objects incapsulating nginx message

Example:

{"app":"mynaughtyapp","log":"178.240.128.46 - - [08/Dec/2022:11:30:15 +0300] \"GET /something HTTP/1.1\" 200 1285 \"https://referrer.com/\" \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36\""}

I also have a plenty of sources that generate json objects without any raw incapsulation, so built-in json decoder fits quite well for corresponding rules.

But I figured that when, for this case, I implement parent json decoder, which applies regex to "log" field, I actually override built-in json decoder and so it breaks all json processing.

I  am, probably, missing something big in the idea of decoders implementation and hierarchy. Could you point me in the right direction? Is it possible to extract and decode raw data from json field?

Thank you!

[Kasim Mustapha]

Hello Pavel,

Thanks for reaching out.

Wazuh incorporates an integrated decoder for JSON logs enabling the extraction of data from any source in this format.

Kindly use the link below as a guide to accomplish the task.

https://documentation.wazuh.com/current/user-manual/ruleset/json-decoder.html

Regards,

Kasim Mustapha IT Security Engineer The Open Source Security Platform

On Thu, Dec 8, 2022 at 3:54 PM Kasim Mustapha <kasim.m...@wazuh.com> wrote:

Wazuh now incorporates an integrated decoder for JSON logs enabling the extraction of data from any source in this format.Hello Pavel,

Thanks for reaching out.

Wazuh incorporates an integrated decoder for JSON logs enabling the extraction of data from any source in this format.

Kindly use the link below as a guide to accomplish the task.

https://documentation.wazuh.com/current/user-manual/ruleset/json-decoder.html

Regards,

Kasim Mustapha IT Security Engineer The Open Source Security Platform

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/d3909a7a-7887-46c2-9e4f-eed8ed47d825n%40googlegroups.com.

[Павел Покровский]

Hi Kasim

Thank you for your feedback

I must have not worded my question correctly. I need to parse string which is part of 'log' field in original json. Is there a way to do so? I understand I can simply create a rule to store this event as an alert, but due to the fact that meaningful information is stored within 'log' field, i am quite limited in rule configuration, that is the reason why I would like to extract values from this field.

четверг, 8 декабря 2022 г. в 17:57:24 UTC+3, kasim.m...@wazuh.com:

[Kasim Mustapha]

Hello again Pavel, and sorry for the late response.

We can not modify the JSON decoder. However, I have researched a few approaches for your case. I think the best one is the one I'm explaining below.This workaround needs to modify the localfile option you use to collect the logs. Then, you will be able to create a custom decoder that both extract json fields and the field you want to be added to the description field.You have to add the out_format option to your localfile section, like this for example, so you can create your custom json decoder by providing the program_name that will be collected:

<out_format>$(timestamp) $(hostname) your_program_name: $(log)</out_format>

With this change (applied after restarting), you will have a log like this one (with syslog format):

Nov 29 15:15:48 managerauth your_program_name: {"app":"mynaughtyapp","log":"178.240.128.46 - - [08/Dec/2022:11:30:15 +0300] \"GET /something HTTP/1.1\" 200 1285 \"https://referrer.com/\" \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36\""}

Now it is time to create the new decoders.

<decoder name="myjson">
    <program_name>your_program_name</program_name>
</decoder>

<decoder name="myjson_child">
    <parent>myjson</parent>
    <plugin_decoder>JSON_Decoder</plugin_decoder>
</decoder>

<decoder name="myjson_child">
    <parent>myjson</parent>
    <regex>{"log":"(\.*)"</regex>
    <order>message_formatted</order>
</decoder>

Then, you have to add the rule that will be triggered with your custom JSON decoder.

Hope this works. Let me know if you have further questions.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/da1f820c-b751-42b7-bf41-4b2fd9e81f8dn%40googlegroups.com.

[Павел Покровский]

Hi, Kasim

sounds like a plan. We will try this out and I will let you know how it went. Thank you very much and have a fabulous day!

четверг, 8 декабря 2022 г. в 22:53:11 UTC+3, kasim.m...@wazuh.com: