docker container logs

[Reza]

Hi
If the type of Docker logging-drive is JSON, then the logs will be in the following format:

{\"log\":\"192.168.132.1 - - [12/May/2023:14:12:14 +0000] \\\"GET /users/?id=SELECT+*+FROM+users HTTP/1.1\\\" 404 153 \\\"-\\\" \\\"curl/7.68.0\\\" \\\"-\\\"\\n\",\"stream\":\"stdout\",\"time\":\"2023-05-12T14:12:14.882936144Z\"}"

I want only the log part to be sent to the server for processing by decoders :

192.168.132.1 - - [12/May/2023:14:12:14 +0000] \\\"GET /users/?id=SELECT+*+FROM+users HTTP/1.1\\\" 404 153 \\\"-\\\" \\\"curl/7.68.0\\\" \\\"-\\\"\\n

How can I do it?

[Md. Nazmur Sakib]

Hi Reza,

Hope you are doing well. Thank you for using Wazuh.

By default, Docker container logs only show stdout and stderr standard streams, which are cleared when the container is destroyed. out_format  in localfile option allows formatting logs from Logcollector using field substitution.

Check this document for details:

https://documentation.wazuh.com/current/user-manual/reference/ossec-conf/localfile.html#out-format

I would also suggest you to check this document for some reference:

https://wazuh.com/blog/docker-container-security-monitoring-with-wazuh/

Also check these documents:

https://wazuh.com/blog/monitoring-docker-container-logs-with-wazuh/

https://documentation.wazuh.com/current/user-manual/capabilities/container-security/monitoring-docker.html

The easiest way to tackle this issue would be to write a new decoder based on your log format.

If you need help with writing decoder check this document:

https://documentation.wazuh.com/current/user-manual/ruleset/ruleset-xml-syntax/decoders.html

I hope this helps. Let me know the update on the issue.

Regards

Md. Nazmur Sakib

[Reza]

Hi
thanks
is it possible that i use following format for extract one field of json-like log

<localfile>
  <location>/path/to/docker/logs</location>
  <log_format>json</log_format>
  <out_format>$(log.log)</out_format>
</localfile>

that log is

{\"log\":\"192.168.132.1 - - [12/May/2023:14:12:14 +0000] \\\"GET /users/?id=SELECT+*+FROM+users HTTP/1.1\\\" 404 153 \\\"-\\\" \\\"curl/7.68.0\\\" \\\"-\\\"\\n\",\"stream\":\"stdout\",\"time\":\"2023-05-12T14:12:14.882936144Z\"}"

and log.log is:

192.168.132.1 - - [12/May/2023:14:12:14 +0000] \\\"GET /users/?id=SELECT+*+FROM+users HTTP/1.1\\\" 404 153 \\\"-\\\" \\\"curl/7.68.0\\\" \\\"-\\\"\\n

indeed i want select only one field of json log record

The reason for this is that field log of json texts are actually the main logs that are produced inside the containers by various services and programs such as nginx, and currently for the logs of these programs and services Various decoders have been written and if this can be done, it is not necessary to write duplicate decoders

Md. Nazmur Sakib در تاریخ دوشنبه ۱۳ نوامبر ۲۰۲۳ ساعت ۸:۲۳:۴۰ (UTC+3:30) نوشت:

[Md. Nazmur Sakib]

Hi  Reza,

I am not sure if using this will help tune your log with this local-file option.

  <out_format>$(log.log)</out_format>

You can alternatively tune your existing decoders with a Regular Expression wildcard. So that it skips the last part of the log while matching.

\.+ To match one or more times anything.

Check this document to learn more about regex syntax:

https://documentation.wazuh.com/current/user-manual/ruleset/ruleset-xml-syntax/regex.html

I hope this information helps.

Regards

Md. Nazmur Sakib

[Reza]

Hi

Is there no other way to do this other than writing duplicate decoders like:
Editing log lines by writing scripts and applying scripts on them before sending
Or
Writing a decoder and passing it to other decoders

Md. Nazmur Sakib در تاریخ دوشنبه ۱۳ نوامبر ۲۰۲۳ ساعت ۱۵:۵۵:۳۲ (UTC+3:30) نوشت:

[Md. Nazmur Sakib]

Hi Reza,

Hope you are doing well. Sorry for the late response.

Can you try changing the log format from JSON to  Syslog?

<localfile>

  <log_format>syslog</log_format>

  <location>/path/to/docker/logs</location>

</localfile>

And check if that makes any changes to the log.

You can also try to write a custom script to trim your log and save it in a new file. And monitor that new file with </localfile>

I do not have expatriates in scripting.

I am sharing a document for your reference:

https://unix.stackexchange.com/questions/572161/linux-command-to-trim-a-log-file

Regards

Md. Nazmur Sakib