Limit log line length in Log Collector

[Lambert Lim]

Hi,

I understand from here that the max line length for a single log event is 65279 bytes. Is there a way to change this into something smaller, like 1000 bytes?

The reason why I ask is we've got some custom logs, some of which have lines that are some 15000+ bytes long. I really don't need Wazuh reading lines that long, just the short lines about security events and other errors that are 1000 bytes or less long.

The simplest solution, I think, would be to just set a shorter line limit. If that's not possible, another solution I came up with is a custom decoder that will prematch​ those long lines and then discard them. I'm trying to write one now, though I'm not sure yet if the discarding part is possible.

Any other ideas would be greatly appreciated.

Thanks,

Lambert

[Lambert Lim]

It just occurred to me that a custom decoder works on the Wazuh Manager, but I want to discard those long lines at the Wazuh Agent, so a custom decoder ain't gonna' fly.

---

From: wa...@googlegroups.com <wa...@googlegroups.com> on behalf of Lambert Lim <ll...@quicksuitetrading.com>
Sent: Monday, July 31, 2023 3:06 PM
To: wa...@googlegroups.com <wa...@googlegroups.com>
Subject: Limit log line length in Log Collector

 

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/PR3P192MB0911AF1C873CF62C9C313F01C905A%40PR3P192MB0911.EURP192.PROD.OUTLOOK.COM.

[Miguel Casares]

Hello Lambert,

Wazuh will need to read the full log to properly assess the pattern recognition performed by the analysis engine. For that reason, there is no possibility to shorten the logs before reaching the Wazuh manager.

However, if you want to limit the full log output in the alert you can use the following option: <options>no_full_log</options>. For instance:

<rule id="87100" level="0">
    <decoded_as>json</decoded_as>
    <field name="integration">virustotal</field>
    <description>VirusTotal integration messages.</description>
    <options>no_full_log</options>
</rule>

Reference: https://documentation.wazuh.com/current/user-manual/ruleset/ruleset-xml-syntax/rules.html#rules-options

I hope this helps. Let me know if you need anything else,

Miguel

[Lambert Lim]

Hi Miguel,

That isn't really a solution to the problem I'm having. We're not getting these 15,000-byte logs in alerts, because we don't care about these logs and therefore we don't have any rules getting triggered by them.

The problem I'm having is that the Wazuh Agent uses a lot of unnecessary CPU because of reading these 15,000-byte lines, and I want to bring that CPU usage down. I'd also like to bring the network usage down, though I'm not having a problem there; just CPU on the endpoint.

Anyway, I've found a coming feature in Wazuh that might just be what I need, assuming I don't have to wait too long for it to arrive.

https://github.com/wazuh/wazuh/pull/14782

I guess I'll just have to wait for localfile​'s ignore​ and restrict​ options to become available.

Lambert

---

From: wa...@googlegroups.com <wa...@googlegroups.com> on behalf of Lambert Lim <ll...@quicksuitetrading.com>

Sent: Monday, July 31, 2023 3:08 PM
To: wa...@googlegroups.com <wa...@googlegroups.com>
Subject: Re: Limit log line length in Log Collector

 

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/PR3P192MB091157CD6A5E2EE30C483A91C905A%40PR3P192MB0911.EURP192.PROD.OUTLOOK.COM.

[Miguel Casares]

Hello Lambert,

If you want to discard the logs before reading them you have different options depending on the OS and the type of logs.

In Windows endpoints for Logcollector, you can filter the logs using queries: https://documentation.wazuh.com/current/user-manual/capabilities/log-data-collection/how-to-collect-wlogs.html#filtering-events-from-windows-event-channel-with-queries

In Linux endpoints for Logcollector, you will need to wait for this feature request: https://github.com/wazuh/wazuh/pull/14782.

For both OS and the rest of the Wazuh capabilities, you have different filtering options. For instance, FIM ignore: https://documentation.wazuh.com/current/user-manual/reference/ossec-conf/syscheck.html#ignore

I hope that helps. Let us know if you have further questions,

Miguel