Wodles sending conflicting events to Filebeat

[Louis Hather]

Hi all,

We're running a Wazuh v4.3.1 cluster with the AWS, and Azure Wodles, and GCP pubsub. All three of these sources occasionally result in illegal state exceptions as they're trying to map objects into keywords or vice versa. I believe this thread describes a similar problem: https://groups.google.com/g/wazuh/c/FkSiLPUcUtQ/m/6NJ0XmkkAQAJ

Azure active directory Graph initially dropped a large number of events, as instead of nesting its fields under data.gcp/data.aws like GCP and AWS do, events fields are mapped directly under data, like data.status. 

I think that particular case is easily addressed with a custom decoder, but we're also seeing the same when pulling logs from GCP and occasionally AWS.

Filebeat complaining about events from gcp:

{"type":"mapper_parsing_exception","reason":"object mapping for [data.gcp.protoPayload.request.instances] tried to parse field [null] as object, but found a concrete value"}, dropping event!

My worry is that we address individual conflicts, but it becomes an ongoing effort to catch and deal with mapping issues in perpetuity. Does anyone have any thoughts on how to deal with conflicts going forwards? 

[Octavio Valle López]

Hi Louis, I hope you are well!

From what I understand in the issue, the drawback is that you don't have the schema in your indexer, and this work must be done manually.

Could you tell me the modifications in the template (wazuh-template.json) and apart from an example of the element in the alert.json?

Thanks

[Louis Hather]

Hi Octavio,

We haven't made any adjustments to our template yet - it seems to me like mapping (for example) data.gcp.protoPayload.status as a string would break any ingestion where data.gcp.protoPayload.status is an object. This JSON is directly from GCP and handled by the JSON decoded, which as I understand is immutable and we're unable to use a custom decoder to make changes.

We only see this with alerts processed by the JSON decoder. Our (bad) workaround involves dropping the offending fields by changing the wodles code.