azure-logs wodle error/skipping the json decoder

[Andrej Smirnov]

Hello!

I am trying to parse Azure sign in logs to Wazuh with native Wazuh wodle called "azure-logs" using graph and as far as I am aware filebeat has issues in parsing sign in events:

2023 - 10 - 11T10: 15: 44.983Z WARN[elasticsearch]elasticsearch / client.go: 408 Cannot index event publisher.Event {
...
(status = 400): {
    "type": "mapper_parsing_exception",
    "reason": "failed to parse field [data.status] of type [keyword] in document with id 'oVY8HosBsfRe_QC-ONxK'. Preview of field's value: '{failureReason=Other., errorCode=0, additionalDetails=null}'",
    "caused_by": {
        "type": "illegal_state_exception",
        "reason": "Can't get text on a START_OBJECT at 1:1198"
    }
}

The same happens when I paste the json log to logtest:

**Phase 3: Completed filtering (rules).
        id: '1002'
        level: '2'
        description: 'Unknown problem somewhere in the system.'
        groups: '['syslog', 'errors']'
        firedtimes: '1'
        gpg13: '['4.3']'
        mail: 'False'

Logtest fails on status field of the log:

{"timestamp":"2023-10-11T10:15:40.705+0000","rule":{"level":3,"description":"Azure: AD ","id":"87802","firedtimes":1,"mail":false,"groups":["azure"]},"agent":{"id":"000","name":"<REDACTED>"},"manager":{"name":"<REDACTED>"},"id":"<REDACTED>","decoder":{"name":"json"},"data":{"id":"<REDACTED>","createdDateTime":"2023-10-11T10:06:02Z","userDisplayName":"<REDACTED>","userPrincipalName":"<REDACTED>@<REDACTED>","userId":"<REDACTED>","appId":"<REDACTED>","appDisplayName":"Azure Portal","ipAddress":"<REDACTED>","clientAppUsed":"Browser","correlationId":"<REDACTED>","conditionalAccessStatus":"notApplied","isInteractive":"true","riskDetail":"hidden","riskLevelAggregated":"hidden","riskLevelDuringSignIn":"hidden","riskState":"none","riskEventTypes":[],"riskEventTypes_v2":[],"resourceDisplayName":"Windows Azure Service Management API","resourceId":"<REDACTED>","status":{"errorCode":"0","failureReason":"Other.","additionalDetails":"null"},"deviceDetail":{"operatingSystem":"Windows10","browser":"<REDACTED>","isCompliant":"false","isManaged":"false"},"location":{"city":"<REDACTED>","state":"<REDACTED>","countryOrRegion":"<REDACTED>","geoCoordinates":{"altitude":"null","latitude":"<REDACTED>","longitude":"<REDACTED>"}},"appliedConditionalAccessPolicies":[],"azure_tag":"azure-ad-graph","azure_aad_tag":"azure-active_directory"},"location":"Azure"}

So I am thinking of creating a custom decoder that will match {"timestamp":"2023-10-11T10:15:40.705+0000","rule":{"level":3,"description":"Azure: AD "  and based on the rule that will match the description full log will be parsed as raw log to Wazuh.

But despite all the attempts that I made whenever I paste the log to logtest that starts with {"timestamp": it treats it as json and uses json decoder instead of my custom made decoder.

Is there a way to somehow configure the custom decoder so that wazuh would skip using json native decoder for log that particularly matches some string that I specify with regex?

Regards,

Andrejs

[Nicolas Zapata]

Hi  Andrej  thanks for using wazuh!

This error  is due to a conflict in your mapping.

If you take a look at the log it says failed to parse field [data.status] of type [keyword] [...]  "reason":"Can't get text on a START_OBJECT, which means that, at some point, one of the indexed events contained a field named data.status mapped as an object, but at a later point in time, a different event contained a field named data.status mapped as a keyword (a type of string).

The best way you can solve this is to understand which events include a field that is decoded as object under data.status and modify the decoder so it is stored under a different field (maybe data.status_object).

Another quicker workaround is to set the field as non-indexable, so that the alert will be visible on the dashboard (the field can be seen in the document), but it will not be searchable.

https://www.elastic.co/guide/en/elasticsearch/reference/7.10/enabled.html
https://www.elastic.co/guide/en/elasticsearch/reference/5.6/mapping-index.html

Regards!