Wazuh Shuffle integration escape characters

[Yago GR]

Hi,

I'm desperate trying to find a solution but I can't find anything that works for my scenario.

First of all I'm using Wazuh 4.7.0 with the default config, nothing it's been parsed AFAIK.

I made an integration in Wazuh to send the alerts for some rules (via webhook to Shuffle and create an alert in DFIR-IRIS) in the file ossec.conf with something like this for the SYSMON events

  <integration>
      <name>custom-shuffle</name>
      <hook_url>XXXXXXXXXXXXXXXXXX</hook_url>
      <rule_id>550,553,554</rule_id>
      <alert_format>json</alert_format>
  </integration>

The problem is what Wazuh receives and what is forwarded to Shuffle is very different. In addition the worst thing happens with the windows agents, because the syslogs uses the backslashes, creating escape characters.

I would like to know what's going on and what to check because I've tried many solutions with no success.

Another possibility that I was thinking about it's to parse the logs in the source, in the Windows agent to replace all the backslashes \ for pipelines | but everything I've tested it's not working either.

This is the original JSON

{ "_index": "xxxxxxxxxxxxxxx", "_id": "ffcbf8a1-dfbb-11ee-9865-005056a4c64a", "_version": 1, "_score": null, "_source": { "agent_id": "005", "agent_name": "xxxxxxxxxxxxxxxxxx", "syscheck_audit_user_id": "S-1-5-21-3686419221-1061318842-3088484869-1113", "gl2_remote_ip": "xxxxxxxxxxxx", "gl2_remote_port": 34080, "rule_tsc": "PI1.4, PI1.5, CC6.1, CC6.8, CC7.2, CC7.3", "source": "xxxxxxxxxxxxxxx", "gl2_source_input": "6578f7605c42f570d936e74e", "rule_level": 7, "syscheck_audit_process_name": "C:\\Windows\\System32\\dllhost.exe", "syscheck_md5_after": "d41d8cd98f00b204e9800998ecf8427e", "rule_description": "File deleted.", "gl2_source_node": "d61a329c-c8d3-4124-94d3-9014dc978bb6", "id": "1710170899.392926447", "syscheck_attrs_after": "ARCHIVE", "rule_mitre_tactic": "Defense Evasion, Impact", "syscheck_mode": "whodata", "syscheck_path": "c:\\users\\test - copia (2).txt", "gl2_accounted_message_size": 4075, "streams": [ "657905bd5c42f570d9370568" ], "rule_mitre_id": "T1070.004, T1485", "gl2_message_id": "01HRQ2259A004VKRR12ZVS8AT1", "agent_ip": "xxxxxxxxxxxx", "syscheck_sha256_after": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "true": 1710170899.762828, "rule_hipaa": "164.312.c.1, 164.312.c.2", "rule_groups": "ossec, syscheck, syscheck_entry_deleted, syscheck_file", "syscheck_audit_user_name": "xxxxxxxxxxxxxxxx", "timestamp_cet": "2024-03-11T16:28:19.307+0100", "rule_gdpr": "II_5.1.f", "syscheck_win_perm_after": "{name=SYSTEM, allowed=[DELETE, READ_CONTROL, WRITE_DAC, WRITE_OWNER, SYNCHRONIZE, READ_DATA, WRITE_DATA, APPEND_DATA, READ_EA, WRITE_EA, EXECUTE, READ_ATTRIBUTES, WRITE_ATTRIBUTES]}, {name=Administradores, allowed=[DELETE, READ_CONTROL, WRITE_DAC, WRITE_OWNER, SYNCHRONIZE, READ_DATA, WRITE_DATA, APPEND_DATA, READ_EA, WRITE_EA, EXECUTE, READ_ATTRIBUTES, WRITE_ATTRIBUTES]}, {name=Usuarios, allowed=[READ_CONTROL, SYNCHRONIZE, READ_DATA, READ_EA, EXECUTE, READ_ATTRIBUTES]}, {name=Todos, allowed=[READ_CONTROL, SYNCHRONIZE, READ_DATA, READ_EA, EXECUTE, READ_ATTRIBUTES]}", "rule_mitre_technique": "File Deletion, Data Destruction", "rule_firedtimes": 1, "full_log": "File 'c:\\users\\test - copia (2).txt' deleted\nMode: whodata", "rule_mail": false, "rule_pci_dss": "11.5", "log_type": "wazuh", "rule_nist_800_53": "SI.7", "decoder_name": "syscheck_deleted", "timestamp": "2024-03-11 15:28:23.850", "syscheck_uname_after": "Administradores", "syscheck_event": "deleted", "gl2_processing_error": "Replaced invalid timestamp value in message <ffcbf8a1-dfbb-11ee-9865-005056a4c64a> with current time - Value <2024-03-11T16:28:19.307+0100> caused exception: Invalid format: \"2024-03-11T16:28:19.307+0100\" is malformed at \"T16:28:19.307+0100\".", "message": "{\"true\":1710170899.762828,\"timestamp\":\"2024-03-11T16:28:19.307+0100\",\"rule\":{\"level\":7,\"description\":\"File deleted.\",\"id\":\"553\",\"mitre\":{\"id\":[\"T1070.004\",\"T1485\"],\"tactic\":[\"Defense Evasion\",\"Impact\"],\"technique\":[\"File Deletion\",\"Data Destruction\"]},\"firedtimes\":1,\"mail\":false,\"groups\":[\"ossec\",\"syscheck\",\"syscheck_entry_deleted\",\"syscheck_file\"],\"pci_dss\":[\"11.5\"],\"gpg13\":[\"4.11\"],\"gdpr\":[\"II_5.1.f\"],\"hipaa\":[\"164.312.c.1\",\"164.312.c.2\"],\"nist_800_53\":[\"SI.7\"],\"tsc\":[\"PI1.4\",\"PI1.5\",\"CC6.1\",\"CC6.8\",\"CC7.2\",\"CC7.3\"]},\"agent\":{\"id\":\"005\",\"name\":\"STGCSMS-WS01\",\"ip\":\"xxxxxxxxxxxxxxx\"},\"manager\":{\"name\":\"xxxxxxxxxxxxx\"},\"id\":\"1710170899.392926447\",\"full_log\":\"File 'c:\\\\users\\\\test - copia (2).txt' deleted\\nMode: whodata\\n\",\"syscheck\":{\"path\":\"c:\\\\users\\\\test - copia (2).txt\",\"mode\":\"whodata\",\"size_after\":\"0\",\"win_perm_after\":[{\"name\":\"SYSTEM\",\"allowed\":[\"DELETE\",\"READ_CONTROL\",\"WRITE_DAC\",\"WRITE_OWNER\",\"SYNCHRONIZE\",\"READ_DATA\",\"WRITE_DATA\",\"APPEND_DATA\",\"READ_EA\",\"WRITE_EA\",\"EXECUTE\",\"READ_ATTRIBUTES\",\"WRITE_ATTRIBUTES\"]},{\"name\":\"Administradores\",\"allowed\":[\"DELETE\",\"READ_CONTROL\",\"WRITE_DAC\",\"WRITE_OWNER\",\"SYNCHRONIZE\",\"READ_DATA\",\"WRITE_DATA\",\"APPEND_DATA\",\"READ_EA\",\"WRITE_EA\",\"EXECUTE\",\"READ_ATTRIBUTES\",\"WRITE_ATTRIBUTES\"]},{\"name\":\"Usuarios\",\"allowed\":[\"READ_CONTROL\",\"SYNCHRONIZE\",\"READ_DATA\",\"READ_EA\",\"EXECUTE\",\"READ_ATTRIBUTES\"]},{\"name\":\"Todos\",\"allowed\":[\"READ_CONTROL\",\"SYNCHRONIZE\",\"READ_DATA\",\"READ_EA\",\"EXECUTE\",\"READ_ATTRIBUTES\"]}],\"uid_after\":\"S-1-5-32-544\",\"md5_after\":\"d41d8cd98f00b204e9800998ecf8427e\",\"sha1_after\":\"da39a3ee5e6b4b0d3255bfef95601890afd80709\",\"sha256_after\":\"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\",\"attrs_after\":[\"ARCHIVE\"],\"uname_after\":\"Administradores\",\"mtime_after\":\"2024-02-14T13:56:02\",\"event\":\"deleted\",\"audit\":{\"user\":{\"id\":\"S-1-5-21-3686419221-1061318842-3088484869-1113\",\"name\":\"xxxxxxxxxxxx\"},\"process\":{\"id\":\"15172\",\"name\":\"C:\\\\Windows\\\\System32\\\\dllhost.exe\"}}},\"decoder\":{\"name\":\"syscheck_deleted\"},\"location\":\"syscheck\"}", "rule_id": "553", "manager_name": "xxxxxxxxxxxxxxxxx", "syscheck_sha1_after": "da39a3ee5e6b4b0d3255bfef95601890afd80709", "syscheck_audit_process_id": "15172", "rule_gpg13": "4.11", "syscheck_uid_after": "S-1-5-32-544", "syscheck_size_after": "0", "location": "syscheck", "rule_group3": "syscheck_entry_deleted", "syscheck_mtime_after": "2024-02-14T13:56:02", "rule_group2": "syscheck", "rule_group1": "ossec" }, "fields": { "timestamp_cet": [ "2024-03-11T15:28:19.307Z" ], "syscheck_mtime_after": [ "2024-02-14T13:56:02.000Z" ], "timestamp": [ "2024-03-11T15:28:23.850Z" ] }, "highlight": { "rule_id": [ "@opensearch-dashboards-highlighted-field@553@/opensearch-dashboards-highlighted-field@" ] }, "sort": [ 1710170903850 ] }

And this is what the Shuffle's webhook receives

"Results for Execution Argument":{8 items
"severity":2
"pretext":"WAZUH Alert"
"title":"File added to the system."
"text":"File 'c:\users\test.txt' added Mode: whodata "
"rule_id":"554"
"timestamp":"2024-03-11T17:15:05.724+0100"
"id":"1710173705.412060679"
"all_fields":{9 items
"timestamp":"2024-03-11T17:15:05.724+0100"
"rule":{12 items
"level":5
"description":"File added to the system."
"id":"554"
"firedtimes":1
"mail":false
"groups":[4 items
0:
"ossec"
1:
"syscheck"
2:
"syscheck_entry_added"
3:
"syscheck_file"
]
"pci_dss":[1 item
0:
"11.5"
]
"gpg13":[1 item
0:
"4.11"
]
"gdpr":[1 item
0:
"II_5.1.f"
]
"hipaa":[2 items
0:
"164.312.c.1"
1:
"164.312.c.2"
]
"nist_800_53":[1 item
0:
"SI.7"
]
"tsc":[6 items
0:
"PI1.4"
1:
"PI1.5"
2:
"CC6.1"
3:
"CC6.8"
4:
"CC7.2"
5:
"CC7.3"
]
}
"agent":{3 items
"id":"005"
"name":"xxxxxxxxxxxxx"
"ip":"xxxxxxx"
}
"manager":{1 item
"name":"xxxxxxx"
}
"id":"1710173705.412060679"
"full_log":"File 'c:\users\test.txt' added Mode: whodata "
"syscheck":{13 items
"path":"c:\users\test.txt"
"mode":"whodata"
"size_after":"0"
"win_perm_after":[4 items
0:{2 items
"name":"SYSTEM"
"allowed":[13 items
0:
"DELETE"
1:
"READ_CONTROL"
2:
"WRITE_DAC"
3:
"WRITE_OWNER"
4:
"SYNCHRONIZE"
5:
"READ_DATA"
6:
"WRITE_DATA"
7:
"APPEND_DATA"
8:
"READ_EA"
9:
"WRITE_EA"
10:
"EXECUTE"
11:
"READ_ATTRIBUTES"
12:
"WRITE_ATTRIBUTES"
]
}
1:{2 items
"name":"Administradores"
"allowed":[13 items
0:
"DELETE"
1:
"READ_CONTROL"
2:
"WRITE_DAC"
3:
"WRITE_OWNER"
4:
"SYNCHRONIZE"
5:
"READ_DATA"
6:
"WRITE_DATA"
7:
"APPEND_DATA"
8:
"READ_EA"
9:
"WRITE_EA"
10:
"EXECUTE"
11:
"READ_ATTRIBUTES"
12:
"WRITE_ATTRIBUTES"
]
}
2:{2 items
"name":"Usuarios"
"allowed":[6 items
0:
"READ_CONTROL"
1:
"SYNCHRONIZE"
2:
"READ_DATA"
3:
"READ_EA"
4:
"EXECUTE"
5:
"READ_ATTRIBUTES"
]
}
3:{2 items
"name":"Todos"
"allowed":[6 items
0:
"READ_CONTROL"
1:
"SYNCHRONIZE"
2:
"READ_DATA"
3:
"READ_EA"
4:
"EXECUTE"
5:
"READ_ATTRIBUTES"
]
}
]
"uid_after":"S-1-5-32-544"
"md5_after":"d41d8cd98f00b204e9800998ecf8427e"
"sha1_after":"da39a3ee5e6b4b0d3255bfef95601890afd80709"
"sha256_after":"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
"attrs_after":[1 item
0:
"ARCHIVE"
]
"uname_after":"Administradores"
"mtime_after":"2024-02-14T13:56:02"
"event":"added"
"audit":{2 items
"user":{2 items
"id":"S-1-5-21-3686419221-1061318842-3088484869-1113"
"name":"xxxxxxxxxxxxxxxxxx"
}
"process":{2 items
"id":"15172"
"name":"C:\Windows\System32\dllhost.exe"
}
}
}
"decoder":{1 item
"name":"syscheck_new_entry"
}
"location":"syscheck"
}
}

Any help would be appreciated.

Thank you

[Rodolfo Arce Sannemann]

Hello Yago

There are two parts to your question.

The first part is the escaped content on the logs from Windows.

This is the expected behavior of the window event channel decoder and does not affect the underlying workings of the Wazuh decoders and alerts. The full Windows log has that content with escaped characters.



Regarding the Shuffle integration, the information sent to the Shuffle service and, hence, what is sent to IRIS is all managed by the /var/ossec/integrations/shuffle.py integration script, regardless of the content on the actual information obtained by the decoder for the Windows event channel. You must customize that script to send additional information to the Shuffle/IRIS or do it appropriately. The information you shared that was received by Shuffle is not escaped for example, you will notice that on the path name for the file:  "File 'c:\users\test.txt"

Regards.. R