Wazuh Manager Error wazuh-states-inventory-groups-wazuh_cluster

[Adrien Di Cristofaro]

Hello everyone.

I'm facing some insue on my main manager since i've updated Wazuh from 4.14.0 -> 4.14.3.

On the manager event logger : 

17/03/2026 10:27:33 indexer-connector WARNING  Document operation failed for index 'wazuh-states-inventory-groups-wazuh_cluster' - type: 'json_parse_exception', reason: 'Unrecognized character escape 'g' (code 103)
17/03/2026 10:27:33 indexer-connector ERROR  Client error, status code: 400, response body: {"error":{"root_cause":[{"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"}],"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"},"status":400}.
17/03/2026 10:27:31 indexer-connector WARNING  Document operation failed for index 'wazuh-states-inventory-groups-wazuh_cluster' - type: 'json_parse_exception', reason: 'Unrecognized character escape 'g' (code 103)
17/03/2026 10:27:31 indexer-connector ERROR  Client error, status code: 400, response body: {"error":{"root_cause":[{"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"}],"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"},"status":400}.
17/03/2026 10:27:30 indexer-connector WARNING  Document operation failed for index 'wazuh-states-inventory-groups-wazuh_cluster' - type: 'json_parse_exception', reason: 'Unrecognized character escape 'g' (code 103)
17/03/2026 10:27:30 indexer-connector ERROR  Client error, status code: 400, response body: {"error":{"root_cause":[{"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"}],"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"},"status":400}.
17/03/2026 10:27:28 indexer-connector WARNING  Document operation failed for index 'wazuh-states-inventory-groups-wazuh_cluster' - type: 'json_parse_exception', reason: 'Unrecognized character escape 'g' (code 103)
17/03/2026 10:27:28 indexer-connector ERROR  Client error, status code: 400, response body: {"error":{"root_cause":[{"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"}],"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"},"status":400}.
17/03/2026 10:27:28 indexer-connector WARNING  Document operation failed for index 'wazuh-states-inventory-users-wazuh_cluster' - type: 'json_parse_exception', reason: 'Unrecognized character escape 's' (code 115)
17/03/2026 10:27:28 indexer-connector ERROR  Client error, status code: 400, response body: {"error":{"root_cause":[{"type":"json_parse_exception","reason":"Unrecognized character escape 's' (code 115)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 84]"}],"type":"json_parse_exception","reason":"Unrecognized character escape 's' (code 115)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 84]"},"status":400}.
17/03/2026 10:27:26 indexer-connector WARNING  Document operation failed for index 'wazuh-states-inventory-groups-wazuh_cluster' - type: 'json_parse_exception', reason: 'Unrecognized character escape 'g' (code 103)
17/03/2026 10:27:26 indexer-connector ERROR  Client error, status code: 400, response body: {"error":{"root_cause":[{"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"}],"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"},"status":400}.
17/03/2026 10:27:25 indexer-connector WARNING  Document operation failed for index 'wazuh-states-inventory-groups-wazuh_cluster' - type: 'json_parse_exception', reason: 'Unrecognized character escape 'g' (code 103)
17/03/2026 10:27:25 indexer-connector ERROR  Client error, status code: 400, response body: {"error":{"root_cause":[{"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"}],"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"},"status":400}.
17/03/2026 10:27:23 indexer-connector WARNING  Document operation failed for index 'wazuh-states-inventory-groups-wazuh_cluster' - type: 'json_parse_exception', reason: 'Unrecognized character escape 'g' (code 103)
17/03/2026 10:27:23 indexer-connector ERROR  Client error, status code: 400, response body: {"error":{"root_cause":[{"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"}],"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"},"status":400}.
17/03/2026 10:27:22 indexer-connector WARNING  Document operation failed for index 'wazuh-states-inventory-users-wazuh_cluster' - type: 'json_parse_exception', reason: 'Unrecognized character escape 's' (code 115)
17/03/2026 10:27:22 indexer-connector ERROR  Client error, status code: 400, response body: {"error":{"root_cause":[{"type":"json_parse_exception","reason":"Unrecognized character escape 's' (code 115)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 84]"}],"type":"json_parse_exception","reason":"Unrecognized character escape 's' (code 115)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 84]"},"status":400}.
17/03/2026 10:27:21 indexer-connector WARNING  Document operation failed for index 'wazuh-states-inventory-groups-wazuh_cluster' - type: 'json_parse_exception', reason: 'Unrecognized character escape 'g' (code 103)
17/03/2026 10:27:21 indexer-connector ERROR  Client error, status code: 400, response body: {"error":{"root_cause":[{"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"}],"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"},"status":400}.
17/03/2026 10:27:20 indexer-connector WARNING  Document operation failed for index 'wazuh-states-inventory-groups-wazuh_cluster' - type: 'json_parse_exception', reason: 'Unrecognized character escape 'g' (code 103)
17/03/2026 10:27:20 indexer-connector ERROR  Client error, status code: 400, response body: {"error":{"root_cause":[{"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"}],"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"},"status":400}.

Enabling debug level 2, got this flood wich seems related to groups name i have in our AD : 

17/03/2026 10:25:23 indexer-connector DEBUG Added document for insertion with id: 191_IREDOM\g^sgdd. 17/03/2026 10:25:23 indexer-connector DEBUG Added document for insertion with id: 191_IREDOM\dl^maintenance^manager. 17/03/2026 10:25:23 indexer-connector DEBUG Added document for insertion with id: 191_IREDOM\g^store. 17/03/2026 10:25:23 indexer-connector DEBUG Added document for insertion with id: 191_IREDOM\a^r&d. 17/03/2026 10:25:23 indexer-connector DEBUG Added document for insertion with id: 191_IREDOM\g^comecer^rw. 17/03/2026 10:25:23 indexer-connector DEBUG Added document for insertion with id: 191_IREDOM\a^cr^rejets. 17/03/2026 10:25:23 indexer-connector DEBUG Added document for insertion with id: 191_IREDOM\helpservicesgroup. 17/03/2026 10:25:23 indexer-connector DEBUG Added document for insertion with id: 191_IREDOM\mail^quality^control. 17/03/2026 10:25:23 indexer-connector DEBUG Added document for insertion with id: 191_IREDOM\g^responsible^monitoring. 17/03/2026 10:25:23 indexer-connector DEBUG Added document for insertion with id: 191_IREDOM\dl^vpn^industrial^limited. 17/03/2026 10:25:23 indexer-connector DEBUG Added document for insertion with id: 191_IREDOM\dl^irelab. 17/03/2026 10:25:23 indexer-connector DEBUG Added document for insertion with id: 191_IREDOM\g^full^supervision. 17/03/2026 10:25:23 indexer-connector DEBUG Added document for insertion with id: 191_IREDOM\g^safety^public. 17/03/2026 10:25:23 indexer-connector DEBUG Added document for insertion with id: 191_IREDOM\g^sgdd^public. 17/03/2026 10:25:23 indexer-connector DEBUG Added document for insertion with id: 191_IREDOM\groupe^projets^spec. 17/03/2026 10:25:23 indexer-connector DEBUG Added document for insertion with id: 191_IREDOM\g^comecer^ro. 17/03/2026 10:25:23 indexer-connector DEBUG Added document for insertion with id: 191_IREDOM\g^metrology^old. 17/03/2026 10:25:23 indexer-connector DEBUG Added document for insertion with id: 191_IREDOM\g^sem^service. 17/03/2026 10:25:23 indexer-connector DEBUG Added document for insertion with id: 191_IREDOM\g^accounting. 17/03/2026 10:25:23 indexer-connector DEBUG Added document for insertion with id: 191_IREDOM\cr. 17/03/2026 10:25:23 indexer-connector DEBUG Added document for insertion with id: 191_IREDOM\cq. 17/03/2026 10:25:23 indexer-connector DEBUG Added document for insertion with id: 191_IREDOM\g^spectrometry. 17/03/2026 10:25:23 indexer-connector DEBUG Added document for insertion with id: 191_IREDOM\g^finance^service. 17/03/2026 10:25:23 indexer-connector DEBUG Added document for insertion with id: 191_IREDOM\a^chefs^de^projets. 17/03/2026 10:25:23 indexer-connector DEBUG Added document for insertion with id: 191_IREDOM\dl^sgdd^public. 17/03/2026 10:25:23 indexer-connector DEBUG Added document for insertion with id: 191_IREDOM\compta.

Any Idea how to fix this issue ?

Everything seems fine, cluster status is green,  all shards ok.

I'm running 3 manager (1 master, 2 slaves) ,1 indexer, 1 dashboard

Filebeat test output is OK on every manager.

I can provide more logs if needed.

Br,

Adrien

[Stuti Gupta]

Hi Adrien,

Please allow me some time. I'm looking into this.

Thank you!

[Stuti Gupta]

Hi

This is a known issue fixed in Wazuh (https://github.com/wazuh/wazuh/pull/33464), where a malformed document gets stuck in the indexerConnector queue, causing the error to loop even after the DB was fixed.

This PR adds escaping for document IDs before sending them to the indexer. However, in your case, the problem persists. This can be because the malformed document was already stored in the queue before the fix, or is not fully covered by that change.

There is also another issue (#34322) in the same release cycle that shows multiple string-handling problems, which suggests there are still edge cases not fully handled:
https://github.com/wazuh/wazuh/issues/34322

As a workaround, you can clear the stuck documents from the queue. This works because the error is caused by the queued document being retried repeatedly:

# Stop Wazuh manager
sudo systemctl stop wazuh-manager

# Remove affected inventory queues
sudo rm -rf /var/ossec/queue/indexer/wazuh-states-inventory-users-*
sudo rm -rf /var/ossec/queue/indexer/wazuh-states-inventory-groups-*

# Start Wazuh manager
sudo systemctl start wazuh-manager

After this, the queue is rebuilt, and the malformed document is removed, so the error should stop.

We are also discussing this internally with the team, since this behaviour should be handled automatically and not require manual cleanup. Will let you know if there is any progress. 

Thank you!

[Adrien Di Cristofaro]

Ok.

Do you want me to apply the fix ?

Or do you need me to provide more logs in order to help you fix the issue ?

Br,

Adrien 

[Stuti Gupta]

No need to apply the fix manually, and no additional logs are required at this point. We have identified the issue internally.

While PR #33464 fixes escaping for insert operations, it does not cover delete operations. We have confirmed that this is the cause in your case, and a separate fix has already been opened for it. https://github.com/wazuh/wazuh/issues/35029

Regarding the workaround: clearing the queue helps because the malformed document is already stored locally and retried. Removing the queue deletes that stuck entry, which may fix the issue.
However, this does not fix the root cause, only the current state.

For now, you can apply the workaround to stop the errors. We will keep you updated once the fix for delete operations is available.

The workaround is :

# Stop Wazuh manager
sudo systemctl stop wazuh-manager

# Remove affected inventory queues
sudo rm -rf /var/ossec/queue/indexer/wazuh-states-inventory-users-*
sudo rm -rf /var/ossec/queue/indexer/wazuh-states-inventory-groups-*

# Start Wazuh manager
sudo systemctl start wazuh-manager

Thank you!

[Adrien Di Cristofaro]

Hello,

Thanks for sharing the informations.

I'll apply the workaround.

Br,

Adrien

[Stuti Gupta]

Hi Adrien

We have opened the issue here https://github.com/wazuh/wazuh/issues/35029?reload=1#top regarding the same. You can please keep track of the progress. Hopefully, it will be resolved in 4.14.5. 

Thank you!

[Adrien Di Cristofaro]

Hello ;)

Just for you to know, i don't know if this is rellevant but error is back since a few days, even after the commands : 

02/04/2026 13:22:43 indexer-connector WARNING Document operation failed for index 'wazuh-states-inventory-groups-wazuh_cluster' - type: 'json_parse_exception', reason: 'Unrecognized character escape 'g' (code 103) 02/04/2026 13:22:43 indexer-connector ERROR Client error, status code: 400, response body: {"error":{"root_cause":[{"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"}],"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"},"status":400}. 02/04/2026 13:22:41 indexer-connector WARNING Document operation failed for index 'wazuh-states-inventory-groups-wazuh_cluster' - type: 'json_parse_exception', reason: 'Unrecognized character escape 'g' (code 103) 02/04/2026 13:22:41 indexer-connector ERROR Client error, status code: 400, response body: {"error":{"root_cause":[{"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"}],"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"},"status":400}. 02/04/2026 13:22:39 indexer-connector WARNING Document operation failed for index 'wazuh-states-inventory-groups-wazuh_cluster' - type: 'json_parse_exception', reason: 'Unrecognized character escape 'g' (code 103) 02/04/2026 13:22:39 indexer-connector ERROR Client error, status code: 400, response body: {"error":{"root_cause":[{"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"}],"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"},"status":400}. 02/04/2026 13:22:37 indexer-connector WARNING Document operation failed for index 'wazuh-states-inventory-groups-wazuh_cluster' - type: 'json_parse_exception', reason: 'Unrecognized character escape 'g' (code 103) 02/04/2026 13:22:37 indexer-connector ERROR Client error, status code: 400, response body: {"error":{"root_cause":[{"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"}],"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"},"status":400}. 02/04/2026 13:22:36 indexer-connector WARNING Document operation failed for index 'wazuh-states-inventory-groups-wazuh_cluster' - type: 'json_parse_exception', reason: 'Unrecognized character escape 'g' (code 103) 02/04/2026 13:22:36 indexer-connector ERROR Client error, status code: 400, response body: {"error":{"root_cause":[{"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"}],"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"},"status":400}. 02/04/2026 13:22:34 indexer-connector WARNING Document operation failed for index 'wazuh-states-inventory-groups-wazuh_cluster' - type: 'json_parse_exception', reason: 'Unrecognized character escape 'g' (code 103) 02/04/2026 13:22:34 indexer-connector ERROR Client error, status code: 400, response body: {"error":{"root_cause":[{"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"}],"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"},"status":400}. 02/04/2026 13:22:32 indexer-connector WARNING Document operation failed for index 'wazuh-states-inventory-groups-wazuh_cluster' - type: 'json_parse_exception', reason: 'Unrecognized character escape 'g' (code 103) 02/04/2026 13:22:32 indexer-connector ERROR Client error, status code: 400, response body: {"error":{"root_cause":[{"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"}],"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"},"status":400}. 02/04/2026 13:22:30 indexer-connector WARNING Document operation failed for index 'wazuh-states-inventory-groups-wazuh_cluster' - type: 'json_parse_exception', reason: 'Unrecognized character escape 'g' (code 103) 02/04/2026 13:22:30 indexer-connector ERROR Client error, status code: 400, response body: {"error":{"root_cause":[{"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"}],"type":"json_parse_exception","reason":"Unrecognized character escape 'g' (code 103)\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 85]"},"status":400}.

I'll live with it for now.

Br,

Adrien