Wazuh Dashboard - constant API timeout

[liam zeb]

Hi.

I have recently dipped my toe into setting up a Wazuh instance.

Until recently, had few problems that I haven't been able to resolve, but since yesterday, every 5-20 mins, the Wazuh Dashboard seems to be unable to contact the API, thows into the health check which still cannot contact the API. After a few minutes, reconnects, to then start the cycle again.

During this time, the opensearch dashboards can still connect just fine, and I can manually connect into the API.

I have tried to search for some information, that could point me towards what I am doing incorrectly, but haven't been able to find much. Any advice or pointers would be greatly appreciated.

Thanks.

[Mateo James]

Hi Liam, thanks for using Wazuh!

Could you please share with me the following information?

- Version of Wazuh

- Architecture being used, and followed procedure of installation of each component

- Logs showing the relevant errors

Kind regards,

Mateo

[liam zeb]

Hi Mateo.

Thank you very much for responding.

- Version is 4.4.0 for manager, dashboard, indexer. Most of the agents still on 4.3.10

- Put together as a 8 node cluster. 1 specifically for the dashboard, 3 for indexer and 4 for manager. Installed everything using the Wazuh installation assistant. All installed fine, and worked for about 3 weeks without issue.

- attached screenshot from the /usr/share/wazuh-dashboard/data/wazuh/logs/wazuhapp.log, but doesn't give much to go off.

Many thanks.

[Mateo James]

Hi Liam, I see. And how many agents do you have deployed? It is complex as a 500 status code can be because of many things, so my recommendation is to first check the logs of the components you have deployed to find more useful information that can guide you in finding the root cause and troubleshoot it.

Best regards

[liam zeb]

Hi Mateo.

We currently have 450 agents.

Could I be a pain and ask which logs would be best to check? Or is it a case of just go through everything?

Many thanks,

[Mateo James]

Hi Liam,

I think you should go through your cluster to see where is the problem. I would start with the logs of the managers, then the indexers, and finally the dashboard.

Kind regards,

Mateo

[liam zeb]

Thank you very much for this advice Mateo.

Having checked the logs, nothing is standing out that might relate, except possibly a number of Errors  "ERROR: at run_worker(): OS_SendSecureTCP(125): Broken pipe (32)".

I have though stopped the agents on a number of machines, and it seems to have stabilised things, when keeping the number reporting back to around 300/350. 

All nodes in the cluster are at least at the recommended spec (I have them all with 16GB RAM, 8 core CPU).

Is it possible that this may be caused by too much coming in from the agents? For reference, I kept everything at the default, but can lower the reporting time/size from them, if it is believed it would help.

Thanks,

Liam.

[Mateo James]

Hi Liam,

I hope you are well. Excuse the delay on my response, I missed your last message.

I wanted to check if you were able to go through the cluster and if you eventually solved this problem. If I can help you with any issues you are having, please let me know.

Kind regards,

Mateo