Many Wazuh Agent disconnected

[sang thanh]

Hi everyone,

Did you guys ever get massive disconnected Wazuh Agents to Manager issue?

I have 100 agents in total, suddenly 70 of them turned to disconnected status, I have checked the Wazuh Manager and Wazuh Agent logs but none of them point the exact issue.

All those 70 disconnected agents in the same network sector with Manager without go through any firewall, ufw on Wazuh Manager was disabled from beginning.

I’m monitoring Manager server performance as well, all the stats is looking good, no anomaly was detected. The Agents success to reconnect after manual restart the agents on each endpoint, then after few days, they back to disconnected once again.

I also removed all of the disconnected agents and make them register with new agents IDs, but the problem still comes back.

Do you have any thoughts on that? Please let me know, I’m really really stuck with this.

Thanks for your help.

[Sebastian Dario Bustos]

Hi sang thanh,

Thank you for using Wazuh!!!

This may happen because of unavailability time of the manager being greater than 1 minute (manager under maintenance or minor network issuefor example), the agents attempt to re-connect for 5 times every 10 seconds and, if failed 5 times in a row, they will attempt to re-register to the manager indefinitely, causing this behavior, they will be seen as disconnected from the manager point of view (if you see the manager's logs on /var/ossec/logs/ossec.log you will notice several "duplicated agent name" failed registration attempts), if the agent is still disconnected (it did not re-register) you can restore the connectivity to the manager by restarting the service (on the agent), this will cause the retries count to reset and the agent to attempt to connect again to the manager.

To restart the agent service:

• Linux: /var/ossec/bin/wazuh-control restart or systemctl restart wazuh-agent
• Windows:
  
  • Powershell: Restart-Service -Name wazuh 
  • or from the win32ui interface found on the installation folder
• MacOS: /Library/Ossec/bin/wazuh-control restart

In order to prevent this issue, of the agents trying re-registering that fast, you can change agent parameters to increase the times the agent retries to connect and the interval between attempts.

These options need to be configured in the client section on the local config of each agent (can’t be distributed through centralized configuration), here are the options.

<max_retries>: The number of connection retries.
Default value: 5
Allowed values: 1 to 1000000000

<retry_interval>: Time interval between connection attempts (seconds).
Default value: 10
Allowed values: 1000000000

And this is a sample of how it may be configured:

<client>

  <server>

     <address>192.168.1.100</address>

     <port>1514</port>

     <protocol>tcp</protocol>

     <max_retries>100</max_retries>

     <retry_interval>15</retry_interval>

   </server>

   <server>

 ...

 </client>

Here is the documentation reference for the available options:

https://documentation.wazuh.com/current/user-manual/reference/ossec-conf/client.html#max-retries

(please change the version of the documentation on the top right corner to the one matching your environment/agent)

Hope this helps.

Regards.

[sang thanh]

Awesome, lets me try that out then response to you guys the result.

Here is the script I use to update the config on Linux for anyone wanna do it after me:

sudo cp /var/ossec/etc/ossec.conf /var/ossec/etc/ossec.conf.backup && sudo sed -i '/<protocol>tcp<\/protocol>/a \      <max_retries>100<\/max_retries>\n      <retry_interval>15<\/retry_interval>' /var/ossec/etc/ossec.conf && sudo systemctl restart wazuh-agent

Regards,

SangNguyen.

--
You received this message because you are subscribed to the Google Groups "Wazuh | Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/fa6ef8f3-b96a-414a-961e-fe4dc558e28dn%40googlegroups.com.