HA Wazuh Master and Worker

[Ami92]

Hi all,

I deployed wazuh cluster and added some agents. I want to setup High Availability plan, so I used keepalived to HA wazuh dashboard and it is work fine. Now I want to HA my master and worker node. I checked it by power off the my master node, But it is not work. 

2020/04/06 15:44:58 wazuh-clusterd: ERROR: [Local Server] [Main] Could not connect to master. Trying again in 10 seconds.

2020/04/06 15:45:00 wazuh-clusterd: INFO: [Local 531263] [Main] Connection received in local server.

2020/04/06 15:45:00 wazuh-clusterd: ERROR: [Local 531263] [Main] Error processing request 'b'dapi'': Error 3023 - Worker node is not connected to master

2020/04/06 15:45:00 wazuh-clusterd: INFO: [Local 531263] [Main] Disconnected.

2020/04/06 15:45:00 wazuh-clusterd: INFO: [Local 702227] [Main] Connection received in local server.

2020/04/06 15:45:00 wazuh-clusterd: ERROR: [Local 702227] [Main] Error processing request 'b'dapi'': Error 3023 - Worker node is not connected to master

2020/04/06 15:45:00 wazuh-clusterd: INFO: [Local 702227] [Main] Disconnected.

2020/04/06 15:45:08 wazuh-clusterd: ERROR: [Local Server] [Main] Could not connect to master. Trying again in 10 seconds.

2020/04/06 15:45:18 wazuh-clusterd: ERROR: [Local Server] [Main] Could not connect to master. Trying again in 10 seconds.

Above mentioned logs are prompt in worker node,

I added all the agents to master.

Then added all of them to worker also.

Use this example method describes in wazuh documentation ,

<client>

    <server>

        <address>172.0.0.4</address>

        <port>1514</port>

        <protocol>udp</protocol>

    </server>

    <server>

        <address>172.0.0.5</address>

        <port>1514</port>

        <protocol>udp</protocol>

    </server>

    <config-profile>ubuntu, ubuntu18, ubuntu18.04</config-profile>

    <notify_time>10</notify_time>

    <time-reconnect>60</time-reconnect>

    <auto_restart>yes</auto_restart>

    <crypto_method>aes</crypto_method>

</client>

 .

It shows Error 3023 - Worker node is not connected to master in the wazuh dashboard also.(I use virtual ip to access dashboard).

Thank You'll.

[Daniel Ruiz]

Hi Ami92,

currently, Wazuh cluster does not support a true high availability mode. The nearest thing you can achieve is to set up a LB to connect all agents (https://documentation.wazuh.com/3.12/user-manual/configuring-cluster/advanced-settings.html#pointing-agents-to-the-cluster-with-a-load-balancer). It will allow agents to jump from one manager to another and carry on reporting.

However, keep in mind that the master node is the only one where you must register new agents (https://documentation.wazuh.com/3.12/user-manual/configuring-cluster/basics.html#types-of-nodes), so if this breaks, you won't be able to register any agents until it gets up again. It does not make sense adding agents to worker nodes as you say because the client.keys file is synchronized from master node to workers, so any changes performed in that file will be overwritten by the master's contents.

In addition, the distributed API won't work if the master node falls.

In the short term, we will be working to improve the cluster to provide true HA, but we are still designing and discussing the final solution.

I'm sorry for the inconveniences.

Do not hesitate to ask any other doubts.

Regards,

Daniel Ruiz

[Amila Sampath]

Hi Daniel,

Thank You very much for your quick response.

Best Regards,

Ami92.

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/32d7cbf1-8088-4ea0-961a-0651f0518da4%40googlegroups.com.

[Amila Sampath]

Hi Daniel,

I want to get more details. I attached my wazuh worker dashboard here. When I power off the mater my worker shows this. Can you please give some instructions.

Thank You.

[Daniel Ruiz]

Hi Ami92,

as I said in my previous answer, if the master is down, the distributed API won't work. Since the master is in charge of distributing API requests and is down you get a 3023 error. The distributed API relies heavily on the 'wazuh-clusterd' service being properly connected between master and worker nodes.

If you want more advanced details you can have a look to development documentation: https://documentation.wazuh.com/3.12/development/wazuh-cluster.html#distributed-api-requests. However, keep in mind that's a very technical documentation specially written for developers.

Regards,

Daniel Ruiz

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+unsubscribe@googlegroups.com.

[Amila Sampath]

Hi Daniel,

Thanks for quick response. I got the point. But I want to get the whole the idea, that is why I asked it again. I want get another thing from you, Is the agents who is report to master but secondary report to worker continues report to worker when master down??? If they report to worker continues, then master power on will they shift to master back??? And their data which stored in worker sift to master???

Thank You.

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/32d7cbf1-8088-4ea0-961a-0651f0518da4%40googlegroups.com.

--

You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/9111da49-80f7-4097-8ea7-6017503e13bd%40googlegroups.com.

[Daniel Ruiz]

Hi Ami92,

yes, as I said in my first answer, the agents continue reporting to the other worker. This happens thanks to the load balancer. Another great resource to help you understand the whole picture is our blog post: https://wazuh.com/blog/nginx-load-balancer-in-a-wazuh-cluster/.

The load balancer is configured with a sticky algorithm, that is, when an agent starts reporting a manager, it keeps assigned to that node until the connection breaks, normally because that node breaks down or is restarted. There is no migration of data from one node to another. Actually, we have just release 3.12 with a huge improvement in our syscheck service to make the agent keeping their own file integrity monitoring status, so false positives are not happening anymore.

The load balancer is the main responsible for balancing the number of agents reporting to each Wazuh manager.

I hope you have a clearer picture now.

Regards,

Daniel Ruiz

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/32d7cbf1-8088-4ea0-961a-0651f0518da4%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+unsubscribe@googlegroups.com.

[Amila Sampath]

Dear Daniel,

Thank you very much for your explanation. I got the points, Appreciate your help.

Thank You,

Ami92.

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/32d7cbf1-8088-4ea0-961a-0651f0518da4%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/9111da49-80f7-4097-8ea7-6017503e13bd%40googlegroups.com.

--

You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/05626f82-2d09-42e4-8f67-7886b3a47111%40googlegroups.com.

[Hrd]

Hi Daniel

in 4.5.2 edition, does wazuh cluster support HA mode?

when i shutdown master node i receive error 3002 from worker node.

Regards,

HamidReza Danehchin