New Server Node Not Receiving Logs from Agents

[Matthew M.]

I've recently added more nodes to our current cluster (5 indexer nodes from 3 and 4 worker nodes from 3) and I do not have logs flowing to our new worker node (worker-4).

I've added wazuh-4 to the nginx configuration on our load balancer:

stream {
    upstream cluster {
        hash $remote_addr consistent;
        server server1.siem.net:1514;
        server server2.siem.net:1514;
        server server3.siem.net:1514;
        server server4.siem.net:1514;
        #server server5.siem.net:1514;
        #server server6.siem.net:1514;
    }

I've waited several days and even started the cluster just to make sure, but I am not getting any log delivery to wazuh-4. How can I spread my connections across to wazuh-4?

[Federico Gustavo Galland]

Hi Matthew,

Have you reloaded the nginx configuration as explained here?

https://documentation.wazuh.com/current/user-manual/configuring-cluster/advanced-settings.html#pointing-agents-to-the-cluster-with-a-load-balancer

Anyway it would be nice to know if that Wazuh Manager node is actually running:

systemctl status wazuh-manager

systemctl status filebeat

filebeat test output

and probably look at its logs as well:

cat /var/ossec/logs/ossec.log

Another thing to double check is the /var/ossec/etc/ossec.conf on the 4th node. Is it equivalent to the others?

Let me know how this goes!

Regards,

Federico

[Matthew M.]

Federico,

I verified everything on the node is working when communicating to the rest of the cluster, but when looking at the ossec.log I get this error message:

2023/04/20 17:39:14 wazuh-authd: INFO: New connection from 10.150.0.107
2023/04/20 17:39:14 wazuh-authd: ERROR: Invalid password provided by 10.150.0.10                    7. Closing connection.

10.150.0.107 is the loadbalancer. Everything else is working so it's like the password set on this one worker node is different than the rest of the cluster. How do I go about resolving that?

Thanks,

Matthew

[Matthew M.]

I was able to figure out that password issue by looking at the password configurations on all of the other servers. That has now been handled. The ossec.conf matches on all of the worker nodes. Is there something else I should be checking. by looking at the alerts.json logs. I can verify that logs are flowing to worker-4, but they don't seem to be making their way into the cluster.

Thanks,

Matthew

[Matthew M.]

Okay, I solved this. For everyone else who runs into this. Federico was spot on when he had me double check those configuration files.

I had originally just copied the ossec.conf from other worker nodes and changed the pertinent information to reflect the new node so I knew it wasn't an issue.

After reviewing the ossec.log the original problem is I didn't have the same authd password on worker-4 that I have on all of the other workers. After adjusting this password I then wasn't getting any more errors from ossec.log, but I also wasn't getting any data from this worker node.

At that point I decided to check the filebeat logs (tail -f /var/log/filebeat/filebeat) and I was getting errors with the account I used for the new node having the proper permissions to write to the indices. After adjusting the permissions for this user in the Dashboard security field I was able to get the logs flowing.

[Federico Gustavo Galland]

Glad to know you fixed it!

Thanks for sharing the solution as well!

Regards,

Federico

--
You received this message because you are subscribed to a topic in the Google Groups "Wazuh mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/wazuh/csOpklk32-4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/d99e2b7b-c10d-40dc-a1fa-ffb362c667bfn%40googlegroups.com.