After update wazuh-indexer not initialize in cluster mode

[Isaac S.]

Hello Wazuh Team

Recently i have updated a Wazuh cluster instalation with 3 nodes from 4.14.1 to 4.14.5. Two of then nodes update correctly and the services started correctly.

But the third one, didn't start correctly and when the service is restarted (systemctl restart wazuh-indexer) it crashed and there no log in /var/log/wazuh-indexer/wazuh-cluster.log

Wazuh version: 4.14.5

Last wazuh-cluster.log (there are not new logs even when the service is restarted)

[2026-06-03T12:20:12,195][WARN ][o.o.s.a.BackendRegistry  ] [node-3] Authentication finally failed for admin from 10.152.74.140:47222
[2026-06-03T12:20:12,195][WARN ][o.o.s.a.BackendRegistry  ] [node-3] Authentication finally failed for admin from 10.152.74.140:47224
[2026-06-03T12:20:12,195][WARN ][o.o.s.a.BackendRegistry  ] [node-3] Authentication finally failed for admin from 10.152.74.140:47244
[2026-06-03T12:20:14,274][WARN ][o.o.s.a.BackendRegistry  ] [node-3] Authentication finally failed for admin from 10.152.74.140:47258
[2026-06-03T12:22:45,074][INFO ][o.o.j.s.JobSweeper       ] [node-3] Running full sweep
[2026-06-03T12:23:46,143][INFO ][o.o.t.t.CronTransportAction] [node-3] Start running hourly cron.
[2026-06-03T12:23:46,143][INFO ][o.o.a.t.ADTaskManager    ] [node-3] Start to maintain running historical tasks
[2026-06-03T12:23:46,157][INFO ][o.o.m.j.JvmGcMonitorService] [node-3] [gc][G1 Concurrent GC][1735255][154381] duration [6.8s], collections [1]/[6.9s], total [6.8s]/[14.9m], memory [876.7mb]->[876.7mb]/[1gb], all_pools {[young] [2mb]->[3mb]/[0b]}{[old] [874.5mb]->[874.5mb]/[1gb]}{[survivor] [247.1kb]->[247.1kb]/[0b]}
[2026-06-03T12:23:46,158][WARN ][o.o.m.j.JvmGcMonitorService] [node-3] [gc][1735255] overhead, spent [6.8s] collecting in the last [6.9s]
[2026-06-03T12:27:45,074][INFO ][o.o.j.s.JobSweeper       ] [node-3] Running full sweep
[2026-06-03T12:28:23,323][INFO ][o.o.n.Node               ] [node-3] stopping ...
[2026-06-03T12:28:23,319][INFO ][o.o.s.a.r.AuditMessageRouter] [node-3] Closing AuditMessageRouter
[2026-06-03T12:28:23,337][INFO ][o.o.s.a.s.SinkProvider   ] [node-3] Closing DebugSink
[2026-06-03T12:28:23,374][INFO ][o.o.c.c.Coordinator      ] [node-3] cluster-manager node [{node-2}{HIhzfGxSSG2ogCweXhrbkQ}{2YKPdnDCR02HTlRoQHDlbQ}{10.152.74.139}{10.152.74.139:9300}{dimr}{shard_indexing_pressure_enabled=true}] failed, restarting discovery
org.opensearch.transport.NodeDisconnectedException: [node-2][10.152.74.139:9300][disconnected] disconnected
[2026-06-03T12:28:24,445][INFO ][o.o.n.Node               ] [node-3] stopped
[2026-06-03T12:28:24,446][INFO ][o.o.n.Node               ] [node-3] closing ...
[2026-06-03T12:28:24,490][INFO ][o.o.s.a.i.AuditLogImpl   ] [node-3] Closing AuditLogImpl
[2026-06-03T12:28:24,543][INFO ][o.o.n.Node               ] [node-3] closed

Isaac S.

[hasitha.u...@wazuh.com]

Hi Isaac,

The crash is occurring before the indexer process fully starts, which is why no new logs appear in wazuh-cluster.log after a restart. The issue needs to be traced from the OS and systemd level instead.

There are 3 problems visible in the logs, ordered by priority:
1. JVM heap exhaustion
Node-3 was using 876 MB out of a 1 GB heap limit right before it stopped. The GC pause of 6.8 seconds confirms the JVM was overwhelmed. When the process crashes at this level, nothing gets written to the application log, which explains the empty log file after restart.
Fix JVM heap if undersized:
# Edit /etc/wazuh-indexer/jvm.options
# Set these to 50% of node-3 RAM, e.g. for 8 GB RAM:
-Xms4g
-Xmx4g

2. Authentication failures
Just before the crash, node-2 attempted to connect to node-3 four times and was rejected each time. This typically happens after a partial upgrade where the security configuration on the upgraded node no longer matches what the rest of the cluster expects.
Check the indexer version if it's upgraded to the latest version.
apt list --installed wazuh-indexer 
or
yum list --installed wazuh-indexer

3. Node disconnection
This is not a separate root cause. Node-3 getting dropped from the cluster is a direct result of the two issues above. Fixing them will resolve this automatically.

Steps to take right now
Run these commands on node-3 and share the output here so the exact cause can be confirmed:
journalctl -u wazuh-indexer -n 500 --no-pager
systemctl status wazuh-indexer -l
dmesg | grep -i "killed\|oom\|out of memory" | tail -30
cat /etc/wazuh-indexer/jvm.options
free -h
 
The dmesg output is especially important here. If the Linux OOM killer terminated the process, it will show up there and confirm the heap issue as the primary cause.

Let me know the update on this.

Ref: https://documentation.wazuh.com/current/user-manual/wazuh-indexer/wazuh-indexer-tuning.html