Wazuh dashboard server is not ready yet

[Wforum Wforum]

Hi,
I have installed wazuh (v4.10.1) from scratch following https://documentation.wazuh.com/current/deployment-options/docker/wazuh-container.html#single-node-deployment in single node

I did not change anything in the docker-compose.yml

After bringing up the containers I get Wazuh dashboard server is not ready yet when I connect to the dashboard.

The logs are telling me

wazuh.manager-1    | 2025-01-17T19:01:49.120Z   WARN    [transport]     transport/tcp.go:52     DNS lookup failure "wazuh.indexer": lookup wazuh.indexer: no such host
wazuh.dashboard-1  | {"type":"log","@timestamp":"2025-01-17T19:01:50Z","tags":["error","opensearch","data"],"pid":53,"message":"[ConnectionError]: getaddrinfo ENOTFOUND wazuh.indexer"}
wazuh.dashboard-1  | {"type":"log","@timestamp":"2025-01-17T19:02:00Z","tags":["error","opensearch","data"],"pid":53,"message":"[ConnectionError]: connect ECONNREFUSED 172.18.0.2:9200"} wazuh.indexer-1 exited with code 137
wazuh.dashboard-1  | {"type":"log","@timestamp":"2025-01-17T19:02:18Z","tags":["error","opensearch","data"],"pid":53,"message":"[ConnectionError]: getaddrinfo ENOTFOUND wazuh.indexer"}
wazuh.dashboard-1  | {"type":"log","@timestamp":"2025-01-17T19:02:20Z","tags":["error","opensearch","data"],"pid":53,"message":"[ConnectionError]: getaddrinfo ENOTFOUND wazuh.indexer"}
wazuh.dashboard-1  | {"type":"log","@timestamp":"2025-01-17T19:02:23Z","tags":["error","opensearch","data"],"pid":53,"message":"[ConnectionError]: getaddrinfo ENOTFOUND wazuh.indexer"}
wazuh.dashboard-1  | {"type":"log","@timestamp":"2025-01-17T19:02:25Z","tags":["error","opensearch","data"],"pid":53,"message":"[ConnectionError]: getaddrinfo ENOTFOUND wazuh.indexer"}
wazuh.manager-1    | 2025-01-17T19:02:25.689Z   ERROR   [publisher_pipeline_output]     pipeline/output.go:154  Failed to connect to backoff(elasticsearch(https://wazuh.indexer:9200)): Get "https://wazuh.indexer:9200": lookup wazuh.indexer: no such host
wazuh.manager-1    | 2025-01-17T19:02:25.698Z   INFO    [publisher_pipeline_output]     pipeline/output.go:145  Attempting to reconnect to backoff(elasticsearch(https://wazuh.indexer:9200)) with 37 reconnect attempt(s)
wazuh.manager-1    | 2025-01-17T19:02:25.723Z   WARN    [transport]     transport/tcp.go:52     DNS lookup failure "wazuh.indexer": lookup wazuh.indexer: no such host
wazuh.dashboard-1  | {"type":"log","@timestamp":"2025-01-17T19:02:28Z","tags":["error","opensearch","data"],"pid":53,"message":"[ConnectionError]: getaddrinfo ENOTFOUND wazuh.indexer"}
wazuh.dashboard-1  | {"type":"log","@timestamp":"2025-01-17T19:02:30Z","tags":["error","opensearch","data"],"pid":53,"message":"[ConnectionError]: getaddrinfo ENOTFOUND wazuh.indexer"}
wazuh.dashboard-1  | {"type":"log","@timestamp":"2025-01-17T19:02:33Z","tags":["error","opensearch","data"],"pid":53,"message":"[ConnectionError]: getaddrinfo ENOTFOUND wazuh.indexer"}
wazuh.dashboard-1  | {"type":"log","@timestamp":"2025-01-17T19:02:35Z","tags":["error","opensearch","data"],"pid":53,"message":"[ConnectionError]: getaddrinfo ENOTFOUND wazuh.indexer"}

Do I overlook something?
Many thanks for the help

[hasitha.u...@wazuh.com]

Hi Wforum,

When you get "Wazuh dashboard server is not ready yet" error it normally indicates that the Wazuh dashboard cannot communicate with the indexer.

Let's investigate the issue about the Wazuh Dashboard:

Check that wazuh.indexer container is accessible and up and running
Make sure that your Wazuh-indexer service is up and running.
systemctl status wazuh-indexer

If not try restart the indexer and check the status.
systemctl restart wazuh-indexer

Then check the status again to see if the dashboard is accessible.
If the Wazuh-indexer is failing please share the indexer logs.
cat /var/log/wazuh-indexer/wazuh-cluster.log | grep -i -E "error|warn"

If the indexer is up and running,

Check if you have the right indexer IP/address and Dashboard certs in the Dashboard configuration file.
/etc/wazuh-dashboard/opensearch_dashboards.yml

Check that wazuh Indexer IP is updated in Wazuh dashboard configuration file opensearch_dashboards.yml
opensearch.hosts: https://<Wazuh-IndexerIP>:9200

Run this command to find the indexer IP
head /etc/wazuh-indexer/opensearch.yml

Run this command to check certificate names. Ensure the paths and filenames match in the configuration
ls -lrt /etc/wazuh-dashboard/certs/

And then restart the Wazuh dashboard service.
systemctl status wazuh-dashboard

Run this command to verify that your Dashboard service can communicate with the indexer service with kibanaserver user.
Need to run this command from Dashboard server
curl -XGET -k -u kibanaserver:pass "https://<Indexer_IP>:9200/_cluster/health"

If the curl request fails this output
curl: (7) Failed to connect to <ip> port 9200 after 0 ms: Connection refused
Check if there is any network connectivity blockage due to the firewall.

If you see no output or authentication error try changing the kibanaserver password.
To update the password for the kibanaserver user, simply changing the <KIBANASERVER_PASSWORD> in the Wazuh Dashboard keystore with the old password might not work. Here's what to do:
Run this command to change the password:
/usr/share/wazuh-indexer/plugins/opensearch-security/tools/wazuh-passwords-tool.sh -u kibanaserver -p '<new_password>'

Make sure the password is between 8 and 64 characters, and includes upper/lowercase letters, numbers, and a symbol (.*+?-).
If you using All in one, it automatically updates the passwords in the Wazuh dashboard node.

Wazuh dashboard node, run the following command to update the kibanaserver password in the Wazuh dashboard keystore. Replace <KIBANASERVER_PASSWORD> with the random password generated in the first step.
echo <KIBANASERVER_PASSWORD> | /usr/share/wazuh-dashboard/bin/opensearch-dashboards-keystore --allow-root add -f --stdin opensearch.password
Ref: https://documentation.wazuh.com/current/user-manual/user-administration/password-management.html

Make sure that if you make any changes to the configuration you need to restart that service to apply changes.

Issue still persists, please provide the output of below-mentioned commands and the above findings to investigate the issue further.
journalctl -u wazuh-dashboard
cat /usr/share/wazuh-dashboard/data/wazuh/logs/wazuhapp.log | grep -i -E "error|warn"

Let me know the update on this.

Regards,
Hasitha Upekshitha

[Wforum Wforum]

Hi,

I see that the wazuh.indexer container is trying to restart after a minute. In the wazuh.indexer logs I don't see any message why.

This is the log

WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/wazuh-indexer/lib/opensearch-2.16.0.jar)
WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
WARNING: System::setSecurityManager will be removed in a future release
Jan 20, 2025 6:41:02 AM sun.util.locale.provider.LocaleProviderAdapter <clinit>
WARNING: COMPAT locale provider will be removed in a future release
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/wazuh-indexer/lib/opensearch-2.16.0.jar)
WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
WARNING: System::setSecurityManager will be removed in a future release
[2025-01-20T06:41:05,561][INFO ][o.o.n.Node               ] [wazuh.indexer] version[2.16.0], pid[1], build[rpm/7149046c7c9c64aa43e437826af0b8b0dcabd730/2025-01-15T11:02:52.707428Z], OS[Linux/6.1.0-30-amd64/amd64], JVM[Eclipse Adoptium/OpenJDK 64-Bit Server VM/21.0.3/21.0.3+9-LTS]
[2025-01-20T06:41:05,586][INFO ][o.o.n.Node               ] [wazuh.indexer] JVM home [/usr/share/wazuh-indexer/jdk], using bundled JDK/JRE [true]
[2025-01-20T06:41:05,593][INFO ][o.o.n.Node               ] [wazuh.indexer] JVM arguments [-Xshare:auto, -Dopensearch.networkaddress.cache.ttl=60, -Dopensearch.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.security.manager=allow, -Djava.locale.providers=SPI,COMPAT, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/tmp/opensearch-6556558646036262468, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=/var/lib/wazuh-indexer, -XX:ErrorFile=/var/log/wazuh-indexer/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/wazuh-indexer/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.security.manager=allow, -Djava.util.concurrent.ForkJoinPool.common.threadFactory=org.opensearch.secure_sm.SecuredForkJoinWorkerThreadFactory, -Dclk.tck=100, -Djdk.attach.allowAttachSelf=true, -Djava.security.policy=file:///usr/share/wazuh-indexer/opensearch-performance-analyzer/opensearch_security.policy, --add-opens=jdk.attach/sun.tools.attach=ALL-UNNAMED, -Xms1g, -Xmx1g, -XX:MaxDirectMemorySize=536870912, -Dopensearch.path.home=/usr/share/wazuh-indexer, -Dopensearch.path.conf=/usr/share/wazuh-indexer, -Dopensearch.distribution.type=rpm, -Dopensearch.bundled_jdk=true]
[2025-01-20T06:41:06,455][WARN ][o.a.l.i.v.VectorizationProvider] [wazuh.indexer] Java vector incubator module is not readable. For optimal vector performance, pass '--add-modules jdk.incubator.vector' to enable Vector API.

And then the container restarts

[hasitha.u...@wazuh.com]

Hi  Wforum,

Let me know before installation, have you followed the requirement section properly to avoid any issues.
https://documentation.wazuh.com/current/deployment-options/docker/docker-installation.html#docker-installation

Wazuh indexer creates many memory-mapped areas. So you need to set the kernel to give a process at least 262,144 memory-mapped areas.
Follow this to set the memory mapped areas.
https://documentation.wazuh.com/current/deployment-options/docker/docker-installation.html#increase-max-map-count-on-your-host-linux

Next, you can follow the official guide to install Wazuh on docker.
https://documentation.wazuh.com/current/deployment-options/docker/wazuh-container.html

Let me know the update on this.