Wazuh dashboard server is not ready yet
Mushahid Bhat
Wazuh dashboard server is not ready yet
logs of wazuh dashboard
:"log","@timestamp":"2024-12-03T07:18:35Z","tags":["info","savedobjects-service"],"pid":12836,"message":"Detected mapping change in \"pr>
:"log","@timestamp":"2024-12-03T07:18:37Z","tags":["info","savedobjects-service"],"pid":12836,"message":"Detected mapping change in \"pr>
:"log","@timestamp":"2024-12-03T07:18:38Z","tags":["info","savedobjects-
My wazuh version
WAZUH_VERSION="v4.9.1"
WAZUH_REVISION="40914"
WAZUH_TYPE="server"
Kinldy shares steps how to fix
hasitha.u...@wazuh.com
Hi Mushahid,
When you get "Wazuh dashboard server is not ready yet" error it normally indicates that the Wazuh dashboard cannot communicate with the indexer.
Let's instigate the issue about the Wazuh Dashboard:
Make sure that your Wazuh-indexer services are up and running.
systemctl status wazuh-indexer
Check if you have right indexer IP/address and Dashboard certs in the Dashboard configuration file.
/etc/wazuh-dashboard/opensearch_dashboards.yml
Check that wazuh Indexer IP is updated in Wazuh dashboard configuration file opensearch_dashboards.yml
opensearch.hosts: https://<Wazuh-IndexerIP>:9200
Run this command to find the indexer IP
head /etc/wazuh-indexer/opensearch.yml
Run this command to check certificate names. Ensure the paths and filenames match in the configuration
ls -lrt /etc/wazuh-dashboard/certs/
And then restart the Wazuh dashboard service.
systemctl status wazuh-dashboard
Run this command to verify that your Dashboard service can communicate with the indexer service with kibanaserver user
Need to run this command from Dashboard server |
curl -XGET -k -u kibanaserver:pass "https://<Indexer_IP>:9200/_cluster/health"
If the curl request fails this output
curl: (7) Failed to connect to <ip> port 9200 after 0 ms: Connection refused
Check if there is any network connectivity blockage due to the firewall.
If you see no output or authentication error try changing the kibanaserver password.
To update the password for the kibanaserver user, simply changing the <KIBANASERVER_PASSWORD> in the Wazuh Dashboard keystore with the old password might not work. Here's what to do:
Run this command to change the password:
/usr/share/wazuh-indexer/plugins/opensearch-security/tools/wazuh-passwords-tool.sh -u kibanaserver -p '<new_password>'
Make sure the password is between 8 and 64 characters, and includes upper/lowercase letters, numbers, and a symbol (.*+?-).
If you using single node setup, it automatically updates the passwords in the Wazuh dashboard node.
In Wazuh dashboard node, run the following command to update the kibanaserver password in the Wazuh dashboard keystore. Replace <KIBANASERVER_PASSWORD> with the random password generated in the first step.
echo <KIBANASERVER_PASSWORD> | /usr/share/wazuh-dashboard/bin/opensearch-dashboards-keystore --allow-root add -f --stdin opensearch.password
Ref: https://documentation.wazuh.com/current/user-manual/user-administration/password-management.html
Issue still persists, please provide the output of below below-mentioned commands and the above findings to investigate the issue further.
journalctl -u wazuh-dashboard
cat /usr/share/wazuh-dashboard/data/wazuh/logs/wazuhapp.log | grep -i -E "error|warn"
cat /var/log/wazuh-indexer/wazuh-cluster.log | grep -i -E "error|warn"
Let me know the update for further assistance.
Regards,Hasitha Upekshitha
Mushahid Bhat
Make sure that your Wazuh-indexer services are up and running.
systemctl status wazuh-indexer
Check if you have right indexer IP/address and Dashboard certs in the Dashboard configuration file.
/etc/wazuh-dashboard/opensearch_dashboards.yml
Check that wazuh Indexer IP is updated in Wazuh dashboard configuration file opensearch_dashboards.yml opensearch.hosts: https://<Wazuh-IndexerIP>:9200
Run this command to find the indexer IP
head /etc/wazuh-indexer/opensearch.yml
Run this command to check certificate names. Ensure the paths and filenames match in the configuration
ls -lrt /etc/wazuh-dashboard/certs/
And then restart the Wazuh dashboard service.
systemctl status wazuh-dashboard
curl -XGET -k -u kibanaserver:***** "https://******9200/_cluster/health"
{"cluster_name":"wazuh-cluster","status":"green","timed_out":false,"number_of_nodes":1,"number_of_data_nodes":1,"discovered_master":true,"discovered_cluster_manager":true,"active_primary_shards":827,"active_shards":827,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"delayed_unassigned_shards":0,"number_of_pending_tasks":0,"number_of_in_flight_fetch":0,"task_max_waiting_in_queue_millis":0,"active_shards_percent_as_number":100.0}
What to do next as I am getting same error dashboard not ready
Mushahid Bhat
I am getting this
journalctl -u wazuh-dashboard
em"],"pid":1144,"message":"Setting up [48] plugins: [usageCollection,opensearchDashboardsUsageCollection,opensearchDashboardsLegacy,ma>
, please use options.freeSocketTimeout instead
, please use options.freeSocketTimeout instead
, please use options.freeSocketTimeout instead
, please use options.freeSocketTimeout instead
, please use options.freeSocketTimeout instead
, please use options.freeSocketTimeout instead
, please use options.freeSocketTimeout instead
, please use options.freeSocketTimeout instead
ts-service"],"pid":1144,"message":"Waiting until all OpenSearch nodes are compatible with OpenSearch Dashboards before starting saved ob>
h","data"],"pid":1144,"message":"[ConnectionError]: connect ECONNREFUSED ***.***.***.***:9200"}
cts-service"],"pid":1144,"message":"Unable to retrieve version information from OpenSearch nodes."}
h","data"],"pid":1144,"message":"[ConnectionError]: connect ECONNREFUSED ***.***.***.***:9200"}
h","data"],"pid":1144,"message":"[ConnectionError]: connect ECONNREFUSED ***.***.***.***:9200"}
h","data"],"pid":1144,"message":"[ConnectionError]: connect ECONNREFUSED ***.***.***.***:9200"}
h","data"],"pid":1144,"message":"[ResponseError]: Response Error"}
h","data"],"pid":1144,"message":"[ResponseError]: Response Error"}
h","data"],"pid":1144,"message":"[ResponseError]: Response Error"}
h","data"],"pid":1144,"message":"[ResponseError]: Response Error"}
h","data"],"pid":1144,"message":"[ResponseError]: Response Error"}
h","data"],"pid":1144,"message":"[ResponseError]: Response Error"}
ts-service"],"pid":1144,"message":"Starting saved objects migrations"}
h","data"],"pid":1144,"message":"[search_phase_execution_exception]: all shards failed"}
jects-service"],"pid":1144,"message":"Unable to connect to OpenSearch. Error: search_phase_execution_exception: "}
h","data"],"pid":1144,"message":"[search_phase_execution_exception]: all shards failed"}
h","data"],"pid":1144,"message":"[search_phase_execution_exception]: all shards failed"}
h","data"],"pid":1144,"message":"[search_phase_execution_exception]: all shards failed"}
h","data"],"pid":1144,"message":"[search_phase_execution_exception]: all shards failed"}
cat /usr/share/wazuh-dashboard/data/wazuh/logs/wazuhapp.log | grep -i -E "error|warn"
{"data":{"dapi_errors":{"node01":{"error":"Error retrieving data from Wazuh DB: There is no database for agent 009. Please check if the agent has connected to the manager"}},"detail":"Error retrieving data from Wazuh DB: There is no database for agent 009. Please check if the agent has connected to the manager","error":2007,"title":"Bad Request"},"date":"2024-10-10T04:53:33.206Z","level":"error","location":"wazuh-api:makeRequest"}
{"data":{"dapi_errors":{"node01":{"error":"Error retrieving data from Wazuh DB: There is no database for agent 009. Please check if the agent has connected to the manager"}},"detail":"Error retrieving data from Wazuh DB: There is no database for agent 009. Please check if the agent has connected to the manager","error":2007,"title":"Bad Request"},"date":"2024-10-10T06:25:51.287Z","level":"error","location":"wazuh-api:makeRequest"}
{"data":{"dapi_errors":{"node01":{"error":"Error retrieving data from Wazuh DB: There is no database for agent 009. Please check if the agent has connected to the manager"}},"detail":"Error retrieving data from Wazuh DB: There is no database for agent 009. Please check if the agent has connected to the manager","error":2007,"title":"Bad Request"},"date":"2024-10-10T06:25:51.311Z","level":"error","location":"wazuh-api:makeRequest"}
{"data":{"dapi_errors":{"node01":{"error":"Wazuh syntax error: Invalid element in the configuration: 'agent_config'. Configuration error at '/var/ossec/tmp/api_tmp_file_pdgsidwy.xml'. Syscheck remote configuration in '/var/ossec/tmp/api_tmp_file_pdgsidwy.xml' is corrupted."}},"detail":"Wazuh syntax error: Invalid element in the configuration: 'agent_config'. Configuration error at '/var/ossec/tmp/api_tmp_file_pdgsidwy.xml'. Syscheck remote configuration in '/var/ossec/tmp/api_tmp_file_pdgsidwy.xml' is corrupted.","error":1114,"title":"Bad Request"},"date":"2024-10-10T06:58:04.678Z","level":"error","location":"wazuh-api:makeRequest"}
{"data":{"detail":"'ctch policy' is not a 'names' - 'name'","title":"Bad Request"},"date":"2024-10-10T07:04:52.194Z","level":"error","location":"wazuh-api:makeRequest"}
{"date":"2024-10-10T12:54:00.404Z","level":"error","location":"queue:delayApiRequest","message":"An error ocurred in the delayed request: \"DELETE /security/user/authenticate\": Request failed with status code 401"}
{"date":"2024-10-10T12:54:00.416Z","level":"error","location":"queue:delayApiRequest","message":"An error ocurred in the delayed request: \"DELETE /security/user/authenticate\": Request failed with status code 401"}
[2024-12-03T15:23:16,267][WARN ][o.o.p.c.u.JsonConverter ] [node-1] Json Mapping Error: Cannot invoke "java.lang.Long.longValue()" because "this.cacheMaxSize" is null (through reference chain: org.opensearch.performanceanalyzer.collectors.CacheConfigMetricsCollector$CacheMaxSizeStatus["Cache_MaxSize"])
[2024-12-03T15:23:21,267][WARN ][o.o.p.c.u.JsonConverter ] [node-1] Json Mapping Error: Cannot invoke "java.lang.Long.longValue()" because "this.cacheMaxSize" is null (through reference chain: org.opensearch.performanceanalyzer.collectors.CacheConfigMetricsCollector$CacheMaxSizeStatus["Cache_MaxSize"])
[2024-12-03T15:23:26,268][WARN ][o.o.p.c.u.JsonConverter ] [node-1] Json Mapping Error: Cannot invoke "java.lang.Long.longValue()" because "this.cacheMaxSize" is null (through reference chain: org.opensearch.performanceanalyzer.collectors.CacheConfigMetricsCollector$CacheMaxSizeStatus["Cache_MaxSize"])
[2024-12-03T15:23:31,269][WARN ][o.o.p.c.u.JsonConverter ] [node-1] Json Mapping Error: Cannot invoke "java.lang.Long.longValue()" because "this.cacheMaxSize" is null (through reference chain: org.opensearch.performanceanalyzer.collectors.CacheConfigMetricsCollector$CacheMaxSizeStatus["Cache_MaxSize"])
hasitha.u...@wazuh.com
Let me know if this happened after the upgrade or suddenly, without making any changes.
Could you please share the output of these commands?
ls -lrt /etc/wazuh-dashboard/certs/
cat /etc/wazuh-dashboard/opensearch.yml
cat /etc/wazuh-indexer/opensearch.yml
Also, could you let me know your Wazuh-manager up and running?
systemctl status wazuh-manager
Also, share ossec.log of the Wazuh manager.
cat /var/ossec/logs/ossec.log | grep -i -E "error|warn|crit"
Because I have found this error "Wazuh syntax error: Invalid element in the configuration: 'agent_config'."
Therefore please share the agent.conf file to check the syntax error.
Regards,
Hasitha Upekshitha