Upgraded 4.7.5 -> 4.8.0, got ERROR3099 - Wazuh not ready yet

[dmitri]

Hello team, 

After upgrade from version 4.7.5 to 4.8.0 got this error:

Could not connect to API id [default]: 3099 - ERROR3099 - Wazuh not ready yet.
Please help.

[David Correa Rodriguez]

Hello. Maybe, the Wazuh API is down and the Wazuh dashboard can not connect to it. Please, follow this related documentation: https://documentation.wazuh.com/current/user-manual/wazuh-dashboard/troubleshooting.html#wazuh-api-seems-to-be-down.

Also, you can check the status of the Wazuh daemons by running `/var/ossec/etc/bin/ossec-control status` command in the Wazuh manager.

Also, could you try to open the Wazuh dashboard from an incognito window to determine if the problem is related to the cookies cache?

[dmitri]

Hello David, 

1. the Wazuh API is up: 

a)

./wazuh-control status
wazuh-clusterd not running...
wazuh-modulesd is running...
wazuh-monitord is running...
wazuh-logcollector is running...
wazuh-remoted is running...
wazuh-syscheckd is running...
wazuh-analysisd is running...
wazuh-maild not running...
wazuh-execd is running...
wazuh-db is running...
wazuh-authd is running...
wazuh-agentlessd not running...
wazuh-integratord is running...
wazuh-dbd not running...
wazuh-csyslogd not running...
wazuh-apid is running...

ALSO, did:
b)
telnet localhost 55000
Trying ::1...
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

c)
curl -k -X GET "https://localhost:55000/" -H "Authorization: Bearer $TOKEN"
{"data": {"title": "Wazuh API REST", "api_version": "4.8.0", "revision": 40812, "license_name": "GPL 2.0", "license_url": "https://github.com/wazuh/wazuh/blob/v4.8.0/LICENSE", "hostname": "ubuntu", "timestamp": "2024-06-17T08:18:11Z"}, "error": 0}

[David Correa Rodriguez]

So then, it seems that the Wazuh API is working fine. Could you please check what I mentioned in the previous message? Try to open the Wazuh dashboard from an incognito window to determine if the problem is related to the cookies cache.

You could try to open the Wazuh dashboard with another web browser too.

Hope it helps.

[dmitri]

I've opened in 3 different browsers: Chrome, Edge, Opera + in Incognito mode on each.
Is the same issue...

[dmitri]

here is the output of the ossec.log file:

2024/06/17 11:44:39 indexer-connector: WARNING: No username and password found in the keystore, using default values.
2024/06/17 11:44:40 indexer-connector: WARNING: IndexerConnector initialization failed for index 'wazuh-states-vulnerabilities-ubuntu', retrying until the connection is successful.

 

I don't know if this may be causing the problem....

[David Correa Rodriguez]

Ok, so the problem is related to the logs you shared. It seems that the Wazuh keystore does not have the username and password values saved, and the Wazuh indexer is in a failed status, that's why the Wazuh dashboard reports that "Wazuh is not ready yet". This message is common when the Wazuh indexer is not running.

About the error you shared, we have dedicated documentation to fix this problem: https://documentation.wazuh.com/current/upgrade-guide/troubleshooting.html#no-username-and-password-found-in-the-keystore

If you have upgraded to 4.8.0, I recommend you to follow every step detailed in the documentation in order to avoid further issues: https://documentation.wazuh.com/current/upgrade-guide/upgrading-central-components.html

[dmitri]

should I set up a new random username and password for Indexer? Or what should I put?

[David Correa Rodriguez]

No, you should not specify randon passwords. You must specify the username and password of the Wazuh indexer. This credentials change depending on the deployment you did, or if you changed the password with the Wazuh password tool.

If you did a step-by-step deployment and you didn't change the passwords, the credentials should be admin:admin, as it is specified in the documentation: https://documentation.wazuh.com/current/installation-guide/wazuh-server/step-by-step.html#configuring-the-wazuh-indexer-connection

Instead, if you deployed Wazuh using the Installation assistant, the password were changed automatically. If you made this deployment, you should have a `wazuh-install-files.tar` file in your system. If this is the case, you can get the Wazuh indexer password with this command: tar -axf wazuh-install-files.tar wazuh-install-files/wazuh-passwords.txt -O | grep -P "\'admin\'" -A 1.

Also, before inserting the credentials in the Wazuh keystore, I recommend you to check which password do you have by testing it in a curl command to the Wazuh indexer: curl -k -u admin:<ADMIN_PASSWORD> https://<WAZUH_INDEXER_IP>:9200/_cat/nodes?v.

Replace <ADMIN_PASSWORD> and <WAZUH_INDEXER_IP> values with your values.

[dmitri]

I accidentally added a random username&password in keystore.
I found the real username and password. Can I replace it with:
/var/ossec/bin/wazuh-keystore -f indexer -k username -v <INDEXER_USERNAME>
/var/ossec/bin/wazuh-keystore -f indexer -k password -v <INDEXER_PASSWORD>
??

[David Correa Rodriguez]

Yes, you can overwrite the introduced values by running the commands again:

/var/ossec/bin/wazuh-keystore -f indexer -k username -v <INDEXER_USERNAME>

/var/ossec/bin/wazuh-keystore -f indexer -k password -v <INDEXER_PASSWORD>.

Hope it helps.

[dmitri]

the error about username&password disappeared, but  IndexConnector still fails:

2024/06/17 13:15:19 indexer-connector: WARNING: IndexerConnector initialization failed for index 'wazuh-states-vulnerabilities-ubuntu', retrying until the connection is successful. 

The service looks like is all fine:

ossec.conf:

location of filebeat certs:

[dmitri]

I'm also getting these warnings:

............................................................

2024/06/17 14:41:52 indexer-connector: WARNING: Failed to sync agent '050' with the indexer.
2024/06/17 14:41:52 indexer-connector: WARNING: Failed to sync agent '018' with the indexer.
2024/06/17 14:41:52 indexer-connector: WARNING: Failed to sync agent '041' with the indexer.
2024/06/17 14:41:52 indexer-connector: WARNING: Failed to sync agent '078' with the indexer.
2024/06/17 14:41:52 indexer-connector: WARNING: Failed to sync agent '128' with the indexer.
2024/06/17 14:41:52 indexer-connector: WARNING: Failed to sync agent '029' with the indexer.
2024/06/17 14:41:52 indexer-connector: WARNING: Failed to sync agent '151' with the indexer.
2024/06/17 14:41:52 indexer-connector: WARNING: Failed to sync agent '048' with the indexer.
2024/06/17 14:41:52 indexer-connector: WARNING: Failed to sync agent '047' with the indexer.
2024/06/17 14:41:52 indexer-connector: WARNING: Failed to sync agent '036' with the indexer.
2024/06/17 14:41:52 indexer-connector: WARNING: Failed to sync agent '122' with the indexer.
2024/06/17 14:41:52 indexer-connector: WARNING: Failed to sync agent '064' with the indexer.
2024/06/17 14:41:52 indexer-connector: WARNING: Failed to sync agent '026' with the indexer.
2024/06/17 14:41:52 indexer-connector: WARNING: Failed to sync agent '005' with the indexer.
2024/06/17 14:41:52 indexer-connector: WARNING: Failed to sync agent '020' with the indexer.
2024/06/17 14:41:52 indexer-connector: WARNING: Failed to sync agent '002' with the indexer.
2024/06/17 14:41:52 indexer-connector: WARNING: Failed to sync agent '093' with the indexer.
2024/06/17 14:41:52 indexer-connector: WARNING: Failed to sync agent '009' with the indexer.
2024/06/17 14:41:52 indexer-connector: WARNING: Failed to sync agent '034' with the indexer.
2024/06/17 14:41:52 indexer-connector: WARNING: Failed to sync agent '042' with the indexer.
2024/06/17 14:41:52 indexer-connector: WARNING: Failed to sync agent '142' with the indexer.
2024/06/17 14:41:52 indexer-connector: WARNING: Failed to sync agent '118' with the indexer.
2024/06/17 14:41:52 indexer-connector: WARNING: Failed to sync agent '095' with the indexer.
2024/06/17 14:41:52 indexer-connector: WARNING: Failed to sync agent '141' with the indexer.
...........................................................

[dmitri]

on Check connection it shows that the Wazuh API is Offile: 

[dmitri]

/usr/share/wazuh-dashboard/data/wazuh/config/wazuh.yml

hosts:
  - default:
      url: https://localhost
      port: 55000
      username: wazuh-wui
      password: "<password>"
      run_as: false

[David Correa Rodriguez]

Hello again.

About the first issue you report, we have also a dedicated documentation in the Troubleshooting Upgrade guide: https://documentation.wazuh.com/current/upgrade-guide/troubleshooting.html#indexerconnector-initialization-failed

Make sure that the IP specified in the indexer block is correct. I noticed that it has specified the default IP, 0.0.0.0, which will probably not work. Replace that IP with the IP of the indexer and restart the Wazuh manager.

This is probably caused because you did not follow the documentation of the Upgrade guide: https://documentation.wazuh.com/current/upgrade-guide/upgrading-central-components.html. Please, next time, make sure you follow every step of the official documentation to avoid these issues.

Regards.

[Craig Szymanski]

I had the same errors. I just figured out that changing the url "https://localhost" to https://127.0.0.1 fixed my issue.

[dmitri]

that didnt work for me, so I just backed up all etc config files and reinstalled.