Timeout of 20000ms exceeded

[Rei Gjata]

Hello, I am getting these type of errors after setting up the Wazuh 4.8 on Ubuntu Server 22.04
It stays a little bit like these and than the API connects successfully if i retry. 
But it will continue to disconnect and connect all the time.
Any idea why these may came from?

[manuel...@wazuh.com]

Hi Rei! In order to properly troubleshoot could you please share with us the following log files found in the manager:

• /var/ossec/logs/ossec.log
• /var/ossec/logs/api.log

Also would you mind sharing with us please the master configuration alongside the API config?

Kind regards,

Manuel

[Rei Gjata]

Hi Manuel
I forgot to mention that this is a AIO setup

NOTE: The error of API does not happen all the time. It happens only when i login at the dashboard. As soon as I retry the API comes back online.
How can i exceed the Timeout time?

. /var/ossec/logs/api.log

2024/09/10 10:39:44 INFO: wazuh-wui 127.0.0.1 "GET /sca/001" with parameters {"offset": "0", "limit": "15", "sort": "+policy_id"} and body {} done in 0.612s: 200
2024/09/10 10:39:44 INFO: wazuh-wui 127.0.0.1 "GET /sca/001/checks/cis_win11_enterprise_21H2" with parameters {"offset": "0", "limit": "10", "sort": "+id"} and body {} done in 0.449s: 200
2024/09/10 10:39:44 INFO: wazuh-wui 127.0.0.1 "GET /cluster/status" with parameters {} and body {} done in 0.303s: 200
2024/09/10 10:39:44 INFO: wazuh-wui 127.0.0.1 "GET /cluster/status" with parameters {} and body {} done in 0.229s: 200
2024/09/10 10:39:44 INFO: wazuh-wui 127.0.0.1 "GET /manager/configuration/request/remote" with parameters {} and body {} done in 0.136s: 200
2024/09/10 10:39:45 INFO: wazuh-wui 127.0.0.1 "GET /manager/configuration/auth/auth" with parameters {} and body {} done in 0.162s: 200
2024/09/10 10:39:45 INFO: wazuh-wui 127.0.0.1 "GET /groups" with parameters {} and body {} done in 0.148s: 200
2024/09/10 10:40:01 INFO: wazuh-wui 127.0.0.1 "GET /manager/stats/remoted" with parameters {} and body {} done in 0.305s: 200
2024/09/10 10:40:01 INFO: wazuh-wui 127.0.0.1 "GET /manager/stats/analysisd" with parameters {} and body {} done in 0.164s: 200
2024/09/10 10:40:32 INFO: wazuh-wui 127.0.0.1 "GET /security/users/me/policies" with parameters {} and body {"idHost": "default"} done in 1.416s: 200
2024/09/10 10:40:32 INFO: wazuh-wui 127.0.0.1 "POST /security/user/authenticate" with parameters {} and body {} done in 4.483s: 200
2024/09/10 10:40:32 INFO: wazuh-wui 127.0.0.1 "GET /" with parameters {} and body {} done in 0.184s: 200
2024/09/10 10:40:33 INFO: wazuh-wui 127.0.0.1 "GET /manager/info" with parameters {} and body {} done in 0.813s: 200
2024/09/10 10:40:33 INFO: wazuh-wui 127.0.0.1 "GET /cluster/status" with parameters {} and body {} done in 0.509s: 200
2024/09/10 10:40:33 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"agents_list": "000"} and body {} done in 0.256s: 200
2024/09/10 10:40:49 INFO: wazuh-wui 127.0.0.1 "GET /cluster/status" with parameters {} and body {} done in 14.673s: 200
2024/09/10 10:40:52 INFO: wazuh-wui 127.0.0.1 "POST /security/user/authenticate" with parameters {} and body {} done in 17.412s: 200
2024/09/10 10:40:52 INFO: wazuh-wui 127.0.0.1 "POST /security/user/authenticate" with parameters {} and body {} done in 15.346s: 200
2024/09/10 10:40:52 INFO: wazuh-wui 127.0.0.1 "POST /security/user/authenticate" with parameters {} and body {} done in 15.423s: 200
2024/09/10 10:40:52 INFO: wazuh-wui 127.0.0.1 "POST /security/user/authenticate" with parameters {} and body {} done in 15.501s: 200
2024/09/10 10:40:56 INFO: wazuh-wui 127.0.0.1 "GET /security/users/me/policies" with parameters {} and body {"idHost": "default"} done in 3.897s: 200
2024/09/10 10:41:05 INFO: wazuh-wui 127.0.0.1 "POST /security/user/authenticate" with parameters {} and body {} done in 12.798s: 200
2024/09/10 10:41:05 INFO: wazuh-wui 127.0.0.1 "POST /security/user/authenticate" with parameters {} and body {} done in 12.847s: 200
2024/09/10 10:41:05 INFO: wazuh-wui 127.0.0.1 "POST /security/user/authenticate" with parameters {} and body {} done in 12.894s: 200
2024/09/10 10:41:05 INFO: wazuh-wui 127.0.0.1 "DELETE /security/user/authenticate" with parameters {} and body {} done in 13.144s: 200
2024/09/10 10:41:05 INFO: unknown_user 127.0.0.1 "GET /manager/info" with parameters {} and body {} done in 0.082s: 401
2024/09/10 10:41:09 INFO: unknown_user 127.0.0.1 "GET /manager/info" with parameters {} and body {} done in 4.465s: 401
2024/09/10 10:41:10 INFO: wazuh-wui 127.0.0.1 "GET /manager/info" with parameters {} and body {} done in 5.526s: 200
2024/09/10 10:41:10 INFO: wazuh-wui 127.0.0.1 "POST /security/user/authenticate" with parameters {} and body {} done in 4.642s: 200
2024/09/10 10:41:14 INFO: unknown_user 127.0.0.1 "GET /" with parameters {} and body {} done in 2.445s: 401
2024/09/10 10:41:15 INFO: unknown_user 127.0.0.1 "GET /agents/summary/status" with parameters {} and body {} done in 2.518s: 401
2024/09/10 10:41:18 INFO: wazuh-wui 127.0.0.1 "POST /security/user/authenticate" with parameters {} and body {} done in 8.420s: 200
2024/09/10 10:41:18 INFO: wazuh-wui 127.0.0.1 "GET /manager/info" with parameters {} and body {} done in 5.997s: 200
2024/09/10 10:41:18 INFO: wazuh-wui 127.0.0.1 "GET /manager/info" with parameters {} and body {} done in 6.002s: 200
2024/09/10 10:41:18 INFO: wazuh-wui 127.0.0.1 "GET /manager/version/check" with parameters {"force_query": "false"} and body {} done in 6.008s: 200
2024/09/10 10:41:19 INFO: unknown_user 127.0.0.1 "DELETE /security/user/authenticate" with parameters {} and body {} done in 1.868s: 401
2024/09/10 10:41:26 INFO: wazuh-wui 127.0.0.1 "POST /security/user/authenticate" with parameters {} and body {} done in 13.992s: 200
2024/09/10 10:41:26 INFO: wazuh-wui 127.0.0.1 "GET /manager/info" with parameters {} and body {} done in 9.335s: 200
2024/09/10 10:41:26 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"agents_list": "000"} and body {} done in 9.344s: 200
2024/09/10 10:41:26 INFO: wazuh-wui 127.0.0.1 "POST /security/user/authenticate" with parameters {} and body {} done in 9.434s: 200
2024/09/10 10:41:26 INFO: wazuh-wui 127.0.0.1 "POST /security/user/authenticate" with parameters {} and body {} done in 9.584s: 200
2024/09/10 10:41:31 INFO: unknown_user 127.0.0.1 "GET /cluster/status" with parameters {} and body {} done in 4.388s: 401
2024/09/10 10:41:34 INFO: wazuh-wui 127.0.0.1 "POST /security/user/authenticate" with parameters {} and body {} done in 7.972s: 200
2024/09/10 10:41:35 INFO: unknown_user 127.0.0.1 "DELETE /security/user/authenticate" with parameters {} and body {} done in 1.752s: 401
2024/09/10 10:41:35 INFO: wazuh-wui 127.0.0.1 "POST /security/user/authenticate" with parameters {} and body {} done in 9.023s: 200
2024/09/10 10:41:35 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"agents_list": "000"} and body {} done in 2.414s: 200
2024/09/10 10:41:35 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"agents_list": "000"} and body {} done in 2.418s: 200
2024/09/10 10:41:41 INFO: wazuh-wui 127.0.0.1 "GET /manager/info" with parameters {} and body {} done in 8.626s: 200
2024/09/10 10:41:43 INFO: unknown_user 127.0.0.1 "GET /security/users/me/policies" with parameters {} and body {"idHost": "default"} done in 7.694s: 401
2024/09/10 10:41:45 INFO: wazuh-wui 127.0.0.1 "GET /manager/info" with parameters {} and body {} done in 9.727s: 200
2024/09/10 10:41:45 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"agents_list": "000"} and body {} done in 9.737s: 200
2024/09/10 10:41:46 INFO: wazuh-wui 127.0.0.1 "POST /security/user/authenticate" with parameters {} and body {} done in 11.124s: 200
2024/09/10 10:41:46 INFO: wazuh-wui 127.0.0.1 "GET /manager/info" with parameters {} and body {} done in 11.130s: 200
2024/09/10 10:41:46 INFO: wazuh-wui 127.0.0.1 "POST /security/user/authenticate" with parameters {} and body {} done in 11.199s: 200
2024/09/10 10:41:46 INFO: wazuh-wui 127.0.0.1 "GET /cluster/status" with parameters {} and body {} done in 11.206s: 200
2024/09/10 10:41:47 INFO: wazuh-wui 127.0.0.1 "POST /security/user/authenticate" with parameters {} and body {} done in 11.498s: 200
2024/09/10 10:41:47 INFO: wazuh-wui 127.0.0.1 "GET /manager/info" with parameters {} and body {} done in 2.239s: 200
2024/09/10 10:41:47 INFO: unknown_user 127.0.0.1 "DELETE /security/user/authenticate" with parameters {} and body {} done in 0.609s: 401
2024/09/10 10:41:47 INFO: wazuh-wui 127.0.0.1 "GET /manager/info" with parameters {} and body {} done in 0.703s: 200
2024/09/10 10:41:52 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"agents_list": "000"} and body {} done in 4.716s: 200
2024/09/10 10:41:52 INFO: wazuh-wui 127.0.0.1 "POST /security/user/authenticate" with parameters {} and body {} done in 4.770s: 200
2024/09/10 10:41:52 INFO: wazuh-wui 127.0.0.1 "GET /cluster/status" with parameters {} and body {} done in 4.774s: 200
2024/09/10 10:41:52 INFO: wazuh-wui 127.0.0.1 "GET /cluster/status" with parameters {} and body {} done in 4.777s: 200
2024/09/10 10:41:53 INFO: wazuh-wui 127.0.0.1 "GET /manager/info" with parameters {} and body {} done in 5.730s: 200
2024/09/10 10:41:53 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"agents_list": "000"} and body {} done in 1.212s: 200
2024/09/10 10:41:53 INFO: wazuh-wui 127.0.0.1 "GET /cluster/status" with parameters {} and body {} done in 1.218s: 200
2024/09/10 10:41:53 INFO: wazuh-wui 127.0.0.1 "GET /manager/info" with parameters {} and body {} done in 1.222s: 200
2024/09/10 10:41:53 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"agents_list": "000"} and body {} done in 1.227s: 200
2024/09/10 10:41:54 INFO: unknown_user 127.0.0.1 "DELETE /security/user/authenticate" with parameters {} and body {} done in 0.743s: 401
2024/09/10 10:41:56 INFO: wazuh-wui 127.0.0.1 "GET /manager/info" with parameters {} and body {} done in 4.124s: 200
2024/09/10 10:41:56 INFO: wazuh-wui 127.0.0.1 "GET /manager/version/check" with parameters {"force_query": "false"} and body {} done in 2.966s: 200
2024/09/10 10:41:56 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"agents_list": "000"} and body {} done in 2.972s: 200
2024/09/10 10:41:58 INFO: wazuh-wui 127.0.0.1 "POST /security/user/authenticate" with parameters {} and body {} done in 4.538s: 200
2024/09/10 10:41:59 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"agents_list": "000"} and body {} done in 2.396s: 200
2024/09/10 10:41:59 INFO: wazuh-wui 127.0.0.1 "GET /manager/info" with parameters {} and body {} done in 0.928s: 200
2024/09/10 10:41:59 INFO: wazuh-wui 127.0.0.1 "GET /cluster/status" with parameters {} and body {} done in 0.933s: 200
2024/09/10 10:41:59 INFO: unknown_user 127.0.0.1 "DELETE /security/user/authenticate" with parameters {} and body {} done in 0.505s: 401
2024/09/10 10:41:59 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"agents_list": "000"} and body {} done in 1.658s: 200
2024/09/10 10:41:59 INFO: wazuh-wui 127.0.0.1 "GET /cluster/status" with parameters {} and body {} done in 0.771s: 200
2024/09/10 10:41:59 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"agents_list": "000"} and body {} done in 0.779s: 200
2024/09/10 10:41:59 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"agents_list": "000"} and body {} done in 0.787s: 200
2024/09/10 10:42:00 INFO: wazuh-wui 127.0.0.1 "GET /cluster/status" with parameters {} and body {} done in 1.005s: 200
2024/09/10 10:42:01 INFO: wazuh-wui 127.0.0.1 "GET /cluster/status" with parameters {} and body {} done in 1.183s: 200
2024/09/10 10:42:01 INFO: wazuh-wui 127.0.0.1 "GET /manager/info" with parameters {} and body {} done in 0.945s: 200
2024/09/10 10:42:01 INFO: unknown_user 127.0.0.1 "DELETE /security/user/authenticate" with parameters {} and body {} done in 0.149s: 401
2024/09/10 10:42:01 INFO: wazuh-wui 127.0.0.1 "GET /cluster/status" with parameters {} and body {} done in 0.988s: 200
2024/09/10 10:42:01 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"agents_list": "000"} and body {} done in 0.965s: 200
2024/09/10 10:42:02 INFO: unknown_user 127.0.0.1 "GET /" with parameters {} and body {} done in 0.445s: 401
2024/09/10 10:42:02 INFO: unknown_user 127.0.0.1 "GET /agents/summary/status" with parameters {} and body {} done in 0.528s: 401
2024/09/10 10:42:02 INFO: unknown_user 127.0.0.1 "DELETE /security/user/authenticate" with parameters {} and body {} done in 0.088s: 401
2024/09/10 10:42:09 INFO: wazuh-wui 127.0.0.1 "GET /security/users/me" with parameters {} and body {} done in 8.754s: 200
2024/09/10 10:42:09 INFO: wazuh-wui 127.0.0.1 "GET /cluster/status" with parameters {} and body {} done in 8.088s: 200
2024/09/10 10:42:09 INFO: wazuh-wui 127.0.0.1 "GET /cluster/status" with parameters {} and body {} done in 7.822s: 200
2024/09/10 10:42:09 INFO: wazuh-wui 127.0.0.1 "GET /cluster/status" with parameters {} and body {} done in 7.829s: 200
2024/09/10 10:42:09 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"agents_list": "000"} and body {} done in 7.292s: 200
2024/09/10 10:42:09 INFO: wazuh-wui 127.0.0.1 "GET /cluster/status" with parameters {} and body {} done in 6.994s: 200
2024/09/10 10:42:11 INFO: wazuh-wui 127.0.0.1 "POST /security/user/authenticate" with parameters {} and body {} done in 8.796s: 200
2024/09/10 10:42:11 INFO: unknown_user 127.0.0.1 "DELETE /security/user/authenticate" with parameters {} and body {} done in 2.096s: 401
2024/09/10 10:42:11 INFO: wazuh-wui 127.0.0.1 "POST /security/user/authenticate" with parameters {} and body {} done in 8.966s: 200
2024/09/10 10:42:15 INFO: unknown_user 127.0.0.1 "DELETE /security/user/authenticate" with parameters {} and body {} done in 0.099s: 401
2024/09/10 10:42:16 INFO: wazuh-wui 127.0.0.1 "POST /security/user/authenticate" with parameters {} and body {} done in 4.444s: 200
2024/09/10 10:42:16 INFO: wazuh-wui 127.0.0.1 "GET /cluster/status" with parameters {} and body {} done in 2.264s: 200
2024/09/10 10:42:16 INFO: wazuh-wui 127.0.0.1 "GET /manager/info" with parameters {} and body {} done in 0.483s: 200
2024/09/10 10:42:16 INFO: wazuh-wui 127.0.0.1 "GET /manager/info" with parameters {} and body {} done in 0.263s: 200
2024/09/10 10:42:16 INFO: unknown_user 127.0.0.1 "DELETE /security/user/authenticate" with parameters {} and body {} done in 0.072s: 401
2024/09/10 10:42:18 INFO: unknown_user 127.0.0.1 "DELETE /security/user/authenticate" with parameters {} and body {} done in 0.218s: 401
2024/09/10 10:42:18 INFO: wazuh-wui 127.0.0.1 "GET /manager/info" with parameters {} and body {} done in 1.480s: 200
2024/09/10 10:42:19 INFO: wazuh-wui 127.0.0.1 "GET /security/users/me" with parameters {} and body {} done in 1.794s: 200
2024/09/10 10:42:19 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"agents_list": "000"} and body {} done in 1.672s: 200
2024/09/10 10:42:19 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"agents_list": "000"} and body {} done in 1.542s: 200
2024/09/10 10:42:19 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"agents_list": "000"} and body {} done in 1.294s: 200
2024/09/10 10:42:19 INFO: wazuh-wui 127.0.0.1 "GET /cluster/status" with parameters {} and body {} done in 0.463s: 200
2024/09/10 10:42:19 INFO: wazuh-wui 127.0.0.1 "GET /cluster/status" with parameters {} and body {} done in 0.351s: 200
2024/09/10 10:42:20 INFO: wazuh-wui 127.0.0.1 "GET /cluster/status" with parameters {} and body {} done in 0.279s: 200
2024/09/10 10:42:29 INFO: unknown_user 127.0.0.1 "GET /" with parameters {} and body {} done in 1.637s: 401
2024/09/10 10:42:29 INFO: wazuh-wui 127.0.0.1 "POST /security/user/authenticate" with parameters {} and body {} done in 9.419s: 200
2024/09/10 10:42:29 INFO: wazuh-wui 127.0.0.1 "POST /security/user/authenticate" with parameters {} and body {} done in 7.458s: 200
2024/09/10 10:42:34 INFO: wazuh-wui 127.0.0.1 "GET /security/users/me/policies" with parameters {} and body {"idHost": "default"} done in 1.819s: 200
2024/09/10 10:42:34 INFO: wazuh-wui 127.0.0.1 "POST /security/user/authenticate" with parameters {} and body {} done in 4.260s: 200
2024/09/10 10:42:34 INFO: wazuh-wui 127.0.0.1 "GET /security/users/me/policies" with parameters {} and body {"idHost": "default"} done in 0.146s: 200
2024/09/10 10:42:35 INFO: wazuh-wui 127.0.0.1 "GET /manager/info" with parameters {} and body {} done in 0.655s: 200
2024/09/10 10:42:37 INFO: wazuh-wui 127.0.0.1 "GET /agents" with parameters {"agents_list": "000"} and body {} done in 0.274s: 200
2024/09/10 10:42:38 INFO: wazuh-wui 127.0.0.1 "GET /cluster/status" with parameters {} and body {} done in 0.234s: 200

. /var/ossec/logs/ossec.log

2024/09/10 09:13:11 sca: INFO: Starting evaluation of policy: '/var/ossec/ruleset/sca/cis_ubuntu22-04.yml'
2024/09/10 09:13:13 wazuh-modulesd:syscollector: INFO: Module started.
2024/09/10 09:13:13 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2024/09/10 09:13:13 wazuh-analysisd: INFO: (7200): Logtest started
2024/09/10 09:13:13 wazuh-analysisd: INFO: EPS limit disabled
2024/09/10 09:13:16 indexer-connector: WARNING: IndexerConnector initialization failed for index 'wazuh-states-vulnerabilities-wazuh-pb', retrying until the connection is successful.
2024/09/10 09:13:21 wazuh-modulesd:vulnerability-scanner: ERROR: Error opening the database: Vendor map can not be found in DB., trying to re-download the feed.
2024/09/10 09:13:22 wazuh-modulesd:vulnerability-scanner: INFO: Vulnerability scanner module started
2024/09/10 09:13:24 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2024/09/10 09:14:07 wazuh-syscheckd: INFO: (6009): File integrity monitoring scan ended.
2024/09/10 09:14:07 wazuh-syscheckd: INFO: FIM sync module started.
2024/09/10 09:14:26 sca: INFO: Evaluation finished for policy '/var/ossec/ruleset/sca/cis_ubuntu22-04.yml'
2024/09/10 09:14:27 sca: INFO: Security Configuration Assessment scan finished. Duration: 76 seconds.
2024/09/10 09:15:59 wazuh-modulesd:vulnerability-scanner: INFO: Initiating update feed process
2024/09/10 09:17:29 rootcheck: INFO: Ending rootcheck scan.
2024/09/10 09:20:20 indexer-connector: INFO: IndexerConnector initialized successfully for index: wazuh-states-vulnerabilities-wazuh-pb.
2024/09/10 10:08:45 wazuh-db: INFO: Created Global database backup "backup/db/global.db-backup-2024-09-10-10:08:45.gz"
2024/09/10 10:08:45 wazuh-db: INFO: Deleted Global database backup: "backup/db/global.db-backup-2024-09-05-10:22:59.gz"
2024/09/10 10:13:25 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2024/09/10 10:13:39 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2024/09/10 10:16:46 wazuh-modulesd:vulnerability-scanner: ERROR: Error updating feed: Invalid line. file: queue/vd_updater/tmp/contents/vd_1.0.0_vd_4.8.0_871035_1725876934.json, trying to re-download the feed.
2024/09/10 10:16:46 wazuh-modulesd:vulnerability-scanner: INFO: Initiating update feed process
2024/09/10 10:16:47 wazuh-modulesd:vulnerability-scanner: ERROR: Error updating feed: Unable to find resource., trying to re-download the feed.
2024/09/10 10:18:33 wazuh-modulesd:vulnerability-scanner: INFO: Initiating update feed process

API CONFIGURATION (I haven't uncommented anything from this file)

# USE THIS FILE AS A TEMPLATE. UNCOMMENT LINES TO APPLY CUSTOM CONFIGURATION

# host: 0.0.0.0
# port: 55000

# Advanced configuration

# https:
#  enabled: yes
#  key: "server.key"
#  cert: "server.crt"
#  use_ca: False
#  ca: "ca.crt"
#  ssl_protocol: "auto"
#  ssl_ciphers: ""

# Modify API's intervals (time in seconds)
# intervals:
#   request_timeout: 10

# Logging configuration
# Values for API log level: disabled, info, warning, error, debug, debug2 (each level includes the previous level).
# Values for API log max_size: <value><unit>. Valid units: K (kilobytes), M (megabytes)
# Enabling the API log max_size will disable the time based rotation (on midnight)
# logs:
#  level: "info"
#  format: "plain"
#  max_size:
#    enabled: False
#    size: "1M"

# Cross-origin resource sharing: https://github.com/aio-libs/aiohttp-cors#usage
# cors:
#  enabled: no
#  source_route: "*"
#  expose_headers: "*"
#  allow_headers: "*"
#  allow_credentials: no

# Cache (time in seconds)
# cache:
#  enabled: yes
#  time: 0.750

# Access parameters
# access:
#  max_login_attempts: 50
#  block_time: 300
#  max_request_per_minute: 300

# Drop privileges (Run as wazuh user)
# drop_privileges: yes

# Enable features under development
# experimental_features: no

# Maximum body size that the API can accept, in bytes (0 -> limitless)
# max_upload_size: 10485760

# Uploadable Wazuh configuration sections
# upload_configuration:
#   remote_commands:
#     localfile:
#       allow: yes
#       exceptions: []
#     wodle_command:
#       allow: yes
#       exceptions: []
#   limits:
#     eps:
#       allow: yes
#   agents:
#     allow_higher_versions:
#       allow: yes
#   indexer:
#     allow: yes
#   integrations:
#     virustotal:
#       public_key:
#         allow: yes
#         minimum_quota: 240



I am noticing that Wazuh Dashboard Service stops sometimes and that it comes back Active after some seconds, maybe this is related to the API failing to Initialize

× wazuh-dashboard.service - wazuh-dashboard
    Loaded: loaded (/etc/systemd/system/wazuh-dashboard.service; enabled; vendor preset: enabled)
    Active: failed (Result: core-dump) since Tue 2024-09-10 10:51:47 UTC; 11s ago
   Process: 5077 ExecStart=/usr/share/wazuh-dashboard/bin/opensearch-dashboards (code=dumped, signal=SEGV)
  Main PID: 5077 (code=dumped, signal=SEGV)
       CPU: 5min 24.535s

Sep 10 10:51:46 wazuh-pb opensearch-dashboards[5077]: {"type":"response","@timestamp":"2024-09-10T10:51:46Z","tags":[],"pid":5077,"method":"post","statusCode":200,"req":{"url":"/auth/logout","method":"post","headers":{"host":"192.168.4>
Sep 10 10:51:46 wazuh-pb opensearch-dashboards[5077]: {"type":"error","@timestamp":"2024-09-10T10:51:45Z","tags":[],"pid":5077,"level":"error","error":{"message":"Internal Server Error","name":"Error","stack":"Error: Internal Server Er>
Sep 10 10:51:46 wazuh-pb opensearch-dashboards[5077]: {"type":"response","@timestamp":"2024-09-10T10:51:45Z","tags":[],"pid":5077,"method":"post","statusCode":500,"req":{"url":"/api/request","method":"post","headers":{"host":"192.168.4>
Sep 10 10:51:46 wazuh-pb opensearch-dashboards[5077]: {"type":"error","@timestamp":"2024-09-10T10:51:45Z","tags":[],"pid":5077,"level":"error","error":{"message":"Internal Server Error","name":"Error","stack":"Error: Internal Server Er>
Sep 10 10:51:46 wazuh-pb opensearch-dashboards[5077]: {"type":"response","@timestamp":"2024-09-10T10:51:45Z","tags":[],"pid":5077,"method":"post","statusCode":500,"req":{"url":"/api/request","method":"post","headers":{"host":"192.168.4>
Sep 10 10:51:46 wazuh-pb opensearch-dashboards[5077]: {"type":"error","@timestamp":"2024-09-10T10:51:45Z","tags":[],"pid":5077,"level":"error","error":{"message":"Internal Server Error","name":"Error","stack":"Error: Internal Server Er>
Sep 10 10:51:46 wazuh-pb opensearch-dashboards[5077]: {"type":"response","@timestamp":"2024-09-10T10:51:45Z","tags":[],"pid":5077,"method":"post","statusCode":500,"req":{"url":"/api/request","method":"post","headers":{"host":"192.168.4>
Sep 10 10:51:47 wazuh-pb systemd[1]: wazuh-dashboard.service: Main process exited, code=dumped, status=11/SEGV
Sep 10 10:51:47 wazuh-pb systemd[1]: wazuh-dashboard.service: Failed with result 'core-dump'.
Sep 10 10:51:47 wazuh-pb systemd[1]: wazuh-dashboard.service: Consumed 5min 24.535s CPU time.

[manuel...@wazuh.com]

Rej,

Hi! Thanks for sharing the info.

Would you mind sharing with us the specs of the AIO server and how many agents its handling? Just to discard any possibility of resource exhaustion.

Kind regards,

Manuel

[manuel...@wazuh.com]

Rej,

Thanks for sharing the requested information.

Just by chance and to rule out any other possibilities: What would be specs of the AIO server and how many agents is it handling?

Such a delay could also be a symptom of elevated resource consumption.

Kind regards,
Manuel

[Rei Gjata]

The AIO specs are: 16 Gb Ram, and 8 Cpu (2 Sockets with 4 Cores) and 130 Gb space.

Currently monitoring only 2 agents but they will go up to 20/25 after everything is okay.

I also did the memory locking for Indexer with a configuration like this:

-Xms8g 

-Xmx8g   

But somehow the indexer service or the dashboard service keeps failing after some period, which i think its related to the API error.


Thanks Manuel

[Rei Gjata]

Hello,

UPDATE to this case

Im noticing that the services fail due to consumed CPU time, and as far as I noticed with HTOP,  CPUS are working at 100% (Not all of them)

× wazuh-indexer.service - Wazuh-indexer
     Loaded: loaded (/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: enabled)
    Drop-In: /etc/systemd/system/wazuh-indexer.service.d
             └─wazuh-indexer.conf
     Active: failed (Result: signal) since Wed 2024-09-11 16:40:54 UTC; 16h ago
       Docs: https://documentation.wazuh.com
    Process: 4331 ExecStart=/usr/share/wazuh-indexer/bin/systemd-entrypoint -p ${PID_DIR}/wazuh-indexer.pid --quiet (code=killed, signal=ABRT)
   Main PID: 4331 (code=killed, signal=ABRT)
        CPU: 25min 57.121s

Sep 11 16:40:45 wazuh-pb systemd-entrypoint[4331]:  dependencies   [0x00007f4e43110c10,0x00007f4e43110c48] = 56
Sep 11 16:40:45 wazuh-pb systemd-entrypoint[4331]:  handler table  [0x00007f4e43110c48,0x00007f4e43110c60] = 24
Sep 11 16:40:45 wazuh-pb systemd-entrypoint[4331]:  nul chk table  [0x00007f4e43110c60,0x00007f4e43110cb8] = 88
Sep 11 16:40:46 wazuh-pb systemd-entrypoint[4331]: #
Sep 11 16:40:46 wazuh-pb systemd-entrypoint[4331]: # If you would like to submit a bug report, please visit:
Sep 11 16:40:46 wazuh-pb systemd-entrypoint[4331]: #   https://github.com/adoptium/adoptium-support/issues
Sep 11 16:40:46 wazuh-pb systemd-entrypoint[4331]: #
Sep 11 16:40:54 wazuh-pb systemd[1]: wazuh-indexer.service: Main process exited, code=killed, status=6/ABRT
Sep 11 16:40:54 wazuh-pb systemd[1]: wazuh-indexer.service: Failed with result 'signal'.
Sep 11 16:40:54 wazuh-pb systemd[1]: wazuh-indexer.service: Consumed 25min 57.121s CPU time.

Any idea why this problem is occurring?
This has never happened to me on versions before 4.8 

[manuel...@wazuh.com]

Rej,

Do you happen to have any integrations running? High resource consumption could lead to API going down temporally.

Besides that, you could try adding a worker node to take off some load from the Manager.

Kind regards,

Manuel