WAZUH INDEXER TIMEOUT

[saman javed]

i have restarted my wazuh indexer and it is giving timeout error.  

i can not find the error or issue. 

Any help would be appreciated

[root@WAZUH ~]# systemctl status wazuh-indexer
● wazuh-indexer.service - Wazuh-indexer
   Loaded: loaded (/usr/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: disabled)
   Active: failed (Result: timeout) since Fri 2023-01-20 19:18:38 PKT; 7min ago
     Docs: https://documentation.wazuh.com
  Process: 11794 ExecStart=/usr/share/wazuh-indexer/bin/systemd-entrypoint -p ${PID_DIR}/wazuh-indexer.pid --quiet (code=exited, status=143)
 Main PID: 11794 (code=exited, status=143)

Jan 20 19:17:21 WAZUH systemd[1]: Starting Wazuh-indexer...
Jan 20 19:18:18 WAZUH systemd-entrypoint[11794]: WARNING: An illegal reflective access operation has occurred
Jan 20 19:18:18 WAZUH systemd-entrypoint[11794]: WARNING: Illegal reflective access by io.protostuff.runtime.PolymorphicThrowableSchema (file:/usr/share/wazuh-indexer/plugins...wable.cause
Jan 20 19:18:18 WAZUH systemd-entrypoint[11794]: WARNING: Please consider reporting this to the maintainers of io.protostuff.runtime.PolymorphicThrowableSchema
Jan 20 19:18:18 WAZUH systemd-entrypoint[11794]: WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
Jan 20 19:18:18 WAZUH systemd-entrypoint[11794]: WARNING: All illegal access operations will be denied in a future release
Jan 20 19:18:36 WAZUH systemd[1]: wazuh-indexer.service start operation timed out. Terminating.
Jan 20 19:18:38 WAZUH systemd[1]: Failed to start Wazuh-indexer.
Jan 20 19:18:38 WAZUH systemd[1]: Unit wazuh-indexer.service entered failed state.
Jan 20 19:18:38 WAZUH systemd[1]: wazuh-indexer.service failed.
Hint: Some lines were ellipsized, use -l to show in full.

[root@WAZUH ~]# journalctl -xe
Jan 20 19:26:14 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:26:14Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:26:16 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:26:16Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:26:19 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:26:19Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:26:21 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:26:21Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:26:24 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:26:24Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:26:26 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:26:26Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:26:29 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:26:29Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:26:31 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:26:31Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:26:34 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:26:34Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:26:36 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:26:36Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:26:39 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:26:39Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:26:41 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:26:41Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:26:44 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:26:44Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:26:46 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:26:46Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:26:49 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:26:49Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:26:51 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:26:51Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:26:54 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:26:54Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:26:56 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:26:56Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:26:59 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:26:59Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:27:01 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:27:01Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:27:04 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:27:04Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:27:06 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:27:06Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:27:09 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:27:09Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:27:11 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:27:11Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:27:14 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:27:14Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:27:16 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:27:16Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:27:19 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:27:19Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:27:21 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:27:21Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:27:24 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:27:24Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:27:26 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:27:26Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:27:29 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:27:29Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:27:31 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:27:31Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:27:34 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:27:34Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:27:36 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:27:36Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:27:39 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:27:39Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:27:41 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:27:41Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:27:44 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:27:44Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:27:46 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:27:46Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:27:49 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:27:49Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:27:51 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:27:51Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:27:54 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:27:54Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:27:56 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:27:56Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:27:59 WAZUH opensearch-dashboards[707]: {"type":"log","@timestamp":"2023-01-20T14:27:59Z","tags":["error","opensearch","data"],"pid":707,"message":"[ConnectionError]: connect ECO
Jan 20 19:28:01 WAZUH opensearch-dashboards[707]: {"type":"log","@time

[Pacome Kemkeu]

Hello Saman,

According to your output, for some reason, the system tries to spin up the wazuh-indexer  service but, due to the service timeout, it doesn't have enough time to get it up.
In this case, let's increase the timeout. For that, kindly run these commands:

# sudo mkdir /etc/systemd/system/wazuh-indexer.service.d
# echo -e "[Service]\nTimeoutStartSec=180" | sudo tee /etc/systemd/system/wazuh-indexer.service.d/startup-timeout.conf 

Then, reload the daemon:
$ sudo systemctl daemon-reload

And finally restart the services:
# sudo systemctl restart wazuh-indexer
# sudo systemctl restart wazuh-manager
# sudo systemctl restart wazuh-dashboard

After a few minutes you can try to access the dashboard again.

Kindly try these and tell me how it goes.

[saman javed]

Hi,

Thanks for your response.

I have disabled SElinux and rebooted my machine and then tried again. 

Indexer got restarted.

Stopped all components first and then started them in sequence indexer - manager -filebeat - dashboard.

For future references can you explain is Selinux may be the cause for wazuh indexer restart failures ??  

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/976ed9b0-d258-4929-be4e-0295fce2e61fn%40googlegroups.com.

[Pacome Kemkeu]

Hello Saman, I am happy to know that it is working well now.

This is an issue that usually occurs and is solved by increasing the timeout or restarting even without disabling SELinux or rebooting. 

I can't provide a proper answer to your question now but I'll reach out to our development team to have more insights and let you know. 

[saman javed]

Thank you for your response.

[Matteo Torta]

How can I disable SElinux??