Issue with Wazuh Dashboard Service Failing - Circuit Breaker Exception

48 views
Skip to first unread message

sale msg

unread,
Feb 3, 2025, 8:36:05 AM2/3/25
to Wazuh | Mailing List
Skip to NavigationI am experiencing an issue with the Wazuh dashboard service failing to start on my system.Both Wazuh manager and indexer are working fine as expected but when i am trying to access the dashboard via browser it say " this site can't be reached" then tried to see if the dashboard is down. ofcourse, when i run systemctl status wazuh-dashboard the result is like below:

× wazuh-dashboard.service - wazuh-dashboard

Loaded: loaded (/etc/systemd/system/wazuh-dashboard.service; enabled; vendor preset: enabled)

Active: failed (Result: exit-code) since Thu 2025-01-30 16:28:43 EAT; 17h ago

Process: 5209 ExecStart=/usr/share/wazuh-dashboard/bin/opensearch-dashboards (code=exited, status=1/FAILURE)

Main PID: 5209 (code=exited, status=1/FAILURE)

CPU: 13.794s

ጃንዩ 30 16:28:42 wazuhdashboard-virtual-machine opensearch-dashboards[5209]: {"type":"log","@timestamp":"2025-01-30T13:28:42Z","tags":["info","savedobjects-service"],"pid":5209,"message":"Waiting until all OpenSearch nodes are compatible with OpenSearch Dashboards before starting saved objects migrations..."}

ጃንዩ 30 16:28:42 wazuhdashboard-virtual-machine opensearch-dashboards[5209]: {"type":"log","@timestamp":"2025-01-30T13:28:42Z","tags":["info","savedobjects-service"],"pid":5209,"message":"Starting saved objects migrations"}

ጃንዩ 30 16:28:42 wazuhdashboard-virtual-machine opensearch-dashboards[5209]: {"type":"log","@timestamp":"2025-01-30T13:28:42Z","tags":["error","opensearch","data"],"pid":5209,"message":"[circuit_breaking_exception]: [parent] Data too large, data for [<http_request>] would be [1048762352/1000.1mb], which is larger than the limit of [1020054732/972.7mb], real usage: [1048762352/1000.1mb], new bytes reserved: [0/0b], usages [request=0/0b, fielddata=58347/56.9kb, in_flight_requests=0/0b]"}

ጃንዩ 30 16:28:42 wazuhdashboard-virtual-machine opensearch-dashboards[5209]: {"type":"log","@timestamp":"2025-01-30T13:28:42Z","tags":["warning","savedobjects-service"],"pid":5209,"message":"Unable to connect to OpenSearch. Error: circuit_breaking_exception: [circuit_breaking_exception] Reason: [parent] Data too large, data for [<http_request>] would be [1048762352/1000.1mb], which is larger than the limit of [1020054732/972.7mb], real usage: [1048762352/1000.1mb], new bytes reserved: [0/0b], usages [request=0/0b, fielddata=58347/56.9kb, in_flight_requests=0/0b]"}

ጃንዩ 30 16:28:42 wazuhdashboard-virtual-machine opensearch-dashboards[5209]: {"type":"log","@timestamp":"2025-01-30T13:28:42Z","tags":["fatal","root"],"pid":5209,"message":"ResponseError: circuit_breaking_exception: [circuit_breaking_exception] Reason: [parent] Data too large, data for [<http_request>] would be [1048762352/1000.1mb], which is larger than the limit of [1020054732/972.7mb], real usage: [1048762352/1000.1mb], new bytes reserved: [0/0b], usages [request=0/0b, fielddata=58347/56.9kb, in_flight_requests=0/0b]\n at onBody (/usr/share/wazuh-dashboard/node_modules/@opensearch-project/opensearch/lib/Transport.js:374:23)\n at IncomingMessage.onEnd (/usr/share/wazuh-dashboard/node_modules/@opensearch-project/opensearch/lib/Transport.js:293:11)\n at IncomingMessage.emit (node:events:529:35)\n at IncomingMessage.emit (node:domain:489:12)\n at endReadableNT (node:internal/streams/readable:1400:12)\n at processTicksAndRejections (node:internal/process/task_queues:82:21) {\n meta: {\n body: { error: [Object], status: 429 },\n statusCode: 429,\n headers: {\n 'content-type': 'application/json; charset=UTF-8',\n 'content-length': '833'\n },\n meta: {\n context: null,\n request: [Object],\n name: 'opensearch-js',\n connection: [Object],\n attempts: 0,\n aborted: false\n }\n }\n}"}

ጃንዩ 30 16:28:42 wazuhdashboard-virtual-machine opensearch-dashboards[5209]: {"type":"log","@timestamp":"2025-01-30T13:28:42Z","tags":["info","plugins-system"],"pid":5209,"message":"Stopping all plugins."}

ጃንዩ 30 16:28:43 wazuhdashboard-virtual-machine opensearch-dashboards[5209]: FATAL {"error":{"root_cause":[{"type":"circuit_breaking_exception","reason":"[parent] Data too large, data for [<http_request>] would be [1048762352/1000.1mb], which is larger than the limit of [1020054732/972.7mb], real usage: [1048762352/1000.1mb], new bytes reserved: [0/0b], usages [request=0/0b, fielddata=58347/56.9kb, in_flight_requests=0/0b]","bytes_wanted":1048762352,"bytes_limit":1020054732,"durability":"PERMANENT"}],"type":"circuit_breaking_exception","reason":"[parent] Data too large, data for [<http_request>] would be [1048762352/1000.1mb], which is larger than the limit of [1020054732/972.7mb], real usage: [1048762352/1000.1mb], new bytes reserved: [0/0b], usages [request=0/0b, fielddata=58347/56.9kb, in_flight_requests=0/0b]","bytes_wanted":1048762352,"bytes_limit":1020054732,"durability":"PERMANENT"},"status":429}

ጃንዩ 30 16:28:43 wazuhdashboard-virtual-machine systemd[1]: wazuh-dashboard.service: Main process exited, code=exited, status=1/FAILURE

ጃንዩ 30 16:28:43 wazuhdashboard-virtual-machine systemd[1]: wazuh-dashboard.service: Failed with result 'exit-code'.

ጃንዩ 30 16:28:43 wazuhdashboard-virtual-machine systemd[1]: wazuh-dashboard.service: Consumed 13.794s CPU time.

IN General:

The wazuh-dashboard service is failing to start, with the following error message in the logs:

circuit_breaking_exception: [parent] Data too large, data for [<http_request>] would be [1048762352/1000.1mb], which is larger than the limit of [1020054732/972.7mb]

This error occurs shortly after the service tries to perform the migration of saved objects, but it appears to be unable to connect to OpenSearch due to resource limitations.

I also tried to increase the heap size to improve performance but not help.
2025-02-03 08_04_21-Screenshot 2025-02-03 071559.pngError Combined_infor.png
Logs:

Here is an excerpt from the log file around the time the issue occurs:

... unable to connect to OpenSearch. Error: circuit_breaking_exception: [parent] Data too large... ... FATAL {"error":{"root_cause":[{"type":"circuit_breaking_exception","reason":"[parent] Data too large, data for [<http_request>] would be [1048762352/1000.1mb], which is larger than the limit of [1020054732/972.7mb]","bytes_wanted":1048762352,"bytes_limit":1020054732,"durability":"PERMANENT"}], "status": 429} System Information:

  • Wazuh Version: v4.10.1

  • Operating System:Ubunut 24

Question:

  1. What is the cause of the circuit_breaking_exception error regarding data size, and why is it occurring in the Wazuh dashboard?

  2. How can I resolve this issue to allow the Wazuh dashboard to start and function correctly?

  3. Are there any configuration changes I can make to avoid hitting the data size limit in the future?

I appreciate any help or guidance you can provide on this issue.

Thank you in advance

Reply all
Reply to author
Forward
0 new messages