Internal Server Error / Error Pattern Handler (getPatternList)

[Jordan Empresas]

Dear All,

Please I need your help and support since I cannot access the dashboard.

run this command:

# cat /var/log/wazuh-indexer/wazuh-indexer-cluster.log | grep -i -E "error|warn"

[2024-12-26T02:12:17,321][ERROR][o.o.a.a.AlertIndices     ] [indexer-1] info deleteOldIndices
[2024-12-26T02:12:17,321][ERROR][o.o.a.a.AlertIndices     ] [indexer-1] info deleteOldIndices
[2024-12-26T14:12:17,323][ERROR][o.o.a.a.AlertIndices     ] [indexer-1] info deleteOldIndices
[2024-12-26T14:12:17,324][ERROR][o.o.a.a.AlertIndices     ] [indexer-1] info deleteOldIndices

Then I proceeded to delete the oldest alerts and restart the indexer and dashboard.

And now it gives me a username and password error for both the admin user and my custom user

After clicking several times, this information appears:

Version: 2.13.0
Build: 49201
Error: Internal Server Error
    at fetch_Fetch.fetchResponse (https:// 10.0.0.3/49201/bundles/core/core.entry.js:15:242047)
    at async interceptResponse (https://10.0.0.3/49201/bundles/core/core.entry.js:15:236801)
    at async https:// 10.0.0.3/49201/bundles/core/core.entry.js:15:239768



Error: Error Pattern Handler (getPatternList)
    at pattern_handler_PatternHandler.getPatternList (https:// 10.0.0.3/49201/bundles/plugin/wazuh/wazuh.chunk.3.js:1:1288529)
    at async WzMenu.load (https:// 10.0.0.3 /49201/bundles/plugin/wazuh/wazuh.chunk.3.js:1:1512577)
    at async WzMenu.componentDidUpdate (https:// 10.0.0.3/49201/bundles/plugin/wazuh/wazuh.chunk.3.js:1:1511323)

As you can see below the status and they are fine:

# systemctl status wazuh-dashboard
● wazuh-dashboard.service - wazuh-dashboard
     Loaded: loaded (/etc/systemd/system/wazuh-dashboard.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2024-12-26 20:12:42 UTC; 49min ago
   Main PID: 2579408 (node)
      Tasks: 11 (limit: 38494)
     Memory: 193.8M
        CPU: 23.883s
     CGroup: /system.slice/wazuh-dashboard.service
             └─2579408 /usr/share/wazuh-dashboard/node/bin/node /usr/share/wazuh-dashboard/src/cli/dist

Dec 26 20:55:00 ip-10-0-0-3 opensearch-dashboards[2579408]: {"type":"log","@timestamp":"2024-12-26T20:55:00Z","tags":["error">
Dec 26 20:55:00 ip-10-0-0-3 opensearch-dashboards[2579408]: {"type":"log","@timestamp":"2024-12-26T20:55:00Z","tags":["error">
Dec 26 20:55:00 ip-10-0-0-3 opensearch-dashboards[2579408]: {"type":"log","@timestamp":"2024-12-26T20:55:00Z","tags":["error">
Dec 26 20:55:00 ip-10-0-0-3 opensearch-dashboards[2579408]: {"type":"log","@timestamp":"2024-12-26T20:55:00Z","tags":["error">
Dec 26 21:00:00 ip-10-0-0-3 opensearch-dashboards[2579408]: {"type":"log","@timestamp":"2024-12-26T21:00:00Z","tags":["error">
Dec 26 21:00:00 ip-10-0-0-3 opensearch-dashboards[2579408]: {"type":"log","@timestamp":"2024-12-26T21:00:00Z","tags":["error">
Dec 26 21:00:00 ip-10-0-0-3 opensearch-dashboards[2579408]: {"type":"log","@timestamp":"2024-12-26T21:00:00Z","tags":["error">
Dec 26 21:00:00 ip-10-0-0-3 opensearch-dashboards[2579408]: {"type":"log","@timestamp":"2024-12-26T21:00:00Z","tags":["error">
Dec 26 21:00:00 ip-10-0-0-3 opensearch-dashboards[2579408]: {"type":"log","@timestamp":"2024-12-26T21:00:00Z","tags":["error">
Dec 26 21:00:00 ip-10-0-0-3 opensearch-dashboards[2579408]: {"type":"log","@timestamp":"2024-12-26T21:00:00Z","tags":["error">

# systemctl status wazuh-indexer
● wazuh-indexer.service - wazuh-indexer
     Loaded: loaded (/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: enabled)
    Drop-In: /etc/systemd/system/wazuh-indexer.service.d
             └─wazuh-indexer.conf
     Active: active (running) since Thu 2024-12-26 19:59:59 UTC; 1h 2min ago
       Docs: https://documentation.wazuh.com
   Main PID: 2578064 (java)
      Tasks: 133 (limit: 38494)
     Memory: 1.8G
        CPU: 11min 16.396s
     CGroup: /system.slice/wazuh-indexer.service
             └─2578064 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch>

Dec 26 19:59:45 ip-10-0-0-3 systemd-entrypoint[2578064]: WARNING: System::setSecurityManager has been called by org.opensearc>
Dec 26 19:59:45 ip-10-0-0-3 systemd-entrypoint[2578064]: WARNING: Please consider reporting this to the maintainers of org.op>
Dec 26 19:59:45 ip-10-0-0-3 systemd-entrypoint[2578064]: WARNING: System::setSecurityManager will be removed in a future rele>
Dec 26 19:59:46 ip-10-0-0-3 systemd-entrypoint[2578064]: Dec 26, 2024 7:59:46 PM sun.util.locale.provider.LocaleProviderAdapt>
Dec 26 19:59:46 ip-10-0-0-3 systemd-entrypoint[2578064]: WARNING: COMPAT locale provider will be removed in a future release
Dec 26 19:59:46 ip-10-0-0-3 systemd-entrypoint[2578064]: WARNING: A terminally deprecated method in java.lang.System has been>
Dec 26 19:59:46 ip-10-0-0-3 systemd-entrypoint[2578064]: WARNING: System::setSecurityManager has been called by org.opensearc>
Dec 26 19:59:46 ip-10-0-0-3 systemd-entrypoint[2578064]: WARNING: Please consider reporting this to the maintainers of org.op>
Dec 26 19:59:46 ip-10-0-0-3 systemd-entrypoint[2578064]: WARNING: System::setSecurityManager will be removed in a future rele>
Dec 26 19:59:59 ip-10-0-0-3 systemd[1]: Started wazuh-indexer.

# systemctl status wazuh-manager
● wazuh-manager.service - Wazuh manager
     Loaded: loaded (/lib/systemd/system/wazuh-manager.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2024-12-26 19:45:46 UTC; 1h 16min ago
    Process: 2574705 ExecStart=/usr/bin/env /var/ossec/bin/wazuh-control start (code=exited, status=0/SUCCESS)
      Tasks: 215 (limit: 38494)
     Memory: 1.8G
        CPU: 8min 8.276s
     CGroup: /system.slice/wazuh-manager.service
             ├─2574780 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
             ├─2574781 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
             ├─2574784 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
             ├─2574787 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
             ├─2574810 /var/ossec/bin/wazuh-integratord
             ├─2574831 /var/ossec/bin/wazuh-authd
             ├─2574847 /var/ossec/bin/wazuh-db
             ├─2574872 /var/ossec/bin/wazuh-execd
             ├─2574888 /var/ossec/bin/wazuh-maild
             ├─2574895 /var/ossec/bin/wazuh-analysisd
             ├─2574907 /var/ossec/bin/wazuh-syscheckd
             ├─2575013 /var/ossec/bin/wazuh-remoted
             ├─2575049 /var/ossec/bin/wazuh-logcollector
             ├─2575068 /var/ossec/bin/wazuh-monitord
             ├─2575092 /var/ossec/bin/wazuh-modulesd
             ├─2575567 /var/ossec/framework/python/bin/python3 /var/ossec/framework/scripts/wazuh_clusterd.py
             ├─2575721 /var/ossec/framework/python/bin/python3 /var/ossec/framework/scripts/wazuh_clusterd.py
             └─2575722 /var/ossec/framework/python/bin/python3 /var/ossec/framework/scripts/wazuh_clusterd.py

Dec 26 19:45:38 ip-10-0-0-3 env[2574705]: Started wazuh-syscheckd...

# systemctl status filebeat
● filebeat.service - Filebeat sends log files to Logstash or directly to Elasticsearch.
     Loaded: loaded (/lib/systemd/system/filebeat.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2024-12-13 14:18:38 UTC; 1 week 6 days ago
       Docs: https://www.elastic.co/products/beats/filebeat
   Main PID: 3352539 (filebeat)
      Tasks: 15 (limit: 38494)
     Memory: 222.5M
        CPU: 51min 24.678s
     CGroup: /system.slice/filebeat.service
             └─3352539 /usr/share/filebeat/bin/filebeat --environment systemd -c /etc/filebeat/filebeat.yml --path.home /usr/sh>

Dec 13 14:18:38 ip-10-0-0-3 systemd[1]: Started Filebeat sends log files to Logstash or directly to Elasticsearch..
lines 1-12/12 (END)

Can someone please help me recover the dashboard?

Kind regards.

[Stuti Gupta]

Hi  Jordan Empresas

Can you please provide the following details:

Please share the wazuh-indexer and dashboard logs:
# cat /var/log/wazuh-indexer/wazuh-cluster.log
# journalctl -u wazuh-dashboard

Also share the out of the following command
curl -XGET -k -u user:pass "https://localhost:9200/_cluster/health"

What's the architecture of your environment? Could you please verify if there is enough disk space

Hope to hear from you soon

[Jordan Empresas]

Estimado,

Muy agradecido por responder.

Te comparto los resultados:

Para el resultado de cluster health me sale esto:

Al darle enter en la segunda opción no me arroja ningún resultado.

Lo demás está en los archivos adjuntos.

La arquitectura es x64 sistema Ubuntu 20.04 LTS

El cluster lo desplegué de la siguiente forma:

 

Te aclaro que en esta configuración no tengo habilitado el balanceo por NGINX, no se si eso era necesario o si me este dando algún problema por no configurarlo?

Estaré atento a tu respuesta y opiniones.

Saludos,

--
You received this message because you are subscribed to the Google Groups "Wazuh | Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/wazuh/5b24c45a-5dff-4130-8345-731d1e52835an%40googlegroups.com.

[Jordan Empresas]

Estimado,

Muy agradecido por responder.

Te comparto los resultados:

Para el resultado de cluster health me sale esto:

Al darle enter en la segunda opción no me arroja ningún resultado.

Lo demás está en los archivos adjuntos.

La arquitectura es x64 sistema Ubuntu 20.04 LTS en AWS

El cluster lo desplegué de la siguiente forma:

Te aclaro que en esta configuración no tengo habilitado el balanceo por NGINX, no se si eso era necesario o si me este dando algún problema por no configurarlo?

Estaré atento a tu respuesta y opiniones.

Saludos,

El vie, 27 dic 2024 a las 0:31, 'Stuti Gupta' via Wazuh | Mailing List (<wa...@googlegroups.com>) escribió:

[Jordan Empresas]

El espacio en disco:

[Jordan Empresas]

Estimado,

Ejecute este comando:

systemctl restart wazuh-dashboard

systemctl status wazuh-dashboard

 wazuh-dashboard.service - wazuh-dashboard
     Loaded: loaded (/etc/systemd/system/wazuh-dashboard.service; enabled; vendor preset: enabled)

     Active: active (running) since Fri 2024-12-27 16:14:29 UTC; 3s ago
   Main PID: 2707679 (node)
      Tasks: 11 (limit: 38494)
     Memory: 134.0M
        CPU: 3.790s
     CGroup: /system.slice/wazuh-dashboard.service
             └─2707679 /usr/share/wazuh-dashboard/node/bin/node /usr/share/wazuh-dashboard/src/cli/dist

y a los segundos se cae nuevamente:

Dec 27 16:09:38 ip-10-0-0-3 opensearch-dashboards[2707105]: {"type":"log","@timestamp":"2024-12-27T16:09:38Z","tags":>
Dec 27 16:09:38 ip-10-0-0-3 opensearch-dashboards[2707105]: {"type":"log","@timestamp":"2024-12-27T16:09:38Z","tags":>
Dec 27 16:09:38 ip-10-0-0-3 opensearch-dashboards[2707105]: {"type":"log","@timestamp":"2024-12-27T16:09:38Z","tags":>
Dec 27 16:09:38 ip-10-0-0-3 opensearch-dashboards[2707105]: {"type":"log","@timestamp":"2024-12-27T16:09:38Z","tags":>
Dec 27 16:09:38 ip-10-0-0-3 opensearch-dashboards[2707105]: {"type":"log","@timestamp":"2024-12-27T16:09:38Z","tags":>
Dec 27 16:09:38 ip-10-0-0-3 opensearch-dashboards[2707105]: {"type":"log","@timestamp":"2024-12-27T16:09:38Z","tags":>
Dec 27 16:09:39 ip-10-0-0-3 opensearch-dashboards[2707105]:  FATAL  {"error":{"root_cause":[{"type":"circuit_breaking>
Dec 27 16:09:39 ip-10-0-0-3 systemd[1]: wazuh-dashboard.service: Main process exited, code=exited, status=1/FAILURE
Dec 27 16:09:39 ip-10-0-0-3 systemd[1]: wazuh-dashboard.service: Failed with result 'exit-code'.
Dec 27 16:09:39 ip-10-0-0-3 systemd[1]: wazuh-dashboard.service: Consumed 7.966s CPU time.

[Jordan Empresas]

Estimado,

Muy agradecido por responder.

Te comparto los resultados:

Para el resultado de cluster health me sale esto:

Al darle enter en la segunda opción no me arroja ningún resultado.

Lo demás está en los archivos adjuntos.

La arquitectura es x64 sistema Ubuntu 20.04 LTS en AWS

El cluster lo desplegué de la siguiente forma:

 

Te aclaro que en esta configuración no tengo habilitado el balanceo por NGINX, no se si eso era necesario o si me este dando algún problema por no configurarlo?

Estaré atento a tu respuesta y opiniones.

Saludos,

El vie, 27 dic 2024 a las 0:31, 'Stuti Gupta' via Wazuh | Mailing List (<wa...@googlegroups.com>) escribió:

[Stuti Gupta]

Hi

The issue you're encountering with the Wazuh Dashboard not starting after upgrading to version 4.9 is related to the circuit_breaking_exception error, which is caused by hitting memory limits during data processing. This error occurs when memory usage surpasses a defined threshold to prevent the system from exceeding available memory, which could lead to crashes or performance issues.

To configure memory efficiently, it's recommended to set both the minimum and maximum heap memory to half of your system's total memory.

Start by checking the total memory on your server with the following command:The output will be something like this: For example, the system has 16 Gi (gigabytes) of total memory, so you should set the heap memory size to 8 Gi
free -h
Next, update the heap memory settings in the /etc/wazuh-indexer/jvm.options file by adjusting the Xms and Xmx values: Set both values to 8g, which is half of the system's total memory.

# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space

-Xms8g
-Xmx8g
Save the changes and restart the Wazuh Indexer to apply the new settings: systemctl restart wazuh-indexer

If the error persists, you may temporarily increase the heap size by 1 or 2 gigabytes as a short-term solution.

However, for long-term resolution, you should consider upgrading your system's memory or adding more indexer nodes to your cluster. You can find more details on scaling your setup here: https://documentation.wazuh.com/current/user-manual/wazuh-indexer-cluster.html

I hope this provides a clearer solution!