Failed to start Wazuh-indexer.

1,520 views
Skip to first unread message

Alex José Velasco Nunes

unread,
May 15, 2023, 8:56:39 AM5/15/23
to Wazuh mailing list
Anybody help me please with this problem?

× wazuh-indexer.service - Wazuh-indexer
     Loaded: loaded (/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Mon 2023-05-15 09:47:38 -03; 22s ago
       Docs: https://documentation.wazuh.com
    Process: 18275 ExecStart=/usr/share/wazuh-indexer/bin/systemd-entrypoint -p ${PID_DIR}/wazuh-indexer.pid --quiet (code=exited, status=1/FAILURE)
   Main PID: 18275 (code=exited, status=1/FAILURE)
        CPU: 2.098s

mai 15 09:47:38 wazuhmaster systemd-entrypoint[18405]: Error: A fatal exception has occurred. Program will exit.
mai 15 09:47:38 wazuhmaster systemd-entrypoint[18405]:         at org.opensearch.tools.launchers.JvmErgonomics.flagsFinal(JvmErgonomics.java:125)
mai 15 09:47:38 wazuhmaster systemd-entrypoint[18405]:         at org.opensearch.tools.launchers.JvmErgonomics.finalJvmOptions(JvmErgonomics.java:87)
mai 15 09:47:38 wazuhmaster systemd-entrypoint[18405]:         at org.opensearch.tools.launchers.JvmErgonomics.choose(JvmErgonomics.java:70)
mai 15 09:47:38 wazuhmaster systemd-entrypoint[18405]:         at org.opensearch.tools.launchers.JvmOptionsParser.jvmOptions(JvmOptionsParser.java:150)
mai 15 09:47:38 wazuhmaster systemd-entrypoint[18405]:         at org.opensearch.tools.launchers.JvmOptionsParser.main(JvmOptionsParser.java:108)
mai 15 09:47:38 wazuhmaster systemd[1]: wazuh-indexer.service: Main process exited, code=exited, status=1/FAILURE
mai 15 09:47:38 wazuhmaster systemd[1]: wazuh-indexer.service: Failed with result 'exit-code'.
mai 15 09:47:38 wazuhmaster systemd[1]: Failed to start Wazuh-indexer.
mai 15 09:47:38 wazuhmaster systemd[1]: wazuh-indexer.service: Consumed 2.098s CPU time.

Javier Bejar

unread,
May 15, 2023, 9:49:49 AM5/15/23
to Wazuh mailing list
Dear Alex,

I hope this email finds you well.

I understand that you are encountering difficulties with the Wazuh indexer service, specifically with its initiation. The error log suggests a potential issue with the JVM options and ergonomics, which indeed can be intricate to address.

In order to systematically diagnose and resolve this issue, I kindly request the following information:

- Wazuh Version: Could you please confirm the version of Wazuh you're currently employing?
- Indexer Configuration: Have you recently updated or altered the indexer configuration file (opensearch.yml)? If yes, could you share the updated configuration? This file is typically found at /etc/wazuh-indexer/opensearch.yml.
- Indexer Logs: Could you provide the error logs from the indexer? These can be obtained by running the command cat /var/log/wazuh-indexer/wazuh-cluster.log | grep error. These logs might contain essential clues to pinpoint the source of the problem.

Please ensure to redact any sensitive information before sharing.

Additionally, if there have been any recent changes or updates to your Wazuh server setup, kindly share those details as well.

I appreciate your cooperation and look forward to your response.

Regards, Javier.

Alex José Velasco Nunes

unread,
May 15, 2023, 3:22:40 PM5/15/23
to Wazuh mailing list
Hi Javier!


Captura de tela de 2023-05-15 16-18-39.png

opensearch.pngossec.pngCaptura de tela de 2023-05-15 16-19-27.png

This command cat /var/log/wazuh-indexer/wazuh-cluster.log | grep error do not exist here.

Javier Bejar

unread,
May 16, 2023, 6:03:04 AM5/16/23
to Wazuh mailing list
Hi Alex,

I appreciate your prompt response and the attached images of the logs and configuration files. However, in order to assist you more effectively, I kindly request the following:

- Wazuh Version: It's crucial for us to know the version of Wazuh you're currently using.
- Plain Text of Files/Logs: While the images provide some insights, having the plain text of the files and logs would be more helpful. The text format allows for easier parsing and searching for specific error patterns. Please attach the logs and configuration files in text format.
- Context and Modifications: If there have been any recent modifications or updates to your Wazuh server setup or configuration, kindly provide those details. Also, any additional context around the issue you're facing would be valuable. This can help us understand the circumstances that might have led to this issue.

I understand that these requests might require extra effort on your part. However, they are essential for us to diagnose and resolve the issue efficiently.

Thank you for your understanding and cooperation.

Best Regards,

Javier
Message has been deleted
Message has been deleted

Alex José Velasco Nunes

unread,
May 18, 2023, 1:06:05 PM5/18/23
to Wazuh mailing list
 wazuh-server-1 (172.16.10.220)
        Version: 4.4.1
        Type: master

Alex José Velasco Nunes

unread,
May 18, 2023, 1:06:54 PM5/18/23
to Wazuh mailing list
at opensearch.yml
node.master: true
node.data: true
node.ingest: true

cluster.name: wazuh-indexer-cluster
cluster.routing.allocation.disk.threshold_enabled: false

node.max_local_storage_nodes: "3"
path.data: /var/lib/wazuh-indexer
path.logs: /var/log/wazuh-indexer


plugins.security.ssl.http.pemcert_filepath: /etc/wazuh-indexer/certs/node-1.pem
plugins.security.ssl.http.pemkey_filepath: /etc/wazuh-indexer/certs/node-1-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: /etc/wazuh-indexer/certs/root-ca.pem
plugins.security.ssl.transport.pemcert_filepath: /etc/wazuh-indexer/certs/node-1.pem
plugins.security.ssl.transport.pemkey_filepath: /etc/wazuh-indexer/certs/node-1-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: /etc/wazuh-indexer/certs/root-ca.pem
plugins.security.ssl.http.enabled: true
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.transport.resolve_hostname: false
plugins.security.ssl.http.enabled_ciphers:
  - "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"
  - "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384"
  - "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256"
  - "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384"
plugins.security.ssl.http.enabled_protocols:
  - "TLSv1.2"
plugins.security.authcz.admin_dn:
- "CN=admin,OU=Wazuh,O=Wazuh,L=California,C=US"
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.restapi.roles_enabled:
- "all_access"
- "security_rest_api_access"

plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: [".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opendistro-notifications-*", ".opendistro-notebooks", ".opensearch-observability", ".opendistro-asynchronous-search-response*", ".replication-metadata-store"]

### Option to allow Filebeat-oss 7.10.2 to work ###
compatibility.override_main_response_version: true
node.name: node-1
network.host: 172.16.10.220
cluster.initial_master_nodes: node-1
plugins.security.nodes_dn:
        - CN=node-1,OU=Wazuh,O=Wazuh,L=California,C=US
Message has been deleted

Alex José Velasco Nunes

unread,
May 18, 2023, 1:07:41 PM5/18/23
to Wazuh mailing list
/var/ossec/bin/cluster_control -i more
Cluster name: wazuh_cluster

Connected nodes (1):


    wazuh-server-1 (172.16.10.220)
        Version: 4.4.1
        Type: master
        Active agents: 0

    wazuh-worker-1 (172.16.10.221)
        Version: 4.4.1
        Type: worker
        Active agents: 0
        Status:
            Last keep Alive:
                Last received: 2023-05-18T16:50:01.807483Z.
            Integrity check:
                Last integrity check: 0.004s (2023-05-18T16:50:10.868332Z - 2023-05-18T16:50:10.872515Z).
                Permission to check integrity: True.
            Integrity sync:
                Last integrity synchronization: n/a (n/a - n/a).
                Synchronized files: Shared: 0 | Missing: 0 | Extra: 0.
            Agents-info:
                Last synchronization: n/a (n/a - n/a).
                Number of synchronized chunks: 0.
                Permission to synchronize agent-info: True.

Alex José Velasco Nunes

unread,
May 18, 2023, 1:08:05 PM5/18/23
to Wazuh mailing list
tail /var/ossec/logs/cluster.log
2023/05/18 13:51:06 INFO: [Master] [Local integrity] Starting.
2023/05/18 13:51:06 INFO: [Master] [Local integrity] Finished in 0.003s. Calculated metadata of 34 files.
2023/05/18 13:51:13 INFO: [Master] [Local agent-groups] Starting.
2023/05/18 13:51:13 INFO: [Master] [Local agent-groups] Finished in 0.001s.
2023/05/18 13:51:13 INFO: [Worker wazuh-worker-1] [Integrity check] Starting.
2023/05/18 13:51:13 INFO: [Worker wazuh-worker-1] [Integrity check] Finished in 0.004s. Received metadata of 34 files. Sync not required.
2023/05/18 13:51:14 INFO: [Worker wazuh-worker-1] [Agent-groups send] Starting.
2023/05/18 13:51:14 INFO: [Worker wazuh-worker-1] [Agent-groups send] Finished in 0.005s. Updated 1 chunks.
2023/05/18 13:51:14 INFO: [Master] [Local integrity] Starting.
2023/05/18 13:51:14 INFO: [Master] [Local integrity] Finished in 0.003s. Calculated metadata of 34 files.

Alex José Velasco Nunes

unread,
May 18, 2023, 1:08:24 PM5/18/23
to Wazuh mailing list
cat /usr/share/wazuh-indexer/plugins/opensearch-security/tools/config.yml
nodes:
  # Wazuh indexer nodes
  indexer:
    - name: node-1
      ip: 172.16.10.220
    #- name: node-2
    #  ip: <indexer-node-ip>
    #- name: node-3
    #  ip: <indexer-node-ip>

  # Wazuh server nodes
  # If there is more than one Wazuh server
  # node, each one must have a node_type
  server:
    - name: wazuh-server-1
      ip: 172.16.10.220
      node_type: master
    - name: wazuh-worker-1
      ip: 172.16.10.221
      node_type: worker
    #- name: wazuh-3
    #  ip: <wazuh-manager-ip>
    #  node_type: worker

  # Wazuh dashboard nodes
  dashboard:
    - name: dashboard
      ip: 172.16.10.220

Alex José Velasco Nunes

unread,
May 18, 2023, 1:09:10 PM5/18/23
to Wazuh mailing list
cat /var/ossec/etc/ossec.conf
<!--
  Wazuh - Manager - Default configuration for ubuntu 22.04
  More info at: https://documentation.wazuh.com
  Mailing list: https://groups.google.com/forum/#!forum/wazuh
-->

<ossec_config>
  <global>
    <jsonout_output>yes</jsonout_output>
    <alerts_log>yes</alerts_log>
    <logall>no</logall>
    <logall_json>no</logall_json>
    <email_notification>no</email_notification>
    <smtp_server>smtp.example.wazuh.com</smtp_server>
    <email_from>wazuhexample.wazuh.com</email_from>
    <email_to>recipientexample.wazuh.com</email_to>
    <email_maxperhour>12</email_maxperhour>
    <email_log_source>alerts.log</email_log_source>
    <agents_disconnection_time>10m</agents_disconnection_time>
    <agents_disconnection_alert_time>0</agents_disconnection_alert_time>
  </global>

  <alerts>
    <log_alert_level>3</log_alert_level>
    <email_alert_level>12</email_alert_level>
  </alerts>

  <!-- Choose between "plain", "json", or "plain,json" for the format of internal logs -->
  <logging>
    <log_format>plain</log_format>
  </logging>

  <remote>
    <connection>secure</connection>
    <port>1514</port>
    <protocol>tcp</protocol>
    <queue_size>131072</queue_size>
  </remote>

  <!-- Policy monitoring -->
  <rootcheck>
    <disabled>no</disabled>
    <check_files>yes</check_files>
    <check_trojans>yes</check_trojans>
    <check_dev>yes</check_dev>
    <check_sys>yes</check_sys>
    <check_pids>yes</check_pids>
    <check_ports>yes</check_ports>
    <check_if>yes</check_if>

    <!-- Frequency that rootcheck is executed - every 12 hours -->
    <frequency>43200</frequency>

    <rootkit_files>etc/rootcheck/rootkit_files.txt</rootkit_files>
    <rootkit_trojans>etc/rootcheck/rootkit_trojans.txt</rootkit_trojans>

    <skip_nfs>yes</skip_nfs>
  </rootcheck>

  <wodle name="cis-cat">
    <disabled>yes</disabled>
    <timeout>1800</timeout>
    <interval>1d</interval>
    <scan-on-start>yes</scan-on-start>

    <java_path>wodles/java</java_path>
    <ciscat_path>wodles/ciscat</ciscat_path>
  </wodle>

  <!-- Osquery integration -->
  <wodle name="osquery">
    <disabled>yes</disabled>
    <run_daemon>yes</run_daemon>
    <log_path>/var/log/osquery/osqueryd.results.log</log_path>
    <config_path>/etc/osquery/osquery.conf</config_path>
    <add_labels>yes</add_labels>
  </wodle>

  <!-- System inventory -->
  <wodle name="syscollector">
    <disabled>no</disabled>
    <interval>1h</interval>
    <scan_on_start>yes</scan_on_start>
    <hardware>yes</hardware>
    <os>yes</os>
    <network>yes</network>
    <packages>yes</packages>
    <ports all="no">yes</ports>
    <processes>yes</processes>

    <!-- Database synchronization settings -->
    <synchronization>
      <max_eps>10</max_eps>
    </synchronization>
  </wodle>

  <sca>
    <enabled>yes</enabled>
    <scan_on_start>yes</scan_on_start>
    <interval>12h</interval>
    <skip_nfs>yes</skip_nfs>
  </sca>

  <vulnerability-detector>
    <enabled>no</enabled>
    <interval>5m</interval>
    <min_full_scan_interval>6h</min_full_scan_interval>
    <run_on_start>yes</run_on_start>

    <!-- Ubuntu OS vulnerabilities -->
    <provider name="canonical">
      <enabled>no</enabled>
      <os>trusty</os>
      <os>xenial</os>
      <os>bionic</os>
      <os>focal</os>
      <os>jammy</os>
      <update_interval>1h</update_interval>
    </provider>

    <!-- Debian OS vulnerabilities -->
    <provider name="debian">
      <enabled>no</enabled>
      <os>buster</os>
      <os>bullseye</os>
      <update_interval>1h</update_interval>
    </provider>

    <!-- RedHat OS vulnerabilities -->
    <provider name="redhat">
      <enabled>no</enabled>
      <os>5</os>
      <os>6</os>
      <os>7</os>
      <os>8</os>
      <os>9</os>
      <update_interval>1h</update_interval>
    </provider>

    <!-- Amazon Linux OS vulnerabilities -->
    <provider name="alas">
      <enabled>no</enabled>
      <os>amazon-linux</os>
      <os>amazon-linux-2</os>
      <update_interval>1h</update_interval>
    </provider>

    <!-- SUSE OS vulnerabilities -->
    <provider name="suse">
      <enabled>no</enabled>
      <os>11-server</os>
      <os>11-desktop</os>
      <os>12-server</os>
      <os>12-desktop</os>
      <os>15-server</os>
      <os>15-desktop</os>
      <update_interval>1h</update_interval>
    </provider>

    <!-- Arch OS vulnerabilities -->
    <provider name="arch">
      <enabled>no</enabled>
      <update_interval>1h</update_interval>
    </provider>

    <!-- Windows OS vulnerabilities -->
    <provider name="msu">
      <enabled>yes</enabled>
      <update_interval>1h</update_interval>
    </provider>

    <!-- Aggregate vulnerabilities -->
    <provider name="nvd">
      <enabled>yes</enabled>
      <update_from_year>2010</update_from_year>
      <update_interval>1h</update_interval>
    </provider>

  </vulnerability-detector>

  <!-- File integrity monitoring -->
  <syscheck>
    <disabled>no</disabled>

    <!-- Frequency that syscheck is executed default every 12 hours -->
    <frequency>43200</frequency>

    <scan_on_start>yes</scan_on_start>

    <!-- Generate alert when new file detected -->
    <alert_new_files>yes</alert_new_files>

    <!-- Don't ignore files that change more than 'frequency' times -->
    <auto_ignore frequency="10" timeframe="3600">no</auto_ignore>

    <!-- Directories to check  (perform all possible verifications) -->
    <directories>/etc,/usr/bin,/usr/sbin</directories>
    <directories>/bin,/sbin,/boot</directories>

    <!-- Files/directories to ignore -->
    <ignore>/etc/mtab</ignore>
    <ignore>/etc/hosts.deny</ignore>
    <ignore>/etc/mail/statistics</ignore>
    <ignore>/etc/random-seed</ignore>
    <ignore>/etc/random.seed</ignore>
    <ignore>/etc/adjtime</ignore>
    <ignore>/etc/httpd/logs</ignore>
    <ignore>/etc/utmpx</ignore>
    <ignore>/etc/wtmpx</ignore>
    <ignore>/etc/cups/certs</ignore>
    <ignore>/etc/dumpdates</ignore>
    <ignore>/etc/svc/volatile</ignore>

    <!-- File types to ignore -->
    <ignore type="sregex">.log$|.swp$</ignore>

    <!-- Check the file, but never compute the diff -->
    <nodiff>/etc/ssl/private.key</nodiff>

    <skip_nfs>yes</skip_nfs>
    <skip_dev>yes</skip_dev>
    <skip_proc>yes</skip_proc>
    <skip_sys>yes</skip_sys>

    <!-- Nice value for Syscheck process -->
    <process_priority>10</process_priority>

    <!-- Maximum output throughput -->
    <max_eps>100</max_eps>

    <!-- Database synchronization settings -->
    <synchronization>
      <enabled>yes</enabled>
      <interval>5m</interval>
      <max_interval>1h</max_interval>
      <max_eps>10</max_eps>
    </synchronization>
  </syscheck>

  <!-- Active response -->
  <global>
    <white_list>127.0.0.1</white_list>
    <white_list>^localhost.localdomain$</white_list>
    <white_list>127.0.0.53</white_list>
  </global>

  <command>
    <name>disable-account</name>
    <executable>disable-account</executable>
    <timeout_allowed>yes</timeout_allowed>
  </command>

  <command>
    <name>restart-wazuh</name>
    <executable>restart-wazuh</executable>
  </command>

  <command>
    <name>firewall-drop</name>
    <executable>firewall-drop</executable>
    <timeout_allowed>yes</timeout_allowed>
  </command>

  <command>
    <name>host-deny</name>
    <executable>host-deny</executable>
    <timeout_allowed>yes</timeout_allowed>
  </command>

  <command>
    <name>route-null</name>
    <executable>route-null</executable>
    <timeout_allowed>yes</timeout_allowed>
  </command>

  <command>
    <name>win_route-null</name>
    <executable>route-null.exe</executable>
    <timeout_allowed>yes</timeout_allowed>
  </command>

  <command>
    <name>netsh</name>
    <executable>netsh.exe</executable>
    <timeout_allowed>yes</timeout_allowed>
  </command>

  <!--
  <active-response>
    active-response options here
  </active-response>
  -->

  <!-- Log analysis -->
  <localfile>
    <log_format>command</log_format>
    <command>df -P</command>
    <frequency>360</frequency>
  </localfile>

  <localfile>
    <log_format>full_command</log_format>
    <command>netstat -tulpn | sed 's/\([[:alnum:]]\+\)\ \+[[:digit:]]\+\ \+[[:digit:]]\+\ \+\(.*\):\([[:digit:]]*\)\ \+\([0-9\.\:\*]\+\).\+\ \([[:digit:]]*\/[[:alnum:]\-]*\).*/\1 \2 == \3 == \4 \5/' | sort -k 4 -g | sed 's/ == \(.*\) ==/:\1/' | sed 1,2d</command>
    <alias>netstat listening ports</alias>
    <frequency>360</frequency>
  </localfile>

  <localfile>
    <log_format>full_command</log_format>
    <command>last -n 20</command>
    <frequency>360</frequency>
  </localfile>

  <ruleset>
    <!-- Default ruleset -->
    <decoder_dir>ruleset/decoders</decoder_dir>
    <rule_dir>ruleset/rules</rule_dir>
    <rule_exclude>0215-policy_rules.xml</rule_exclude>
    <list>etc/lists/audit-keys</list>
    <list>etc/lists/amazon/aws-eventnames</list>
    <list>etc/lists/security-eventchannel</list>

    <!-- User-defined ruleset -->
    <decoder_dir>etc/decoders</decoder_dir>
    <rule_dir>etc/rules</rule_dir>
  </ruleset>

  <rule_test>
    <enabled>yes</enabled>
    <threads>1</threads>
    <max_sessions>64</max_sessions>
    <session_timeout>15m</session_timeout>
  </rule_test>

  <!-- Configuration for wazuh-authd -->
  <auth>
    <disabled>no</disabled>
    <port>1515</port>
    <use_source_ip>no</use_source_ip>
    <purge>yes</purge>
    <use_password>no</use_password>
    <ciphers>HIGH:!ADH:!EXP:!MD5:!RC4:!3DES:!CAMELLIA:@STRENGTH</ciphers>
    <!-- <ssl_agent_ca></ssl_agent_ca> -->
    <ssl_verify_host>no</ssl_verify_host>
    <ssl_manager_cert>etc/sslmanager.cert</ssl_manager_cert>
    <ssl_manager_key>etc/sslmanager.key</ssl_manager_key>
    <ssl_auto_negotiate>no</ssl_auto_negotiate>
  </auth>

  <cluster>
    <name>wazuh_cluster</name>
    <node_name>wazuh-server-1</node_name>
    <node_type>master</node_type>
    <key>3a6827917a73ea16d672d32efc842638</key>
    <port>1516</port>
    <bind_addr>172.16.10.220</bind_addr>
    <nodes>
        <node>172.16.10.220</node>
    </nodes>
    <hidden>no</hidden>
    <disabled>no</disabled>
  </cluster>

</ossec_config>

<ossec_config>
  <localfile>
    <log_format>syslog</log_format>
    <location>/var/ossec/logs/active-responses.log</location>
  </localfile>

  <localfile>
    <log_format>syslog</log_format>
    <location>/var/log/auth.log</location>
  </localfile>

  <localfile>
    <log_format>syslog</log_format>
    <location>/var/log/syslog</location>
  </localfile>

  <localfile>
    <log_format>syslog</log_format>
    <location>/var/log/dpkg.log</location>
  </localfile>

  <localfile>
    <log_format>syslog</log_format>
    <location>/var/log/kern.log</location>
  </localfile>

</ossec_config>

Alex José Velasco Nunes

unread,
May 18, 2023, 1:09:34 PM5/18/23
to Wazuh mailing list
systemctl status wazuh-indexer

× wazuh-indexer.service - Wazuh-indexer
     Loaded: loaded (/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Thu 2023-05-18 13:19:32 -03; 36min ago
       Docs: https://documentation.wazuh.com
    Process: 3032 ExecStart=/usr/share/wazuh-indexer/bin/systemd-entrypoint -p ${PID_DIR}/wazuh-indexer.pid --quiet (code=exited, status=1/FAILURE)
   Main PID: 3032 (code=exited, status=1/FAILURE)
        CPU: 2.007s

mai 18 13:19:32 wazuhmaster systemd-entrypoint[3162]: Error: A fatal exception has occurred. Program will exit.
mai 18 13:19:32 wazuhmaster systemd-entrypoint[3162]:         at org.opensearch.tools.launchers.JvmErgonomics.flagsFinal(JvmErgonomics.java:125)
mai 18 13:19:32 wazuhmaster systemd-entrypoint[3162]:         at org.opensearch.tools.launchers.JvmErgonomics.finalJvmOptions(JvmErgonomics.java:87)
mai 18 13:19:32 wazuhmaster systemd-entrypoint[3162]:         at org.opensearch.tools.launchers.JvmErgonomics.choose(JvmErgonomics.java:70)
mai 18 13:19:32 wazuhmaster systemd-entrypoint[3162]:         at org.opensearch.tools.launchers.JvmOptionsParser.jvmOptions(JvmOptionsParser.java:150)
mai 18 13:19:32 wazuhmaster systemd-entrypoint[3162]:         at org.opensearch.tools.launchers.JvmOptionsParser.main(JvmOptionsParser.java:108)
mai 18 13:19:32 wazuhmaster systemd[1]: wazuh-indexer.service: Main process exited, code=exited, status=1/FAILURE
mai 18 13:19:32 wazuhmaster systemd[1]: wazuh-indexer.service: Failed with result 'exit-code'.
mai 18 13:19:32 wazuhmaster systemd[1]: Failed to start Wazuh-indexer.
mai 18 13:19:32 wazuhmaster systemd[1]: wazuh-indexer.service: Consumed 2.007s CPU time.

Alex José Velasco Nunes

unread,
May 18, 2023, 1:09:57 PM5/18/23
to Wazuh mailing list
journalctl -xeu wazuh-indexer.service
░░
░░ The job identifier is 1404 and the job result is failed.

mai 18 13:19:32 wazuhmaster systemd[1]: wazuh-indexer.service: Consumed 2.007s CPU time.
░░ Subject: Resources consumed by unit runtime
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ The unit wazuh-indexer.service completed and consumed the indicated resources.
mai 18 13:56:41 wazuhmaster systemd[1]: Starting Wazuh-indexer...
░░ Subject: A start job for unit wazuh-indexer.service has begun execution
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ A start job for unit wazuh-indexer.service has begun execution.
░░
░░ The job identifier is 1491.
mai 18 13:56:42 wazuhmaster systemd-entrypoint[3490]: Exception in thread "main" java.lang.RuntimeException: starting java failed with [1]
mai 18 13:56:42 wazuhmaster systemd-entrypoint[3490]: output:
mai 18 13:56:42 wazuhmaster systemd-entrypoint[3490]: [0.000s][error][logging] Error opening log file '/var/log/wazuh-indexer/gc.log': No such file or directory
mai 18 13:56:42 wazuhmaster systemd-entrypoint[3490]: [0.000s][error][logging] Initialization of output 'file=/var/log/wazuh-indexer/gc.log' using options 'filecount=32,filesize=64m' failed.
mai 18 13:56:42 wazuhmaster systemd-entrypoint[3490]: error:
mai 18 13:56:42 wazuhmaster systemd-entrypoint[3490]: Invalid -Xlog option '-Xlog:gc*,gc+age=trace,safepoint:file=/var/log/wazuh-indexer/gc.log:utctime,pid,tags:filecount=32,filesize=64m', see error log for details.
mai 18 13:56:42 wazuhmaster systemd-entrypoint[3490]: Error: Could not create the Java Virtual Machine.
mai 18 13:56:42 wazuhmaster systemd-entrypoint[3490]: Error: A fatal exception has occurred. Program will exit.
mai 18 13:56:42 wazuhmaster systemd-entrypoint[3490]:         at org.opensearch.tools.launchers.JvmErgonomics.flagsFinal(JvmErgonomics.java:125)
mai 18 13:56:42 wazuhmaster systemd-entrypoint[3490]:         at org.opensearch.tools.launchers.JvmErgonomics.finalJvmOptions(JvmErgonomics.java:87)
mai 18 13:56:42 wazuhmaster systemd-entrypoint[3490]:         at org.opensearch.tools.launchers.JvmErgonomics.choose(JvmErgonomics.java:70)
mai 18 13:56:42 wazuhmaster systemd-entrypoint[3490]:         at org.opensearch.tools.launchers.JvmOptionsParser.jvmOptions(JvmOptionsParser.java:150)
mai 18 13:56:42 wazuhmaster systemd-entrypoint[3490]:         at org.opensearch.tools.launchers.JvmOptionsParser.main(JvmOptionsParser.java:108)
mai 18 13:56:42 wazuhmaster systemd[1]: wazuh-indexer.service: Main process exited, code=exited, status=1/FAILURE
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ An ExecStart= process belonging to unit wazuh-indexer.service has exited.
░░
░░ The process' exit code is 'exited' and its exit status is 1.
mai 18 13:56:42 wazuhmaster systemd[1]: wazuh-indexer.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ The unit wazuh-indexer.service has entered the 'failed' state with result 'exit-code'.
mai 18 13:56:42 wazuhmaster systemd[1]: Failed to start Wazuh-indexer.
░░ Subject: A start job for unit wazuh-indexer.service has failed
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ A start job for unit wazuh-indexer.service has finished with a failure.
░░
░░ The job identifier is 1491 and the job result is failed.
mai 18 13:56:42 wazuhmaster systemd[1]: wazuh-indexer.service: Consumed 2.103s CPU time.
░░ Subject: Resources consumed by unit runtime
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ The unit wazuh-indexer.service completed and consumed the indicated resources.

Alex José Velasco Nunes

unread,
May 18, 2023, 1:12:06 PM5/18/23
to Wazuh mailing list
filebeat test output
elasticsearch: https://172.16.10.220:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 172.16.10.220
    dial up... ERROR dial tcp 172.16.10.220:9200: connect: connection refused


Em terça-feira, 16 de maio de 2023 às 07:03:04 UTC-3, Javier Bejar escreveu:

Alex José Velasco Nunes

unread,
May 18, 2023, 1:14:48 PM5/18/23
to Wazuh mailing list
lsof /var/ossec/logs/alerts/alerts.json
COMMAND    PID  USER   FD   TYPE DEVICE SIZE/OFF   NODE NAME
wazuh-ana 1203 wazuh   12w   REG  253,0    12048 812100 /var/ossec/logs/alerts/alerts.json
Em terça-feira, 16 de maio de 2023 às 07:03:04 UTC-3, Javier Bejar escreveu:

Javier Bejar

unread,
May 22, 2023, 10:21:26 AM5/22/23
to Wazuh mailing list
Dear Alex,

Thank you for your continued cooperation.

Based on the output from the journalctl command you provided, it appears that there may be an issue with opening the file '/var/log/wazuh-indexer/gc.log'. This could potentially be caused by incorrect file permissions or insufficient disk space.

To diagnose this further, could you please do the following?

Verify File Permissions: Check the permissions of the file '/var/log/wazuh-indexer/gc.log' using the command 'ls -l /var/log/wazuh-indexer/gc.log'. The Wazuh indexer service should have the necessary permissions to read and write to this file.

I.e:
root@engine:/home/vagrant/engine/wazuh/src/engine# ls -l /var/log/wazuh-indexer/
total 1076
-rw-r--r-- 1 wazuh-indexer wazuh-indexer 162274 Apr 28 16:21 gc.log
-rw-r--r-- 1 wazuh-indexer wazuh-indexer   2045 Apr 14 08:01 gc.log.00
-rw-r--r-- 1 wazuh-indexer wazuh-indexer  56124 Apr 14 08:04 gc.log.01
-rw-r--r-- 1 wazuh-indexer wazuh-indexer   2069 Apr 24 13:34 gc.log.02
-rw-r--r-- 1 wazuh-indexer wazuh-indexer  54887 Apr 24 13:37 gc.log.03
-rw-r--r-- 1 wazuh-indexer wazuh-indexer   2069 Apr 24 15:44 gc.log.04
-rw-r--r-- 1 wazuh-indexer wazuh-indexer 122710 Apr 24 16:27 gc.log.05
-rw-r--r-- 1 wazuh-indexer wazuh-indexer   2069 Apr 27 11:10 gc.log.06
-rw-r--r-- 1 wazuh-indexer wazuh-indexer 131076 Apr 27 12:45 gc.log.07
-rw-r--r-- 1 wazuh-indexer wazuh-indexer   2069 Apr 27 13:51 gc.log.08
-rw-r--r-- 1 wazuh-indexer wazuh-indexer 223253 Apr 27 16:47 gc.log.09
-rw-r--r-- 1 wazuh-indexer wazuh-indexer   2069 Apr 28 15:23 gc.log.10
-rw-r--r-- 1 wazuh-indexer wazuh-indexer   8726 Apr 24 13:34 wazuh-cluster-2023-04-14-1.json.gz
-rw-r--r-- 1 wazuh-indexer wazuh-indexer   8090 Apr 24 13:34 wazuh-cluster-2023-04-14-1.log.gz
-rw-r--r-- 1 wazuh-indexer wazuh-indexer  14401 Apr 27 11:10 wazuh-cluster-2023-04-24-1.json.gz
-rw-r--r-- 1 wazuh-indexer wazuh-indexer  13166 Apr 27 11:10 wazuh-cluster-2023-04-24-1.log.gz
-rw-r--r-- 1 wazuh-indexer wazuh-indexer  16549 Apr 28 15:23 wazuh-cluster-2023-04-27-1.json.gz
-rw-r--r-- 1 wazuh-indexer wazuh-indexer  14939 Apr 28 15:23 wazuh-cluster-2023-04-27-1.log.gz
-rw-r----- 1 wazuh-indexer wazuh-indexer  18719 Apr 28 15:40 wazuh-cluster_deprecation.json
-rw-r----- 1 wazuh-indexer wazuh-indexer   9749 Apr 28 15:40 wazuh-cluster_deprecation.log
-rw-r----- 1 wazuh-indexer wazuh-indexer      0 Apr 14 08:01 wazuh-cluster_index_indexing_slowlog.json
-rw-r----- 1 wazuh-indexer wazuh-indexer      0 Apr 14 08:01 wazuh-cluster_index_indexing_slowlog.log
-rw-r----- 1 wazuh-indexer wazuh-indexer      0 Apr 14 08:01 wazuh-cluster_index_search_slowlog.json
-rw-r----- 1 wazuh-indexer wazuh-indexer      0 Apr 14 08:01 wazuh-cluster_index_search_slowlog.log
-rw-r--r-- 1 wazuh-indexer wazuh-indexer  48399 Apr 28 16:18 wazuh-cluster.log
-rw-r--r-- 1 wazuh-indexer wazuh-indexer 103846 Apr 28 16:18 wazuh-cluster_server.json
-rw-r----- 1 wazuh-indexer wazuh-indexer      0 Apr 14 08:01 wazuh-cluster_task_detailslog.json
-rw-r----- 1 wazuh-indexer wazuh-indexer      0 Apr 14 08:01 wazuh-cluster_task_detailslog.log


Check Disk Space: Please ensure that your disk has enough free space. You can check the available disk space using the command df -h. Insufficient disk space can often lead to unexpected behavior in services like Wazuh indexer.

Once we have this information, we can determine the next steps to resolve your issue.

Thank you and looking forward to your reply.

Best Regards,

Javier
Reply all
Reply to author
Forward
0 new messages