Kibana Server is not ready

1,105 views
Skip to first unread message

Bill Green

unread,
Mar 30, 2021, 8:30:29 AM3/30/21
to Wazuh mailing list
Good morning,

I appreciate any help you can provide - I've read through a number of support sites and I'm still at an impasse.  After a reboot of our Ubuntu 20.04 server to restart all services, when I attempted to log into the Wazuh portal I got the dreaded Kibana Server is Not Ready message.  I waited some time in case I had been too hasty attempting to login, but hours later (and now a day later) the message remains.

My apologies if the solution is right in front of me - I'm just getting up to speed on Linux and Wazuh.  Below are the .yml.files for Elasticsearch, filebeat and kibana, along with a variety of logs, but, I'm happy to provide additional information if needed.

The issue appears to be an authentication issue, but, I'm a little at a loss as I've rebooted the server before after applying patches and did not receive the error.  I didn't patch anything yesterday, but, I do tend to reboot boxes only I use on Monday mornings to free up resources, etc. (Guess that is the Windows guy in me).

Greatly appreciate any help!
Thanks,
Bill

Elasticsearch.yml

network.host: xxx.xxx.32.33
node.name: node-1
cluster.initial_master_nodes: node-1

opendistro_security.ssl.transport.pemcert_filepath: /etc/elasticsearch/certs/elasticsearch.pem
opendistro_security.ssl.transport.pemkey_filepath: /etc/elasticsearch/certs/elasticsearch.key
opendistro_security.ssl.transport.pemtrustedcas_filepath: /etc/elasticsearch/certs/root-ca.pem
opendistro_security.ssl.transport.enforce_hostname_verification: false
opendistro_security.ssl.transport.resolve_hostname: false
opendistro_security.ssl.http.enabled: true
opendistro_security.ssl.http.pemcert_filepath: /etc/elasticsearch/certs/elasticsearch_http.pem
opendistro_security.ssl.http.pemkey_filepath: /etc/elasticsearch/certs/elasticsearch_http.key
opendistro_security.ssl.http.pemtrustedcas_filepath: /etc/elasticsearch/certs/root-ca.pem
opendistro_security.nodes_dn:
- CN=node-1,OU=Docu,O=Wazuh,L=California,C=US
opendistro_security.authcz.admin_dn:
- CN=admin,OU=Docu,O=Wazuh,L=California,C=US

opendistro_security.audit.type: internal_elasticsearch
opendistro_security.enable_snapshot_restore_privilege: true
opendistro_security.check_snapshot_restore_write_privileges: true
opendistro_security.restapi.roles_enabled: ["all_access", "security_rest_api_access"]
cluster.routing.allocation.disk.threshold_enabled: false
node.max_local_storage_nodes: 3

path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch


Filebeat.yml
# Wazuh - Filebeat configuration file
output.elasticsearch:
  hosts: ["xxx.xxx.32.33:9200"]
  username: "admin"
  password: "xxxxxxxxx"
  ssl.verification_mode: none
  protocol: https
  #username: "Internal Filebeat User"
  #password: "xxxxxxxxxxxxxxxxxxxxxxxx"
  ssl.certificate_authorities:
    - /etc/filebeat/certs/root-ca.pem
  ssl.certificate: "/etc/filebeat/certs/filebeat.pem"
  ssl.key: "/etc/filebeat/certs/filebeat.key"
setup.template.json.enabled: true
setup.template.json.path: '/etc/filebeat/wazuh-template.json'
setup.ilm.overwrite: true
setup.ilm.enabled: false

filebeat.modules:
  - module: wazuh
    alerts:
      enabled: true
    archives:
      enabled: false


Kibana.yml
server.host: xxx.xxx.32.33
server.port: 443
elasticsearch.hosts: https://xxx.xxx.32.33:9200
elasticsearch.ssl.verificationMode: certificate
elasticsearch.username: kibanaserver
elasticsearch.password: kibanaserver
elasticsearch.requestHeadersWhitelist: ["securitytenant","Authorization"]
opendistro_security.multitenancy.enabled: true
opendistro_security.readonly_mode.roles: ["kibana_read_only"]
server.ssl.enabled: true
server.ssl.key: "/etc/kibana/certs/kibana.key"
server.ssl.certificate: "/etc/kibana/certs/kibana.pem"
elasticsearch.ssl.certificateAuthorities: ["/etc/kibana/certs/root-ca.pem"]
server.defaultRoute: /app/wazuh

systemctl status kibana
● kibana.service - Kibana
     Loaded: loaded (/etc/systemd/system/kibana.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2021-03-29 16:18:11 CDT; 14h ago
   Main PID: 1195 (node)
      Tasks: 11 (limit: 43202)
     Memory: 168.9M
     CGroup: /system.slice/kibana.service
             └─1195 /usr/share/kibana/bin/../node/bin/node /usr/share/kibana/bin/../src/cli/dist -c /etc/kibana/kibana.yml

Mar 30 06:47:16 secon kibana[1195]: {"type":"log","@timestamp":"2021-03-30T11:47:16Z","tags":["error","elasticsearch","data"],"pid":1195,"message":"[ResponseError]: Response Error"}
Mar 30 06:47:19 secon kibana[1195]: {"type":"log","@timestamp":"2021-03-30T11:47:19Z","tags":["error","elasticsearch","data"],"pid":1195,"message":"[ResponseError]: Response Error"}
Mar 30 06:47:21 secon kibana[1195]: {"type":"log","@timestamp":"2021-03-30T11:47:21Z","tags":["error","elasticsearch","data"],"pid":1195,"message":"[ResponseError]: Response Error"}
Mar 30 06:47:24 secon kibana[1195]: {"type":"log","@timestamp":"2021-03-30T11:47:24Z","tags":["error","elasticsearch","data"],"pid":1195,"message":"[ResponseError]: Response Error"}
Mar 30 06:47:26 secon kibana[1195]: {"type":"log","@timestamp":"2021-03-30T11:47:26Z","tags":["error","elasticsearch","data"],"pid":1195,"message":"[ResponseError]: Response Error"}
Mar 30 06:47:29 secon kibana[1195]: {"type":"log","@timestamp":"2021-03-30T11:47:29Z","tags":["error","elasticsearch","data"],"pid":1195,"message":"[ResponseError]: Response Error"}
Mar 30 06:47:31 secon kibana[1195]: {"type":"log","@timestamp":"2021-03-30T11:47:31Z","tags":["error","elasticsearch","data"],"pid":1195,"message":"[ResponseError]: Response Error"}
Mar 30 06:47:34 secon kibana[1195]: {"type":"log","@timestamp":"2021-03-30T11:47:34Z","tags":["error","elasticsearch","data"],"pid":1195,"message":"[ResponseError]: Response Error"}
Mar 30 06:47:36 secon kibana[1195]: {"type":"log","@timestamp":"2021-03-30T11:47:36Z","tags":["error","elasticsearch","data"],"pid":1195,"message":"[ResponseError]: Response Error"}
Mar 30 06:47:39 secon kibana[1195]: {"type":"log","@timestamp":"2021-03-30T11:47:39Z","tags":["error","elasticsearch","data"],"pid":1195,"message":"[ResponseError]: Response Error"}
Mar 30 06:47:41 secon kibana[1195]: {"type":"log","@timestamp":"2021-03-30T11:47:41Z","tags":["error","elasticsearch","data"],"pid":1195,"message":"[ResponseError]: Response Error"}
Mar 30 06:47:44 secon kibana[1195]: {"type":"log","@timestamp":"2021-03-30T11:47:44Z","tags":["error","elasticsearch","data"],"pid":1195,"message":"[ResponseError]: Response Er


root@secon:/# filebeat test output
elasticsearch: https://xxx.xxx.32.33:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 150.201.32.33
    dial up... OK
  TLS...
    security... WARN server's certificate chain verification is disabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
  talk to server... OK
  version: 7.10.0
root@secon:/#



root@secon:/etc/kibana# systemctl status filebeat
● filebeat.service - Filebeat sends log files to Logstash or directly to Elasticsearch.
     Loaded: loaded (/lib/systemd/system/filebeat.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2021-03-29 16:18:33 CDT; 14h ago
   Main PID: 1701 (filebeat)
      Tasks: 19 (limit: 43202)
     Memory: 76.3M
     CGroup: /system.slice/filebeat.service
             └─1701 /usr/share/filebeat/bin/filebeat --environment systemd -c /etc/filebeat/filebeat.yml --path.home /usr/share/filebeat --path.config /etc/filebeat --path.data /var/lib/filebeat --path.logs /var/log/filebeat

Mar 30 06:44:10 secon filebeat[1701]: 2021-03-30T06:44:10.209-0500        INFO        [monitoring]        log/log.go:145        Non-zero metrics in the last 30s        {"monitoring": {"metrics": {"beat":{"cgroup":{"cpuacct":{"total":{"ns":163201919}},"memory":{"mem":{"usage":{"bytes":8192}}}},"cpu":{"system":{"tick>
Mar 30 06:44:40 secon filebeat[1701]: 2021-03-30T06:44:40.210-0500        INFO        [monitoring]        log/log.go:145        Non-zero metrics in the last 30s        {"monitoring": {"metrics": {"beat":{"cgroup":{"cpuacct":{"total":{"ns":191861224}},"memory":{"mem":{"usage":{"bytes":12288}}}},"cpu":{"system":{"tic>
Mar 30 06:45:10 secon filebeat[1701]: 2021-03-30T06:45:10.209-0500        INFO        [monitoring]        log/log.go:145        Non-zero metrics in the last 30s        {"monitoring": {"metrics": {"beat":{"cgroup":{"cpuacct":{"total":{"ns":204897306}},"memory":{"mem":{"usage":{"bytes":8192}}}},"cpu":{"system":{"tick>
Mar 30 06:45:40 secon filebeat[1701]: 2021-03-30T06:45:40.210-0500        INFO        [monitoring]        log/log.go:145        Non-zero metrics in the last 30s        {"monitoring": {"metrics": {"beat":{"cgroup":{"cpuacct":{"total":{"ns":177108874}},"memory":{"mem":{"usage":{"bytes":8192}}}},"cpu":{"system":{"tick>
Mar 30 06:46:10 secon filebeat[1701]: 2021-03-30T06:46:10.209-0500        INFO        [monitoring]        log/log.go:145        Non-zero metrics in the last 30s        {"monitoring": {"metrics": {"beat":{"cgroup":{"cpuacct":{"total":{"ns":180737381}},"memory":{"mem":{"usage":{"bytes":139264}}}},"cpu":{"system":{"ti>
Mar 30 06:46:40 secon filebeat[1701]: 2021-03-30T06:46:40.209-0500        INFO        [monitoring]        log/log.go:145        Non-zero metrics in the last 30s        {"monitoring": {"metrics": {"beat":{"cgroup":{"cpuacct":{"total":{"ns":229196684}},"memory":{"mem":{"usage":{"bytes":-114688}}}},"cpu":{"system":{"t>
Mar 30 06:47:10 secon filebeat[1701]: 2021-03-30T06:47:10.210-0500        INFO        [monitoring]        log/log.go:145        Non-zero metrics in the last 30s        {"monitoring": {"metrics": {"beat":{"cgroup":{"cpuacct":{"total":{"ns":255212518}},"memory":{"mem":{"usage":{"bytes":8192}}}},"cpu":{"system":{"tick>
Mar 30 06:47:40 secon filebeat[1701]: 2021-03-30T06:47:40.209-0500        INFO        [monitoring]        log/log.go:145        Non-zero metrics in the last 30s        {"monitoring": {"metrics": {"beat":{"cgroup":{"cpuacct":{"total":{"ns":176423137}},"memory":{"mem":{"usage":{"bytes":8192}}}},"cpu":{"system":{"tick>
Mar 30 06:48:10 secon filebeat[1701]: 2021-03-30T06:48:10.209-0500        INFO        [monitoring]        log/log.go:145        Non-zero metrics in the last 30s        {"monitoring": {"metrics": {"beat":{"cgroup":{"cpuacct":{"total":{"ns":144247987}},"memory":{"mem":{"usage":{"bytes":8192}}}},"cpu":{"system":{"tick>
Mar 30 06:48:40 secon filebeat[1701]: 2021-03-30T06:48:40.210-0500        INFO        [monitoring]        log/log.go:145        Non-zero metrics in the last 30s        {"monitoring": {"metrics": {"beat":{"cgroup":{"cpuacct":{"total":{"ns":166625309}},"memory":{"mem":{"usage":{"bytes":8192}}}},"cpu":{"system":{"tick>
lines 1-20/20 (END)


root@secon:/etc/kibana# systemctl status elasticsearch
● elasticsearch.service - Elasticsearch
     Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2021-03-29 16:19:16 CDT; 14h ago
       Docs: https://www.elastic.co
   Main PID: 1700 (java)
      Tasks: 171 (limit: 43202)
     Memory: 21.1G
     CGroup: /system.slice/elasticsearch.service
             └─1700 /usr/share/elasticsearch/jdk/bin/java -Xshare:auto -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -XX:+ShowCodeDetailsInExceptionMessages -Dio.net>

Mar 29 16:18:33 secon systemd[1]: Starting Elasticsearch...
Mar 29 16:19:15 secon systemd-entrypoint[1700]: WARNING: An illegal reflective access operation has occurred
Mar 29 16:19:15 secon systemd-entrypoint[1700]: WARNING: Illegal reflective access by com.amazon.opendistro.elasticsearch.performanceanalyzer.collectors.MasterServiceEventMetrics (file:/usr/share/elasticsearch/plugins/opendistro_performance_analyzer/opendistro_performance_analyzer-1.12.0.0.jar) to field java.util.c>
Mar 29 16:19:15 secon systemd-entrypoint[1700]: WARNING: Please consider reporting this to the maintainers of com.amazon.opendistro.elasticsearch.performanceanalyzer.collectors.MasterServiceEventMetrics
Mar 29 16:19:15 secon systemd-entrypoint[1700]: WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
Mar 29 16:19:15 secon systemd-entrypoint[1700]: WARNING: All illegal access operations will be denied in a future release
Mar 29 16:19:16 secon systemd[1]: Started Elasticsearch.
lines 1-17/17 (END)


root@secon:/var/ossec/logs# tail -f ossec.log
2021/03/30 04:19:44 ossec-syscheckd: INFO: (6008): File integrity monitoring scan started.
2021/03/30 04:19:52 ossec-syscheckd: INFO: (6009): File integrity monitoring scan ended.
2021/03/30 04:21:59 rootcheck: INFO: Starting rootcheck scan.
2021/03/30 04:23:08 rootcheck: INFO: Ending rootcheck scan.
2021/03/30 05:13:09 ossec-analysisd: ERROR: The new permissions could not be added to the JSON alert.
2021/03/30 05:13:09 ossec-analysisd: ERROR: The new permissions could not be added to the JSON alert.
2021/03/30 05:18:52 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2021/03/30 05:19:10 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2021/03/30 06:18:52 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2021/03/30 06:19:11 wazuh-modulesd:syscollector: INFO: Evaluation finished.

root@secon:/var/log/elasticsearch# tail -f elasticsearch.log
EOE
[2021-03-30T06:53:49,314][WARN ][c.a.o.s.a.BackendRegistry] [node-1] Authentication finally failed for kibanaserver from xxx.xxx.32.33:58472
[2021-03-30T06:53:51,816][WARN ][c.a.o.s.a.BackendRegistry] [node-1] Authentication finally failed for kibanaserver from xxx.xxx.32.33:58474
[2021-03-30T06:53:54,319][WARN ][c.a.o.s.a.BackendRegistry] [node-1] Authentication finally failed for kibanaserver from xxx.xxx.32.33:58472
[2021-03-30T06:53:56,820][WARN ][c.a.o.s.a.BackendRegistry] [node-1] Authentication finally failed for kibanaserver from xxx.xxx.32.33:58474
[2021-03-30T06:53:59,322][WARN ][c.a.o.s.a.BackendRegistry] [node-1] Authentication finally failed for kibanaserver from xxx.xxx.32.33:58472
[2021-03-30T06:54:01,827][WARN ][c.a.o.s.a.BackendRegistry] [node-1] Authentication finally failed for kibanaserver from xxx.xxx.32.33:58474
[2021-03-30T06:54:04,330][WARN ][c.a.o.s.a.BackendRegistry] [node-1] Authentication finally failed for kibanaserver from xxx.xxx.32.33:58472
[2021-03-30T06:54:06,828][WARN ][c.a.o.s.a.BackendRegistry] [node-1] Authentication finally failed for kibanaserver from xxx.xxx.32.33:58474
[2021-03-30T06:54:09,328][WARN ][c.a.o.s.a.BackendRegistry] [node-1] Authentication finally failed for kibanaserver from xxx.xxx.32.33:58472
[2021-03-30T06:54:11,831][WARN ][c.a.o.s.a.BackendRegistry] [node-1] Authentication finally failed for kibanaserver from xxx.xxx.32.33:58474
[2021-03-30T06:54:14,332][WARN ][c.a.o.s.a.BackendRegistry] [node-1] Authentication finally failed for kibanaserver from xxx.xxx.32.33:58472
[2021-03-30T06:54:15,049][INFO ][c.a.o.j.s.JobSweeper     ] [node-1] Running full sweep
[2021-03-30T06:54:16,836][WARN ][c.a.o.s.a.BackendRegistry] [node-1] Authentication finally failed for kibanaserver from xxx.xxx.32.33:58474
[2021-03-30T06:54:19,338][WARN ][c.a.o.s.a.BackendRegistry] [node-1] Authentication finally failed for kibanaserver from xxx.xxx.32.33:58472
[2021-03-30T06:54:21,839][WARN ][c.a.o.s.a.BackendRegistry] [node-1] Authentication finally failed for kibanaserver from xxx.xxx.32.33:58474
[2021-03-30T06:54:24,341][WARN ][c.a.o.s.a.BackendRegistry] [node-1] Authentication finally failed for kibanaserver from xxx.xxx.32.33:58472
[2021-03-30T06:54:26,843][WARN ][c.a.o.s.a.BackendRegistry] [node-1] Authentication finally failed for kibanaserver from xxx.xxx.32.33:58474
[2021-03-30T06:54:29,363][WARN ][c.a.o.s.a.BackendRegistry] [node-1] Authentication finally failed for kibanaserver from xxx.xxx.32.33:58472
[2021-03-30T06:54:31,845][WARN ][c.a.o.s.a.BackendRegistry] [node-1] Authentication finally failed for kibanaserver from xxx.xxx.32.33:58474
[2021-03-30T06:54:34,350][WARN ][c.a.o.s.a.BackendRegistry] [node-1] Authentication finally failed for kibanaserver from xxx.xxx.32.33:58472
[2021-03-30T06:54:36,850][WARN ][c.a.o.s.a.BackendRegistry] [node-1] Authentication finally failed for kibanaserver from xxx.xxx.32.33:58474
[2021-03-30T06:54:39,351][WARN ][c.a.o.s.a.BackendRegistry] [node-1] Authentication finally failed for kibanaserver from xxx.xxx.32.33:58472
[2021-03-30T06:54:41,853][WARN ][c.a.o.s.a.BackendRegistry] [node-1] Authentication finally failed for kibanaserver from xxx.xxx.32.33:58474
[2021-03-30T06:54:44,356][WARN ][c.a.o.s.a.BackendRegistry] [node-1] Authentication finally failed for kibanaserver from xxx.xxx.32.33:58472
[2021-03-30T06:54:46,856][WARN ][c.a.o.s.a.BackendRegistry] [node-1] Authentication finally failed for kibanaserver from xxx.xxx.32.33:58474
[2021-03-30T06:54:47,722][INFO ][stats_log                ] [node-1] ------------------------------------------------------------------------
Program=PerformanceAnalyzerPlugin
rca-version=0.0.1
StartTime=1617105227.714
EndTime=Tue, 30 Mar 2021 06:54:47 CDT
Time=60008 msecs
Timing=total-time:60008.0/1
Counters=TotalError=0
EOE
[2021-03-30T06:54:47,722][INFO ][stats_log                ] [node-1] ------------------------------------------------------------------------
Program=PerformanceAnalyzerPlugin
rca-version=0.0.1
Metrics=
StartTime=0.000
EndTime=Tue, 30 Mar 2021 06:54:47 CDT
Time=1617105287722 msecs
Timing=total-time:1.617105287722E12/1
Counters=
EOE
[2021-03-30T06:54:47,722][INFO ][stats_log                ] [node-1] ------------------------------------------------------------------------
Program=PerformanceAnalyzerPlugin
rca-version=0.0.1
Metrics=
StartTime=0.000
EndTime=Tue, 30 Mar 2021 06:54:47 CDT
Time=1617105287722 msecs
Timing=total-time:1.617105287722E12/1
Counters=
EOE
[2021-03-30T06:54:47,722][INFO ][stats_log                ] [node-1] ------------------------------------------------------------------------
Program=PerformanceAnalyzerPlugin
rca-version=0.0.1
Metrics=
StartTime=0.000
EndTime=Tue, 30 Mar 2021 06:54:47 CDT
Time=1617105287722 msecs
Timing=total-time:1.617105287722E12/1
Counters=
EOE
[2021-03-30T06:54:47,722][INFO ][stats_log                ] [node-1] ------------------------------------------------------------------------
Program=PerformanceAnalyzerPlugin
rca-version=0.0.1
Metrics=
StartTime=0.000
EndTime=Tue, 30 Mar 2021 06:54:47 CDT
Time=1617105287722 msecs
Timing=total-time:1.617105287722E12/1
Counters=
EOE
[2021-03-30T06:54:47,722][INFO ][stats_log                ] [node-1] ------------------------------------------------------------------------
Program=PerformanceAnalyzerPlugin
rca-version=0.0.1
Metrics=
StartTime=0.000
EndTime=Tue, 30 Mar 2021 06:54:47 CDT
Time=1617105287722 msecs
Timing=total-time:1.617105287722E12/1
Counters=
EOE
[2021-03-30T06:54:47,722][INFO ][stats_log                ] [node-1] ------------------------------------------------------------------------
Program=PerformanceAnalyzerPlugin
rca-version=0.0.1
Metrics=
StartTime=0.000
EndTime=Tue, 30 Mar 2021 06:54:47 CDT
Time=1617105287722 msecs
Timing=total-time:1.617105287722E12/1
Counters=
EOE
[2021-03-30T06:54:47,723][INFO ][stats_log                ] [node-1] ------------------------------------------------------------------------
Program=PerformanceAnalyzerPlugin
rca-version=0.0.1
Metrics=
StartTime=0.000
EndTime=Tue, 30 Mar 2021 06:54:47 CDT
Time=1617105287722 msecs
Timing=total-time:1.617105287722E12/1
Counters=
EOE
[2021-03-30T06:54:49,360][WARN ][c.a.o.s.a.BackendRegistry] [node-1] Authentication finally failed for kibanaserver from xxx.xxx.32.33:58472
[2021-03-30T06:54:51,861][WARN ][c.a.o.s.a.BackendRegistry] [node-1] Authentication finally failed for kibanaserver from xxx.xxx.32.33:58474
[


root@secon:/var/log/filebeat# tail -f filebeat
2021-03-29T09:17:31.654-0500    INFO    instance/beat.go:645    Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]
2021-03-29T09:17:31.654-0500    INFO    instance/beat.go:653    Beat ID: 76cfb8ed-490c-4047-9a2d-e769f1acb967
2021-03-29T09:17:31.654-0500    INFO    [index-management]      idxmgmt/std.go:184      Set output.elasticsearch.index to 'filebeat-7.10.0' as ILM is enabled.
2021-03-29T09:17:31.655-0500    INFO    eslegclient/connection.go:99    elasticsearch url: https://xxx.xxx.32.33:9200
2021-03-29T09:17:31.655-0500    WARN    [tls]   tlscommon/tls_config.go:93      SSL/TLS verifications disabled.
2021-03-29T09:17:31.656-0500    WARN    [tls]   tlscommon/tls_config.go:93      SSL/TLS verifications disabled.
2021-03-29T09:17:31.683-0500    WARN    [tls]   tlscommon/tls_config.go:93      SSL/TLS verifications disabled.
2021-03-29T09:17:31.714-0500    INFO    [esclientleg]   eslegclient/connection.go:314   Attempting to connect to Elasticsearch version 7.10.0



root@secon:/usr/share/kibana/data/wazuh/logs# tail -f wazuhapp.log
{"date":"2021-03-29T13:30:01.259Z","level":"info","location":"Cron-scheduler","message":{"name":"ResponseError","meta":{"body":"Unauthorized","statusCode":401,"headers":{"www-authenticate":"Basic realm=\"Open Distro Security\"","content-type":"text/plain; charset=UTF-8","content-length":"12"},"meta":{"context":null,"request":{"params":{"method":"POST","path":"/wazuh-statistics-2021.13w/_doc/_bulk","bulkBody":[{"index":{"_index":"wazuh-statistics-2021.13w"}},{"analysisd":[{"total_events_decoded":131,"syscheck_events_decoded":0,"syscheck_edps":0,"syscollector_events_decoded":0,"syscollector_edps":0,"rootcheck_events_decoded":0,"rootcheck_edps":0,"sca_events_decoded":0,"sca_edps":0,"hostinfo_events_decoded":0,"hostinfo_edps":0,"winevt_events_decoded":97,"winevt_edps":19,"dbsync_messages_dispatched":0,"dbsync_mdps":0,"other_events_decoded":34,"other_events_edps":6,"events_processed":131,"events_edps":26,"events_received":131,"events_dropped":0,"alerts_written":86,"firewall_written":0,"fts_written":0,"syscheck_queue_usage":0,"syscheck_queue_size":16384,"syscollector_queue_usage":0,"syscollector_queue_size":16384,"rootcheck_queue_usage":0,"rootcheck_queue_size":16384,"sca_queue_usage":0,"sca_queue_size":16384,"hostinfo_queue_usage":0,"hostinfo_queue_size":16384,"winevt_queue_usage":0,"winevt_queue_size":16384,"dbsync_queue_usage":0,"dbsync_queue_size":16384,"upgrade_queue_usage":0,"upgrade_queue_size":16384,"event_queue_usage":0,"event_queue_size":16384,"rule_matching_queue_usage":0,"rule_matching_queue_size":16384,"alerts_queue_usage":0,"alerts_queue_size":16384,"firewall_queue_usage":0,"firewall_queue_size":16384,"statistical_queue_usage":0,"statistical_queue_size":16384,"archives_queue_usage":0,"archives_queue_size":16384}],"apiName":"production","cluster":"false","timestamp":"2021-03-29T13:30:00.896Z"}],"querystring":"","body":"{\"index\":{\"_index\":\"wazuh-statistics-2021.13w\"}}\n{\"analysisd\":[{\"total_events_decoded\":131,\"syscheck_events_decoded\":0,\"syscheck_edps\":0,\"syscollector_events_decoded\":0,\"syscollector_edps\":0,\"rootcheck_events_decoded\":0,\"rootcheck_edps\":0,\"sca_events_decoded\":0,\"sca_edps\":0,\"hostinfo_events_decoded\":0,\"hostinfo_edps\":0,\"winevt_events_decoded\":97,\"winevt_edps\":19,\"dbsync_messages_dispatched\":0,\"dbsync_mdps\":0,\"other_events_decoded\":34,\"other_events_edps\":6,\"events_processed\":131,\"events_edps\":26,\"events_received\":131,\"events_dropped\":0,\"alerts_written\":86,\"firewall_written\":0,\"fts_written\":0,\"syscheck_queue_usage\":0,\"syscheck_queue_size\":16384,\"syscollector_queue_usage\":0,\"syscollector_queue_size\":16384,\"rootcheck_queue_usage\":0,\"rootcheck_queue_size\":16384,\"sca_queue_usage\":0,\"sca_queue_size\":16384,\"hostinfo_queue_usage\":0,\"hostinfo_queue_size\":16384,\"winevt_queue_usage\":0,\"winevt_queue_size\":16384,\"dbsync_queue_usage\":0,\"dbsync_queue_size\":16384,\"upgrade_queue_usage\":0,\"upgrade_queue_size\":16384,\"event_queue_usage\":0,\"event_queue_size\":16384,\"rule_matching_queue_usage\":0,\"rule_matching_queue_size\":16384,\"alerts_queue_usage\":0,\"alerts_queue_size\":16384,\"firewall_queue_usage\":0,\"firewall_queue_size\":16384,\"statistical_queue_usage\":0,\"statistical_queue_size\":16384,\"archives_queue_usage\":0,\"archives_queue_size\":16384}],\"apiName\":\"production\",\"cluster\":\"false\",\"timestamp\":\"2021-03-29T13:30:00.896Z\"}\n","headers":{"user-agent":"elasticsearch-js/7.10.0-rc.1 (linux 5.4.0-66-generic-x64; Node.js v10.22.1)","x-elastic-product-origin":"kibana","content-type":"application/x-ndjson","content-length":"1419"},"timeout":30000},"options":{},"id":399540},"name":"elasticsearch-js","connection":{"url":"https://xxx.xxx.32.33:9200/","id":"https://xxx.xxx.32.33:9200/","headers":{},"deadCount":0,"resurrectTimeout":0,"_openRequests":1,"status":"alive","roles":{"master":true,"data":true,"ingest":true,"ml":false}},"attempts":0,"aborted":false}}}}
{"date":"2021-03-29T13:30:01.604Z","level":"info","location":"Cron-scheduler","message":{"name":"ResponseError","meta":{"body":"Unauthorized","statusCode":401,"headers":{"www-authenticate":"Basic realm=\"Open Distro Security\"","content-type":"text/plain; charset=UTF-8","content-length":"12"},"meta":{"context":null,"request":{"params":{"method":"POST","path":"/wazuh-statistics-2021.13w/_doc/_bulk","bulkBody":[{"index":{"_index":"wazuh-statistics-2021.13w"}},{"remoted":[{"queue_size":0,"total_queue_size":131072,"tcp_sessions":64,"evt_count":10908168,"ctrl_msg_count":2167212,"discarded_count":0,"msg_sent":2176172,"recv_bytes":9488408187,"dequeued_after_close":0}],"apiName":"production","cluster":"false","timestamp":"2021-03-29T13:30:01.257Z"}],"querystring":"","body":"{\"index\":{\"_index\":\"wazuh-statistics-2021.13w\"}}\n{\"remoted\":[{\"queue_size\":0,\"total_queue_size\":131072,\"tcp_sessions\":64,\"evt_count\":10908168,\"ctrl_msg_count\":2167212,\"discarded_count\":0,\"msg_sent\":2176172,\"recv_bytes\":9488408187,\"dequeued_after_close\":0}],\"apiName\":\"production\",\"cluster\":\"false\",\"timestamp\":\"2021-03-29T13:30:01.257Z\"}\n","headers":{"user-agent":"elasticsearch-js/7.10.0-rc.1 (linux 5.4.0-66-generic-x64; Node.js v10.22.1)","x-elastic-product-origin":"kibana","content-type":"application/x-ndjson","content-length":"338"},"timeout":30000},"options":{},"id":399541},"name":"elasticsearch-js","connection":{"url":"https://xxx.xxx.32.33:9200/","id":"https://xxx.xxx.32.33:9200/","headers":{},"deadCount":0,"resurrectTimeout":0,"_openRequests":0,"status":"alive","roles":{"master":true,"data":true,"ingest":true,"ml":false}},"attempts":0,"aborted":false}}}}
{"date":"2021-03-29T13:35:01.029Z","level":"error","location":"cron-scheduler|SaveDocument","message":"Error searching or creating 'wazuh-statistics-2021.13w' due to 'Response Error'"}
{"date":"2021-03-29T13:35:01.031Z","level":"error","location":"cron-scheduler|SaveDocument","message":"Error searching or creating 'wazuh-statistics-2021.13w' due to 'Response Error'"}
{"date":"2021-03-29T13:35:01.377Z","level":"info","location":"Cron-scheduler","message":{"name":"ResponseError","meta":{"body":"Unauthorized","statusCode":401,"headers":{"www-authenticate":"Basic realm=\"Open Distro Security\"","content-type":"text/plain; charset=UTF-8","content-length":"12"},"meta":{"context":null,"request":{"params":{"method":"POST","path":"/wazuh-statistics-2021.13w/_doc/_bulk","bulkBody":[{"index":{"_index":"wazuh-statistics-2021.13w"}},{"analysisd":[{"total_events_decoded":148,"syscheck_events_decoded":0,"syscheck_edps":0,"syscollector_events_decoded":0,"syscollector_edps":0,"rootcheck_events_decoded":0,"rootcheck_edps":0,"sca_events_decoded":0,"sca_edps":0,"hostinfo_events_decoded":0,"hostinfo_edps":0,"winevt_events_decoded":115,"winevt_edps":23,"dbsync_messages_dispatched":0,"dbsync_mdps":0,"other_events_decoded":33,"other_events_edps":6,"events_processed":148,"events_edps":29,"events_received":148,"events_dropped":0,"alerts_written":90,"firewall_written":0,"fts_written":0,"syscheck_queue_usage":0,"syscheck_queue_size":16384,"syscollector_queue_usage":0,"syscollector_queue_size":16384,"rootcheck_queue_usage":0,"rootcheck_queue_size":16384,"sca_queue_usage":0,"sca_queue_size":16384,"hostinfo_queue_usage":0,"hostinfo_queue_size":16384,"winevt_queue_usage":0,"winevt_queue_size":16384,"dbsync_queue_usage":0,"dbsync_queue_size":16384,"upgrade_queue_usage":0,"upgrade_queue_size":16384,"event_queue_usage":0,"event_queue_size":16384,"rule_matching_queue_usage":0,"rule_matching_queue_size":16384,"alerts_queue_usage":0,"alerts_queue_size":16384,"firewall_queue_usage":0,"firewall_queue_size":16384,"statistical_queue_usage":0,"statistical_queue_size":16384,"archives_queue_usage":0,"archives_queue_size":16384}],"apiName":"production","cluster":"false","timestamp":"2021-03-29T13:35:01.030Z"}],"querystring":"","body":"{\"index\":{\"_index\":\"wazuh-statistics-2021.13w\"}}\n{\"analysisd\":[{\"total_events_decoded\":148,\"syscheck_events_decoded\":0,\"syscheck_edps\":0,\"syscollector_events_decoded\":0,\"syscollector_edps\":0,\"rootcheck_events_decoded\":0,\"rootcheck_edps\":0,\"sca_events_decoded\":0,\"sca_edps\":0,\"hostinfo_events_decoded\":0,\"hostinfo_edps\":0,\"winevt_events_decoded\":115,\"winevt_edps\":23,\"dbsync_messages_dispatched\":0,\"dbsync_mdps\":0,\"other_events_decoded\":33,\"other_events_edps\":6,\"events_processed\":148,\"events_edps\":29,\"events_received\":148,\"events_dropped\":0,\"alerts_written\":90,\"firewall_written\":0,\"fts_written\":0,\"syscheck_queue_usage\":0,\"syscheck_queue_size\":16384,\"syscollector_queue_usage\":0,\"syscollector_queue_size\":16384,\"rootcheck_queue_usage\":0,\"rootcheck_queue_size\":16384,\"sca_queue_usage\":0,\"sca_queue_size\":16384,\"hostinfo_queue_usage\":0,\"hostinfo_queue_size\":16384,\"winevt_queue_usage\":0,\"winevt_queue_size\":16384,\"dbsync_queue_usage\":0,\"dbsync_queue_size\":16384,\"upgrade_queue_usage\":0,\"upgrade_queue_size\":16384,\"event_queue_usage\":0,\"event_queue_size\":16384,\"rule_matching_queue_usage\":0,\"rule_matching_queue_size\":16384,\"alerts_queue_usage\":0,\"alerts_queue_size\":16384,\"firewall_queue_usage\":0,\"firewall_queue_size\":16384,\"statistical_queue_usage\":0,\"statistical_queue_size\":16384,\"archives_queue_usage\":0,\"archives_queue_size\":16384}],\"apiName\":\"production\",\"cluster\":\"false\",\"timestamp\":\"2021-03-29T13:35:01.030Z\"}\n","headers":{"user-agent":"elasticsearch-js/7.10.0-rc.1 (linux 5.4.0-66-generic-x64; Node.js v10.22.1)","x-elastic-product-origin":"kibana","content-type":"application/x-ndjson","content-length":"1420"},"timeout":30000},"options":{},"id":399667},"name":"elasticsearch-js","connection":{"url":"https://xxx.xxx.32.33:9200/","id":"https://xxx.xxx.32.33:9200/","headers":{},"deadCount":0,"resurrectTimeout":0,"_openRequests":1,"status":"alive","roles":{"master":true,"data":true,"ingest":true,"ml":false}},"attempts":0,"aborted":false}}}}
{"date":"2021-03-29T13:35:01.382Z","level":"info","location":"Cron-scheduler","message":{"name":"ResponseError","meta":{"body":"Unauthorized","statusCode":401,"headers":{"www-authenticate":"Basic realm=\"Open Distro Security\"","content-type":"text/plain; charset=UTF-8","content-length":"12"},"meta":{"context":null,"request":{"params":{"method":"POST","path":"/wazuh-statistics-2021.13w/_doc/_bulk","bulkBody":[{"index":{"_index":"wazuh-statistics-2021.13w"}},{"remoted":[{"queue_size":0,"total_queue_size":131072,"tcp_sessions":64,"evt_count":10915051,"ctrl_msg_count":2169131,"discarded_count":0,"msg_sent":2178106,"recv_bytes":9494792871,"dequeued_after_close":0}],"apiName":"production","cluster":"false","timestamp":"2021-03-29T13:35:01.031Z"}],"querystring":"","body":"{\"index\":{\"_index\":\"wazuh-statistics-2021.13w\"}}\n{\"remoted\":[{\"queue_size\":0,\"total_queue_size\":131072,\"tcp_sessions\":64,\"evt_count\":10915051,\"ctrl_msg_count\":2169131,\"discarded_count\":0,\"msg_sent\":2178106,\"recv_bytes\":9494792871,\"dequeued_after_close\":0}],\"apiName\":\"production\",\"cluster\":\"false\",\"timestamp\":\"2021-03-29T13:35:01.031Z\"}\n","headers":{"user-agent":"elasticsearch-js/7.10.0-rc.1 (linux 5.4.0-66-generic-x64; Node.js v10.22.1)","x-elastic-product-origin":"kibana","content-type":"application/x-ndjson","content-length":"338"},"timeout":30000},"options":{},"id":399668},"name":"elasticsearch-js","connection":{"url":"https://xxx.xxx.32.33:9200/","id":"https://xxx.xxx.32.33:9200/","headers":{},"deadCount":0,"resurrectTimeout":0,"_openRequests":0,"status":"alive","roles":{"master":true,"data":true,"ingest":true,"ml":false}},"attempts":0,"aborted":false}}}}
{"date":"2021-03-29T13:40:01.469Z","level":"error","location":"cron-scheduler|SaveDocument","message":"Error searching or creating 'wazuh-statistics-2021.13w' due to 'Response Error'"}
{"date":"2021-03-29T13:40:01.470Z","level":"error","location":"cron-scheduler|SaveDocument","message":"Error searching or creating 'wazuh-statistics-2021.13w' due to 'Response Error'"}
{"date":"2021-03-29T13:40:01.816Z","level":"info","location":"Cron-scheduler","message":{"name":"ResponseError","meta":{"body":"Unauthorized","statusCode":401,"headers":{"www-authenticate":"Basic realm=\"Open Distro Security\"","content-type":"text/plain; charset=UTF-8","content-length":"12"},"meta":{"context":null,"request":{"params":{"method":"POST","path":"/wazuh-statistics-2021.13w/_doc/_bulk","bulkBody":[{"index":{"_index":"wazuh-statistics-2021.13w"}},{"remoted":[{"queue_size":0,"total_queue_size":131072,"tcp_sessions":64,"evt_count":10922173,"ctrl_msg_count":2171048,"discarded_count":0,"msg_sent":2180023,"recv_bytes":9501273747,"dequeued_after_close":0}],"apiName":"production","cluster":"false","timestamp":"2021-03-29T13:40:01.469Z"}],"querystring":"","body":"{\"index\":{\"_index\":\"wazuh-statistics-2021.13w\"}}\n{\"remoted\":[{\"queue_size\":0,\"total_queue_size\":131072,\"tcp_sessions\":64,\"evt_count\":10922173,\"ctrl_msg_count\":2171048,\"discarded_count\":0,\"msg_sent\":2180023,\"recv_bytes\":9501273747,\"dequeued_after_close\":0}],\"apiName\":\"production\",\"cluster\":\"false\",\"timestamp\":\"2021-03-29T13:40:01.469Z\"}\n","headers":{"user-agent":"elasticsearch-js/7.10.0-rc.1 (linux 5.4.0-66-generic-x64; Node.js v10.22.1)","x-elastic-product-origin":"kibana","content-type":"application/x-ndjson","content-length":"338"},"timeout":30000},"options":{},"id":399796},"name":"elasticsearch-js","connection":{"url":"https://xxx.xxx.32.33:9200/","id":"https://xxx.xxx.32.33:9200/","headers":{},"deadCount":0,"resurrectTimeout":0,"_openRequests":1,"status":"alive","roles":{"master":true,"data":true,"ingest":true,"ml":false}},"attempts":0,"aborted":false}}}}
{"date":"2021-03-29T13:40:01.820Z","level":"info","location":"Cron-scheduler","message":{"name":"ResponseError","meta":{"body":"Unauthorized","statusCode":401,"headers":{"www-authenticate":"Basic realm=\"Open Distro Security\"","content-type":"text/plain; charset=UTF-8","content-length":"12"},"meta":{"context":null,"request":{"params":{"method":"POST","path":"/wazuh-statistics-2021.13w/_doc/_bulk","bulkBody":[{"index":{"_index":"wazuh-statistics-2021.13w"}},{"analysisd":[{"total_events_decoded":114,"syscheck_events_decoded":0,"syscheck_edps":0,"syscollector_events_decoded":0,"syscollector_edps":0,"rootcheck_events_decoded":0,"rootcheck_edps":0,"sca_events_decoded":0,"sca_edps":0,"hostinfo_events_decoded":0,"hostinfo_edps":0,"winevt_events_decoded":72,"winevt_edps":14,"dbsync_messages_dispatched":0,"dbsync_mdps":0,"other_events_decoded":42,"other_events_edps":8,"events_processed":114,"events_edps":22,"events_received":114,"events_dropped":0,"alerts_written":56,"firewall_written":0,"fts_written":0,"syscheck_queue_usage":0,"syscheck_queue_size":16384,"syscollector_queue_usage":0,"syscollector_queue_size":16384,"rootcheck_queue_usage":0,"rootcheck_queue_size":16384,"sca_queue_usage":0,"sca_queue_size":16384,"hostinfo_queue_usage":0,"hostinfo_queue_size":16384,"winevt_queue_usage":0,"winevt_queue_size":16384,"dbsync_queue_usage":0,"dbsync_queue_size":16384,"upgrade_queue_usage":0,"upgrade_queue_size":16384,"event_queue_usage":0,"event_queue_size":16384,"rule_matching_queue_usage":0,"rule_matching_queue_size":16384,"alerts_queue_usage":0,"alerts_queue_size":16384,"firewall_queue_usage":0,"firewall_queue_size":16384,"statistical_queue_usage":0,"statistical_queue_size":16384,"archives_queue_usage":0,"archives_queue_size":16384}],"apiName":"production","cluster":"false","timestamp":"2021-03-29T13:40:01.471Z"}],"querystring":"","body":"{\"index\":{\"_index\":\"wazuh-statistics-2021.13w\"}}\n{\"analysisd\":[{\"total_events_decoded\":114,\"syscheck_events_decoded\":0,\"syscheck_edps\":0,\"syscollector_events_decoded\":0,\"syscollector_edps\":0,\"rootcheck_events_decoded\":0,\"rootcheck_edps\":0,\"sca_events_decoded\":0,\"sca_edps\":0,\"hostinfo_events_decoded\":0,\"hostinfo_edps\":0,\"winevt_events_decoded\":72,\"winevt_edps\":14,\"dbsync_messages_dispatched\":0,\"dbsync_mdps\":0,\"other_events_decoded\":42,\"other_events_edps\":8,\"events_processed\":114,\"events_edps\":22,\"events_received\":114,\"events_dropped\":0,\"alerts_written\":56,\"firewall_written\":0,\"fts_written\":0,\"syscheck_queue_usage\":0,\"syscheck_queue_size\":16384,\"syscollector_queue_usage\":0,\"syscollector_queue_size\":16384,\"rootcheck_queue_usage\":0,\"rootcheck_queue_size\":16384,\"sca_queue_usage\":0,\"sca_queue_size\":16384,\"hostinfo_queue_usage\":0,\"hostinfo_queue_size\":16384,\"winevt_queue_usage\":0,\"winevt_queue_size\":16384,\"dbsync_queue_usage\":0,\"dbsync_queue_size\":16384,\"upgrade_queue_usage\":0,\"upgrade_queue_size\":16384,\"event_queue_usage\":0,\"event_queue_size\":16384,\"rule_matching_queue_usage\":0,\"rule_matching_queue_size\":16384,\"alerts_queue_usage\":0,\"alerts_queue_size\":16384,\"firewall_queue_usage\":0,\"firewall_queue_size\":16384,\"statistical_queue_usage\":0,\"statistical_queue_size\":16384,\"archives_queue_usage\":0,\"archives_queue_size\":16384}],\"apiName\":\"production\",\"cluster\":\"false\",\"timestamp\":\"2021-03-29T13:40:01.471Z\"}\n","headers":{"user-agent":"elasticsearch-js/7.10.0-rc.1 (linux 5.4.0-66-generic-x64; Node.js v10.22.1)","x-elastic-product-origin":"kibana","content-type":"application/x-ndjson","content-length":"1419"},"timeout":30000},"options":{},"id":399797},"name":"elasticsearch-js","connection":{"url":"https://xxx.xxx.32.33:9200/","id":"https://xxx.xxx.32.33:9200/","headers":{},"deadCount":0,"resurrectTimeout":0,"_openRequests":0,"status":"alive","roles":{"master":true,"data":true,"ingest":true,"ml":false}},"attempts":0,"aborted":false}}}}

Nicolas Koziuk

unread,
Mar 30, 2021, 11:00:34 AM3/30/21
to Wazuh mailing list
Hello Bill, I hope you're having a great day!

Before anything, thank you very much for providing such detailed logs and info to help us identify the issue you are experiencing, this is very helpful and saves us a lot of time.

I have been looking at the provided info, and as you mentioned, everyhting seems to be configured properly. The issue is definetly related to authentication on elasticsearch, as seen in this log:


[2021-03-30T06:54:21,839][WARN ][c.a.o.s.a.BackendRegistry] [node-1] Authentication finally failed for kibanaserver from xxx.xxx.32.33:58474

I would like to know if there have been any changes to the kibanaserver user login credentials, as the issue might be related to this. From the info you provided, I noticed that you're still using default credentials for this user, which is not the reccomended configuration

elasticsearch.username: kibanaserver
elasticsearch.password: kibanaserver

Kind regards

Bill Green

unread,
Mar 30, 2021, 12:03:16 PM3/30/21
to Wazuh mailing list
Hi Nicolas,

Thank you for the reply - I really appreciate it.

I did make a change, that at the time I didn't think would be an issue.  While logged into the Wazuh portal, I created a new account called "general" and applied a password to it.  My thinking was that it would be good to have a non-admin account I could provide to my supervisor or others to let them log into the portal but not have any admin permissions to change things, etc. 

When I opened an incognito window in another browser and attempted to login with the new user and creds, I received a message indicating the user name or password was invalid.  I believe that was when I assumed I needed to reboot the server for the new account to be usable, but, when everything appeared to be up I had the Kibana server error.  Although that seemed innocuous, that has to be what the issue is since that is the only change I'm aware of and that just can't be coincidental. Is there a .yml or .conf file I can remove that account from possibly to see if that would resolve it?

And I will definitely change the elasicsearch defaults - that's a huge oversight on my part.

Thank you again for the help!
Bill

Nicolas Koziuk

unread,
Mar 30, 2021, 1:14:27 PM3/30/21
to Bill Green, Wazuh mailing list
Hey Bill, 
 
Regarding the "general" account you mention, was it created in wazuh>security>users on Kibana? Something like this:

image.png
If so, this should not be related to this issue, as these users are not related to kibana / elastic

On the other hand, I was able to reproduce the issue by simply changing the following in /etc/kibana/kibana.yml on my test box, and restarting the kibana service (# systemctl restart kibana)

elasticsearch.username: kibanaserver
elasticsearch.password: kibanaserver

To 

elasticsearch.username: kibanaserver2
elasticsearch.password: kibanaserver2

Screenshot from 2021-03-30 13-56-20.png

After that, I reverted the changes and restarted kibana service again, and everything was back to working normally.

What this means is that the user/password you have set on your kibana.yml file is most likely not the correct one, so I suggest you check that. It makes sense that the issue only appeared after a reboot, since the service needs to be restarted for new credentials to take effect. 
My guess is that you need to add your newly created credentials to this file, and after restarting the service again, it should work :D

I hope this helps!

Best regards

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/4a50104e-2065-4674-a768-16053f2d4c78n%40googlegroups.com.

Bill Green

unread,
Mar 30, 2021, 1:23:08 PM3/30/21
to Wazuh mailing list
Thanks again for the help - yes, I created the general account in the portal at wazuh>security>users.  Good to know that wasn't the issue.

However, I'll track down what the password is for the elasticsearch user now and test that.

Really appreciate the help!
Bill

Reply all
Reply to author
Forward
0 new messages