elasticsearch

[OD]

I installed wazuh management server without issues but after a few days elasticsearch crashed and when I try to start it up I get this:

root@wazuh1:~# service elasticsearch status -l

● elasticsearch.service - Elasticsearch

   Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)

  Drop-In: /etc/systemd/system/elasticsearch.service.d

           └─elasticsearch.conf

   Active: failed (Result: exit-code) since Sun 2018-07-15 09:20:57 EDT; 4h 14min ago

     Docs: http://www.elastic.co

  Process: 3038 ExecStart=/usr/share/elasticsearch/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid --quiet (code=exited, status=1/FAILURE)

 Main PID: 3038 (code=exited, status=1/FAILURE)

Jul 15 09:20:57 wazuh1 elasticsearch[3038]: Likely root cause: java.nio.file.AccessDeniedException: /etc/elasticsearch/elasticsearch.keystore.tmp

Jul 15 09:20:57 wazuh1 elasticsearch[3038]:         at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)

Jul 15 09:20:57 wazuh1 elasticsearch[3038]:         at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)

Jul 15 09:20:57 wazuh1 elasticsearch[3038]:         at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)

Jul 15 09:20:57 wazuh1 elasticsearch[3038]:         at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)

Jul 15 09:20:57 wazuh1 elasticsearch[3038]:         at java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)

Jul 15 09:20:57 wazuh1 elasticsearch[3038]:         at java.nio.file.Files.newOutputStream(Files.java:216)

Jul 15 09:20:57 wazuh1 systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE

Jul 15 09:20:57 wazuh1 systemd[1]: elasticsearch.service: Unit entered failed state.

Jul 15 09:20:57 wazuh1 systemd[1]: elasticsearch.service: Failed with result 'exit-code'.

root@wazuh1:/usr/share/elasticsearch/bin# ./elasticsearch --version

Version: 6.3.1, Build: default/deb/eb782d0/2018-06-29T21:59:26.107521Z, JVM: 1.8.0_171

not sure how to fix.  I am also wondering if I can use the elasticsearch service with aws. I would rather not pay for the service but I keep on having issues with elasticsearch in my ELK setups in previous experience.

Thanks,

Olivier

[jesus.g...@wazuh.com]

Hi Oliver,

From this line:

java.nio.file.AccessDeniedException: /etc/elasticsearch/elasticsearch.keystore.tmp

my bet is Elasticsearch has a permission issue. Can you set by yourself the permission for that file? 

Also note other files could be affected, but we are going to start from that file. Try the next commands to set the permissions:

# chown root:elasticsearch /etc/elasticsearch/elasticsearch.keystore.tmp
# chmod 660 /etc/elasticsearch/elasticsearch.keystore.tmp

Let us know once done.

Regards,

Jesús

[Olivier Doisneau]

That file did not exist. So I had to chmod +gw /etc/elasticsearch but then port 5601 never came up even after kibana started so I rebuilt from scratch.  I really need to have agents connecting ok to it today as I have an audit July 31 which has many parts handled by wazuh.  I just find it odd that a server that is not touched broke  without any agents connecting to it or anything. I had not even setup the api yet.

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To post to this group, send email to wa...@googlegroups.com.
Visit this group at https://groups.google.com/group/wazuh.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/6665db5b-59ac-4871-89f3-c4939eb6853e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[jesus.g...@wazuh.com]

Hello again Oliver,

The port 5601 is related to Kibana itself and not Elasticsearch. If you have never touched the file /etc/kibana/kibana.yml, it's right

to not see that port out of localhost. Please edit that file, look for "server.host" key and replace by this:

server.host: "0.0.0.0"

then restart Kibana:

# systemctl restart kibana

Also if you are building it from scratch, my suggestion is to follow our guide step by step:

1. Wazuh manager, Wazuh API and Filebeat if needed https://documentation.wazuh.com/current/installation-guide/installing-wazuh-server/index.html

2. Elasticsearch, Logstash, Kibana, Wazuh App https://documentation.wazuh.com/current/installation-guide/installing-elastic-stack/index.html

3. Wazuh agents https://documentation.wazuh.com/current/installation-guide/installing-wazuh-agent/index.html

A wrong order following our guide can cause problems. Let us know once done.

Regards,

Jesús

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+unsubscribe@googlegroups.com.

[Olivier Doisneau]

so I was looking at the working setup and the one that does not work and I believe I have an incompatibility issue.  the kibana that is installed with my latest ansible scripts is 

./kibana --version

6.3.1

the one that works in dev is 

./kibana --version

6.2.3

so I see this in the kibana logs:

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:til...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.1

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:wat...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.1

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:index_ma...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:gr...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.1)"

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:secu...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:grokde...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:logs...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:repo...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0

Jul 16 06:37:02 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:37:02Z","tags":["listening","info"],"pid":12659,"message":"Server running at http://0.0.0.0:5601"}

Jul 16 06:37:20 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:37:20Z","tags":["license","warning","xpack"],"pid":12659,"message":"License information from the X-Pack plugin could not be obtained from Elasticsearch for the [data] cluster. [invalid_index_name_exception] Invalid index name [_xpack], must not start with '_'., with { index_uuid=\"_na_\" & index=\"_xpack\" } :: {\"path\":\"/_xpack\",\"statusCode\":400,\"response\":\"{\\\"error\\\":

~

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/cecd248f-c24d-40e8-9856-ce475174fe79%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

- Olivier

Olivier Doisneau

Sr. IT Engineer

Payrailz  

95 Glastonbury Blvd., Suite 105

Glastonbury, CT 06033

413.313.5017 mobile

NOTICE OF CONFIDENTIALITY:

This e-mail is intended solely for the use of the individual to whom it is addressed and may contain information that is privileged, confidential or otherwise exempt from disclosure. If the reader of this e-mail is not the intended recipient or the employee or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, please immediately notify us by replying to the original message at the listed e-mail address.  Thank You.

[jesus.g...@wazuh.com]

That's right Oliver,

All the Elastic stack components must be using the exactly same version, otherwise you'll see an unexpected behaviour.

This includes Logstash, Filebeat, Elasticsearch, Kibana...

Regards,

Jesús

El lunes, 16 de julio de 2018, 12:46:06 (UTC+2), OD escribió:

so I was looking at the working setup and the one that does not work and I believe I have an incompatibility issue.  the kibana that is installed with my latest ansible scripts is 

./kibana --version

6.3.1

the one that works in dev is 

./kibana --version

6.2.3

so I see this in the kibana logs:

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:til...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.1

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:wat...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.1

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:index_manag...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:gr...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.1)"

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:security@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:grokdebugger@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:logstash@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:reporting@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0

[Olivier Doisneau]

trying it again with new version in default files.  oddly I saw this and had to change to kibana:kibana. 

ls -alrt /usr/share/kibana/optimize/.babelcache.json

-rw-rw-r-- 1 root root 2 Jun 29 18:10 /usr/share/kibana/optimize/.babelcache.json

There is nothing that you would know about that would change the perm on this file do you?

On Mon, Jul 16, 2018 at 6:49 AM, <jesus.g...@wazuh.com> wrote:

That's right Oliver,

All the Elastic stack components must be using the exactly same version, otherwise you'll see an unexpected behaviour.

This includes Logstash, Filebeat, Elasticsearch, Kibana...

Regards,

Jesús

El lunes, 16 de julio de 2018, 12:46:06 (UTC+2), OD escribió:

so I was looking at the working setup and the one that does not work and I believe I have an incompatibility issue.  the kibana that is installed with my latest ansible scripts is 

./kibana --version

6.3.1

the one that works in dev is 

./kibana --version

6.2.3

so I see this in the kibana logs:

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:til...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.1

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:wat...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.1

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:index_managemen...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:gr...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.1)"

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:secu...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:grokdebugger@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:logs...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/b095a777-aba0-443d-9e5b-d216de4e6170%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[jesus.g...@wazuh.com]

Hello Oliver,

We have no package for Wazuh 3.2.1 + Kibana 6.3.1, you can check our compatible version at the next link:

- https://documentation.wazuh.com/current/installation-guide/compatibility_matrix/index.html#api-and-app

What are you meaning with the Kibana role? Are you trying to configure X-Pack RBAC features from Elasticsearch RBAC? If

so, we can help you, but first of all you should have all your environment with the proper version installed. From Wazuh to Elasticsearch, all 

components must be installed using a compatible version.

Regards,

Jesús

El lunes, 16 de julio de 2018, 13:00:38 (UTC+2), OD escribió:

what do I put for kibana version in kibana role defaults/main.yml?

elastic_stack_version: 6.3.1

wazuh_version: 3.2.1

fatal: [10.199.20.152]: FAILED! => {"changed": true, "cmd": "/usr/share/kibana/bin/kibana-plugin install https://packages.wazuh.com/wazuhapp/wazuhapp-3.2.1_6.3.1.zip", "delta": "0:00:01.019625", "end": "2018-07-16 06:56:54.554669", "msg": "non-zero return code", "rc": 70, "start": "2018-07-16 06:56:53.535044", "stderr": "Plugin installation was unsuccessful due to error \"No valid url specified.\"", "stderr_lines": ["Plugin installation was unsuccessful due to error \"No valid url specified.\""], "stdout": "Attempting to transfer from https://packages.wazuh.com/wazuhapp/wazuhapp-3.2.1_6.3.1.zip\nAttempting to transfer from https://artifacts.elastic.co/downloads/kibana-plugins/https://packages.wazuh.com/wazuhapp/wazuhapp-3.2.1_6.3.1.zip/https://packages.wazuh.com/wazuhapp/wazuhapp-3.2.1_6.3.1.zip-6.3.1.zip", "stdout_lines": ["Attempting to transfer from https://packages.wazuh.com/wazuhapp/wazuhapp-3.2.1_6.3.1.zip", "Attempting to transfer from https://artifacts.elastic.co/downloads/kibana-plugins/https://packages.wazuh.com/wazuhapp/wazuhapp-3.2.1_6.3.1.zip/https://packages.wazuh.com/wazuhapp/wazuhapp-3.2.1_6.3.1.zip-6.3.1.zip"]}

On Mon, Jul 16, 2018 at 6:56 AM, Olivier Doisneau <odoi...@payrailz.com> wrote:

trying it again with new version in default files.  oddly I saw this and had to change to kibana:kibana. 

ls -alrt /usr/share/kibana/optimize/.babelcache.json

-rw-rw-r-- 1 root root 2 Jun 29 18:10 /usr/share/kibana/optimize/.babelcache.json

There is nothing that you would know about that would change the perm on this file do you?

On Mon, Jul 16, 2018 at 6:49 AM, <jesus.g...@wazuh.com> wrote:

That's right Oliver,

All the Elastic stack components must be using the exactly same version, otherwise you'll see an unexpected behaviour.

This includes Logstash, Filebeat, Elasticsearch, Kibana...

Regards,

Jesús

El lunes, 16 de julio de 2018, 12:46:06 (UTC+2), OD escribió:

so I was looking at the working setup and the one that does not work and I believe I have an incompatibility issue.  the kibana that is installed with my latest ansible scripts is 

./kibana --version

6.3.1

the one that works in dev is 

./kibana --version

6.2.3

so I see this in the kibana logs:

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:til...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.1

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:wat...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.1

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:index_manag...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:gr...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.1)"

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:security@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:grokdebugger@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:logstash@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.

[jesus.g...@wazuh.com]

Hello again Oliver,

Yes, that error is coming from Ansible, take a look at this file: 

• https://github.com/wazuh/wazuh-ansible/blob/master/ansible-wazuh-agent/tasks/Linux.yml
  

And it's related to the registration procedure using the RESTful API (https://documentation.wazuh.com/current/user-manual/agents/restful-api/register.html#restful-api-register)

Maybe the Wazuh API is down? 

Also read the next docs https://documentation.wazuh.com/current/deploying-with-ansible/index.html if you haven't read them before, 

there we explain Ansible deployments.

Regards,

Jesús

El lunes, 16 de julio de 2018, 15:32:55 (UTC+2), OD escribió:

do you know what this is coming from?

fatal: [10.10.50.131]: FAILED! => {"msg": "The conditional check 'newagent_api.json.error == 0' failed. The error was: error while evaluating conditional (newagent_api.json.error == 0): 'dict object' has no attribute 'json'"}

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/1e801288-beb4-45e3-9da5-eb92a1e40fd7%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Olivier Doisneau]

Yes I think I got it to work. I messed up and ran the agent ansible script on the management server which messed up everything. so now that is fixed.  The agent is running but the management server is telling me the agent never connected.  Where do I look to find reason for not connecting?

On Mon, Jul 16, 2018 at 10:06 AM, <jesus.g...@wazuh.com> wrote:

Hello again Oliver,

Jesús

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:index_managemen...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:gr...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.1)"

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:secu...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:grokdebugger@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:logs...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/b011c250-a373-47dc-bd7e-dae1127eaccf%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[jesus.g...@wazuh.com]

Hello Oliver,

Let's see the ossec.log file from both Wazuh manager and Wazuh agent:

SSH into the agent machine and execute the next command:

cat /var/ossec/logs/ossec.log | grep -i -E '(error|warn)'

SSH into the manager machine and execute the next command:

cat /var/ossec/logs/ossec.log | grep -i -E '(error|warn)'

The above command will look for errors and warnings in the main log file, where we usually store any incidents from the software.

Regards,

Jesús

Jesús

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:index_manag...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:gr...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.1)"

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:security@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:grokdebugger@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:logstash@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.

[Olivier Doisneau]

It was a silly acl issue with udp coming back.

Now just struggling with postfix and I think I will have my vanilla setup done.

I am very happy with this tool and I am impressed by your support for all those emails and I am glad to see you have answers to almost all issues.

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.

To post to this group, send email to wa...@googlegroups.com.
Visit this group at https://groups.google.com/group/wazuh.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/90483088-9d20-47ae-aeec-b27125d31b0c%40googlegroups.com.

[jesus.g...@wazuh.com]

You are welcome Oliver. Let us know once you are done.

Best regards,

Jesús

[Olivier Doisneau]

so woke up this am and tried to login and says it is not up and running. Kibana seems to have just stopped.

root@wazuh1:/var/log/elasticsearch# systemctl status kibana

● kibana.service - Kibana

   Loaded: loaded (/etc/systemd/system/kibana.service; enabled; vendor preset: enabled)

   Active: inactive (dead) since Tue 2018-07-17 06:25:15 EDT; 2h 37min ago

 Main PID: 1806 (code=killed, signal=TERM)

Jul 16 22:41:34 wazuh1 kibana[1806]: {"type":"response","@timestamp":"2018-07-17T02:41:34Z","tags":[],"pid":1806,"method":"get","statusCode":304,"req":{"url":"/ui/favicons/favicon-32x32.png","method":"get","headers":{"host":"wazuh.prod.com:5601","connection":"keep-alive","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36","accept":"image/webp,image/apng,image/*,*/*;q=0.8","referer":"http://wazuh.prod.com:5601/app/wazuh","accept-encoding":"gzip, deflate","accept-language":"en-US,en;q=0.9","if-none-match":"\"8e183c2e644fb050707d89402e1f7a120a95e4d2\"","if-modified-since":"Mon, 11 Jun 2018 23:48:58 GMT"},"remoteAddress":"10.199.8.48","userAgent":"10.199.8.48","referer":"http://wazuh.prod.com:5601/app/wazuh"},"res":{"statusCode":304,"responseTime":2,"contentLength":9},"message":"GET /ui/favicons/favicon-32x32.png 304 2ms - 9.0B"}

Jul 16 22:41:34 wazuh1 kibana[1806]: {"type":"response","@timestamp":"2018-07-17T02:41:34Z","tags":[],"pid":1806,"method":"get","statusCode":304,"req":{"url":"/ui/favicons/favicon-16x16.png","method":"get","headers":{"host":"wazuh.prod.com:5601","connection":"keep-alive","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36","accept":"image/webp,image/apng,image/*,*/*;q=0.8","referer":"http://wazuh.prod.com:5601/app/wazuh","accept-encoding":"gzip, deflate","accept-language":"en-US,en;q=0.9","if-none-match":"\"13b869be5df4bdc56920edc16a28e67a7c08203b\"","if-modified-since":"Mon, 11 Jun 2018 23:48:58 GMT"},"remoteAddress":"10.199.8.48","userAgent":"10.199.8.48","referer":"http://wazuh.prod.com:5601/app/wazuh"},"res":{"statusCode":304,"responseTime":1,"contentLength":9},"message":"GET /ui/favicons/favicon-16x16.png 304 1ms - 9.0B"}

Jul 16 22:41:34 wazuh1 kibana[1806]: {"type":"response","@timestamp":"2018-07-17T02:41:34Z","tags":[],"pid":1806,"method":"post","statusCode":200,"req":{"url":"/elasticsearch/_msearch","method":"post","headers":{"host":"wazuh.prod.com:5601","connection":"keep-alive","content-length":"862","accept":"application/json, text/plain, */*","origin":"http://wazuh.prod.com:5601","kbn-version":"6.3.0","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36","content-type":"application/x-ndjson","referer":"http://wazuh.prod.com:5601/app/wazuh","accept-encoding":"gzip, deflate","accept-language":"en-US,en;q=0.9"},"remoteAddress":"10.199.8.48","userAgent":"10.199.8.48","referer":"http://wazuh.prod.com:5601/app/wazuh"},"res":{"statusCode":200,"responseTime":134,"contentLength":9},"message":"POST /elasticsearch/_msearch 200 134ms - 9.0B"}

Jul 16 22:41:51 wazuh1 kibana[1806]: {"type":"response","@timestamp":"2018-07-17T02:41:51Z","tags":[],"pid":1806,"method":"get","statusCode":200,"req":{"url":"/bundles/448c34a56d699c29117adc64c43affeb.woff2","method":"get","headers":{"host":"wazuh.prod.com:5601","connection":"keep-alive","origin":"http://wazuh.prod.com:5601","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36","accept":"*/*","referer":"http://wazuh.prod.com:5601/bundles/commons.style.css","accept-encoding":"gzip, deflate","accept-language":"en-US,en;q=0.9"},"remoteAddress":"10.199.8.48","userAgent":"10.199.8.48","referer":"http://wazuh.prod.com:5601/bundles/commons.style.css"},"res":{"statusCode":200,"responseTime":3,"contentLength":9},"message":"GET /bundles/448c34a56d699c29117adc64c43affeb.woff2 200 3ms - 9.0B"}

Jul 16 22:41:58 wazuh1 kibana[1806]: {"type":"response","@timestamp":"2018-07-17T02:41:58Z","tags":[],"pid":1806,"method":"get","statusCode":304,"req":{"url":"/ui/favicons/favicon-32x32.png","method":"get","headers":{"host":"wazuh.prod.com:5601","connection":"keep-alive","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36","accept":"image/webp,image/apng,image/*,*/*;q=0.8","referer":"http://wazuh.prod.com:5601/app/wazuh","accept-encoding":"gzip, deflate","accept-language":"en-US,en;q=0.9","if-none-match":"\"8e183c2e644fb050707d89402e1f7a120a95e4d2\"","if-modified-since":"Mon, 11 Jun 2018 23:48:58 GMT"},"remoteAddress":"10.199.8.48","userAgent":"10.199.8.48","referer":"http://wazuh.prod.com:5601/app/wazuh"},"res":{"statusCode":304,"responseTime":1,"contentLength":9},"message":"GET /ui/favicons/favicon-32x32.png 304 1ms - 9.0B"}

Jul 16 22:41:58 wazuh1 kibana[1806]: {"type":"response","@timestamp":"2018-07-17T02:41:58Z","tags":[],"pid":1806,"method":"get","statusCode":304,"req":{"url":"/ui/favicons/favicon-16x16.png","method":"get","headers":{"host":"wazuh.prod.com:5601","connection":"keep-alive","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36","accept":"image/webp,image/apng,image/*,*/*;q=0.8","referer":"http://wazuh.prod.com:5601/app/wazuh","accept-encoding":"gzip, deflate","accept-language":"en-US,en;q=0.9","if-none-match":"\"13b869be5df4bdc56920edc16a28e67a7c08203b\"","if-modified-since":"Mon, 11 Jun 2018 23:48:58 GMT"},"remoteAddress":"10.199.8.48","userAgent":"10.199.8.48","referer":"http://wazuh.prod.com:5601/app/wazuh"},"res":{"statusCode":304,"responseTime":1,"contentLength":9},"message":"GET /ui/favicons/favicon-16x16.png 304 1ms - 9.0B"}

Jul 16 22:41:58 wazuh1 kibana[1806]: {"type":"response","@timestamp":"2018-07-17T02:41:58Z","tags":[],"pid":1806,"method":"post","statusCode":200,"req":{"url":"/elasticsearch/_msearch","method":"post","headers":{"host":"wazuh.prod.com:5601","connection":"keep-alive","content-length":"906","accept":"application/json, text/plain, */*","origin":"http://wazuh.prod.com:5601","kbn-version":"6.3.0","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36","content-type":"application/x-ndjson","referer":"http://wazuh.prod.com:5601/app/wazuh","accept-encoding":"gzip, deflate","accept-language":"en-US,en;q=0.9"},"remoteAddress":"10.199.8.48","userAgent":"10.199.8.48","referer":"http://wazuh.prod.com:5601/app/wazuh"},"res":{"statusCode":200,"responseTime":144,"contentLength":9},"message":"POST /elasticsearch/_msearch 200 144ms - 9.0B"}

Jul 17 06:25:15 wazuh1 systemd[1]: Stopping Kibana...

Jul 17 06:25:15 wazuh1 systemd[1]: Stopped Kibana.

Jul 17 06:25:16 wazuh1 systemd[1]: Stopped Kibana.

root@wazuh1:/var/log# /var/ossec/bin/ossec-control status

wazuh-clusterd not running...

ossec-monitord is running...

ossec-logcollector is running...

ossec-remoted is running...

ossec-syscheckd is running...

ossec-analysisd is running...

ossec-maild is running...

ossec-execd is running...

wazuh-modulesd is running...

wazuh-db is running...

ossec-authd is running...

Elasticsearch wazuh.log

[2018-07-17T00:00:00,438][INFO ][o.e.c.m.MetaDataCreateIndexService] [node-1] [wazuh-monitoring-3.x-2018.07.17] creating index, cause [api], templates [wazuh-agent], shards [5]/[1], mappings [wazuh-agent]

[2018-07-17T00:00:00,557][INFO ][o.e.c.m.MetaDataMappingService] [node-1] [wazuh-monitoring-3.x-2018.07.17/haQ0GTDGQf6ymQigjBe_sg] update_mapping [wazuh-agent]

[2018-07-17T00:00:00,559][INFO ][o.e.c.m.MetaDataMappingService] [node-1] [wazuh-monitoring-3.x-2018.07.17/haQ0GTDGQf6ymQigjBe_sg] update_mapping [wazuh-agent]

[2018-07-17T00:29:43,211][INFO ][o.e.c.m.MetaDataCreateIndexService] [node-1] [wazuh-alerts-3.x-2018.07.17] creating index, cause [auto(bulk api)], templates [wazuh], shards [5]/[1], mappings [wazuh]

[2018-07-17T00:29:43,313][INFO ][o.e.c.m.MetaDataMappingService] [node-1] [wazuh-alerts-3.x-2018.07.17/9XJKqOkXR9ufPem2l8n09A] update_mapping [wazuh]

[2018-07-17T00:48:41,535][INFO ][o.e.c.m.MetaDataMappingService] [node-1] [wazuh-alerts-3.x-2018.07.17/9XJKqOkXR9ufPem2l8n09A] update_mapping [wazuh]

[2018-07-17T00:48:41,573][INFO ][o.e.c.m.MetaDataMappingService] [node-1] [wazuh-alerts-3.x-2018.07.17/9XJKqOkXR9ufPem2l8n09A] update_mapping [wazuh]

[2018-07-17T01:11:30,100][INFO ][o.e.c.m.MetaDataMappingService] [node-1] [wazuh-alerts-3.x-2018.07.17/9XJKqOkXR9ufPem2l8n09A] update_mapping [wazuh]

[2018-07-17T01:20:32,310][INFO ][o.e.c.m.MetaDataMappingService] [node-1] [wazuh-alerts-3.x-2018.07.17/9XJKqOkXR9ufPem2l8n09A] update_mapping [wazuh]

[2018-07-17T01:45:00,001][INFO ][o.e.x.m.MlDailyMaintenanceService] triggering scheduled [ML] maintenance tasks

[2018-07-17T01:45:00,002][INFO ][o.e.x.m.a.TransportDeleteExpiredDataAction] [node-1] Deleting expired data

[2018-07-17T01:45:00,030][INFO ][o.e.x.m.a.TransportDeleteExpiredDataAction] [node-1] Completed deletion of expired data

[2018-07-17T01:45:00,030][INFO ][o.e.x.m.MlDailyMaintenanceService] Successfully completed [ML] maintenance tasks

[2018-07-17T02:50:50,950][INFO ][o.e.c.m.MetaDataMappingService] [node-1] [wazuh-alerts-3.x-2018.07.17/9XJKqOkXR9ufPem2l8n09A] update_mapping [wazuh]

[2018-07-17T10:25:20,241][INFO ][o.e.x.s.a.f.FileUserPasswdStore] [node-1] users file [/etc/elasticsearch/users] changed. updating users... )

[2018-07-17T10:25:20,242][INFO ][o.e.x.s.a.f.FileUserRolesStore] [node-1] users roles file [/etc/elasticsearch/users_roles] changed. updating users roles...

[2018-07-17T10:25:20,243][INFO ][o.e.x.s.a.s.FileRolesStore] [node-1] parsed [0] roles from file [/etc/elasticsearch/roles.yml]

[2018-07-17T10:25:20,243][INFO ][o.e.x.s.a.s.FileRolesStore] [node-1] updated roles (roles file [/etc/elasticsearch/roles.yml] changed)

Jesús

Jesús

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:index_managemen...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:gr...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.1)"

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:secu...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:grokdebugger@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:logs...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/22ca3e35-f6ea-44a7-b7ae-fa5b41ffb398%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Olivier Doisneau]

started kibana and now wazuh is not showing up.

Jesús

Jesús

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:index_management...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:gr...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.1)"

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:secu...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:grokdebugger@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:logs...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:repo...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0

[jesus.g...@wazuh.com]

Hello Oliver,

When Kibana is ignoring a plugin it usually means the plugin expected a different Kibana version from the installed version

or it's invalid or it's removed.

1. Check for Kibana errors:

journalctl -u kibana > /tmp/kibanalog; cat /tmp/kibanalog | grep -i -E "(error|warn)"

2. Check if there is a Wazuh plugin:

ls /usr/share/kibana/plugins/ | grep wazuh

3. Check Kibana version:

cat /usr/share/kibana/package.json | grep version

4. Check app expected version:

cat /usr/share/kibana/plugins/wazuh/package.json | grep kibana -A1 | grep version

Please paste the output of the above commands, thanks in advance.

Regards,

Jesús

Jesús

Jesús

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:index_manag...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:gr...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.1)"

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:security@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:grokdebugger@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:logstash@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:reporting@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0

...

[Olivier Doisneau]

not sure you need #1.  I have 2,3,4 which tells me you are right on.

root@wazuh1:/var/ossec/etc# ls /usr/share/kibana/plugins/ | grep wazuh

wazuh

root@wazuh1:/var/ossec/etc# cat /usr/share/kibana/package.json | grep version

  "version": "6.3.1",

root@wazuh1:/var/ossec/etc# cat /usr/share/kibana/plugins/wazuh/package.json | grep kibana -A1 | grep version

        "version": "6.3.0"

root@wazuh1:/var/ossec/etc# 

why would it work yesterda and not today?  How do I fix?

Hello Oliver,

Jesús

Jesús

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:index_managemen...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:gr...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.1)"

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:secu...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:grokdebugger@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:logs...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.

...

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/88217b6c-eafd-4cd1-9778-5c33571d555d%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[jesus.g...@wazuh.com]

Hello again Oliver,

You have a Wazuh App for Kibana 6.3.0 but 6.3.1 is installed. The Wazuh App package to be installed must be https://packages.wazuh.com/wazuhapp/wazuhapp-3.3.1_6.3.1.zip 

but you need to have Wazuh 3.3.1 installed too to be compatible. 

This means your environment should be as follow:

- Wazuh manager 3.3.1 + Wazuh API 3.3.1

- Elasticsearch, Logstash, Kibana and Filebeat(if distributed architecture) 6.3.1

- Wazuh App for Kibana 3.3.1-6.3.1

To upgrade the Wazuh App you can do it as follow:

# systemctl stop kibana
# /usr/share/kibana/bin/kibana-plugin remove wazuh
# rm -rf /usr/share/kibana/optimize/bundles
# /usr/share/kibana/bin/kibana-plugin install https://packages.wazuh.com/wazuhapp/wazuhapp-3.3.1_6.3.1.zip
# systemctl restart kibana

Once done, wait few minutes and open a new incognito window in your browser or remove cache/cookies from your browser.

That's all, let us know if you need help upgrading some component.

Regards,

Jesús

Hello Oliver,

Jesús

Jesús

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:index_manag...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:gr...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.1)"

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:security@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:grokdebugger@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:logstash@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.

...

[Olivier Doisneau]

Nice it is back. Any idea why it worked and stopped working because I did not touch anything

Hello Oliver,

Jesús

Jesús

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:index_managemen...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:gr...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.1)"

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:secu...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:grokdebugger@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:logs...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.

...

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/d0cf9e3c-11b0-41cb-8ee5-ef58cc7eea6d%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[jesus.g...@wazuh.com]

Hi Oliver,

It's pretty weird but I think maybe you or someone has upgraded all packages from your machine (yum update, apt-get upgrade) then Kibana 

upgraded to 6.3.1 and your plugin went down. It's my only reasonable answer to this question.

Regards,

Jesús

Hello Oliver,

Jesús

Jesús

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:index_manag...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:gr...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.1)"

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:security@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:grokdebugger@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:logstash@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.

...

--

...

[jesus.g...@wazuh.com]

Forgot to say, a useful tip to disable the Elastic repository, this way you prevent from non-desired upgrades:

Debian / Ubuntu

# sed -i "s/^deb/#deb/" /etc/apt/sources.list.d/elastic-6.x.list
# apt-get update

CentOS / RHEL

# sed -i "s/^enabled=1/enabled=0/" /etc/yum.repos.d/elastic.repo

Regards,

Jesús

[Olivier Doisneau]

oh right probably from my postfix issues.  Good point.  thanks

Jesús

Hi Oliver,

Hello Oliver,

Jesús

Jesús

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:index_managemen...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:gr...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.1)"

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:secu...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:grokdebugger@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.

Jul 16 06:36:50 wazuh1 kibana[12659]: {"type":"log","@timestamp":"2018-07-16T10:36:50Z","tags":["status","plugin:logs...@6.3.1","error"],"pid":12659,"state":"red","message":"Status changed from red to red - X-Pack plugin is not installed on the [data] Elasticsearch cluster.","prevState":"red","prevMsg":"This version of Kibana requires Elasticsearch v6.3.1 on all nodes. I found the following incompatible nodes in your cluster: v6.2.3 @ 127.0.0.1:9200 (127.0.0.

...

--

...

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/c45825c3-3be0-4763-aa9b-55aed2348779%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[jesus.g...@wazuh.com]

You are welcome!

Regards,

Jesús