Elasticsearch 6.3.0 Request TImeout after 300000ms

635 views
Skip to first unread message

Adiel Jesus Navarro Rosado

unread,
Sep 6, 2018, 12:36:58 PM9/6/18
to wa...@googlegroups.com

When I started Kibana, only this page appears:

 

I increase the timeout value but not changed

 

cid:image001.png@01D445D5.5CD23E60

 

jesus.g...@wazuh.com

unread,
Sep 6, 2018, 12:46:42 PM9/6/18
to Wazuh mailing list
Hi Adiel,

It appears that Elasticsearch is down or that Kibana is not able to communicate properly with Elasticsearch.

Please, check if Elasticsearch is running. SSH into the Elasticsearch machine and execute the next command please:

systemctl status elasticsearch

If it's running, let's do a few more checks described in the next steps.

If your Elasticsearch machine is the same as the Kibana machine

Try to execute a curl command pointing to Elasticsearch.

curl elastic_ip:9200


If your Elasticsearch machine is different from the Kibana machine

Can you ssh into the 10.209.41.56 machine (Kibana machine)? Once you are logged in, please try to ping your Elasticsearch machine
and try to execute a curl pointing to Elasticsearch machine.

curl elastic_ip:9200
ping elastic_ip

Last question: 

- Is Elasticsearch using X-Pack security feature?

I hope it helps.

Regards,
Jesús

Adiel Jesus Navarro Rosado

unread,
Sep 6, 2018, 1:37:38 PM9/6/18
to jesus.g...@wazuh.com, Wazuh mailing list

Hi Jesus.

 

Elasticsearch machine is the same as the Kibana and Logstash machine

 

Send you the output of the commands:

 

[root@porsdmelk04 ossec]# systemctl status elasticsearch -l

● elasticsearch.service - Elasticsearch

   Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)

   Active: active (running) since jue 2018-09-06 12:15:12 CDT; 12min ago

     Docs: http://www.elastic.co

Main PID: 16757 (java)

   CGroup: /system.slice/elasticsearch.service

           ├─16757 /bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.io.tmpdir=/tmp/elasticsearch.fiAYHY5i -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/lib/elasticsearch -XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:/var/log/elasticsearch/gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=32 -XX:GCLogFileSize=64m -Des.path.home=/usr/share/elasticsearch -Des.path.conf=/etc/elasticsearch -Des.distribution.flavor=default -Des.distribution.type=rpm -cp /usr/share/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch -p /var/run/elasticsearch/elasticsearch.pid --quiet

           └─16838 /usr/share/elasticsearch/modules/x-pack/x-pack-ml/platform/linux-x86_64/bin/controller

 

sep 06 12:15:12 porsdmelk04 systemd[1]: Started Elasticsearch.

sep 06 12:15:12 porsdmelk04 systemd[1]: Starting Elasticsearch...

[root@porsdmelk04 ossec]# curl server_ip:9200

{

  "name" : "porsdmelk04",

  "cluster_name" : "PORSDMtry4",

  "cluster_uuid" : "Cgv-skFeTWugXhX_FuEpCA",

  "version" : {

    "number" : "6.3.0",

    "build_flavor" : "default",

    "build_type" : "rpm",

    "build_hash" : "424e937",

    "build_date" : "2018-06-11T23:38:03.357887Z",

    "build_snapshot" : false,

    "lucene_version" : "7.3.1",

    "minimum_wire_compatibility_version" : "5.6.0",

    "minimum_index_compatibility_version" : "5.0.0"

  },

  "tagline" : "You Know, for Search"

}

 

 

Is Elasticsearch using X-Pack security feature?

NO... we don’t install any X-Pack feature

 

 

 

De: wa...@googlegroups.com [mailto:wa...@googlegroups.com] En nombre de jesus.g...@wazuh.com
Enviado el: jueves, 06 de septiembre de 2018 11:47 a.m.
Para: Wazuh mailing list
Asunto: [SOCIAL NETWORK] Re: Elasticsearch 6.3.0 Request TImeout after 300000ms

 

Hi Adiel,

 

It appears that Elasticsearch is down or that Kibana is not able to communicate properly with Elasticsearch.

 

Please, check if Elasticsearch is running. SSH into the Elasticsearch machine and execute the next command please:

 

systemctl status elasticsearch

 

If it's running, let's do a few more checks described in the next steps.

 

If your Elasticsearch machine is the same as the Kibana machine

 

Try to execute a curl command pointing to Elasticsearch.

 

curl elastic_ip:9200

 

 

If your Elasticsearch machine is different from the Kibana machine

 

Can you ssh into the machine (Kibana machine)? Once you are logged in, please try to ping your Elasticsearch machine

and try to execute a curl pointing to Elasticsearch machine.

 

curl elastic_ip:9200
ping elastic_ip


Last question: 

 

- Is Elasticsearch using X-Pack security feature?

 

I hope it helps.

 

Regards,

Jesús

El jueves, 6 de septiembre de 2018, 18:36:58 (UTC+2), Adiel Jesus Navarro Rosado escribió:

When I started Kibana, only this page appears:

 

I increase the timeout value but not changed

 

cid:image001.png@01D445D5.5CD23E60

 

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To post to this group, send email to wa...@googlegroups.com.
Visit this group at https://groups.google.com/group/wazuh.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/85aefe1c-00b8-4b5f-ba26-942f45cac5d7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

jesus.g...@wazuh.com

unread,
Sep 7, 2018, 3:28:21 AM9/7/18
to Wazuh mailing list
Hello again Adiel,

It seems strange for me, let's try to check the Kibana configuration and on the other hand the Elasticsearch logs:

1. Please copy and paste the contents of your Kibana configuration file:

cat /etc/kibana/kibana.yml

2. Please copy and paste the output of the following command:

cat /var/log/elasticsearch/elasticsearch.log | grep -E "(ERROR|WARN|CRITICAL|FATAL)"

Regards,
Jesús

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+unsubscribe@googlegroups.com.

Adiel Jesus Navarro Rosado

unread,
Sep 11, 2018, 11:01:27 AM9/11/18
to jesus.g...@wazuh.com, Wazuh mailing list

Is Elasticsearch using X-Pack security feature?

NO... we don’t install any X-Pack feature

 

 

 

De: wa...@googlegroups.com [mailto:wa...@googlegroups.com] En nombre de jesus.g...@wazuh.com
Enviado el: jueves, 06 de septiembre de 2018 11:47 a.m.
Para: Wazuh mailing list
Asunto: [SOCIAL NETWORK] Re: Elasticsearch 6.3.0 Request TImeout after 300000ms

 

Hi Adiel,

 

It appears that Elasticsearch is down or that Kibana is not able to communicate properly with Elasticsearch.

 

Please, check if Elasticsearch is running. SSH into the Elasticsearch machine and execute the next command please:

 

systemctl status elasticsearch

 

If it's running, let's do a few more checks described in the next steps.

 

If your Elasticsearch machine is the same as the Kibana machine

 

Try to execute a curl command pointing to Elasticsearch.

 

curl elastic_ip:9200

 

 

If your Elasticsearch machine is different from the Kibana machine

 

Can you ssh into the machine (Kibana machine)? Once you are logged in, please try to ping your Elasticsearch machine

and try to execute a curl pointing to Elasticsearch machine.

 

curl elastic_ip:9200
ping elastic_ip


Last question: 

 

- Is Elasticsearch using X-Pack security feature?

 

I hope it helps.

 

Regards,

Jesús

El jueves, 6 de septiembre de 2018, 18:36:58 (UTC+2), Adiel Jesus Navarro Rosado escribió:

When I started Kibana, only this page appears:

 

I increase the timeout value but not changed

 

cid:image001.png@01D445D5.5CD23E60

 

--

You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.

jesus.g...@wazuh.com

unread,
Sep 12, 2018, 7:02:15 AM9/12/18
to Wazuh mailing list
Hello again Adiel,

As I said in our last message, I need the output of the next commands:

cat /etc/kibana/kibana.yml
cat /var/log/elasticsearch/elasticsearch.log | grep -E "(ERROR|WARN|CRITICAL|FATAL)"

Please, paste the output of the above commands to continue helping you, thanks.

Regards,
Jesús

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+unsubscribe@googlegroups.com.

Adiel Jesus Navarro Rosado

unread,
Sep 12, 2018, 10:30:47 AM9/12/18
to jesus.g...@wazuh.com, Wazuh mailing list, Jose Carlos Ortiz Hernandez

Jesus, I send you the output of the commands.

 

Waiting for your responses and comments.

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.

--

You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.


To post to this group, send email to wa...@googlegroups.com.
Visit this group at https://groups.google.com/group/wazuh.

elasticsearch.log
kibana_yml.txt

jesus.g...@wazuh.com

unread,
Sep 13, 2018, 11:15:39 AM9/13/18
to Wazuh mailing list
Hello Adiel,

Elasticsearch is having troubles, see next logs extracted from your log dump:

[2018-09-12T09:23:00,539][WARN ][o.e.m.j.JvmGcMonitorService] [porsdmelk05] [gc][23624] overhead, spent [3s] collecting in the last [3.2s]
[2018-09-12T09:23:04,296][ERROR][o.e.x.m.c.n.NodeStatsCollector] [porsdmelk05] collector [node_stats] timed out when collecting data
[2018-09-12T09:23:04,302][WARN ][o.e.m.j.JvmGcMonitorService] [porsdmelk05] [gc][23625] overhead, spent [3.6s] collecting in the last [3.7s]

This usually means you have a low in resources machine. Can you verify we are not in a low RAM or low CPU environment?

Regards,
Jesús

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+unsubscribe@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+unsubscribe@googlegroups.com.

Reply all
Reply to author
Forward
0 new messages