Question about 5 of 110 shards failed error in kibana

1,151 views
Skip to first unread message

맹서기

unread,
May 22, 2019, 8:52:13 PM5/22/19
to Wazuh mailing list

5 of 110.JPG



Your previous answer was very helpful in resolving this issue.

I'd like to ask you a question.

As shown above, I searched for an error of " 5 of 110 Shard failed."
as a result of one's search 

First, the search results show that increasing the size of threadpool.search.stack solves it, but I don't know where the file is setting it.
If the solution is the same as I said, please let me know where the setup file is.

Secondly, if that is not a solution, I would appreciate it if you could give me a proper solution.

Thank you every time.

Jose Miguel Garcia Ramos

unread,
May 24, 2019, 5:38:15 AM5/24/19
to 맹서기, wa...@googlegroups.com

Hi, 맹서기

Elasticsearch nodes hold several thread pools that allow pending requests to be held and processed later instead of discarding them. In case the search would be big enough to overflow that pools, then it would be discarded and the shard would reply with a fail.
ES official documentation about Thread pools: https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-threadpool.html#modules-threadpool

But in your case, the number of shards failed is exactly the same on every search which makes me think it’s not a problem related to pools.

I recommend you to check your Elasticsearch logs (/var/log/elasticsearch/) and look for errors or warnings. Feel free to share them with me in your reply so we can review them together!

Also, try with bigger searches (less specific searches that give you more results) and with smaller searches (more specific searches that give fewer results) and check if the number of shards that fail is exactly the same.
Anyway, if you want to modify your thread pool attributes you have to modify the elasticsearch.yml configuration file of every node and add the following:

thread_pool:
    search:
        size: 30
        queue_size: 1000

Where size parameter specify the number of threads (default is number of cores * 5) and queue_size controls the number of threads that can be set on hold for later execution in that note.

Feel free to ask any other questions or further information about this anytime!

Best Regards,
Jose


--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To post to this group, send email to wa...@googlegroups.com.
Visit this group at https://groups.google.com/group/wazuh.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/2fc41048-a9e6-47a1-b43e-9d57b0f66de0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply all
Reply to author
Forward
0 new messages