Wazuh Single Node Cluster is Yellow
Prachi Katakwar
Hi Team,
Happy Christmas Eve.
My Wazuh Cluster seems to be yellow and below are the details , I don’t want to increase the nodes and have configured Index Lifecycle management Policy using below link:
https://wazuh.com/blog/wazuh-index-management/
Please could you help me in changing the Cluster health from yellow to green?
[root@sekaissecdetection elasticsearch]# curl -X GET "10.64.97.71:9200/_cluster/health?pretty"
{
"cluster_name" : "sekaissecdetection-elasticsearch",
"status" : "yellow",
"timed_out" : false,
"number_of_nodes" : 1,
"number_of_data_nodes" : 1,
"active_primary_shards" : 929,
"active_shards" : 929,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 1,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 99.89247311827957
}
[root@sekaissecdetection elasticsearch]# curl -XGET 'http://10.64.97.71:9200/_cluster/allocation/explain'
{"index":"ilm-history-3-000010","shard":0,"primary":false,"current_state":"unassigned","unassigned_info":{"reason":"INDEX_CREATED","at":"2021-12-16T10:57:51.804Z","last_allocation_status":"no_attempt"},"can_allocate":"no","allocate_explanation":"cannot allocate because allocation is not permitted to any of the nodes","node_allocation_decisions":[{"node_id":"mnkwUCnISKWHi_hnNeDewA","node_name":"sekaissecdetection.hubseka.ericsson.net","transport_address":"10.64.97.71:9300","node_attributes":{"ml.machine_memory":"33511694336","xpack.installed":"true","transform.node":"true","ml.max_open_jobs":"512","ml.max_jvm_size":"17179869184"},"node_decision":"no","weight_ranking":1,"deciders":[{"decider":"same_shard","decision":"NO","explanation":"a copy of this shard is already allocated to this node [[ilm-history-3-000010][0], node[mnkwUCnISKWHi_hnNeDewA], [P], s[STARTED], a[id=nDf6VnNYSmuXaVhfoG7hAA]]"}]}]}
BR
//Prachi
José Fernández
Prachi Katakwar
Hej Jose
In this guide https://wazuh.com/blog/wazuh-index-management/, we have followed Elastic ILM where it is not given to set the “number of replicas” : 0.
How to do that?
BR
//Prachi
--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
wazuh+un...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/wazuh/b533c250-330e-456f-bf7f-e4cebb1c8897n%40googlegroups.com.
José Fernández
.png?part=0.1&view=1)
Then you will need to reconfigure the template to take effect as the guide points out.
Prachi Katakwar
Hej Jose,
The Steps which you are pointing are for Opendistro and my setup is not Opendistro.
At the start of the blog post , below are the lines mentioned, I have configured through Elastic ILM.
In this blog post you will learn how to configure Elastic
ILM and OpenDistro
ISM to automatically manage the data in those indices over time.
BR
//Prachi
From: wa...@googlegroups.com <wa...@googlegroups.com>
On Behalf Of José Fernández
Sent: den 21 december 2021 12:03
To: Wazuh mailing list <wa...@googlegroups.com>
Subject: Re: Wazuh Single Node Cluster is Yellow
We can change that value at Configuration of Index State Policies
step where you have to specify manually the policy. There is a field called
number_of_replicas that must be changed to 0, by default is set to 1 because is a good practice to have at least one replica set. Take a look at the attached screenshot.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/3bb04c79-2e7b-49fb-9371-b057e80bff1dn%40googlegroups.com.
José Fernández
.png?part=0.1&view=1)
Prachi Katakwar
Hi Jose,
Yes, I just now checked out my settings for ILM policy , it was greyed out and now I just set to “0”.
Soon after that checked the below commands, but the cluster state is still yellow, do I need to restart any of the services?
[root@sekaissecdetection ~]# curl -X GET "10.64.97.71:9200/_cluster/health?pretty"
{
"cluster_name" : "sekaissecdetection-elasticsearch",
"status" : "yellow",
"timed_out" : false,
"number_of_nodes" : 1,
"number_of_data_nodes" : 1,
"active_primary_shards" : 931,
"active_shards" : 931,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 1,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 99.89270386266095
}
[root@sekaissecdetection ~]# curl -XGET 'http://10.64.97.71:9200/_cluster/allocation/explain'
{"index":"ilm-history-3-000010","shard":0,"primary":false,"current_state":"unassigned","unassigned_info":{"reason":"INDEX_CREATED","at":"2021-12-16T10:57:51.804Z","last_allocation_status":"no_attempt"},"can_allocate":"no","allocate_explanation":"cannot allocate because allocation is not permitted to any of the nodes","node_allocation_decisions":[{"node_id":"mnkwUCnISKWHi_hnNeDewA","node_name":"sekaissecdetection.hubseka.ericsson.net","transport_address":"10.64.97.71:9300","node_attributes":{"ml.machine_memory":"33511694336","xpack.installed":"true","transform.node":"true","ml.max_open_jobs":"512","ml.max_jvm_size":"17179869184"},"node_decision":"no","weight_ranking":1,"deciders":[{"decider":"same_shard","decision":"NO","explanation":"a copy of this shard is already allocated to this node [[ilm-history-3-000010][0], node[mnkwUCnISKWHi_hnNeDewA], [P], s[STARTED], a[id=n[root@sekaissecdetection ~]#
BR
//Prachi
From: wa...@googlegroups.com <wa...@googlegroups.com>
On Behalf Of José Fernández
Sent: den 22 december 2021 13:15
To: Wazuh mailing list <wa...@googlegroups.com>
Subject: Re: Wazuh Single Node Cluster is Yellow
Hello Prachi,
Yeah, in Elastic environment you have an option to manage this field. Take a look at the attached image and check if you have set the number of replicas field to 0.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/4d54904f-e7be-4fea-a270-c777c8a917c9n%40googlegroups.com.
Prachi Katakwar
Sent: Wednesday, December 22, 2021 9:31:20 PM
To: José Fernández <jose.fe...@wazuh.com>; Wazuh mailing list <wa...@googlegroups.com>
Subject: RE: Wazuh Single Node Cluster is Yellow
Alberto Rodriguez
Prachi Katakwar
Hi Alberto,
I am sorry for the delay in reply and bothering you again and again.
Yes, you are right and I saw the link also given by you in the trail email to update the yellow indices replicas setting to 0.
So just to get the yellow indices , ran this command but nothing was displayed
[root@sekaissecdetection ~]# curl -X GET "10.64.97.71:9200/_cat/indices/my-index-*?v=true&s=index&pretty"
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size //no output displayed here ,
[root@sekaissecdetection ~]# curl -X GET "10.64.97.71:9200/_cluster/health?pretty"
{
"cluster_name" : "sekaissecdetection-elasticsearch",
"status" : "yellow",
"timed_out" : false,
"number_of_nodes" : 1,
"number_of_data_nodes" : 1,
"active_primary_shards" : 935,
"active_shards" : 935,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 1,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 99.8931623931624
}
What should I do now?
BR
//Prachi
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/07027b08-e81e-406e-b6ec-592ee04ffad2n%40googlegroups.com.
Prachi Katakwar
Hi Team,
Merry Christmas and Happy New Year 😊
The issue is resolved now, below are the steps I took , please let me know if my understanding is correct?
First I hit the command:
[root@sekaissecdetection ~]# curl -XGET 'http://10.64.97.71:9200/_cluster/allocation/explain'
{"index":"ilm-history-3-000010","shard":0,"primary":false,"current_state":"unassigned","unassigned_info":{"reason":"INDEX_CREATED","at":"2021-12-16T10:57:51.804Z","last_allocation_status":"no_attempt"},"can_allocate":"no","allocate_explanation":"cannot allocate because allocation is not permitted to any of the nodes","node_allocation_decisions":[{"node_id":"mnkwUCnISKWHi_hnNeDewA","node_name":"sekaissecdetection.hubseka.ericsson.net","transport_address":"10.64.97.71:9300","node_attributes":{"ml.machine_memory":"33511694336","xpack.installed":"true","transform.node":"true","ml.max_open_jobs":"512","ml.max_jvm_size":"17179869184"},"node_decision":"no","weight_ranking":1,"deciders":[{"decider":"same_shard","decision":"NO","explanation":"a copy of this shard is already allocated to this node [[ilm-history-3-000010][0], node[mnkwUCnISKWHi_hnNeDewA], [P], s[STARTED], a[id=n[root@sekaissecdetection ~]#
Here I understood that the index :"ilm-history-3-000010" is having a replica and so the Wazuh single Node Cluter is yellow.
Then I just hit this command:
PUT /ilm-history-3-000010/_settings
{
"index" : {
"number_of_replicas" : 0
}
}
After this cluster state is green.
[root@sekaissecdetection ~]# curl -X GET "10.64.97.71:9200/_cluster/health?pretty"
{
"cluster_name" : "sekaissecdetection-elasticsearch",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 1,
"number_of_data_nodes" : 1,
"active_primary_shards" : 933,
"active_shards" : 933,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 100.0
}
BR
//Prachi
Alberto Rodriguez
Sorry for the late response. Yes, you are completely right, your cluster created an index with a replica. This replica was an orphan because, with only one node, the cluster doesn't know where to place it. By setting the index replicas to 0, the cluster knows that the replica must be removed, and the cluster goes green.
You do have not high availability, but I see that you configured index snapshots which is a good idea. If your Elasticsearch node goes down, you can't use and ingest data in Elasticsearch but at least, you will have a backup. Having a cluster gives you an advantage: if one node goes down, you can still ingest data, so it's recommended for some productions environments depending on your needs, of course.
Prachi Katakwar
Thank you Alberto so much for a positive response, being using Wazuh for almost 2 years I feel now atleast I know something 😊
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/dc78f814-32d6-4a46-9a71-6c6b875a0b2en%40googlegroups.com.