Events not showing on Kibana dashborad

[siddha...@gmail.com]

Hello Team,

I'm using Wazuh 4.2 All-in-one on Ubuntu 20.04 VM.

when i login into the kibana web interface it does not show any event.

i have checked the status of wazuh-manager,filebeat,elasticsearch and kibana services all are showing running.

please suggest.

Any help would be appreciated.

Thank yo.

[Alfonso Ruiz-Bravo]

Hello Siddharth,

Could you give us more information?
In which Dashboard can't you see the events?
Can you see Wazuh alerts in the Discover section of Kibana?
Can you check for Wazuh alert indexes in Elasticsearch? GET /_cat/indices/wazuh-alerts*?s=index

Thank you very much for your patience, we are waiting for your answer.

Best regards,

Alfonso Ruiz-Bravo

[siddha...@gmail.com]

Hello Alfonso,

Thanks for your support.

Yes, I can't see any alerts on the discover dashboard and also on agent security events. i run this cmd and not getting any output. GET /_cat/indices/wazuh-alerts*?s=index 

I have also noticed that disk space is getting consumed more, before it's not working like this.

please suggest.

[Alfonso Ruiz-Bravo]

Hello Siddharth, 

Sorry, I didn't express it well. Could you run the following request against the Elasticsearch API?

GET /_cat/indices/wazuh-alerts*?s=index

Example:

curl -k -u <user> "https://<elastic_ip>:9200/_cat/indices/wazuh-alerts*?s=index"

Additionally, if you do not see alerts in Kibana, neither in Discover nor in Security Events, it could be that the alerts are not being generated, that they are not being indexed in Elasticsearch or that Kibana has a problem.

Could you confirm if you have created the index-pattern of wazuh-alerts-*?

Best regards,

Alfonso Ruiz-Bravo

[siddha...@gmail.com]

Hello Alfonso,
Thanks for your support.

I have run this cmd curl -k -u <user> "https://<elastic_ip>:9200/_cat/indices/wazuh-alerts*?s=index"
and see the alerts stored there a few months back. I have also attached a file,please check.
i don't know about this index-pattern of wazuh-alerts-*?
please suggest.

[siddha...@gmail.com]

miss the file, now attached please check.

[Alfonso Ruiz-Bravo]

Hello Siddharth

I see quite a few indexes and it looks like I stopped indexing alerts at the end of November last year.

Could you please do this check?

curl -k -u <user> "https://<elastic_ip>:9200/_cluster/health?pretty"

Regards

[siddha...@gmail.com]

Hello Alfonso,

i have run the same cmd and got this output

{
  "cluster_name" : "wazuh-cluster",
  "status" : "yellow",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 864,
  "active_shards" : 864,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 136,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 86.4
}

please suggest.

[Alfonso Ruiz-Bravo]

Hello Siddharth,

Well, it looks like the Open Distro cluster is fine. It has a high number of shards, when they reach 1000 it is possible that it will stop indexing, you should increase that number or merge or delete old indexes. But that doesn't seem to be the root of the current problem.

Could you run the following commands on the Filebeat host? is the Filebeat service running?

- filebeat test config

- filebeat test output

If you re-run the request that shows the indexes, could you see an increase in documents from the last index?

wazuh-alerts-4.x-2021.11.29 -> 73002

Regards,

[siddha...@gmail.com]

Hello Alfonso,

i have run the cmd as you suggested please check.

root@WAZUHAIO:~#  filebeat test config
Config OK

root@WAZUHAIO:~# filebeat test output
elasticsearch: https://127.0.0.1:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 127.0.0.1
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
  talk to server... OK
  version: 7.10.2

yet trying to learn, i don't have much understanding.

please suggest.

[Alfonso Ruiz-Bravo]

Hello Siddharth,

Apparently, there is a problem in the data flow, our intention is to see where it resides, it seems that Filebeat has no problem reaching Elasticsearch.

If you re-run the request that shows the indexes, could you see an increase in documents from the last index?

curl -k -u <user> "https://<elastic_ip>:9200/_cat/indices/wazuh-alerts*?s=index"

wazuh-alerts-4.x-2021.11.29 -> 73002

If the number of documents does not increase, Filebeat may not have the service running, or the Wazuh manager may not be generating alerts.

If the number of documents does increase, the problem lies with Kibana.

Regards,

Alfonso Ruiz-Bravo Cloud computing engineer The Open Source Security Platform

--
You received this message because you are subscribed to a topic in the Google Groups "Wazuh mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/wazuh/F3BaDqffjAA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/e42a5ac4-0950-4c81-83ca-2c7d2c71ec1an%40googlegroups.com.

[siddha...@gmail.com]

Hello Alfonso,

when i run this cmd curl -k -u <user> "https://<elastic_ip>:9200/_cat/indices/wazuh-alerts*?s=index"

Below output please check.

{"error":{"root_cause":[{"type":"master_not_discovered_exception","reason":null}],"type":"master_not_discovered_exception","reason":null},"status":503}root@WAZUHAIO:~#

please suggest.

[Alfonso Ruiz-Bravo]

Hi Siddharth,

This error means that your Elasticsearch cluster has not started correctly, because it cannot find a master node to start and coordinate the cluster startup.

Have you restarted the Elasticsearch service or implemented any changes? Previously you had no problems communicating with the Elasticsearch API, something must have happened.

Regards,

Alfonso Ruiz-Bravo Cloud computing engineer The Open Source Security Platform

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/ce1f8c90-581f-4d0e-986d-5c1327d1fd60n%40googlegroups.com.

[siddharth jha]

Hi Alfonso,

I have not made any changes automatically showing this.

I have just checked the service and it shows this.

root@WAZUHAIO:~# systemctl status elasticsearch
● elasticsearch.service - Elasticsearch
     Loaded: loaded (/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2022-02-22 17:33:11 IST; 1 day 1h ago
       Docs: https://www.elastic.co
   Main PID: 11952 (java)
      Tasks: 134 (limit: 19125)
     Memory: 6.1G
     CGroup: /system.slice/elasticsearch.service
             └─11952 /usr/share/elasticsearch/jdk/bin/java -Xshare:auto -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch >

Feb 23 18:56:22 WAZUHAIO systemd-entrypoint[11952]:         at org.apache.logging.log4j.core.layout.TextEncoderHelper.writeAndEncodeAsMuchAsPossible(TextEncoderHelper.>
Feb 23 18:56:22 WAZUHAIO systemd-entrypoint[11952]:         at org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeChunkedText(TextEncoderHelper.java:147)
Feb 23 18:56:22 WAZUHAIO systemd-entrypoint[11952]:         at org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(TextEncoderHelper.java:58)
Feb 23 18:56:22 WAZUHAIO systemd-entrypoint[11952]:         at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:68)
Feb 23 18:56:22 WAZUHAIO systemd-entrypoint[11952]:         at org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:32)
Feb 23 18:56:22 WAZUHAIO systemd-entrypoint[11952]:         at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:220)
Feb 23 18:56:22 WAZUHAIO systemd-entrypoint[11952]:         at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58)
Feb 23 18:56:22 WAZUHAIO systemd-entrypoint[11952]:         at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStre>
Feb 23 18:56:22 WAZUHAIO systemd-entrypoint[11952]:         at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppend>
Feb 23 18:56:22 WAZUHAIO systemd-entrypoint[11952]:         at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.

please suggest.

[Alfonso Ruiz-Bravo]

Hi Siddharth,

Could you run the request again? You can also try the request that listed the health of the Elasticsearch cluster to check different requests. Remember that they are requests against the Elasticsearch API.

Regards,

Alfonso Ruiz-Bravo Cloud computing engineer The Open Source Security Platform

[siddharth jha]

Hi Alfonso,

i run this cmd curl -k -u <user> "https://<elastic_ip>:9200/_cat/indices/wazuh-alerts*?s=index"

below is output

{"error":{"root_cause":[{"type":"master_not_discovered_exception","reason":null}],"type":"master_not_discovered_exception","reason":null},"status":503}

checked elasticsearch service is showing active and running .

please suggest.

[Alfonso Ruiz-Bravo]

Hi Siddharth,

Here are some possible options, you can find more in the Elasticsearch forum to fix this new bug:

- https://discuss.elastic.co/t/im-getting-master-not-discovered-exception-on-remote-server/188508

- https://stackoverflow.com/questions/56790620/master-not-discovered-exception-when-trying-to-configure-elastic-search-on-rem

- https://discuss.elastic.co/t/getting-master-not-discovered-exception-from-node/269840/2

If you find a solution to this problem, maybe you can try searching for the error returned by the Request and checking your version of Elasticsearch.

When you get Elasticsearch up and running again, it will be interesting to see if the Wazuh alert indices continue to grow in number of documents.

Regards,

Alfonso Ruiz-Bravo Cloud computing engineer The Open Source Security Platform

[siddharth jha]

HI Alfonso,

I have rebooted the machine and checked that elasticsearch service is now active and running.

also run this cmd   curl -k -u <user> "https://<elastic_ip>:9200/_cat/indices/wazuh-alerts*?s=index"   and get output which file also i attached please check.

please suggest.

[Alfonso Ruiz-Bravo]

Hi Siddharth,

As we can see, the documents in its latest index have not increased. We can see in both files (this one and the previously attached) that you have the same number of documents. This means that Elasticsearch is not indexing, it is not receiving the alerts from Wazuh to store them in the indexes, since quite some time ago it seems. Therefore we can locate the problem in Wazuh or Filebeat.

wazuh-alerts-4.x-2021.11.29 -> 73002

Let's continue to narrow the circle, you can check if your Wazuh manager is generating alerts? for this, you can track the file /var/ossec/logs/alerts/alerts.json.

We are interested in knowing if it is generating new alerts (you can do a tail -f /var/ossec/logs/alerts/alerts.json to observe the evolution), that is to say if this file is increasing and if it is not increasing, it can also be interesting to see the date of the last alert generated.

Regards,

Alfonso Ruiz-Bravo Cloud computing engineer The Open Source Security Platform

[siddharth jha]

Hi Alfonso,

yes i have checked /var/ossec/logs/alerts/2022/Feb/ its showing alerts are stored there.

I have also checked this nano /var/ossec/logs/alerts/alerts.log and they are also showing alerts.

please suggest.

[Alfonso Ruiz-Bravo]

Hi Siddharth,

What is the date of the last alerts generated?

Regards

Alfonso Ruiz-Bravo Cloud computing engineer The Open Source Security Platform

[siddharth jha]

Hi Alfonso,

yes it's showing 23 and 24 feb.

[Alfonso Ruiz-Bravo]

Hi Siddharth,

Well, that means that Wazuh is working correctly, so we can check that the error has to be between Filebeat and Elasticsearch. If you check the logs of each service (usually located in the /var/log/ directory), can you see any errors?

Regards,

Alfonso Ruiz-Bravo Cloud computing engineer The Open Source Security Platform

[siddharth jha]

Hi Alfonso.,

can you please guide me? I have checked and found some logs, the same I have attached please check.

[Alfonso Ruiz-Bravo]

Hi Siddharth,

[2022-02-24T18:07:36,625][ERROR][c.a.o.s.a.s.InternalESSink] [node-1] Unable to index audit log {"audit_cluster_name":"wazuh-cluster","audit_transport_headers":{"_syst>
org.elasticsearch.common.ValidationException: Validation Failed: 1: this action would add [2] total shards, but this cluster currently has [1000]/[1000] maximum shards>

This is the problematic log. The Elasticsearch cluster has already 1000/1000 shards assigned to it, so no more indexes are created. Here you can act in two ways:

- Delete or close old indexes that you don't need to free up shards, as well as I advise you to manage your index replicas. If you have only one Elasticsearch node you do not need more replicas of the primary shard of each index. You can also reindex the indexes per week/month/year into a single index and gain shards without having to delete data.

- The second option is to increase the limit to 1000 shards, this is not always advisable because it can worsen the performance of the environment, but it is a possibility if you are running low on resources.

Regards,

Alfonso Ruiz-Bravo Cloud computing engineer The Open Source Security Platform

[siddharth jha]

Hi Alfonso,

Thanks again for your kind support.

Could you please help me to fix this as I am still trying to learn and I don't have the required knowledge.

can i follow this article where its mention Setting the number of shards and replica

https://documentation.wazuh.com/current/user-manual/elasticsearch/elastic-tuning.html

or please suggest.

[Alfonso Ruiz-Bravo]

Hi Siddharth,

The Elasticsearch cluster has reached its shard limit and cannot create new shards, so it has stopped indexing Wazuh alerts. In this case, I propose the following actions:

1. Change the number of replicas that are created by default. Having a single Elasticsearch node having more than 1 replica is superfluous, it is best to set it to 0.

https://documentation.wazuh.com/current/user-manual/elasticsearch/elastic-tuning.html#changing-the-number-of-replicas

This should free up a lot of the shards created. Now, there are more options to avoid this in the future.

2. Group indexes by time. For example, when a month or a week has passed, you can group the indices as follows:

https://www.elastic.co/guide/en/elasticsearch/reference/7.10/docs-reindex.html

The idea is, to add in a single index the content of several indexes, as we have mentioned, for example, to create an index wazuh-alerts-4.x-2022.02-week.1 and to add there the indexes corresponding to the first week. Once reindexed you can delete the original ones, thus freeing up shards.

3. Finally, you can increase the shard limit of the cluster, but do not abuse this practice as it may slow down the performance of your Elasticsearch cluster.

curl -k -u <user> -XPUT  "https://<Elastic_IP>:9200//_cluster/settings"  -H 'Content-Type: application/json' -d'
{
  "persistent": {
"cluster.max_shards_per_node": "<NEW_LIMIT>"
  }
}'

I hope you find this information helpful.

Best regards,

Alfonso Ruiz-Bravo Cloud computing engineer The Open Source Security Platform

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/CAOY1NSYku%2BocRWuTTMUnwBKa4sLJaHo1QrCRvSuo8bVOgwMZxw%40mail.gmail.com.

[siddharth jha]

Hi Alfonso,

Thanks much for your kind support.

As you suggested, I am following action no 1 .

but again facing some challenges.

please look on it once again,

root@WAZUHAIO:~# curl -X PUT "http://localhost:9200/wazuh-alerts-\*/_settings?pretty" -H 'Content-Type: application/json' -d'

> {

>   "settings" : {

>     "number_of_replicas" : 0

>   }

> }'

curl: (52) Empty reply from server

getting this output curl: (52) Empty reply from server.

After that try curl -X PUT -u username:password -k "http://localhost:9200/wazuh-alerts-\*/_settings?pretty" -H 'Content-Type: application/json' -d'

but getting the same output.

please suggest.

[Alfonso Ruiz-Bravo]

Hi Siddharth,

Try with:

curl -k -u <USER>-X PUT "https://<ELASTIC_IP>:9200/wazuh-alerts-*/_settings?pretty" -H 'Content-Type: application/json' -d'
{
  "settings" : {
    "number_of_replicas" : 0
  }
}'

Regards,

Alfonso Ruiz-Bravo Cloud computing engineer The Open Source Security Platform

[siddharth jha]

Hi Alfonso,

Thanks for your support.

As you suggested I have run that cmd and that output is acknowledged as true.

and now i have checked that logs are still showing the same error message .

 

 nano /var/log/elasticsearch/wazuh-cluster_server.json

{"type": "server", "timestamp": "2022-03-03T19:42:12,082+05:30", "level": "ERROR", "component": "c.a.o.s.a.s.InternalESSink", "cluster.name": "wazuh-cluster", "node.na>

"stacktrace": ["org.elasticsearch.common.ValidationException: Validation Failed: 1: this action would add [2] total shards, but this cluster currently has [1000]/[1000>

"at org.elasticsearch.indices.ShardLimitValidator.validateShardLimit(ShardLimitValidator.java:80) ~[elasticsearch-7.10.2.jar:7.10.2]",

"at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.aggregateIndexSettings(MetadataCreateIndexService.java:765) ~[elasticsearch-7.10.2.jar:7.10.2]",

"at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexRequestWithV1Templates(MetadataCreateIndexService.java:489) ~[elasticsearch-7.10.2.ja>

"at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexRequest(MetadataCreateIndexService.java:370) ~[elasticsearch-7.10.2.jar:7.10.2]",

"at org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexRequest(MetadataCreateIndexService.java:377) ~[elasticsearch-7.10.2.jar:7.10.2]",

"at org.elasticsearch.action.admin.indices.create.AutoCreateAction$TransportAction$1.execute(AutoCreateAction.java:137) ~[elasticsearch-7.10.2.jar:7.10.2]",

"at org.elasticsearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:47) ~[elasticsearch-7.10.2.jar:7.10.2]",

"at org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:702) ~[elasticsearch-7.10.2.jar:7.10.2]",

"at org.elasticsearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:324) ~[elasticsearch-7.10.2.jar:7.10.2]",

"at org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:219) ~[elasticsearch-7.10.2.jar:7.10.2]",

"at org.elasticsearch.cluster.service.MasterService.access$000(MasterService.java:73) ~[elasticsearch-7.10.2.jar:7.10.2]",

"at org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:151) ~[elasticsearch-7.10.2.jar:7.10.2]",

"at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150) ~[elasticsearch-7.10.2.jar:7.10.2]",

"at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188) ~[elasticsearch-7.10.2.jar:7.10.2]",

"at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:684) ~[elasticsearch-7.10.2.jar:7.10.2]",

"at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:252) ~[ela>

"at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:215) ~[elasticsear>

"at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]",

"at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]",

"at java.lang.Thread.run(Thread.java:832) [?:?]"] }

curl -k -u siddharth "https://localhost:9200/_cluster/health?pretty"

{

  "cluster_name" : "wazuh-cluster",

  "status" : "yellow",

  "timed_out" : false,

  "number_of_nodes" : 1,

  "number_of_data_nodes" : 1,

  "active_primary_shards" : 864,

  "active_shards" : 864,

  "relocating_shards" : 0,

  "initializing_shards" : 0,

  "unassigned_shards" : 136,

  "delayed_unassigned_shards" : 0,

  "number_of_pending_tasks" : 0,

  "number_of_in_flight_fetch" : 0,

  "task_max_waiting_in_queue_millis" : 0,

  "active_shards_percent_as_number" : 86.4

}

please suggest.

[Alfonso Ruiz-Bravo]

Hi Siddharth,

Yes, this change is for future indices. You now should choose between  the option 2 or 3.

Regards,

Alfonso Ruiz-Bravo Cloud computing engineer The Open Source Security Platform

[siddharth jha]

Hello Alfonso,

Thanks again for your support.

Now I am trying to increase shards as you suggested in option 3.

and getting this error, could you please check and suggest.

{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index []","resource.type":"index_or_alias","resource.id":"","index_uuid":"_na_","index":""}],"type":"index_not_found_exception","reason":"no such index []","resource.type":"index_or_alias","resource.id":"","index_uuid":"_na_","index":""},"status":404}root@WAZUHAIO:~#

[Alfonso Ruiz-Bravo]

Hello Siddharth,

I understand that you have used the following request:

curl -k -u <user> -XPUT  "https://<Elastic_IP>:9200//_cluster/settings"  -H 'Content-Type: application/json' -d'
{
  "persistent": {
"cluster.max_shards_per_node": "<NEW_LIMIT>"
  }
}'

Could you check if the command you have executed is well-formed? because the error output is that it is looking for a specific index and it does not find it, when in this request what we are doing is to modify a setting of the Elasticsearch cluster configuration.

If you could re-run the command and provide us with everything, both the command and the output, we could better study the error.

Regards,

Alfonso Ruiz-Bravo Cloud computing engineer The Open Source Security Platform

[siddha...@gmail.com]

Hello Alfonso,

Thanks again for your kind support.

please check and suggest.

root@WAZUHAIO:~# curl -k -u siddharth -XPUT  "https://localhost:9200//_cluster/settings"  -H 'Content-Type: application/json' -d'
> {
>   "persistent": {
> "cluster.max_shards_per_node": "1500"
>   }
> }'
Enter host password for user 'siddharth':
{"error":{"root_cause":[{"type":"cluster_block_exception","reason":"blocked by: [SERVICE_UNAVAILABLE/2/no master];","suppressed":[{"type":"index_not_found_exception","reason":"no such index []","resource.type":"index_or_alias","resource.id":"","index_uuid":"_na_","index":""}]}],"type":"cluster_block_exception","reason":"blocked by: [SERVICE_UNAVAILABLE/2/no master];","suppressed":[{"type":"index_not_found_exception","reason":"no such index []","resource.type":"index_or_alias","resource.id":"","index_uuid":"_na_","index":""}]},"status":503}root@WAZUHAIO:~#

[Alfonso Ruiz-Bravo]

Hello Siddharth,

There seems to be a problem with your Elasticsearch cluster.

Error -> blocked by: [SERVICE_UNAVAILABLE/2/no master]

Maybe the cluster is not running?

Could you have run out of disk space?

Regards,

Alfonso Ruiz-Bravo Cloud computing engineer The Open Source Security Platform

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/d4147f2f-cbf2-4da4-87a4-fe49852db63an%40googlegroups.com.

[siddha...@gmail.com]

Hi,

yes sometimes all disk space is consumed.

i have delete some files from alerts.log folder now disk space is available and run that cmd again.

root@WAZUHAIO:~# systemctl status elasticsearch
● elasticsearch.service - Elasticsearch
     Loaded: loaded (/lib/systemd/system/elasticsearch.service; enabled; vendor>

     Active: active (running) since Wed 2022-03-09 11:53:01 IST; 9min ago
       Docs: https://www.elastic.co
   Main PID: 1136 (java)
      Tasks: 174 (limit: 19125)
     Memory: 9.2G
     CGroup: /system.slice/elasticsearch.service
             └─1136 /usr/share/elasticsearch/jdk/bin/java -Xshare:auto -Des.net>

Mar 09 11:51:55 WAZUHAIO systemd[1]: Starting Elasticsearch...
Mar 09 11:53:01 WAZUHAIO systemd[1]: Started Elasticsearch.

root@WAZUHAIO:~# curl -k -u siddharth -XPUT  "https://localhost:9200//_cluster/s                                                                                        ettings"  -H 'Content-Type: application/json' -d'
> {
>   "persistent": {
> "cluster.max_shards_per_node": "3000"

>   }
> }'
Enter host password for user 'siddharth':

{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such in                                                                                        dex []","resource.type":"index_or_alias","resource.id":"","index_uuid":"_na_","i                                                                                        ndex":""}],"type":"index_not_found_exception","reason":"no such index []","resou                                                                                        rce.type":"index_or_alias","resource.id":"","index_uuid":"_na_","index":""},"sta

please suggest. 

[Alfonso Ruiz-Bravo]

Hello Siddharth,

There is a typo behind the port number, there is one character leftover /

Try:

curl -k -u <user> -XPUT  "https://<Elastic_IP>:9200/_cluster/settings"  -H 'Content-Type: application/json' -d'
{
  "persistent": {
"cluster.max_shards_per_node": "<NEW_LIMIT>"
  }
}'

Regards,

Alfonso Ruiz-Bravo Cloud computing engineer The Open Source Security Platform

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/33df38e2-e403-4949-b63a-fff38c78ea91n%40googlegroups.com.