New indices and problems with logstash

[Felipe Andres Concha Sepúlveda]

Hello,

Someone in our team created new indexes in elasticsearch and we started to see an error in the logstash and since that day we no longer see alerts in kibana.

Will they have any idea of the problem?

Eliminating the indices could be solved?

Or do you have some record of a similar problem reported?

The error in logstash is as follows

[2018-11-22T17:13:52,271][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({"type"=>"cluster_block_exception", "reason"=>"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"})

[2018-11-22T17:13:52,271][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({"type"=>"cluster_block_exception", "reason"=>"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"})

[2018-11-22T17:13:52,271][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({"type"=>"cluster_block_exception", "reason"=>"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"})

[2018-11-22T17:13:52,271][INFO ][logstash.outputs.elasticsearch] Retrying individual bulk actions that failed or were rejected by the previous bulk request. {:count=>125}

New indices in elasticsearch

[elw...@wazuh.com]

Hello Felipe,

Elasticsearch has changed your indices settings to read_only. This implies you can't index new data in them.

This is related to low storage in your node. You should either increase your storage capacity or delete old indices to prevent this from happening again.

For the time being, you can disable the read_only setting with the following command in Kibana dev tools:

PUT */_settings
{
 "index.blocks.read_only_allow_delete": null
}

Make sure to restart kibana:

systemctl restart kibana

Hope this will help.
Best regards,

[Felipe Andres Concha Sepúlveda]

Thanks for your help!!!

I have executed the script that you sent me and I have received some alerts in kibana, but only until yesterday, today I can not see them, today's alert was red (see photo 3), I think I still have the problem, log logstash (See photo 1)

Let's try changing the data that saves elasticsearch to a larger disk, to do this, it is only necessary to change the address in each node of elasticsearch (see photo 2)?

Or do we have to make a change somewhere else so that kibana can read the info?

FOTO1

Foto 2

FOTO3

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To post to this group, send email to wa...@googlegroups.com.
Visit this group at https://groups.google.com/group/wazuh.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/3908c70c-c332-4740-9042-7bd2602c27fb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[elw...@wazuh.com]

Hello Felipe,

Indeed, you have the possibility to change the path or add multiple paths as mentioned HERE.

Make sure to stop elasticsearch before making this change :

systemctl stop elasticsearch

Then edit the path.data to the one you want, But make sure that this path has the elasticsearch user and permissions as shown below :

drwxr-x---. 3 elasticsearch elasticsearch 4096 Nov 15 19:53 /var/lib/elasticsearch/

Then start elasticSearch again.

systemctl start elasticsearch

No changes are required to perform in kibana.

Hope this will help.
Best regards,

Wali.k

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+unsubscribe@googlegroups.com.

[Felipe Andres Concha Sepúlveda]

Thank you very much!!!!!  :)

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.

To post to this group, send email to wa...@googlegroups.com.
Visit this group at https://groups.google.com/group/wazuh.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/3908c70c-c332-4740-9042-7bd2602c27fb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.

To post to this group, send email to wa...@googlegroups.com.
Visit this group at https://groups.google.com/group/wazuh.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/6dfbf6c3-4b15-401c-ae37-b8e42b4696f1%40googlegroups.com.

[Felipe Andres Concha Sepúlveda]

Thank you very much I made the change and it works correctly !!!

regards

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.

To post to this group, send email to wa...@googlegroups.com.
Visit this group at https://groups.google.com/group/wazuh.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/6dfbf6c3-4b15-401c-ae37-b8e42b4696f1%40googlegroups.com.

[elw...@wazuh.com]

You're welcome, we are always glad to help.

regards,