deleting indices for a period

[nik]

Hello Team,

I read the documentation and could not find instructions how to delete an old information (indices, documents) from the wazuh manager.

How to do it correctly?  For example for a specified period ?

Best regards,

Nik

[jesus.g...@wazuh.com]

Hi Nik,

This question was answered at https://groups.google.com/d/topic/wazuh/M5AfXmX1TTI/discussion but I'm going to copy/paste because I think

you are in the same issue. If you are not referring to Elasticsearch, let us know.

Don't know if the next solution matches your goal but since Wazuh generates daily indices you 

can delete specific index for specific days. 

How to see your indices list:

# curl elastic_ip:9200/_cat/indices/wazuh-alerts*

// Example output
yellow open wazuh-alerts-3.x-2018.07.11 HMRNDMv_TEW7_bkXu5gWMA 5 1 139908 0  20.5mb  20.5mb
yellow open wazuh-alerts-3.x-2018.07.12 f6yjp6AjRkKoRIAkk63NAQ 5 1 938926 0 482.1mb 482.1mb

As you can see, a Wazuh index name is composed using the next pattern:

wazuh-alerts-3.x-YYYY.MM.DD

This means you can delete specific days as I said. 

How to delete specific day:

curl -XDELETE elastic_ip:9200/wazuh-alerts-3.x-2018.07.12

How to delete two specific days:

curl -XDELETE elastic_ip:9200/wazuh-alerts-3.x-2018.07.11,wazuh-alerts-3.x-2018.07.11

How to delete all July:

curl -XDELETE elastic_ip:9200/wazuh-alerts-3.x-2018.07.11,wazuh-alerts-3.x-2018.07*

Note: since you can use wildcard (*), delete a mont is pretty easy as you can see in the above curl command.

If the above commands don't match your desired goal because you want to delete specific documents from specific day, let us know

and we can drive you in the right way.

Regards,

Jesús

[nik]

All clear, Jesus, thanks a lot.

One more question.

If I directly delete some folders from /var/ossec/logs/alerts/* and from /var/ossec/logs/archives/* ?

For example, /var/ossec/logs/archives/2018/Apr  and /var/ossec/logs/logs/2018/Apr ?

Whether it will be right?

Nik

[jesus.g...@wazuh.com]

Hello again Nik,

It doesn't matter if you delete some archives. The only files to be deleted will be files under:

/var/ossec/logs/archives/<YEAR>/*

Other files under /var/ossec/logs don't need to be removed and they are used by the Wazuh manager.

Before remove archives files, please stop your Wazuh manager:

systemctl stop wazuh-manager

Once you are done, restart it:

systemctl restart wazuh-manager

I hope it helps!

Regards,

Jesús

[nik]

Hello Jesus,

as far as I uderstand there are two variants of deleting of unusable indices:

 - curl -XDELETE elastic_ip:9200/.....

and

 - /var/ossec/logs/archives/<YEAR>/*

I didn't understand about the files under /var/ossec/logs... 

Ok, these files are not related to deleting indices in elastic.

But they can not assemble indefinitely.

Do these files have to be deleted sometime? I mean old log files.

Nik

[jesus.g...@wazuh.com]

Hello Nik,

as far as I uderstand there are two variants of deleting of unusable indices:
 - curl -XDELETE elastic_ip:9200/.....
and
 - /var/ossec/logs/archives/<YEAR>/*

That's not true, our Wazuh manager generates alerts, those alerts are stored in /var/ossec/logs/alerts/alerts.json file.

If that file grows in size, the Wazuh manager rotates the file and it creates a backup under /var/ossec/logs/archives/<YEAR>, once rotated, it clears the 

file /var/ossec/logs/alerts/alerts.json.

On the other hand, Logstash/Filebeat reads the file /var/ossec/logs/alerts/alerts.json and send those alerts to Elasticsearch. Alerts in Elasticsearch are the data used by Kibana

to show you the visualizations. You can free disk usage removing /var/ossec/logs/archives/<YEAR>/* or old indices in Elasticsearch. If you don't need them anymore it's safe to remove them in both

places. Keep in mind that removing /var/ossec/logs/archives/<YEAR>/* don't implies they are removed from Elasticsearch.

I hope it's now clearer for you.

Regards,

Jesús

[nik]

Jesus,

Not all clear for me with /var/ossec/logs/alerts/<YEAR>/* folders.

As you said after rotation of "/var/ossec/logs/alerts/alerts.json" file the alerts move into /var/ossec/logs/archives/<YEAR>/*.

OK.

And when the /var/ossec/logs/alerts/<YEAR>/* folder is cleared ? Never?

And the second question.

Except wazuh-alerts* I found wazuh-monitoring* indices.

Should I delete them like wazuh-alerts* ?

Regards,

Nik

[jesus.g...@wazuh.com]

Hello again Nik,

And when the /var/ossec/logs/alerts/<YEAR>/* folder is cleared ? Never?

/var/ossec/logs/alerts/<YEAR>/* is a backup, Wazuh manager doesn't remove it never, you must delete it manually if you want to free disk usage.

Except wazuh-alerts* I found wazuh-monitoring* indices.
Should I delete them like wazuh-alerts* ?

wazuh-monitoring-* indices are generated by the Wazuh App, there are no files out of Elasticsearch with that information. These indices store periodically you agents statuses.

If you don't need old indices you can delete them in the same way as wazuh-alerts indices.

Regards,

Jesús

[nik]

Jesus,

I asked to many questions today.. and sorry for such quantity... Now all is clear.

Many thanks for your help and a good job.

Regards,

Nik

[jesus.g...@wazuh.com]

We are here to help!

Regards,

Jesús

[Robert H]

Hi, here's some information, if it applies.

Related to data retention in Elastic, I modified the example from the Elastic Curator page and this is working during my testing with it.  Note:  you need to install Curator and create both a config file and a actions file.  A crontab runs once a day.  This removes the index and all primary and replica shards.  

**Note: There might have been a bug in Elastic version 6.3.0.  The same code run on that version reported that it deleted the indices successfully, but the monitoring indices shards were left unassigned.  But after I upgraded the stack to 6.3.2, the same code is working correctly.

**Note:  I have 3 master nodes.  To make this run only on the currently elected master, there is a little more config required.

0 3 * * *       curator --config /etc/curator/config.yml /etc/curator/alert_actions.yml

////////////////////////

# cat /etc/curator/alert_actions.yml

---

actions:

  1:

    action: delete_indices

    description: >-

      Delete indices older than 30 days (based on index name), for logstash-

      prefixed indices. Ignore the error if the filter does not result in an

      actionable list of indices (ignore_empty_list) and exit cleanly.

    options:

      ignore_empty_list: True

      disable_action: False

    filters:

    - filtertype: pattern

      kind: prefix

      value: wazuh-alerts-3.x-*

    - filtertype: age

      source: name

      direction: older

      timestring: '%Y.%m.%d'

      unit: days

      unit_count: 30

  2:

    action: delete_indices

    description: >-

      Delete indices older than 30 days (based on index name), for logstash-

      prefixed indices. Ignore the error if the filter does not result in an

      actionable list of indices (ignore_empty_list) and exit cleanly.

    options:

      ignore_empty_list: True

      disable_action: False

    filters:

    - filtertype: pattern

      kind: prefix

      value: wazuh-monitoring-3.x-*

    - filtertype: age

      source: name

      direction: older

      timestring: '%Y.%m.%d'

      unit: days

      unit_count: 30

Regards,

Robert

[jesus.g...@wazuh.com]

Hi Robert, 

Thanks for your point!

Regards,

Jesús