PCI requirements: settings to view logs online for 3 months

[Odie]

Hi everyone!

How to configure logging to satisfies PCI DSS log requirements? It should be searchable/viewable in kibana apps.

Log retention: logs must be stored for one year with the last three months available in an easily accessible storage

Thanks!

[Borja Arroba]

Hi Odie.

In kibana you can access the elastic indexed logs from 5 years ago.

The logs are not automatically deleted by default. As for the rotation of logs depends on the version of wazuh you have installed, for the latest version, you can see this link to see how to configure it:

https://documentation.wazuh.com/3.x/user-manual/reference/internal-options.html?highlight=log%20rotation#monitord

Depending on your needs you can rotate it by time or size, but this does not affect the display of alerts from kibana.

Hope it help.
Regards.
Borja Arroba.

[Odie]

Thanks!

so how can I configure the log rotation? let say I need to view logs in Kibana dashboard for 90 days, but want to archived logs for 1 year

Thanks!

[Borja Arroba]

Hi, Odie.

To rotate elastic alerts and view them in kibana you can use the "Elasticsearch Curator". This tool allows you many possibilities I propose two different ones:

1. You can simply store the 90 days of indexes you need and delete the oldest ones.
2. Or, you can do snapshots every day. When the 90th day comes, remove the old ones, so you can restore an older snapshot if necessary.

As for the logs stored in the mananger are managed independently from the Elastic indexes, to maintain 365 days you must configure the option "monitord.keep_log_days=365". In the folder"/var/ossec/logs/2018" you will have all the logs for that year, later:

1. You can erase the oldest ones.
2. Create a zip file and move it wherever you want.

Best regards.
Borja Arroba.

[Odie]

Hi Borja,

Thanks! Can you please give me example how to do this using curator. 

Appreciate your help.. thanks!

[alberto....@wazuh.com]

Hello Odie

  Please, take a look into https://www.elastic.co/guide/en/elasticsearch/client/curator/5.1/index.html, happily, Elasticsearch provides a great documentation in all their components :)

So, you'll need a curator.yml, something like this:

[root@wazuh-server .curator]# cat curator.yml
# Remember, leave a key empty if there is no value.  None will be a string,
# not a Python "NoneType"
client:
  hosts:
    - localhost
  port: 9200
  url_prefix:
  use_ssl: False
  certificate:
  client_cert:
  client_key:
  ssl_no_validate: False
  http_auth:
  timeout: 30
  master_only: False


logging:
  loglevel: INFO
  logfile: /var/log/curator/curator.log
  logformat: json
  blacklist: ['elasticsearch', 'urllib3']

replacing your values if necessary like the host, port, certificates, http_auth, etc. After that, I recommend you define an action yml file for deletions, like this one: 

[root@wazuh-server .curator]# cat delete.yml
actions:
  1:
    action: 'delete_indices'
    description: |
      Delete all indices whose name starts with 'wazuh-alerts-3.x-' and that have
      been created more than 30 days ago.
    options:
      continue_if_exception: False
      disable_action: False
      ignore_empty_list: True
    filters:
    - filtertype: 'pattern'
      kind: 'prefix'
      value: 'wazuh-alerts-3.x-'
    - filtertype: 'age'
      source: 'name'
      timestring: '%Y.%m.%d'
      direction: 'older'
      unit: 'days'
      unit_count: 365

So, this action file will define a rule in order to remove all indices with pattern wazuh-alerts-3.x- and older than 365. If we want to execute it:

curator delete.yml --dry-run

You will need to have a configuration file (described in the top of this message) in ~/.curator folder. And the --dry-run allow you to execute the command without no actions, only for review the possible results that you'll be able to see in /var/log/curator/curator.log. If you are sure about the execution of this command and you want to definitively remove the older than 365 indices, then: 

curator delete.yml

By removing --dry-run we skip the "simulation mode". 

The last step that you'll need is automatizing the process every day/month. We could use cron e.g:

crontab -e

and then add the line:

0 6 1 * * root curator /path/delete_index.yml --config /path/curator.yml

This cron will execute the removing task every day 1 of every month at 06:00 AM. 

 Hope it help.

Best regards, 

Alberto R.