Wazuh index-management

[Nuno Campos]

Hi all, 

I am new in Wazuh and trying to configure OpenDistro for Elasticsearch to automatically manage the data in those indices over time.

I'm following the configuration steps OpenDistro ISM:

https://wazuh.com/blog/wazuh-index-management/

The snippets below:

1- The policy created -> wazuh-alert-retention-policy

{

    "policy_id": "wazuh-alert-retention-policy",

    "description": "Wazuh index state management for OpenDistro to delete after a 30d.",

    "last_updated_time": 1636046836506,

    "schema_version": 1,

    "error_notification": null,

    "default_state": "hot",

    "states": [

        {

            "name": "hot",

            "actions": [

                {

                    "replica_count": {

                        "number_of_replicas": 1

                    }

                }

            ],

            "transitions": [

                {

                    "state_name": "delete",

                    "conditions": {

                        "min_index_age": "30d"

                    }

                }

            ]

        },

        {

            "name": "delete",

            "actions": [

                {

                    "delete": {}

                }

            ],

            "transitions": []

        }

    ],

    "ism_template": null

}

2- Run the following commands: 

sed -i 's/  "settings": {/  "settings": {\n    "opendistro.index_state_management.policy_id": "wazuh-alert-retention-policy",/g' /etc/filebeat/wazuh-template.json

filebeat setup --index-management

Output:

ILM policy and write alias loading not enabled.

Index setup finished.

When I check the Managed Indices the wazuh-alerts-4.x is not assigned to the policy.

Anyone can help? 

Thanks!

NC

[Alfonso Ruiz-Bravo]

Hello NC!!

I think the problem is because your policy wazuh-alert-retention-policy has no associated ism_template, that is, it will not apply to any index because that field is empty:

   "ism_template": null

Here is an example: https://opendistro.github.io/for-elasticsearch-docs/docs/im/ism/policies/#sample-policy-with-ism-template. 

If you want to add this policy to all your alert indexes, you should perform the following steps:

1. Get policy seq_no. (In the steps I add the use of jq to speed up the process.)

curl -s -k -u admin /usr/share/elasticsearch/config/admin-key.pem https://odfe:9200/_opendistro/_ism/policies/wazuh-alert-retention-policy | jq '._seq_no'

2. Get policy primary_term.

curl -s -k -u admin https://odfe:9200/_opendistro/_ism/policies/ wazuh-alert-retention-policy   | jq '._primary_term'

3. Add ism template to your policy.

curl -s -k -u admin PUT "https://odfe:9200/_opendistro/_ism/policies/ wazuh-alert-retention-policy  ?if_seq_no=YOUR_SEQ_NO&if_primary_term=YOUR_PRIMARY_TERM" -H 'Content-Type: application/json' -d'

{

    "policy_id": "wazuh-alert-retention-policy",

    "description": "Wazuh index state management for OpenDistro to delete after a 30d.",

    "last_updated_time": 1636046836506,

    "schema_version": 1,

    "error_notification": null,

    "default_state": "hot",

    "states": [

        {

            "name": "hot",

            "actions": [

                {

                    "replica_count": {

                        "number_of_replicas": 1

                    }

                }

            ],

            "transitions": [

                {

                    "state_name": "delete",

                    "conditions": {

                        "min_index_age": "30d"

                    }

                }

            ]

        },

        {

            "name": "delete",

            "actions": [

                {

                    "delete": {}

                }

            ],

            "transitions": []

        }

    ],

    "ism_template": {

       "index_patterns": ["wazuh-alerts-4.x*"],

       "priority": "SET_YOUR_DESIREED_PRIORITY"

    }

}

 '

I hope this information is helpful to you. 

Best regards,

Alfonso Ruiz-Bravo

[Alfonso Ruiz-Bravo]

Curls with blanks, I attach them fixed.

1. Get policy seq_no. (In the steps I add the use of jq to speed up the process.)

curl -s -k -u admin  https://odfe:9200/_opendistro/_ism/policies/wazuh-alert-retention-policy | jq '._seq_no'

2. Get policy primary_term.

curl -s -k -u admin https://odfe:9200/_opendistro/_ism/policies/wazuh-alert-retention-policy   | jq '._primary_term'

3. Add ism template to your policy.

curl -s -k -u admin PUT "https://odfe:9200/_opendistro/_ism/policies/wazuh-alert-retention-policy  ?if_seq_no=YOUR_SEQ_NO&if_primary_term=YOUR_PRIMARY_TERM" -H 'Content-Type: application/json' -d'

Regards,

Alfonso Ruiz-Bravo

[Nuno Campos]

Hello Alfonso,

the information you share was very helpful.

Step1-2. Get policy seq_no and policy primary_term I needed to change:

curl -s -k -u admin  https://odfe:9200 -> https://localhost:9200

Output: 1

Step3. Add ism template to your policy.

curl -s -k -u admin PUT “https://localhost:9200/_opendistro/_ism/policies/wazuh-alert-retention-policy?if_seq_no=1&if_primary_term=1” -H 'Content-Type: application/json' -d'

Output:

>        "priority": 0

>     }

> }

>  '

[1] 3500489

-H: command not found

[1]+  Stopped                 curl -s -k -u admin PUT “https://localhost:9200/_opendistro/_ism/policies/wazuh-alert-retention-policy ?if_seq_no=1

note: I don't know what value should I put for priority, the defaults 0?

Thanks!

NC

[Alfonso Ruiz-Bravo]

Hello NC!

Try with:

curl -s -k -u admin PUT "https://localhost:9200/_opendistro/_ism/policies/wazuh-alert-retention-policy?if_seq_no=1&if_primary_term=1"  -H  'Content-Type: application/json' -d '

       "priority": "50"
    }
}
'

In your case it has failed because you have not taken the -H flag of the curl, which is used to indicate the format of the body header (-d flag). You still have to obtain the values seq_no and primary_term again.

We usually use a standard priority of 50.

Regards!

Alfonso Ruiz-Bravo Cloud computing engineer The Open Source Security Platform

--
You received this message because you are subscribed to a topic in the Google Groups "Wazuh mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/wazuh/MqL_r1t7XP8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/3ad70e56-cd0f-43f2-a9ac-46342af0c14bn%40googlegroups.com.

[Nuno Campos]

Hi Alfonso,

I'm trying to obtain the values seq_no and primary_term again with:
curl -s -k -u admin https://localhost:9200/_opendistro/_ism/policies/wazuh-alert-retention-policy | jq '._seq_no'

Enter host password for user 'admin':

1

curl -s -k -u admin https://localhost:9200/_opendistro/_ism/policies/wazuh-alert-retention-policy | jq '._primary_term'

Enter host password for user 'admin':

1

The values are the same -> 1 it's correct?

Thanks.

Nuno Campos

[Alfonso Ruiz-Bravo]

Hi Nuno!!

Yes, it is possible that they have the same value.

Best regards, 

Alfonso Ruiz-Bravo Cloud computing engineer The Open Source Security Platform

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/63a3cfb7-4436-42b6-bb13-8440661ef36fn%40googlegroups.com.

[Nuno Campos]

Hi Alfonso,

please check the snippets below:

Enter host password for user 'admin':

{"error":"Incorrect HTTP method for uri [/_opendistro/_ism/policies/wazuh-alert-retention-policy?if_seq_no=1&if_primary_term=1] and method [POST], allowed: [PUT, HEAD, DELETE, GET]","status":405}

Thanks.

Nuno Campos

[Alfonso Ruiz-Bravo]

Hi Nuno!

My fault,

miss -X flag for PUT

curl -s -k -u admin -X PUT "https://localhost:9200/_opendistro/_ism/policies/wazuh-alert-retention-policy?if_seq_no=1&if_primary_term=1"  -H  'Content-Type: application/json' -d '

Best regards,

Alfonso Ruiz-Bravo Cloud computing engineer The Open Source Security Platform

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/ee8268ed-98cd-45c9-bb47-b8b946439588n%40googlegroups.com.

[Nuno Campos]

Hi Alfonso,

Output:

Enter host password for user 'admin':

{"error":{"root_cause":[{"type":"parsing_exception","reason":"Failed to parse object: expecting token of type [START_OBJECT] but found [VALUE_STRING]","line":3,"col":18}],"type":"parsing_exception","reason":"Failed to parse object: expecting token of type [START_OBJECT] but found [VALUE_STRING]","line":3,"col":18},"status":400}

Thank you for your attention!

Nuno Campos

[Alfonso Ruiz-Bravo]

Hi Nuno!

try it without the following line

"last_updated_time": 1636046836506,

Best regards,

Alfonso Ruiz-Bravo Cloud computing engineer The Open Source Security Platform

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/11486c47-b14d-4fcc-8acb-ee9147bb0847n%40googlegroups.com.

[Nuno Campos]

Hello Alfonso,

please check the snippets below:

curl -s -k -u admin -X PUT "https://localhost:9200/_opendistro/_ism/policies/wazuh-alert-retention-policy?if_seq_no=1&if_primary_term=1"  -H  'Content-Type: application/json' -d '

> {

>     "policy_id": "wazuh-alert-retention-policy",

>     "description": "Wazuh index state management for OpenDistro to delete after a 30d.",

>     "schema_version": 1,

Question(?)- It's possible to add the "ism_template"  at Kibana UI (Screenshot attached)

"ism_template": {

       "index_patterns": ["wazuh-alerts-4.x*"],

       "priority": "50"

    }

Thanks.

Nuno Campos

[Alfonso Ruiz-Bravo]

Hi Nuno!!

Yes, it is possible to add it from the UI and it may be a better option.  Try it if it works for you.

Best regards,

Alfonso Ruiz-Bravo Cloud computing engineer The Open Source Security Platform

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/a26ebb32-d4ed-463b-a726-c36cf5f8c737n%40googlegroups.com.