Per Bucket monitor Error.

89 views
Skip to first unread message

никита какдела

unread,
Jan 20, 2026, 6:16:31 AMJan 20
to Wazuh | Mailing List
Hello! Do somebody know what does mean this error? (Per Bucket Monitor)
Error: Failed fetching inputs: GeneralScriptException[Failed to compile inline script [{"size":0,"query":{"bool":{"filter":[{"range":{"@timestamp":{"from":"{{period_end}}||-300s","to":"{{period_end}}","include_lower":true,"include_upper":true,"format":"epoch_millis","boost":1.0}}},{"term":{"rule.id":{"value":"100014","boost":1.0}}}],"must_not":[{"terms":{"data.win.eventdata.targetUserName":["ANONYMOUS LOGON","АНОНИМНЫЙ ВХОД"],"boost":1.0}}],"adjust_pure_negative":true,"boost":1.0}},"aggregations":{"composite_agg":{"composite":{"size":10,"sources":[{"data.win.eventdata.targetUserName":{"terms":{"field":"data.win.eventdata.targetUserName","missing_bucket":false,"order":"asc"}}}],"after":{"data.win.eventdata.targetUserName":"atitova"}},"aggregations":{"unique_ips":{"cardinality":{"field":"data.win.eventdata.ipAddress"}}}},"xGdjA5sBoeamHjFYf3Hq":{"bucket_selector_ext":{"buckets_path":{"uniq":"unique_ips.value"},"parent_bucket_path":"composite_agg","script":{"source":"params.uniq > 4","lang":"painless"},"gap_policy":"skip"}}}}] using lang [mustache]]; nested: CircuitBreakingException[[script] Too many dynamic script compilations within, max: [300/5m]; please use indexed, or scripts with parameters instead; this limit can be changed by the [script.context.template.max_compilations_rate] setting];; CircuitBreakingException[[script] Too many dynamic script compilations within, max: [300/5m]; please use indexed, or scripts with parameters instead; this limit can be changed by the [script.context.template.max_compilations_rate] setting]

musbau....@wazuh.com

unread,
Jan 20, 2026, 6:58:49 AMJan 20
to Wazuh | Mailing List
Hi,

The error indicates that there's an issue script inside your bucket selector aggregation. The error message shows your script got cut off. It ends with "source":"params.uniq but is missing the actual condition.

In a bucket selector, the script needs to be a complete expression that returns true or false. For example, if you're trying to filter buckets where the unique IP count is greater than 1, your script should look like:

json
"script": {
  "source": "params.uniq > 1"
}

It looks like the script source is incomplete or was truncated. Please double-check the full script in your monitor configuration and make sure it's a valid boolean expression.

You can reference the documentation and issues below

https://docs.opensearch.org/latest/observing-your-data/alerting/monitors/
https://github.com/opensearch-project/alerting/issues/915
https://github.com/opensearch-project/alerting/issues/706

никита какдела

unread,
Jan 20, 2026, 7:07:42 AMJan 20
to Wazuh | Mailing List
I have no idea why i am getting this error. I have ~30 Per Bucket monitors.
вторник, 20 января 2026 г. в 14:16:31 UTC+3, никита какдела:

никита какдела

unread,
Jan 20, 2026, 7:07:43 AMJan 20
to Wazuh | Mailing List
What happens if i raise the limit to 1000/5m?


вторник, 20 января 2026 г. в 14:16:31 UTC+3, никита какдела:
Hello! Do somebody know what does mean this error? (Per Bucket Monitor)

musbau....@wazuh.com

unread,
Jan 20, 2026, 7:58:47 AMJan 20
to Wazuh | Mailing List
Hi,

Increasing the limit to 1000/5m might help temporarily by giving the monitor more resources, but it may not fix the root cause if it's the script truncation bug.

Since you have many monitors, I would suggest you

1. First check if the scripts in your failing monitors look complete or cut off in the dashboard

2. Try simplifying one script to something very short like params.uniq > 0 as a test

The GitHub issues show this is a known bug where scripts get cut off at 140 characters when saved

https://github.com/opensearch-project/alerting/issues/915
https://github.com/opensearch-project/alerting/issues/706

никита какдела

unread,
Jan 20, 2026, 8:03:06 AMJan 20
to Wazuh | Mailing List
Now i again got this Error. Please, look at my Trigger Condition
{
    "buckets_path": {
        "_count": "_count"

    },
    "parent_bucket_path": "composite_agg",
    "script": {
        "source": "params._count > 0",

        "lang": "painless"
    },
    "gap_policy": "skip"
}


And Monitor
{

    "size": 0,
    "query": {
        "bool": {
            "filter": [
                {
                    "range": {
                        "@timestamp": {
                            "from": "{{period_end}}||-1m",

                            "to": "{{period_end}}",
                            "include_lower": true,
                            "include_upper": true,
                            "format": "epoch_millis",
                            "boost": 1
                        }
                    }
                },
                {
                    "term": {
                        "rule.id": {
                            "value": "100026",
                            "boost": 1
                        }
                    }
                }
            ],
            "adjust_pure_negative": true,
            "boost": 1

        }
    },
    "aggregations": {
        "composite_agg": {
            "composite": {
                "size": 10,
                "sources": [
                    {
                        "agent.name": {
                            "terms": {
                                "field": "agent.name",
                                "missing_bucket": false,
                                "order": "asc"
                            }
                        }
                    }
                ]
            }
        }
    }
}
Can you tell me what's wrong and why do i get this error?
вторник, 20 января 2026 г. в 14:58:49 UTC+3, musbau....@wazuh.com:

никита какдела

unread,
Jan 20, 2026, 8:47:09 AMJan 20
to Wazuh | Mailing List
This error occurs "sometimes" on a working monitor. It's as if something is missing at some point. Look at my trigger; the script isn't truncated. And it's as simple as can be.

вторник, 20 января 2026 г. в 15:58:47 UTC+3, musbau....@wazuh.com:

musbau....@wazuh.com

unread,
Jan 20, 2026, 1:30:44 PMJan 20
to Wazuh | Mailing List
Hi,

Your configuration looks correct. The fact it works sometimes suggests an environmental issue rather than a configuration bug.

Would you be able to check your cluster's resource usage during the failure times? This intermittent "sometimes" failure could likely be as a result of resource constraints rather than script errors.

I would advise that you do the below to get more insights

1. Check cluster health when the error occurs - look for high CPU, memory pressure, or circuit breaker trips

2. Monitor timing - Does the error happen during peak load times?

3. OpenSearch logs - Search for GeneralScriptException in the cluster logs around the failure time 

никита какдела

unread,
Jan 21, 2026, 1:27:14 AMJan 21
to Wazuh | Mailing List
I noticed the following errors in wazuh-cluster.log.
Perhaps these are the reason why actions in monitors sometimes don't work?Снимок экрана 2026-01-21 090607.png

вторник, 20 января 2026 г. в 21:30:44 UTC+3, musbau....@wazuh.com:

musbau....@wazuh.com

unread,
Jan 21, 2026, 5:46:44 AMJan 21
to Wazuh | Mailing List
Hi,

Thank you for the information shared as it gives more insights. This screenshot you provided shows that your cluster has a corrupted monitor definition where someone saved a query as 3=[] (which is invalid). This malformed query is being loaded and causing parsing failures.

QueryShardException[Failed to parse query [*3=[*]]; nested: ParseException[Cannot parse '3=[': Encountered "<EOF>"

I would advise to first search for monitors containing this malformed query then either repair or delete those corrupted monitors. 

Regards,

никита какдела

unread,
Jan 21, 2026, 7:21:20 AMJan 21
to Wazuh | Mailing List
I can show you a possibly problematic monitor

#! Deprecation: [interval] on [date_histogram] is deprecated, use [fixed_interval] or [calendar_interval] in the future.

{

  "_id": "xWdjA5sBoeamHjFYf3Hw",

  "_version": 31,

  "_seq_no": 37920,

  "_primary_term": 39,

  "monitor": {

    "type": "monitor",

    "schema_version": 8,

    "name": "MS Windows: Успешное подключение одной УЗ с разных IP адресов",

    "monitor_type": "bucket_level_monitor",

    "enabled": true,

    "enabled_time": 1768918477969,

    "schedule": {

      "period": {

        "interval": 1,

        "unit": "MINUTES"

      }

    },

    "inputs": [

      {

        "search": {

          "indices": [

            "wazuh-alerts-current"

          ],

          "query": {

            "size": 0,

            "query": {

              "bool": {

                "filter": [

                  {

                    "range": {

                      "@timestamp": {

                        "from": "{{period_end}}||-300s",

                        "to": "{{period_end}}",

                        "include_lower": true,

                        "include_upper": true,

                        "format": "epoch_millis",

                        "boost": 1

                      }

                    }

                  },

                  {

                    "term": {

                      "rule.id": {

                        "value": "100014",

                        "boost": 1

                      }

                    }

                  }

                ],

                "must_not": [

                  {

                    "terms": {

                      "data.win.eventdata.targetUserName": [

                        "ANONYMOUS LOGON",

                        "АНОНИМНЫЙ ВХОД"

                      ],

                      "boost": 1

                    }

                  }

                ],

                "adjust_pure_negative": true,

                "boost": 1

              }

            },

            "aggregations": {

              "composite_agg": {

                "composite": {

                  "size": 20,

                  "sources": [

                    {

                      "data.win.eventdata.targetUserName": {

                        "terms": {

                          "field": "data.win.eventdata.targetUserName",

                          "missing_bucket": false,

                          "order": "asc"

                        }

                      }

                    }

                  ]

                },

                "aggregations": {

                  "unique_ips": {

                    "cardinality": {

                      "field": "data.win.eventdata.ipAddress"

                    }

                  }

                }

              }

            }

          }

        }

      }

    ],

    "triggers": [

      {

        "bucket_level_trigger": {

          "id": "xGdjA5sBoeamHjFYf3Hq",

          "name": "MS Windows: Успешное подключение одной УЗ с разных рабочих станций",

          "severity": "2",

          "condition": {

            "buckets_path": {

              "uniq": "unique_ips.value"

            },

            "parent_bucket_path": "composite_agg",

            "script": {

              "source": "params.uniq > 4",

              "lang": "painless"

            },

            "gap_policy": "skip"

          },

          "actions": [

            {

              "id": "notification327539",

              "name": "Send to Kaiten",

              "destination_id": "xF8JnJoBovKpQ5b8ijIc",

              "message_template": {

                "source": """{

  "title": "Целевая УЗ: {{#ctx.newAlerts}}{{bucket_keys}}{{/ctx.newAlerts}}",

  "tags": ["{{ctx.monitor.name}}"],

  "links": [

    {

      "url": "{{#ctx.newAlerts}}https://wazuh.ovp.ru/app/data-explorer/discover#?_a=(discover:(columns:!(_source),isDirty:!f,sort:!()),metadata:(indexPattern:'wazuh-alerts-*',view:discover))&_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-1h,to:now))&_q=(filters:!(('$state':(store:appState),meta:(alias:!n,disabled:!f,index:'wazuh-alerts-*',key:rule.id,negate:!f,params:(query:'100014'),type:phrase),query:(match_phrase:(rule.id:'100014'))),('$state':(store:appState),meta:(alias:!n,disabled:!f,index:'wazuh-alerts-*',key:data.win.eventdata.targetUserName,negate:!f,params:(query:{{bucket_keys}}),type:phrase),query:(match_phrase:(data.win.eventdata.targetUserName:{{bucket_keys}})))),query:(language:kuery,query:'')){{/ctx.newAlerts}}",

      "description": "Открыть в Wazuh Discover"

    }

  ],

  "description": "\n- 🚨 Событие:  {{ctx.monitor.name}}\n- 🚨 Приоритет:  {{ctx.trigger.severity}}\n- ⏳ Время начала: {{ctx.periodStart}} UTC\n- ⌛ Время окончания: {{ctx.periodEnd}} UTC {{#ctx.newAlerts}}\n---{{#sample_documents.0}}\n- 🙎‍♂️ Инициатор:  {{_source.data.win.eventdata.targetUserName}}\n- 👁‍🗨 Агент:  {{_source.agent.name}}\n- 🔎 Правило: {{_source.rule.description}} (id: {{_source.rule.id}})\n- 🚨 Level: {{_source.rule.level}}\n [Открыть в Wazuh](https://wazuh.ovp.ru/app/data-explorer/discover#?_a=(discover:(columns:!(_source),isDirty:!t,sort:!()),metadata:(indexPattern:'wazuh-alerts-*',view:discover))&_q=(filters:!(('$state':(store:appState),meta:(alias:!n,disabled:!f,index:'wazuh-alerts-*',key:rule.id,negate:!f,params:(query:'100014'),type:phrase),query:(match_phrase:(rule.id:'100014'))),('$state':(store:appState),meta:(alias:!n,disabled:!f,index:'wazuh-alerts-*',key:data.win.eventdata.targetUserName,negate:!f,params:(query:{{bucket_keys}}),type:phrase),query:(match_phrase:(data.win.eventdata.targetUserName:{{bucket_keys}})))),query:(language:kuery,query:''))&_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-24h,to:now)))\n---{{/sample_documents.0}}{{/ctx.newAlerts}}"

}

""",

                "lang": "mustache"

              },

              "throttle_enabled": false,

              "subject_template": {

                "source": "Alerting Notification action",

                "lang": "mustache"

              },

              "action_execution_policy": {

                "action_execution_scope": {

                  "per_alert": {

                    "actionable_alerts": [

                      "NEW"

                    ]

                  }

                }

              }

            },

            {

              "id": "notification810195",

              "name": "Send to Yandex",

              "destination_id": "X4L_5pkBS6jN-8SDuQFi",

              "message_template": {

                "source": """{

  "chat_id": "1/0/191a25c4-b3f1-4e10-a6b1-a412c17b48e5",

  "text": "WAZUH\n\n- 🚨 Событие:  {{ctx.monitor.name}}\n- 🚨 Приоритет:  {{ctx.trigger.severity}}\n- ⏳ Время начала: {{ctx.periodStart}} UTC\n- ⌛ Время окончания: {{ctx.periodEnd}} UTC {{#ctx.newAlerts}}\n---{{#sample_documents.0}}\n- 🙎‍♂️ Инициатор:  {{_source.data.win.eventdata.targetUserName}}\n- 👁‍🗨 Агент:  {{_source.agent.name}}\n- 🔎 Правило: {{_source.rule.description}} (id: {{_source.rule.id}})\n- 🚨 Level: {{_source.rule.level}}\n [Открыть в Wazuh](https://wazuh.ovp.ru/app/data-explorer/discover#?_a=(discover:(columns:!(_source),isDirty:!t,sort:!()),metadata:(indexPattern:'wazuh-alerts-*',view:discover))&_q=(filters:!(('$state':(store:appState),meta:(alias:!n,disabled:!f,index:'wazuh-alerts-*',key:rule.id,negate:!f,params:(query:'100014'),type:phrase),query:(match_phrase:(rule.id:'100014'))),('$state':(store:appState),meta:(alias:!n,disabled:!f,index:'wazuh-alerts-*',key:data.win.eventdata.targetUserName,negate:!f,params:(query:{{bucket_keys}}),type:phrase),query:(match_phrase:(data.win.eventdata.targetUserName:{{bucket_keys}})))),query:(language:kuery,query:''))&_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-24h,to:now)))\n---{{/sample_documents.0}}{{/ctx.newAlerts}}"

}

""",

                "lang": "mustache"

              },

              "throttle_enabled": false,

              "subject_template": {

                "source": "Alerting Notification action",

                "lang": "mustache"

              },

              "action_execution_policy": {

                "action_execution_scope": {

                  "per_alert": {

                    "actionable_alerts": [

                      "NEW"

                    ]

                  }

                }

              }

            },

            {

              "id": "notification225546",

              "name": "Send to TG",

              "destination_id": "tAAOOpoBAqvA3MNHy-lM",

              "message_template": {

                "source": """{

  "chat_id": "-1002403153612",

  "text": "WAZUH\n\n- 🚨 Событие:  {{ctx.monitor.name}}\n- 🚨 Приоритет:  {{ctx.trigger.severity}}\n- ⏳ Время начала: {{ctx.periodStart}} UTC\n- ⌛ Время окончания: {{ctx.periodEnd}} UTC {{#ctx.newAlerts}}\n---{{#sample_documents.0}}\n- 🙎‍♂️ Инициатор:  {{_source.data.win.eventdata.targetUserName}}\n- 👁‍🗨 Агент:  {{_source.agent.name}}\n- 🔎 Правило: {{_source.rule.description}} (id: {{_source.rule.id}})\n- 🚨 Level: {{_source.rule.level}}\n [Открыть в Wazuh](https://wazuh.ovp.ru/app/data-explorer/discover#?_a=(discover:(columns:!(_source),isDirty:!t,sort:!()),metadata:(indexPattern:'wazuh-alerts-*',view:discover))&_q=(filters:!(('$state':(store:appState),meta:(alias:!n,disabled:!f,index:'wazuh-alerts-*',key:rule.id,negate:!f,params:(query:'100014'),type:phrase),query:(match_phrase:(rule.id:'100014'))),('$state':(store:appState),meta:(alias:!n,disabled:!f,index:'wazuh-alerts-*',key:data.win.eventdata.targetUserName,negate:!f,params:(query:{{bucket_keys}}),type:phrase),query:(match_phrase:(data.win.eventdata.targetUserName:{{bucket_keys}})))),query:(language:kuery,query:''))&_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-24h,to:now)))\n---{{/sample_documents.0}}{{/ctx.newAlerts}}"

}

""",

                "lang": "mustache"

              },

              "throttle_enabled": false,

              "subject_template": {

                "source": "Alerting Notification action",

                "lang": "mustache"

              },

              "action_execution_policy": {

                "action_execution_scope": {

                  "per_alert": {

                    "actionable_alerts": [

                      "NEW"

                    ]

                  }

                }

              }

            }

          ]

        }

      }

    ],

    "last_update_time": 1768918477969,

    "data_sources": {

      "query_index": ".opensearch-alerting-queries",

      "findings_index": ".opensearch-alerting-finding-history-write",

      "findings_index_pattern": "<.opensearch-alerting-finding-history-{now/d}-1>",

      "alerts_index": ".opendistro-alerting-alerts",

      "alerts_history_index": ".opendistro-alerting-alert-history-write",

      "alerts_history_index_pattern": "<.opendistro-alerting-alert-history-{now/d}-1>",

      "comments_index": ".opensearch-alerting-comments-history-write",

      "comments_index_pattern": "<.opensearch-alerting-comments-history-{now/d}-1>",

      "query_index_mappings_by_type": {},

      "findings_enabled": false

    },

    "delete_query_index_in_every_run": false,

    "should_create_single_alert_for_findings": false,

    "owner": "alerting"

  },

  "associated_workflows": []

}

среда, 21 января 2026 г. в 13:46:44 UTC+3, musbau....@wazuh.com:

никита какдела

unread,
Jan 21, 2026, 7:21:20 AMJan 21
to Wazuh | Mailing List
[2026-01-21T11:02:02,707][ERROR][o.o.a.BucketLevelMonitorRunner] [node-1] Failed to retrieve sample documents for alert Vmg24JsB-dPPuwmWbwYj from trigger f7QlspsBd-k2aqqxq1jt of monitor gbQlspsBd-k2aqqxq1j8 during execution gbQlspsBd-k2aqqxq1j8_2026-01-21T11:00:22.364989286_0a4cfd4f-4c47-47e4-91f9-4ca3da4d64df.
[2026-01-21T11:02:22,465][ERROR][o.o.a.BucketLevelMonitorRunner] [node-1] Failed to retrieve sample documents for alert Z2g44JsB-dPPuwmWQzas from trigger f7QlspsBd-k2aqqxq1jt of monitor gbQlspsBd-k2aqqxq1j8 during execution gbQlspsBd-k2aqqxq1j8_2026-01-21T11:02:22.362117358_7d4e78d8-efaf-4592-82df-7aa4e7b92de7.
[2026-01-21T11:02:22,657][ERROR][o.o.a.BucketLevelMonitorRunner] [node-1] Failed to retrieve sample documents for alert Z2g44JsB-dPPuwmWQzas from trigger f7QlspsBd-k2aqqxq1jt of monitor gbQlspsBd-k2aqqxq1j8 during execution gbQlspsBd-k2aqqxq1j8_2026-01-21T11:02:22.362117358_7d4e78d8-efaf-4592-82df-7aa4e7b92de7.
[2026-01-21T11:02:22,828][ERROR][o.o.a.BucketLevelMonitorRunner] [node-1] Failed to retrieve sample documents for alert Z2g44JsB-dPPuwmWQzas from trigger f7QlspsBd-k2aqqxq1jt of monitor gbQlspsBd-k2aqqxq1j8 during execution gbQlspsBd-k2aqqxq1j8_2026-01-21T11:02:22.362117358_7d4e78d8-efaf-4592-82df-7aa4e7b92de7.


I am getting this ERRORS from this monitor. Can u explain on my monitor whats wrong, please? 


#! Deprecation: [interval] on [date_histogram] is deprecated, use [fixed_interval] or [calendar_interval] in the future.
{
  "_id": "gbQlspsBd-k2aqqxq1j8",
  "_version": 31,
  "_seq_no": 37918,

  "_primary_term": 39,
  "monitor": {
    "type": "monitor",
    "schema_version": 8,
    "name": "MS Windows: входы более чем с 3-х или на более 3 хоста под одной и той же учетной записью",

    "monitor_type": "bucket_level_monitor",
    "enabled": true,
    "enabled_time": 1768918462358,

    "schedule": {
      "period": {
        "interval": 1,
        "unit": "MINUTES"
      }
    },
    "inputs": [
      {
        "search": {
          "indices": [
            "wazuh-alerts-current"
          ],
          "query": {
            "size": 0,
            "query": {
              "bool": {
                "filter": [
                  {
                    "range": {
                      "@timestamp": {
                        "from": "{{period_end}}||-30m",

                        "to": "{{period_end}}",
                        "include_lower": true,
                        "include_upper": true,
                        "format": "epoch_millis",
                        "boost": 1
                      }
                    }
                  },
                  {
                    "term": {
                      "rule.id": {
                        "value": "100014",
                        "boost": 1
                      }
                    }
                  }
                ],
                "must_not": [
                  {
                    "wildcard": {
                      "data.win.eventdata.targetUserName": {
                        "wildcard": "*$",
                        "boost": 1
                      }
                    }
                  },
                  {
                    "terms": {
                      "data.win.eventdata.targetUserName": [
                        "Replicaton_Shuttle",
                        "elmasys",
                        "esbservice",
                        "service_1c_1"
                      ],
                      "boost": 1
                    }
                  },
                  {
                    "term": {
                      "data.win.eventdata.authenticationPackageName": {
                        "value": "Kerberos",

                        "boost": 1
                      }
                    }
                  }
                ],
                "adjust_pure_negative": true,
                "boost": 1
              }
            },
            "aggregations": {
              "composite_agg": {
                "composite": {
                  "size": 20,

                  "sources": [
                    {
                      "data.win.eventdata.targetUserName": {
                        "terms": {
                          "field": "data.win.eventdata.targetUserName",
                          "missing_bucket": false,
                          "order": "asc"
                        }
                      }
                    }
                  ]
                },
                "aggregations": {
                  "unique_hosts": {
                    "cardinality": {
                      "field": "agent.name"
                    }
                  }
                }
              }
            }
          }
        }
      }
    ],
    "triggers": [
      {
        "bucket_level_trigger": {
          "id": "f7QlspsBd-k2aqqxq1jt",
          "name": "MS Windows: входы более чем с 3-х или на более 3-хоста под одной и той же учетной записью",
          "severity": "3",
          "condition": {
            "buckets_path": {
              "uniq": "unique_hosts.value"

            },
            "parent_bucket_path": "composite_agg",
            "script": {
              "source": "params.uniq > 2",

              "lang": "painless"
            },
            "gap_policy": "skip"
          },
          "actions": [
            {
              "id": "notification928046",
              "name": "Send to Telegram",

              "destination_id": "tAAOOpoBAqvA3MNHy-lM",
              "message_template": {
                "source": """{
  "chat_id": "-1002403153612",
  "text": "WAZUH\n\n- 🚨 Событие:  {{ctx.monitor.name}}\n- 🚨 Приоритет:  {{ctx.trigger.severity}}\n- ⏳ Время начала: {{ctx.periodStart}} UTC\n- ⌛ Время окончания: {{ctx.periodEnd}} UTC {{#ctx.newAlerts}}\n---\n- 🙎‍♂️ Инициатор:  {{bucket_keys}}\n{{/ctx.newAlerts}}"

}
""",
                "lang": "mustache"
              },
              "throttle_enabled": false,
              "subject_template": {
                "source": "Alerting Notification action",
                "lang": "mustache"
              },
              "action_execution_policy": {
                "action_execution_scope": {
                  "per_alert": {
                    "actionable_alerts": [
                      "NEW"
                    ]
                  }
                }
              }
            },
            {
              "id": "notification096773",

              "name": "Send to Yandex",
              "destination_id": "X4L_5pkBS6jN-8SDuQFi",
              "message_template": {
                "source": """{
  "chat_id": "1/0/191a25c4-b3f1-4e10-a6b1-a412c17b48e5",
  "text": "WAZUH\n\n- 🚨 Событие:  {{ctx.monitor.name}}\n- 🚨 Приоритет:  {{ctx.trigger.severity}}\n- ⏳ Время начала: {{ctx.periodStart}} UTC\n- ⌛ Время окончания: {{ctx.periodEnd}} UTC {{#ctx.newAlerts}}\n---\n- 🙎‍♂️ Инициатор:  {{bucket_keys}}{{/ctx.newAlerts}}"

}
""",
                "lang": "mustache"
              },
              "throttle_enabled": false,
              "subject_template": {
                "source": "Alerting Notification action",
                "lang": "mustache"
              },
              "action_execution_policy": {
                "action_execution_scope": {
                  "per_alert": {
                    "actionable_alerts": [
                      "NEW"
                    ]
                  }
                }
              }
            },
            {
              "id": "notification902119",

              "name": "Send to Kaiten",
              "destination_id": "xF8JnJoBovKpQ5b8ijIc",
              "message_template": {
                "source": """{
  "title": "Целевая УЗ: {{#ctx.newAlerts}}{{bucket_keys}}{{/ctx.newAlerts}}",
  "tags": ["{{ctx.monitor.name}}"],
  "links": [
    {
      "url": "{{#ctx.newAlerts}}https://wazuh.ovp.ru/app/data-explorer/discover#?_a=(discover:(columns:!(_source),isDirty:!f,sort:!()),metadata:(indexPattern:'wazuh-alerts-*',view:discover))&_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-24h,to:now))&_q=(filters:!(('$state':(store:appState),meta:(alias:!n,disabled:!f,index:'wazuh-alerts-*',key:rule.id,negate:!f,params:(query:'100014'),type:phrase),query:(match_phrase:(rule.id:'100014'))),('$state':(store:appState),meta:(alias:!n,disabled:!f,index:'wazuh-alerts-*',key:data.win.eventdata.targetUserName,negate:!f,params:(query:{{bucket_keys}}),type:phrase),query:(match_phrase:(data.win.eventdata.targetUserName:{{bucket_keys}})))),query:(language:kuery,query:'')){{/ctx.newAlerts}}",
      "description": "Открыть в Wazuh"
    }
  ],
  "description": "\n- 🚨 Событие:  {{ctx.monitor.name}}\n- 🚨 Приоритет:  {{ctx.trigger.severity}}\n- ⏳ Время начала: {{ctx.periodStart}} UTC\n- ⌛ Время окончания: {{ctx.periodEnd}} UTC {{#ctx.newAlerts}}\n---\n- 🙎‍♂️ Инициатор:  {{bucket_keys}}{{/ctx.newAlerts}}"

}""",
                "lang": "mustache"
              },
              "throttle_enabled": false,
              "subject_template": {
                "source": "Alerting Notification action",
                "lang": "mustache"
              },
              "action_execution_policy": {
                "action_execution_scope": {
                  "per_alert": {
                    "actionable_alerts": [
                      "NEW"
                    ]
                  }
                }
              }
            }
          ]
        }
      }
    ],
    "last_update_time": 1768918462358,

никита какдела

unread,
Jan 21, 2026, 7:21:20 AMJan 21
to Wazuh | Mailing List
How can i do it through DevTools? I have almost 100 monitors.

среда, 21 января 2026 г. в 13:46:44 UTC+3, musbau....@wazuh.com:
Hi,

musbau....@wazuh.com

unread,
Jan 21, 2026, 9:44:40 AMJan 21
to Wazuh | Mailing List
Hi,

Please check this out

GET /_plugins/_alerting/monitors/_search
{
  "query": {
    "wildcard": {
      "monitor.inputs.search.query.query": "*3=[]*"

musbau....@wazuh.com

unread,
Jan 21, 2026, 9:46:40 AMJan 21
to Wazuh | Mailing List
Hi,
Please check this out.


GET /_plugins/_alerting/monitors/_search
{
  "query": {
    "wildcard": {
      "monitor.inputs.search.query.query": "*3=[]*"
    }
  }
}


Regards,

никита какдела

unread,
Jan 22, 2026, 2:47:44 AMJan 22
to Wazuh | Mailing List
{
  "took": 1,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 0,
      "relation": "eq"
    },
    "max_score": null,
    "hits": []
  }
}

среда, 21 января 2026 г. в 17:46:40 UTC+3, musbau....@wazuh.com:

никита какдела

unread,
Jan 22, 2026, 2:47:44 AMJan 22
to Wazuh | Mailing List
I keep getting errors, here is an example of one of them ([2026-01-22T06:39:22,345][ERROR][o.o.n.c.t.WebhookDestinationTransport] [node-1] Exception sending webhook message X4L_5pkBS6jN-8SDuQFi: org.opensearch.notifications.spi.model.MessageContent@e87df90)
I don't understand how to figure out what exactly the error is? Which monitor? The errors are not constant.
среда, 21 января 2026 г. в 17:46:40 UTC+3, musbau....@wazuh.com:
Hi,

musbau....@wazuh.com

unread,
Jan 22, 2026, 7:13:31 AMJan 22
to Wazuh | Mailing List
Hi,

For every time you see new errors is any change being made on your end? I would advise to restore your Wazuh server to a point before this initial challenge occurred as the previous log you shared showed that it was a change that was made to one of the monitors that caused the error. You can also temporarily disable groups of monitors (by tag or pattern) to see if errors stop when certain monitors are off and narrow down to the problematic ones

Also, based on the last log you shared, you should use the below to check if it gives you more insights. The webhook errors mean monitors are triggering but failing to send notifications.

GET /_plugins/_alerting/alerts/_search
{
  "query": {
    "term": {
      "_id": "X4L_5pkBS6jN-8SDuQFi"

никита какдела

unread,
Jan 22, 2026, 7:40:05 AMJan 22
to Wazuh | Mailing List
{
  "error": "no handler found for uri [/_plugins/_alerting/alerts/_search] and method [GET]"
}

dev tools response

четверг, 22 января 2026 г. в 15:13:31 UTC+3, musbau....@wazuh.com:

никита какдела

unread,
Jan 22, 2026, 7:57:07 AMJan 22
to Wazuh | Mailing List
Let's use an example.
I have one of the logs.
What command in Dev Tools can I use to find this alert using alert_id?

[2026-01-22T12:51:21,848][ERROR][o.o.a.BucketLevelMonitorRunner] [node-1] Failed to retrieve sample documents for alert JufC5ZsB-dPPuwmWZ1ug from trigger xGdjA5sBoeamHjFYf3Hq of monitor xWdjA5sBoeamHjFYf3Hw during execution xWdjA5sBoeamHjFYf3Hw_2026-01-22T12:51:21.574506534_be3e555b-9d4f-4b1a-ae52-57eeabb96502.

четверг, 22 января 2026 г. в 15:13:31 UTC+3, musbau....@wazuh.com:

musbau....@wazuh.com

unread,
Jan 22, 2026, 3:19:16 PMJan 22
to Wazuh | Mailing List
Hi,

Can you run this command GET /_cat/indices/.opendistro*?v so we can know the exact index name as it might vary depending on your OpenSearch version.

никита какдела

unread,
Jan 23, 2026, 1:18:05 AMJan 23
to Wazuh | Mailing List
health status index                                                   uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   .opendistro-ism-managed-index-history-2025.12.25-000069 EZIM8_hvSYiyoipz26LBQg   1   1         16            0     84.4kb         84.4kb
yellow open   .opendistro-alerting-alert-history-2026.01.03-000017    nV2DmiWbTEq52jXyqZEmvQ   1   1       3640            0      1.4mb          1.4mb
yellow open   .opendistro-ism-managed-index-history-2025.12.24-000068 LfvbvUuKQiiKgcOArNVJXg   1   1          8            0     60.6kb         60.6kb
yellow open   .opendistro-ism-managed-index-history-2026.01.06-000080 ixSI_YFbSdiYmUQT5ieaHA   1   1         12            0     36.7kb         36.7kb
yellow open   .opendistro-ism-managed-index-history-2026.01.07-000081 VJzCCbCsSdeyENIdhmnEzw   1   1          8            0     46.8kb         46.8kb
yellow open   .opendistro-ism-managed-index-history-2026.01.08-000082 fCei8uq0QguDVk5G7hibDA   1   1          8            0     53.6kb         53.6kb
green  open   .opendistro-reports-definitions                         rBFKmF1SRPm4ndWjYFJ5Wg   1   0          0            0       228b           228b
yellow open   .opendistro-ism-managed-index-history-2026.01.09-000083 kZZxO60VQCu5VSnR89t5Mg   1   1          8            0     49.4kb         49.4kb
green  open   .opendistro-reports-instances                           yZ2i-V9wTXSesjJJyEe83g   1   0          1            0      6.2kb          6.2kb
green  open   .opendistro_security                                    HCMRREk5TfO_J6w0s3gSZQ   1   0         10            2     72.4kb         72.4kb
yellow open   .opendistro-alerting-alert-history-2026.01.22-000023    Xlun4DmPR1mGO1PEQaWJwA   1   1        675            0    697.7kb        697.7kb
yellow open   .opendistro-alerting-config                             22tOiYJqRlWt7689cQWqNg   1   1        318           53    351.3kb        351.3kb
yellow open   .opendistro-alerting-alert-history-2025.12.16-000013    SEB9ChaySsGyYz5kZ5ih_w   1   1       1182            0    719.4kb        719.4kb
yellow open   .opendistro-alerting-alert-history-2025.12.19-000014    vOjlmceJTOKYaOqEKjyS9w   1   1       1071            0    686.8kb        686.8kb
yellow open   .opendistro-ism-managed-index-history-2026.01.22-000094 UtR5dFdISpa65gMByeXklw   1   1          8            0     61.5kb         61.5kb
yellow open   .opendistro-ism-managed-index-history-2025.12.31-000074 2Iy3ZH14QIu0KGJti14uUA   1   1         10            0     50.8kb         50.8kb
yellow open   .opendistro-ism-config                                  uNRcMYoGTZ2rB0rwlUeIpw   1   1         44        11596      1.1mb          1.1mb
yellow open   .opendistro-ism-managed-index-history-2026.01.01-000075 RhqwRxW0Td-ibP8aaTn4Zw   1   1          8            0     39.3kb         39.3kb
yellow open   .opendistro-anomaly-checkpoints                         kJLMk6f5TZib3N6cqVa6FQ   1   1          1            0    655.8kb        655.8kb
yellow open   .opendistro-alerting-alert-history-2025.12.29-000016    Hv6A_Jh-TfKgPGP7v4qrYA   1   1       1049            0    749.5kb        749.5kb
yellow open   .opendistro-ism-managed-index-history-2026.01.05-000079 wfCifDVZR-SFFWitRZEUSg   1   1          8            0     34.9kb         34.9kb
yellow open   .opendistro-alerting-alerts                             Qn0M3QjZSc2AC8aSFhtXjg   1   1       4628         4609      2.1mb          2.1mb
yellow open   .opendistro-ism-managed-index-history-2025.12.28-000071 RTdycJW2SQaNZp4tnfs6zQ   1   1         10            0     28.4kb         28.4kb
yellow open   .opendistro-ism-managed-index-history-2026.01.03-000077 HJVWRB4zQ4G62kTOfvsC1Q   1   1          8            0     42.5kb         42.5kb
yellow open   .opendistro-ism-managed-index-history-2026.01.15-000088 MqEuUphXS5GDKIfXG23Cuw   1   1          8            0     53.6kb         53.6kb
yellow open   .opendistro-anomaly-results-history-2025.11.20-1        yHAfGzSKSbWqyqGhL8OmCQ   1   1    6866267            0    344.4mb        344.4mb
yellow open   .opendistro-ism-managed-index-history-2026.01.13-000087 SHlX5evhQRSNbdWvhJl3Lw   1   1         20            0     63.8kb         63.8kb
yellow open   .opendistro-ism-managed-index-history-2026.01.12-000086 oBnpugjJRo2hKXQ0F0GyfA   1   1          8            0     46.8kb         46.8kb
yellow open   .opendistro-alerting-alert-history-2026.01.17-000020    3mTBY0EfTaSr7e6_sj502g   1   1       1551            0    840.3kb        840.3kb
yellow open   .opendistro-ism-managed-index-history-2026.01.16-000089 BVmKgpOOQzONOm9S2jSRTQ   1   1          8            0     42.6kb         42.6kb
yellow open   .opendistro-alerting-alert-history-2026.01.19-000021    CHKiJZr8SJyDogNPKjpWwQ   1   1       1078            0    754.1kb        754.1kb
yellow open   .opendistro-alerting-alert-history-2025.12.24-000015    1OqbcOOmST2PaEQFPw5TnQ   1   1       1109            0    674.1kb        674.1kb
yellow open   .opendistro-ism-managed-index-history-2026.01.11-000085 grQeNd9YSEC2kAdKC7uwzA   1   1         10            0     47.2kb         47.2kb
yellow open   .opendistro-anomaly-detectors                           CPf3l-o3RmydSHO2JEY56g   1   1         68            0     86.9kb         86.9kb
yellow open   .opendistro-alerting-alert-history-2026.01.13-000018    1StJuBcMT6-xlLReK7Rb-g   1   1       1408            0    818.3kb        818.3kb
yellow open   .opendistro-ism-managed-index-history-2026.01.10-000084 izN2kWlYR-Ol0YsROxon3A   1   1          8            0     42.4kb         42.4kb
yellow open   .opendistro-ism-managed-index-history-2026.01.19-000092 siodfaj1So2T00uBxO1Q1A   1   1         20            0    109.9kb        109.9kb
yellow open   .opendistro-ism-managed-index-history-2026.01.18-000091 8pWipWq7TMOwp9FkLz2lhw   1   1         10            0     36.3kb         36.3kb
yellow open   .opendistro-alerting-alert-history-2025.12.09-000012    QakxSMIHRIaAvQhveM3IrQ   1   1       1056            0    780.8kb        780.8kb
yellow open   .opendistro-ism-managed-index-history-2026.01.17-000090 ikYG-6h7SWicHiUPiaXGpA   1   1          8            0     34.9kb         34.9kb
yellow open   .opendistro-alerting-alert-history-2025.11.29-000007    MhDHipRETAyzGQa3hQ3vWg   1   1       1085            0    445.4kb        445.4kb
yellow open   .opendistro-alerting-alert-history-2025.12.07-000011    -gV1KfbJTSSi8JrPo8Bf9w   1   1       1589            0    816.8kb        816.8kb
yellow open   .opendistro-anomaly-detector-jobs                       WnIC1OE0T9anWs3mG9JjRg   1   1         10            0       49kb           49kb
yellow open   .opendistro-alerting-alert-history-2025.11.27-000006    oxAnJW74SyqVM6Wuk9DH4g   1   1       1010            0    403.8kb        403.8kb
yellow open   .opendistro-alerting-alert-history-2025.11.25-000005    IN0TEVDjSqOw1zlwLi8f7Q   1   1       1120            0    518.1kb        518.1kb
yellow open   .opendistro-alerting-alert-history-2025.12.05-000010    Y2Bwy0xBRz-nQ6a6eIaSSg   1   1       1018            0    660.3kb        660.3kb
yellow open   .opendistro-ism-managed-index-history-2026.01.21-000093 0pA0aqiwRbaiPUsDmNyMNQ   1   1          6            0     40.9kb         40.9kb
yellow open   .opendistro-alerting-alert-history-2025.12.03-000009    2zarVgATSmaYSiJS3tXUQA   1   1       1142            0    575.6kb        575.6kb
yellow open   .opendistro-ism-managed-index-history-2026.01.02-000076 OdgK-vhYRCuWbx8XjR1jnw   1   1          8            0     53.8kb         53.8kb
yellow open   .opendistro-ism-managed-index-history-2025.12.27-000070 xg4N94o3S_mUZjgybLgHXA   1   1          8            0     49.4kb         49.4kb
yellow open   .opendistro-ism-managed-index-history-2025.12.30-000073 h5JPgF4uRQGuWdzkdeskMg   1   1         12            0     36.6kb         36.6kb
yellow open   .opendistro-anomaly-detection-state                     ciczOzDmQoqxbOnVV9hxgg   1   1         97           19    151.5kb        151.5kb
yellow open   .opendistro-ism-managed-index-history-2026.01.04-000078 DzS5zl65SuuU8bsEtwC0kw   1   1         10            0       55kb           55kb
yellow open   .opendistro-ism-managed-index-history-2025.12.29-000072 IDmvgswpSe6zYWmEBWEccA   1   1          8            0     34.9kb         34.9kb
yellow open   .opendistro-alerting-alert-history-2026.01.21-000022    P5oXQpiaQOmDRY-2tLzTzg   1   1       1659            0    888.9kb        888.9kb
yellow open   .opendistro-alerting-alert-history-2026.01.16-000019    jsDNDuRUShKhPZHkpMC-lw   1   1       1011            0    578.6kb        578.6kb
yellow open   .opendistro-job-scheduler-lock                          xdKq_vfmSFirx4NP-b4KQg   1   1         27         4902    333.4kb        333.4kb
yellow open   .opendistro-alerting-alert-history-2025.12.01-000008    t_imUSr6Q8W5rwm7qg5TCQ   1   1       1443            0    659.2kb        659.2kb

четверг, 22 января 2026 г. в 23:19:16 UTC+3, musbau....@wazuh.com:

musbau....@wazuh.com

unread,
Jan 23, 2026, 5:18:40 AMJan 23
to Wazuh | Mailing List
Hi,

Thank you for the information.

- For the specific alert from your error log e.g (alert ID: JufC5ZsB-dPPuwmWZ1ug)
  GET /.opendistro-alerting-alerts/_doc/xxxxxxxxxxxxxxxxx

- For the monitor from your error log e.g (monitor ID: xWdjA5sBoeamHjFYf3Hw)
GET /.opendistro-alerting-config/_doc/xxxxxxxxxxxxxxx

Try searching for your specific alert first. You need to use the specific IDs from your error log. This should tell you which monitor created that alert, what trigger fired, and give you more context about the failure.

никита какдела

unread,
Jan 26, 2026, 8:32:29 AMJan 26
to Wazuh | Mailing List
{
  "error": {
    "root_cause": [
      {
        "type": "routing_missing_exception",
        "reason": "routing is required for [.opendistro-alerting-alerts]/[JufC5ZsB-dPPuwmWZ1ug]",
        "index": ".opendistro-alerting-alerts",
        "index_uuid": "_na_"
      }
    ],
    "type": "routing_missing_exception",
    "reason": "routing is required for [.opendistro-alerting-alerts]/[JufC5ZsB-dPPuwmWZ1ug]",
    "index": ".opendistro-alerting-alerts",
    "index_uuid": "_na_"
  },
  "status": 400
}

{
  "_index": ".opendistro-alerting-config",
  "_id": "xWdjA5sBoeamHjFYf3Hw",
  "found": false
}

What does it mean? 


пятница, 23 января 2026 г. в 13:18:40 UTC+3, musbau....@wazuh.com:
Reply all
Reply to author
Forward
0 new messages