Monitor performance

[никита какдела]

Hello, dear Wazuh team.  I really need your help, because I've already spent all my energy trying to solve this problem. Below I will attach my monitor, which counts "Successful login attempts from 3 or more hosts". The fact is that when I "Run" the query, I get about 10 thousand events, which is a huge load on the monitor and, as a result, incorrect operation of mustache and, in general, skipping events. I need some advice, how can I fix this? Also, I want to display an example of the IPAddress, target/subjectusername fields in this and in all my monitors. agent.name , rule.name , rule.description. I want to output 1 event to mustache that will contain all these fields. Right now, it turns out that if a backet is triggered and it contains 5-10 events, they are all sent to mustache. How can I fix this correctly? Please help me. Also, please explain what I'm doing wrong, what the problem is, and how to fix it.

Monitor Query:
{
    "size": 0,
    "query": {
        "bool": {
            "filter": [
                {
                    "range": {
                        "@timestamp": {
                            "from": "{{period_end}}||-30m",
                            "to": "{{period_end}}",
                            "include_lower": true,
                            "include_upper": true,
                            "format": "epoch_millis",
                            "boost": 1
                        }
                    }
                },
                {
                    "term": {
                        "rule.id": {
                            "value": "100014",
                            "boost": 1
                        }
                    }
                }
            ],
            "must_not": [
                {
                    "wildcard": {
                        "data.win.eventdata.targetUserName": {
                            "wildcard": "*$",
                            "boost": 1
                        }
                    }
                },
                {
                    "terms": {
                        "data.win.eventdata.targetUserName": [
                            "Replicaton_Shuttle",
                            "elmasys",
                            "esbservice",
                            "service_1c_1",
                            "InfoSec",
                            "service_illinois"
                        ],
                        "boost": 1
                    }
                }
            ],
            "adjust_pure_negative": true,
            "boost": 1
        }
    },
    "aggregations": {
        "composite_agg": {
            "composite": {
                "size": 20,
                "sources": [
                    {
                        "data.win.eventdata.ipAddress": {
                            "terms": {
                                "field": "data.win.eventdata.ipAddress",
                                "missing_bucket": false,
                                "order": "asc"
                            }
                        }
                    }
                ]
            },
            "aggregations": {
                "unique_hosts": {
                    "cardinality": {
                        "field": "agent.name"
                    }
                }
            }
        }
    }
}

Trigger:
{
    "buckets_path": {
        "uniq": "unique_hosts.value"
    },
    "parent_bucket_path": "composite_agg",
    "script": {
        "source": "params.uniq > 2",
        "lang": "painless"
    },
    "gap_policy": "skip"
}

Mustache (Action):
{
  "chat_id": "123123",
  "text": "WAZUH\n\n- 🚨 Событие:  {{ctx.monitor.name}}\n- 🚨 Приоритет:  {{ctx.trigger.severity}}\n- ⏳ Время начала: {{ctx.periodStart}} UTC\n- ⌛ Время окончания: {{ctx.periodEnd}} UTC {{#ctx.newAlerts.0.sample_documents}}\n---\n- 🙎‍♂️ Адрес источника:  {{_source.data.win.eventdata.ipAddress}}\n- 👨‍💻 Инициатор:  {{_source.data.win.eventdata.targetUserName}}\n- 👁‍🗨 Агент:  {{_source.agent.name}}\n- 🔎 Правило: {{_source.rule.description}}\n- 🚨 Level: {{_source.rule.level}}\n [Открыть в Wazuh](https://wazuh.ovp.ru/app/data-explorer/discover#?_a=(discover:(columns:!(agent.name,data.win.eventdata.targetUserName,data.win.eventdata.ipAddress,rule.id,rule.description),isDirty:!f,sort:!()),metadata:(indexPattern:'wazuh-alerts-*',view:discover))&_q=(filters:!(('$state':(store:appState),meta:(alias:!n,disabled:!f,index:'wazuh-alerts-*',key:rule.id,negate:!f,params:!('100015','100016','100017','100018','100019','100020'),type:phrases,value:'100015,%20100016,%20100017,%20100018,%20100019,%20100020'),query:(bool:(minimum_should_match:1,should:!((match_phrase:(rule.id:'100015')),(match_phrase:(rule.id:'100016')),(match_phrase:(rule.id:'100017')),(match_phrase:(rule.id:'100018')),(match_phrase:(rule.id:'100019')),(match_phrase:(rule.id:'100020')))))),('$state':(store:appState),meta:(alias:!n,disabled:!f,index:'wazuh-alerts-*',key:data.win.eventdata.ipAddress,negate:!f,params:(query:'{{bucket_keys}}'),type:phrase),query:(match_phrase:(data.win.eventdata.ipAddress:'{{bucket_keys}}')))),query:(language:kuery,query:''))&_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:'{{ctx.periodStart}}',to:now)))\n---{{/ctx.newAlerts.0.sample_documents}}"
}

[Isaiah Daboh]

Hello,

Please note that I am taking a look at this. I will revert as soon as possible.

Regards,

[Isaiah Daboh]

Hello,

The current query has huge query load (10k events) because despite having the top-level "size":0, the monitor till processes all matching docs inside the aggregation window. Which means if 10,000 events match the rule.id=100014 in 30m, the monitor must load them, aggregate them, evaluate the bucket condition and build sample_documents per bucket.

The composite aggregation used here is best for pagination which is not needed, here we need to use TERMS aggregation which will help exclude buckets with few events early.

Please try this instead:

{
  "size": 0,
  "query": {
    "bool": {
      "filter": [
        {
          "range": {
            "@timestamp": {

              "from": "{{period_end}}||-15m",

              "to": "{{period_end}}",
              "include_lower": true,
              "include_upper": true,
              "format": "epoch_millis"
            }
          }
        },

        {
          "term": { "rule.id": { "value": "100014" } }
        }
      ],

      "must_not": [
        {
          "wildcard": {
            "data.win.eventdata.targetUserName": { "wildcard": "*$" }
          }
        },

        {
          "terms": {
            "data.win.eventdata.targetUserName": [
              "Replicaton_Shuttle",
              "elmasys",
              "esbservice",
              "service_1c_1",
              "InfoSec",
              "service_illinois"
            ]
          }
        }
      ]
    }
  },

  "aggregations": {
    "ip_buckets": {
      "terms": {
        "field": "data.win.eventdata.ipAddress",
        "size": 20,
        "min_doc_count": 3

      },
      "aggregations": {
        "unique_hosts": {
          "cardinality": {
            "field": "agent.name"
          }

        },
        "sample_event": {
          "top_hits": {
            "size": 1,
            "_source": {
              "includes": [
                "data.win.eventdata.ipAddress",
                "data.win.eventdata.targetUserName",
                "agent.name",
                "rule.name",
                "rule.description",
                "rule.level"
              ]
            }
          }
        }
      }
    }
  }
}

The top_hits aggregation added (sample_event with size: 1) will make sure each bucket contains exactly one representative document with all the fields you need. This should resolve the issue with the Mustache too.  

Regards,

[никита какдела]

I wrote this monitor and added the following trigger to it:
{
    "buckets_path": {
        "uniq": "unique_hosts.value"
    },
    "parent_bucket_path": "ip_buckets",

    "script": {
        "source": "params.uniq > 2",
        "lang": "painless"
    },
    "gap_policy": "skip"
}

When I click on the preview condition response button, I get:
[
    {},
    {}
]


That is, there is a feeling that the trigger is working correctly. But nevertheless, the alert does not appear, no matter how I change the input data, the viewport, or the count. The alert simply does not work with such a monitor. I'm sending you a screenshot. (No existings alerts.) I have no idea why is it.

среда, 25 февраля 2026 г. в 21:15:35 UTC+3, Isaiah Daboh:

[Isaiah Daboh]

Hello,

The trigger looks fine however no match for the specified condition.

Few troubleshooting steps below:

- Debug the issue from dev tools to see the raw aggregation results directly. This will confirm if there are any buckets at all, and to see if unique_hosts values is up to 2.  params.uniq > 2 will never trigger if there is no unique_host > 2.

From Dev Tools

GET your-index-*/_search
{
  "size": 0,

  "aggregations": {
    "ip_buckets": {
      "terms": {
        "field": "data.win.eventdata.ipAddress",
        "size": 20,
        "min_doc_count": 3
      },
      "aggregations": {
        "unique_hosts": {
          "cardinality": {
            "field": "agent.name"
          }
        }
      }
    }
  }
}

Note: There is no time filter so that we can be sure there are data to process. If the first running was successful, then you cn include the time filter to detect if it is a time filter issue, hence the time can be increased.

Other thing to check:

- Change the Index from wazuh-alerts-current to wazuh-alerts-*

- Lower the threshold temporarily to confirm the trigger mechanism works. "source": "params.uniq > 0"

Let me know if the outlined steps works for you.

Regards,

[никита какдела]

I got this response:

{
  "took": 37,
  "timed_out": false,
  "_shards": {
    "total": 3,
    "successful": 3,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 10000,
      "relation": "gte"
    },
    "max_score": null,
    "hits": []
  },
  "aggregations": {
    "ip_buckets": {
      "doc_count_error_upper_bound": 1119,
      "sum_other_doc_count": 165505,
      "buckets": [
        {
          "key": "10.254.0.160",
          "doc_count": 141924,
          "unique_hosts": {
            "value": 1
          }
        },
        {
          "key": "::ffff:10.110.0.116",
          "doc_count": 19945,
          "unique_hosts": {
            "value": 1
          }
        },
        {
          "key": "10.254.0.173",
          "doc_count": 19240,
          "unique_hosts": {
            "value": 2
          }
        },
        {
          "key": "::ffff:10.254.0.116",
          "doc_count": 17031,
          "unique_hosts": {
            "value": 1
          }
        },
        {
          "key": "::ffff:10.254.0.110",
          "doc_count": 10082,
          "unique_hosts": {
            "value": 1
          }
        },
        {
          "key": "10.115.21.233",
          "doc_count": 8368,
          "unique_hosts": {
            "value": 1
          }
        },
        {
          "key": "::ffff:10.115.22.101",
          "doc_count": 6011,
          "unique_hosts": {
            "value": 1
          }
        },
        {
          "key": "10.3.21.59",
          "doc_count": 5909,
          "unique_hosts": {
            "value": 4
          }
        },
        {
          "key": "10.6.0.96",
          "doc_count": 5718,
          "unique_hosts": {
            "value": 2
          }
        },
        {
          "key": "10.250.0.61",
          "doc_count": 5686,
          "unique_hosts": {
            "value": 4
          }
        },
        {
          "key": "::ffff:10.115.21.130",
          "doc_count": 4732,
          "unique_hosts": {
            "value": 1
          }
        },
        {
          "key": "10.250.20.31",
          "doc_count": 3698,
          "unique_hosts": {
            "value": 5
          }
        },
        {
          "key": "::1",
          "doc_count": 3626,
          "unique_hosts": {
            "value": 46
          }
        },
        {
          "key": "::ffff:10.113.3.113",
          "doc_count": 3064,
          "unique_hosts": {
            "value": 2
          }
        },
        {
          "key": "10.70.1.156",
          "doc_count": 3037,
          "unique_hosts": {
            "value": 4
          }
        },
        {
          "key": "10.70.1.81",
          "doc_count": 2960,
          "unique_hosts": {
            "value": 3
          }
        },
        {
          "key": "10.70.0.158",
          "doc_count": 2826,
          "unique_hosts": {
            "value": 2
          }
        },
        {
          "key": "10.70.1.52",
          "doc_count": 2780,
          "unique_hosts": {
            "value": 3
          }
        },
        {
          "key": "10.30.0.171",
          "doc_count": 2676,
          "unique_hosts": {
            "value": 3
          }
        },
        {
          "key": "10.70.1.177",
          "doc_count": 2667,
          "unique_hosts": {
            "value": 4
          }
        }
      ]
    }
  }
}

My Dev Tools Query:

GET wazuh-alerts-current/_search
{
  "size": 0,
  "query": {
    "range": {
      "@timestamp": {
        "gte": "now-500m",
        "lte": "now"

      }
    }
  },
  "aggregations": {
    "ip_buckets": {
      "terms": {
        "field": "data.win.eventdata.ipAddress",
        "size": 20,
        "min_doc_count": 3
      },
      "aggregations": {
        "unique_hosts": {
          "cardinality": {
            "field": "agent.name"
          }
        }
      }
    }
  }
}

But nevertheless, the alert does not appear, no matter how I change the input data, the viewport, or the count. The alert simply does not work with such a monitor. I'm sending you a screenshot. (No existings alerts.) I have no idea why is it. 

I mean in Per Bucket Monitor. But Dev Tools responsed correctly as you can see. Any thoughts? 

вторник, 3 марта 2026 г. в 21:34:44 UTC+3, Isaiah Daboh:

[Isaiah Daboh]

Hello,

Apologies for the delayed response. I was away for a while.

I can see from your Dev tools result that there are buckets that satisfy the condition and should trigger alerts. However, it looks like unique_hosts.value doesn't exist as a bucket path.

Please try the following trigger instead and let me know if that works.


{
  "buckets_path": {
    "uniq": "unique_hosts"

  },
  "parent_bucket_path": "ip_buckets",
  "script": {
    "source": "params.uniq > 2",
    "lang": "painless"
  },
  "gap_policy": "skip"
}

Regards,