How to prevent "queue flooding" and scale log ingestion to 500,000+ events?

[Alex]

Hi everyone,

I am experiencing a persistent "queue flooding" issue on one of my Wazuh agents. To resolve the issue, I modified the <client_buffer> configuration in the agent's ossec.conf file, significantly increasing the queue size and EPS limits:

<client_buffer>
  <disabled>no</disabled>
  <queue_size>100000</queue_size>
  <events_per_second>1000</events_per_second>
</client_buffer>

Despite applying this configuration, I am still getting the queue flooding logs.

My goal here is not to reduce the noise or filter these logs out. I actually want to ingest this massive volume of logs successfully. For example, I need the architecture to be able to capture and process around 500,000 logs without causing any flooding on the agent side or dropping any events.

How can I achieve this? What are the necessary advanced tuning steps on both the agent side and the manager side (such as wazuh-remoted worker pools, internal queues, etc.) to handle this level of log volume seamlessly without triggering flood warnings?

Thanks in advance,

Alex

[Md. Nazmur Sakib]

Hi Alex,

You are getting this error due to the anti-flooding mechanism placed on the agent and your agent is generaating more events than the limit you have configured.

You can disable the event queue if you like from the agent’s ossec.conf.

<client_buffer>

  <disabled>yes</disabled>

  <queue_size>100000</queue_size>

  <events_per_second>1000</events_per_second>

</client_buffer>

Ref: client_buffer

Keep in mind that disabling the client buffer will allow sending bursts of alerts, and the agent loses its shock absorber, so the manager and network must be able to sustain the rate of logs per second.

Configure the Wazuh agent communication queue by editing the <queue_size> in the <remote> block of the /var/ossec/etc/ossec.conf file on the Wazuh server

And adjust the worker_pool size in the /var/ossec/etc/internal_options.conf file.

Agent queuing mechanisms


Keep an eye on the static files to monitor if there are any potential drops of events.

cat /var/ossec/var/run/wazuh-analysisd.state

cat /var/ossec/var/run/wazuh-remoted.state

Let me know if this works for you.

[Alex]

Hi,

Thanks for the suggestions! I applied the configurations you recommended:

• remoted.worker_pool: 16

• <queue_size>262144</queue_size>

• <disabled>yes</disabled>

This method helped a little bit, but when I monitor the queues, I can still see that event drops are occurring.

It looks like wazuh-remoted is successfully handling the load now (the remoted/event queue is at 0%), but the bottleneck has clearly shifted to analysisd. Specifically, the syscollector queue is completely maxed out at 100%, which is causing a massive amount of dropped events.

Here are the statistics I am seeing right now:

Queue usage (all monitored queues, highest first): analysisd/syscollector: 100.00% (size 16384) analysisd/dbsync: 3.00% (size 16384) analysisd/syscheck: 0.00% (size 16384) analysisd/rootcheck: 0.00% (size 16384) analysisd/sca: 0.00% (size 16384) analysisd/hostinfo: 0.00% (size 16384) analysisd/winevt: 0.00% (size 16384) analysisd/upgrade: 0.00% (size 16384) analysisd/event: 0.00% (size 16384) analysisd/rule_matching: 0.00% (size 16384) analysisd/alerts: 0.00% (size 16384) analysisd/firewall: 0.00% (size 16384) analysisd/statistical: 0.00% (size 16384) analysisd/archives: 0.00% (size 16384) remoted/event: 0.00% (size 262144) Data-loss counters that grew since the previous check: analysisd:events_dropped: +117009 (total 117009) Current data-loss counters: analysisd:events_dropped: 117009 remoted:dequeued_after_close: 0 remoted:discarded_count: 0

How can I resolve this new bottleneck in analysisd? Is there a specific parameter in internal_options.conf to increase the syscollector queue size, or any other tuning you would recommend to prevent analysisd from dropping these events?

Thanks again,

Alex

23 Temmuz 2026 Perşembe tarihinde saat 14:07:55 UTC+3 itibarıyla Md. Nazmur Sakib şunları yazdı:

[Md. Nazmur Sakib]

You can increase the analysis engine queue following this doc:
Wazuh analysis engine queue (queue_and)

Go to this file:
/var/ossec/etc/local_internal_options.conf  

And update the specific queue sizes in the analysisd

Keep in mind that increasing this can overwarm your server. In this case, you can add more Wazuh manager nodes to balance the load. Ref: Adding new Wazuh server nodes

Let me know if this solves your issue.

[Alex]

Hi,

I tried what you mentioned, but sometimes events still drop as shown below.

Queue usage (all monitored queues, highest first): analysisd/event: 95.00% (size 2000000) analysisd/syscheck: 0.00% (size 2000000) analysisd/syscollector: 0.00% (size 2000000) analysisd/rootcheck: 0.00% (size 16384) analysisd/sca: 0.00% (size 16384) analysisd/hostinfo: 0.00% (size 16384) analysisd/winevt: 0.00% (size 16384) analysisd/dbsync: 0.00% (size 16384) analysisd/upgrade: 0.00% (size 16384) analysisd/rule_matching: 0.00% (size 16384) analysisd/alerts: 0.00% (size 16384) analysisd/firewall: 0.00% (size 16384) analysisd/statistical: 0.00% (size 16384) analysisd/archives: 0.00% (size 16384)

remoted/event: 0.00% (size 262144) Data-loss counters that grew since the previous check: analysisd:events_dropped: +1326008 (total 1326008) remoted:discarded_count: +20146 (total 20146) Current data-loss counters: analysisd:events_dropped: 1326008 remoted:dequeued_after_close: 0 remoted:discarded_count: 20146

Is there any other setting I can adjust?

Thanks again,

Alex

24 Temmuz 2026 Cuma tarihinde saat 13:43:29 UTC+3 itibarıyla Md. Nazmur Sakib şunları yazdı:

[Md. Nazmur Sakib]

From the
/var/ossec/etc/local_internal_options.conf  

You can increase the
analysisd.event_threads — general event decoders

analysisd.syscollector_threads — relevant to syscollector event decoders


The default value is 0, which sets the number of threads according to the number of CPU cores, and it accepts any integer between 0 and 32.

Even if you increase the threads, if you do not have the physical core count needed, analysis will not be able to process the events.


I highly recommend that you add an additional Wazuh manager node to your Wazuh manager cluster and refer some of the agents to the other nodes in the same wazuh manager cluster.

Adding new Wazuh server nodes

Let me know the update on this issue.

[Alex]

Hi,

Thank you for the recommendations.

Currently, my infrastructure consists of the following resources:

• 3x Manager nodes (8 CPU / 8GB RAM each)

• 1x Indexer node (16 CPU / 32GB RAM)

I have updated the configuration and increased the values for all analysisd components to 2,000,000, but unfortunately, we are still experiencing log drops.

Given this current setup and the ongoing issue, should I scale up this architecture even further? Would you recommend increasing the number of Manager and Indexer servers, or should I take a different approach (such as upgrading the hardware of the existing ones)?

Please let me know what path I should follow to completely resolve these drop issues.

Looking forward to your guidance.

Best regards,

Alex

28 Temmuz 2026 Salı tarihinde saat 09:40:48 UTC+3 itibarıyla Md. Nazmur Sakib şunları yazdı:

[Md. Nazmur Sakib]

Wazuh easily scales horizontally rather than vertically. We recommend adding a new node when you see drops in the number of events. 

Check the document for adding new nodes.

Adding new Wazuh server nodes

[Francisco Sousa]

Before you add a fourth node, there are three numbers already in this thread that decide whether a fourth node helps.

The queues grew and so did the loss. On July 24 the analysisd queues were at 16384 and events_dropped was 117009. On July 27 they were at 2000000 and events_dropped was 1326008. You raised the depth by roughly 122 times and lost about 11 times more events. That is what a consumer slower than its producer looks like: a bigger queue does not add throughput, it adds the amount of work in flight before the same drop happens, and it moves the bottleneck rather than removing it. You watched that happen three times, from remoted to syscollector to event.

Work out what 2,000,000 costs you in RAM. You have 8 GB per manager node, a queue is sized in events, and the memory is your average event size times the depth, across every analysisd queue you raised. If the queues alone want more than the box has, the drops you are seeing now are a different failure from the one you started with in July, and another node does not fix a manager that cannot hold its own configuration.

remoted:discarded_count appeared on July 27 at 20146 and has not been mentioned since. That counter is loss before analysisd. It showed up after you set disabled to yes on the client buffer. The buffer was the only thing rate limiting the agents, so without it they send in bursts and remoted drops what it cannot take. Turning it off did not create capacity, it removed the back pressure that was keeping the shortfall in one place where you could see it.

The number nobody has asked for is your actual rate. "500,000 events" has been in this thread since July with no unit on it. At 500,000 per second, three eight core nodes are not close and the answer is a different architecture. At 500,000 per day that is under six per second and none of this should be dropping anything, which would point at configuration rather than capacity. events_dropped growing by 1,326,008 is not a rate either, because the interval between the two checks is not in the thread.

One more thing from your July 24 output, because it went by without comment: analysisd/syscollector was at 100 percent while every other queue sat at 0. Syscollector is inventory, not security events. If inventory is what fills your pipeline, the lever is the scan interval on the agents, and that one is free.

Regards,

Francisco Sousa — Systems in Production, systemsinproduction.substack.com