Delay in AWS log fetching on services without even high EPS - "aws-s3" woodle

[CJK]

Hi Team,

I have found some issues with my wazuh "aws-s3" woodle integration.I have configured 16 buckets and 3 services. Buckets includes VPC,WAF,Guardduty,cloudtrail etc. And 3 services includes 2 cloudwatch and 1 inspector profile. 

The issue is i have conigured "interval 5m" in config but there is a delay around 15 minutes to complete log fetching mostly from these 3 services which have only 1 EPS. Where from all the buckets it complete in 2 minutes with around 450 EPS. 

on ossec log i have this warning : "wazuh-modulesd:aws-s3: WARNING: Interval overtaken." 

Can someone help me to reduce this delay on log fetcing from service like inspector and cloudwatch? Even without much logs why this is talking around 15 -20 minutes to complete? 

And is it possible to give seperate interval for bucket & service to continue logs from bucket at every 5minutes? And i found as per the below link it says "next run will not start until the previous one is finished."
https://documentation.wazuh.com/current/cloud-security/amazon/services/troubleshooting.html#interval-overtaken-message-is-present-in-the-log-file

Wazuh version: 4.3.11
Config and logs sample ref attached.

Thanks in advance

Clint

 

[Benjamin Nworah]

Hello CJK,

This behavior is expected when using the aws-s3 wodle. Buckets and services are processed sequentially within a single execution, and a new run will not start until the previous one has fully completed. While S3 buckets are generally fast to process, CloudWatch and Inspector are API-based services and are significantly slower, even with very low EPS. As a result, the total execution time can exceed the configured 5-minute interval, triggering the “Interval overtaken” warning.

It is not possible to assign separate intervals for buckets and services within the same aws-s3 wodle configuration. To reduce delays, it is recommended to split buckets and services into separate aws-s3 configurations, keeping S3 buckets on a 5-minute interval and increasing the interval for CloudWatch and Inspector (for example, to 15–20 minutes).

It is also important to note that the interval setting does not control how fast Wazuh pulls logs from a bucket or service. Instead, it defines the minimum amount of time Wazuh waits before attempting the next execution. If a log collection cycle takes longer than the configured interval, the next run is delayed until the current one completes, which results in the “Interval overtaken” warning.

https://documentation.wazuh.com/current/user-manual/reference/ossec-conf/wodle-s3.html#interval

[CJK]

Hi Benjamin,

To reduce delays, it is recommended to split buckets and services into separate aws-s3 configurations, keeping S3 buckets on a 5-minute interval and increasing the interval for CloudWatch and Inspector (for example, to 15–20 minutes).

Can you help me on this how to do the same? Does this require another manager node or can we do this on the same ossec config?

[Benjamin Nworah]

Hello CK,

You can configure more than one wodle on a single Wazuh manager, as shown below. Please note that this is only a sample configuration; you may need to include additional parameters depending on your setup.

 <wodle name="aws-s3">
  <disabled>no</disabled>
  <interval>5m</interval>

  <bucket type="cloudtrail">
    <name>cloudtrail</name>
  </bucket>
 </wodle>

  <wodle name="aws-s3">
  <disabled>no</disabled>
  <interval>20m</interval>

  <service type="inspector">
    <regions>us-east-1</regions>
  </service>
  </wodle>

[CJK]

Hi Banjamin,

Thanks, That helps. Will try the same. 
Does this multiple wodle support on other wodles like : wodle name="azure-logs" as well?