Calculate sum over multiple events

[Fabio Zuber]

Heya Wazuh team and community

I would like to detect if a user downloads / steals a large amount of data in a short period of time.

For a simple case I would sum up / calculate the total of `bytes_sent` in a ssh access log.

My question is if / how this could be done with wazuh.

Many thanks and BR

Fabio

[Fabio Zuber]

For some context, this is an example log, where I'd like to calculate the total:

{"time_local":"06/Oct/2022:16:41:19 +0000","remote_addr":"123.456.78.10","status": "200","protocol": "TCP","bytes_sent": "1441","bytes_received": "1250","session_time": "1.459"}
{"time_local":"06/Oct/2022:16:42:05 +0000","remote_addr":"123.456.78.10","status": "200","protocol": "TCP","bytes_sent": "1441","bytes_received": "1250","session_time": "1.406"}
{"time_local":"06/Oct/2022:16:42:52 +0000","remote_addr":"123.456.78.10","status": "200","protocol": "TCP","bytes_sent": "1441","bytes_received": "1250","session_time": "1.437"}
{"time_local":"06/Oct/2022:16:43:38 +0000","remote_addr":"123.456.78.10","status": "200","protocol": "TCP","bytes_sent": "1441","bytes_received": "1250","session_time": "1.430"}
{"time_local":"06/Oct/2022:16:44:26 +0000","remote_addr":"123.456.78.10","status": "200","protocol": "TCP","bytes_sent": "1441","bytes_received": "1250","session_time": "1.509"}
{"time_local":"06/Oct/2022:16:45:14 +0000","remote_addr":"123.456.78.10","status": "200","protocol": "TCP","bytes_sent": "1441","bytes_received": "1250","session_time": "1.431"}
{"time_local":"06/Oct/2022:16:46:01 +0000","remote_addr":"123.456.78.10,"status": "200","protocol": "TCP","bytes_sent": "1441","bytes_received": "1250","session_time": "1.440"}
{"time_local":"06/Oct/2022:16:46:47 +0000","remote_addr":"123.456.78.10","status": "200","protocol": "TCP","bytes_sent": "1441","bytes_received": "1250","session_time": "1.405"}

[Juan Nicolás Asselle (Nico Asselle)]

Hi Fabio,

Unfortunately, Wazuh is not capable of accumulating values extracted from certain fields (in this case sum every byte_sent value in certain time windows). 

It's possible to detect several interactions from the same remote_addr in certain windows that involves "byte_sents" (even asking that event has at least N byte_sents, not counting them), but seems tricky because could lead to false positives. If this still help you, please let me know to give you an example

I was wondering if ssh could log scp interaction, but is not available.

[Ranjith Kesavan]

Hello Fabio, 

We use Logstash and Elastalert for this purpose. Logstash is used only to convert the  bytes_sent field to number/long. If you already have this configured in number format, you can ignore logstash. Then use elastalert2 metric aggragation rule to aggragate the sum of bytes_sent over a period and alert if its above the threashold. 

Logstash config if bytes_sent is not in number format: 

1) Add a new field in elasticsearch index in number format. lets call it sent_bytes for example. You can do it from Devtools. 

PUT wazuh*/_mapping/doc
{
  "properties": {
    "sent_bytes": {
      "type":"long"
    }
  }
}

2) Add the following Logstash filter 

filter {

     if bytes_sent {

         mutate {

             copy => { "[data][bytes_sent]" => "sent_bytes" }

             convert => { "sent_bytes" => "integer" }

         }

      }

}

3) As now we have the  sent_bytes field as an integer, you can configure elastalert ( Rule Types and Configuration Options — ElastAlert 2 0.0.1 documentation) for agragation and alerting. You can see the sample rule here elastalert2/example_single_metric_agg.yaml at master · jertel/elastalert2 (github.com).  Have the proper filter and change the metric_agg_type to "sum". 

Here is the working rule from our environment: 

---------------------------------------------------------------------------------------------------------------------------

name: Possible data exfiltration
type: metric_aggregation

########################Query Details################################
index: wazuh-archives*

buffer_time:
  hours: 1

metric_agg_key: sent_bytes
metric_agg_type: sum
query_key: source.ip
max_threshold: 2147483648

filter:
- query:
    query_string:
       query: "decoder.name: fortigate-firewall AND event.action: allow"
include:
  - source.ip
########################Define Alert Details#########################
alert_subject: "Possible data exfiltration to internet from host: <{}>"
alert_subject_args:
  - source.ip

alert_text: |-
  More than 2GB of data uploaded to internet from host {} in last one hour.
  Details:
  Source host: {}
alert_text_args:
- source.ip
- source.ip

alert:
 - email:
      from_addr: "sen...@example.com"
      email: "reci...@example.com"

---------------------------------------------------------------------------------------------------------------------------

Thank you,

Ranjith Kesavan