AWS VPC Flow Log Issue

[david.p...@gmail.com]

Good afternoon,

I have set Wazuh up to monitor some VPC flow logs.  Access to the logs seems to be good, but I never see any data in the AWS section of the Wazuh gui.

I ran the rule test script a little bit ago, and I think I know why - it looks like the rules for Squid are firing:

2020/12/31 18:51:35 ossec-testrule: INFO: Started (pid: 9087).
ossec-testrule: Type one log per line.

2 652845508278 eni-0884a2d29b0c3b2f7 10.133.0.152 10.129.1.95 37066 514 6 6 360 1608681257 1608681317 REJECT OK


**Phase 1: Completed pre-decoding.
       full event: '2 652845508278 eni-0884a2d29b0c3b2f7 10.133.0.152 10.129.1.95 37066 514 6 6 360 1608681257 1608681317 REJECT OK'
       timestamp: '(null)'
       hostname: 'ip-10-129-1-198'
       program_name: '(null)'
       log: '2 652845508278 eni-0884a2d29b0c3b2f7 10.133.0.152 10.129.1.95 37066 514 6 6 360 1608681257 1608681317 REJECT OK'

**Phase 2: Completed decoding.
       decoder: 'squid-accesslog'

**Phase 3: Completed filtering (rules).
       Rule id: '35000'
       Level: '0'
       Description: 'Squid messages grouped.

Am I correct in thinking that I need to write a decoder for the flow log?

[Alexander Bohorquez]

Hello David,

Thanks for using Wazuh!

Taking the log you share as a reference. Alerts are not being generated because they match a rule with level "0". Alerts will only be generated if they are higher than level 3.

On the other hand, the log you share matches with a Squid decoder. There are already default decoders for AWS and each of its services. Therefore, it is not necessary to generate custom decoders for these logs.

Testing with the log you share. I'm not getting the same results when using ossec-logtest. Could you share different lines of the VPC log?

As well as the following:

• Wazuh Manager version
• From Menu > Kibana > Discover > Add a filter "Vpc". This to verify if we are receiving events related to VPC.

Did you follow this guide to configure the monitoring?

Please share the aforementioned and I will be assisting you to identify what the problem would be in this case.

You can hide any private data in the logs that you are going to share.

Regards,
Alexander Bohorquez

[david.p...@gmail.com]

Thank you for getting back to me Alexander.

I did follow the monitoring guide.  I am using Wazuh Manager 3.9.1-1

Kibana under the discover section I do not see a way to add a filter.  I'll research how to do that.

Here are additional flow logs:

2 652845508278 eni-0b9c0c0d9ffda8f2f 10.133.1.10 10.129.1.198 36244 1514 17 1 129 1609804652 1609804653 REJECT OK
2 652845508278 eni-0884a2d29b0c3b2f7 10.133.0.152 10.129.1.198 50168 1514 17 5 613 1609804655 1609804715 REJECT OK
2 652845508278 eni-0884a2d29b0c3b2f7 10.133.0.152 10.129.1.198 49301 1514 17 3 387 1609804655 1609804715 REJECT OK
2 652845508278 eni-0884a2d29b0c3b2f7 10.133.0.152 10.129.1.198 55461 1514 17 1 129 1609804655 1609804715 REJECT OK
2 652845508278 eni-016421ac9343cac68 10.133.0.110 10.129.1.198 37359 1514 17 1 129 1609804665 1609804678 REJECT OK
2 652845508278 eni-0b9c0c0d9ffda8f2f 10.133.1.10 10.129.1.198 45789 1514 17 2 226 1609804673 1609804680 REJECT OK
2 652845508278 eni-0b9c0c0d9ffda8f2f 10.133.1.10 10.129.1.198 45789 1514 17 1 129 1609804685 1609804696 REJECT OK
2 652845508278 eni-0b9c0c0d9ffda8f2f 10.133.1.10 10.129.1.198 60611 1514 17 1 129 1609804685 1609804696 REJECT OK
2 652845508278 eni-0a242255e89f1df3c 10.133.0.239 172.19.0.2 137 137 17 3 234 1609804693 1609804697 REJECT OK
2 652845508278 eni-0b9c0c0d9ffda8f2f 10.133.1.10 10.129.1.198 60611 1514 17 2 242 1609804701 1609804706 REJECT OK
2 652845508278 eni-0b9c0c0d9ffda8f2f 10.133.1.10 10.129.1.198 60611 1514 17 2 226 1609804710 1609804722 REJECT OK
2 652845508278 eni-016421ac9343cac68 10.133.0.110 193.227.197.2 42035 123 17 1 76 1609804719 1609804729 REJECT OK
2 652845508278 eni-0b9c0c0d9ffda8f2f 10.133.1.10 10.129.1.198 40209 1514 17 2 242 1609804727 1609804734 REJECT OK
2 652845508278 eni-0884a2d29b0c3b2f7 10.133.0.152 10.129.1.198 34145 1514 17 5 645 1609804715 1609804775 REJECT OK
2 652845508278 eni-0884a2d29b0c3b2f7 10.133.0.152 10.129.1.198 55461 1514 17 4 516 1609804715 1609804775 REJECT OK
2 652845508278 eni-0884a2d29b0c3b2f7 10.133.0.152 10.129.1.198 43410 1514 17 1 129 1609804715 1609804775 REJECT OK
2 652845508278 eni-016421ac9343cac68 10.133.0.110 10.129.1.198 41868 1514 17 1 129 1609804741 1609804752 REJECT OK
2 652845508278 eni-016421ac9343cac68 10.133.0.110 67.205.162.81 36291 123 17 1 76 1609804741 1609804752 REJECT OK
2 652845508278 eni-0b9c0c0d9ffda8f2f 10.133.1.10 10.129.1.198 40209 1514 17 3 371 1609804737 1609804750 REJECT OK
2 652845508278 eni-0a242255e89f1df3c 10.133.0.239 172.19.0.2 137 137 17 3 234 1609804761 1609804765 REJECT OK

[Alexander Bohorquez]

Hello David,

Sorry for the delay in replying.

In this case, the script to extract the information from AWS, introduces the message in JSON format to the queue. Checking the matches with Amazon rules by looking for specific fields.

In this case, we have to verify if these events are reaching the manager, as follows:

Please enable archives logging in the Wazuh manager.

1. Open /var/ossec/etc/ossec.conf  and set the option logall_json  to "yes".

2. Then, restart the manager to apply changes.

# Systemctl restart wazuh-manager.service

That should be saving all events received into the file /var/ossec/logs/archives/archives.json, even if they do not trigger an alert.

3. If logs are being generated in your bucket, give it some time and check if there are vpcflow alerts there: grep -i vpc /var/ossec/logs/archives/archives.json.

Please indicate what the result is after performing the aforementioned test.

If we do not receive events in the archives.json. We must perform another test to identify if there are any errors when running the AWS script:

1. Restart the manager

# Systemctl restart wazuh-manager.service

2. Check the aws-s3 logs in ossec.log:

# grep aws /var/ossec/logs/ossec.log

3. Please share the output of the last command.

Please let me know if you have any questions.

Regards.

Alexander Bohorquez

[david.p...@gmail.com]

Thanks for getting back to me.  Don't feel bad about the delayed reply, I'm sure like everyone else you have plenty on your plate.

Anyway, we are getting VPC flow data in:

{"timestamp":"2021-01-06T19:24:27.412+0000","rule":{"level":4,"description":"AWS VPC Flow: [REJECT] - Interface: eni-0b9c0c0d9ffda8f2f - Protocol: 17","id":"80402","firedtimes":47,"mail":false,"groups":["amazon","aws","aws_vpcflow"]},"agent":{"id":"000","name":"ip-10-129-1-198.corios.org"},"manager":{"name":"ip-10-129-1-198.corios.org"},"id":"1609961067.273060624","decoder":{"name":"json"},"data":{"integration":"aws","aws":{"log_info":{"aws_account_alias":"UMP_Legato","log_file":"AWSLogs/652845508278/vpcflowlogs/us-west-2/2021/01/06/652845508278_vpcflowlogs_us-west-2_fl-099e8547e909977c0_20210106T1920Z_32d9aa6f.log.gz","s3bucket":"wazuh-cloudtrail-ump-legato"},"version":"2","account_id":"652845508278","interface_id":"eni-0b9c0c0d9ffda8f2f","srcaddr":"10.133.1.10","dstaddr":"10.129.1.198","srcport":"43783","dstport":"1514","protocol":"17","packets":"2","bytes":"242","start":"1609960935","end":"1609960942","action":"REJECT","log_status":"OK","source":"vpc","aws_account_id":"652845508278"}},"location":"Wazuh-AWS"}

While I don't want to get an alert every time  a packet is blocked, is this data supposed to show up in the AWS portion of the Wazuh console?

[Alexander Bohorquez]

Hello David,

In the event you send, it is verified that the Wazuh Manager is processing the events that come from Amazon VPC flow and are matching with alerts. In this case with a level 4 rule alert.

By default, Wazuh will create an alert when a log matches a rule with a level higher or equal to 3.

The alerting rule is the rule with ID "80402". Since the integration is being done from the Manager. The reports will come from the agent with ID "000" which is the Manager by default.

These events can be checked from:

Wazuh Menu> Modules> Security Events.

Or also from:

Kibana> Discover> Searching by "Vpc".
    
Regarding your requirement, if you don't want to see the "Reject" events. What you could do in this case is to lower the level of this rule that Wazuh has by default. At a level less than 3 so that it is not sent to Kibana.

The default rules are located at: /var/ossec/ruleset/rules/. Specifically, those from Amazon in the file "0350-amazon_rules.xml". Rules should not be edited in that location because they are overwritten when you upgrade Wazuh manager or perform a Wazuh Ruleset update.

Custom changes to the ruleset must be done within files in the /var/ossec/etc/rules/ folder. In order to change a default rule, then the overwrite = "yes" option must be used when declaring the rule.

Here I'll explain to you the procedure:

1. Copy the existing rule "80402" from /var/ossec/ruleset/rules/0350-amazon_rules.xml
3. Paste it into /var/ossec/etc/rules/local_rules.xml
4. Change the level to a lower level than 3 and add the tag "overwrite=yes" the rule should look like this:

   <group name="amazon,aws,">
    <rule id="80402" level="2" overwrite="yes">
        <if_sid>80400</if_sid>
        <field name="aws.action">REJECT</field>
        <description>AWS VPC Flow: [$(aws.action)] - Interface: $(aws.interface_id) - Protocol: $(aws.protocol)</description>
        <group>aws_vpcflow,</group>
        <options>no_full_log</options>
    </rule>
  </group>

Make sure to insert it before the closing </group> tag, as all rules must be located inside of a <group> section.

5.After this a restart of the Manager is necessary to save the changes:

Sytemctl restart wazuh-manager.service.

I leave you this guide as a reference:

https://documentation.wazuh.com/3.11/learning-wazuh/replace-stock-rule.html

I hope this helps. Please let me know if you have any other questions!

Regards.

Alexander Bohorquez

[david.p...@gmail.com]

It looks like the issue actually is that no security events are showing up in the console, at all.  I am getting emailed alerts for some other things though.
I'm going to head down that road.  Thank you for your help.

[Alexander Bohorquez]

Hello David,

Sorry for the delay. It seems you aren't indexing data.

First, let's check a few things.

Filebeat is the tool on the Wazuh server that securely forwards alerts and archived events to Elasticsearch. If alerts are being generated in /var/ossec/logs/alerts/alerts.json

1. Please go to your Wazuh Server and run this command:

# filebeat test output

Please share with me the output of the command to check if filebeat is reaching elasticsearch.

2.Check if Filebeat is reading alerts:

# lsof /var/ossec/logs/alerts/alerts.json

The output should look like this:

lsof /var/ossec/logs/alerts/alerts.json
COMMAND    PID  USER   FD   TYPE DEVICE SIZE/OFF  NODE NAME
filebeat   682  root    9r   REG    8,1  4920718 14966 /var/ossec/logs/alerts/alerts.json
ossec-ana 1266 ossec   10w   REG    8,1  4920718 14966 /var/ossec/logs/alerts/alerts.json

Also,

3. Try making a simple request to your Elasticsearch from your Wazuh server to rule out visibility problems:

curl -XGET http://[YOUR_ELASTIC]:9200

If Elasticsearch on the same server as your filebeat you could use: curl -XGET http://localhost:9200

Please share with me the output of both commands.

Regards.

Alexander Bohorquez