Cannot see alerts created by AWS in alerts.json

[Anish]

Wazuh Version: 3.6.1

Wazuh Wodle: aws-s3

Operating system: Ubuntu 14.04

Node type: Wazuh Manager

Hello everyone,

Its a pretty strange behaviour I observe in the Wazuh AWS wodle. I have setup the aws-s3 wodle using a very basic configuration.

<wodle name="aws-s3">

    <disabled>no</disabled>

    <interval>10m</interval>

    <run_on_start>yes</run_on_start>

    <skip_on_error>no</skip_on_error>

    <remove_from_bucket>no</remove_from_bucket>

    <bucket type="cloudtrail">

      <name>redacted</name>

      <only_logs_after>2018-AUG-31</only_logs_after>

    </bucket>

</wodle>

I am using Instance IAM Roles to perform the authentication. 

When running modules.d in debug mode, it is seen that the AWS python script is successfully reading logs from the CloudTrail bucket without any errors and forwarding them to the ossec queue.

However, the AWS events from 1st September (eg: ConsoleLogin, TerminateInstance, etc) are not visible in alerts.json. When I looked up the bucket in S3 manually, I found that these events do exist there.

The ruleset for AWS is the default one coming with Wazuh 3.6.1. 

The Wazuh setup I have is a default one which comes with wazuh-ansible. 

Is there some configuration that I am missing here?

[Marta Gómez]

Hi Anish,

You can try running the python script that fetches logs directly, 

# /var/ossec/wodles/aws/aws-s3 --bucket redacted --type cloudtrail --debug 3 --only_logs_after 2018-AUG-31

Please check the events you're looking for are parsed and if there is any error during the process. You can also use --reparse option which will reparse already processed events.

Best regards,

Marta

[Anish]

Hello Marta,

After running the command, I see that a lot of CloudTrail JSONs are printed on the console. There are no messages that imply that an error has occurred parsing any of the rules. Also, for an example, I can one of the logs display as:

{"aws": {"eventVersion": "1.05", "eventID": "", "eventTime": "2018-09-05T04:36:23Z", "aws_account_id": "", "additionalEventData": {"MFAUsed": "Yes", "LoginTo": "", "MobileVersion": ""}, "source_ip_address": "", "eventType": "AwsConsoleSignIn", "responseElements": {"ConsoleLogin": "Success"}, "awsRegion": "eu-west-3", "eventName": "ConsoleLogin", "source": "cloudtrail", "userIdentity": {"userName": "", "type": "IAMUser", "": "", "principalId": "", "accountId": ""}, "eventSource": "signin.aws.amazon.com", "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36", "log_info": {"s3bucket": "redacted", "aws_account_alias": "", "log_file": "somefile.json.gz"}, "sourceIPAddress": "", "recipientAccountId": ""}, "integration": "aws"}

As seen in in the log above, AWS CloudTrail eventName was ConsoleLogin, typically matches the rules 80200(aws.eventSource exists), 800202 (eventSource is signin.aws.amazon.com), and 80203 (eventType: ConsoleLogin). However, I don't see a level 4 alert generated. (https://github.com/wazuh/wazuh-ruleset/blob/master/rules/0350-amazon_rules.xml) in alerts.json. Is there anywhere else I should look for more information?

[Marta Gómez]

Hi Anish,

You can try putting the event you've posted (or any other) into logtest and check if it matches any rule:

# /var/ossec/bin/ossec-logtest

In this case, the above event matches the following:

**Phase 3: Completed filtering (rules).

       Rule id: '80200'

       Level: '0'

       Description: 'Amazon cloudtrail alerts.'

Since it has matched a rule with level 0, no alert will be raised. Why has this happened? It seems the field  aws.eventSource: 'signin.aws.amazon.com' is not present in our CDB list to match event sources (https://github.com/wazuh/wazuh/blob/master/etc/lists/amazon/aws-sources) so rule 80201 is not being matched.

You can try updating that CDB list and recompiling it (https://documentation.wazuh.com/current/user-manual/ruleset/cdb-list.html):

# echo signin.aws.amazon.com: >> /var/ossec/etc/lists/amazon/aws-sources
# /var/ossec/bin/ossec-makelists

And running again the script with the option --reparse.

Best regards,

Marta

[Anish]

Hello Marta,

Thank you for guiding me on the process of debugging rules.

I did some more tests, but I am not getting the expected results. Here is another example of a Cloudtrail log RunInstances.

**Phase 2: Completed decoding.

       decoder: 'json'

       aws.eventVersion: '1.05'

       aws.eventID: ''

       ...aws.responseElements....

       aws.eventType: 'AwsApiCall'

       aws.source_ip_address: ''

       aws.eventSource: 'ec2.amazonaws.com' #matches eventSource in https://github.com/wazuh/wazuh/blob/master/etc/lists/amazon/aws-sources

       aws.requestID: ''

       aws.userAgent: 'Boto/2.48.0 Python/2.7.15 Darwin/17.7.0'

       aws.aws_account_id: ''

       aws.recipientAccountId: ''

       aws.eventTime: ''

       aws.requestParameters.userData: '<sensitiveDataRemoved>'

       aws.requestParameters.instancesSet.items.minCount: '1'

       aws.requestParameters.instancesSet.items.maxCount: '1'

       aws.requestParameters.instancesSet.items.keyName: ''

       aws.requestParameters.instancesSet.items.imageId: ''

       aws.requestParameters.instanceInitiatedShutdownBehavior: 'stop'

       aws.requestParameters.ebsOptimized: 'true'

       aws.requestParameters.tenancy: 'default'

       aws.requestParameters.subnetId: ''

       aws.requestParameters.monitoring.enabled: 'false'

       aws.requestParameters.disableApiTermination: 'false'

       aws.requestParameters.instanceType: 'm4.2xlarge'

       aws.requestParameters.iamInstanceProfile.name: ''

       aws.awsRegion: 'us-east-1'

       aws.eventName: 'RunInstances' #matches eventName https://github.com/wazuh/wazuh/blob/master/etc/lists/amazon/aws-eventnames#L61

       aws.source: 'cloudtrail'

       aws.userIdentity.userName: ''

       aws.userIdentity.principalId: ''

       aws.userIdentity.accessKeyId: ''

       aws.userIdentity.type: 'IAMUser'

       aws.userIdentity.arn: ''

       aws.userIdentity.accountId: ''

       aws.log_info.s3bucket: ''

       aws.log_info.aws_account_alias: ''

       aws.log_info.log_file: '20180909T0225Z.json.gz'

       aws.sourceIPAddress: ''

       integration: 'aws'

**Phase 3: Completed filtering (rules).

       Rule id: '80200'

       Level: '0'

       Description: 'Amazon cloudtrail alerts.'

So, the above log should've been classified at least a level 3 by the rule 80202 https://github.com/wazuh/wazuh/blob/master/etc/rules/0350-amazon_rules.xml#L28
But, the completed filtering only shows a match with RuleId 80200. 

Also, the event did not appear in alerts.json after I ran the script with --reparse option.

Am I getting something wrong here?

[Jeremy Phillips]

Hi Marta,

Is there any specific purpose in filtering based upon the eventSource in 80201, to then filter again based upon eventName in 80202?

I would think it would be more intuitive to use a single CDB in 80202 against the eventName, and then replace 80201 with a generic filter to catch all CloudTrail events.  Something like:

<!-- Filter 2: Only CloudTrail events -->

<rule id="80201" level="0">

        <if_sid>80200</if_sid>

        <field name="aws.source">cloudtrail</field>

        <description>Amazon CloudTrail event</description>

</rule>

Thanks,

Jeremy

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To post to this group, send email to wa...@googlegroups.com.
Visit this group at https://groups.google.com/group/wazuh.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/9e16b7c9-6591-48a8-a1d3-273b797a1f1a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jesus Linares]

Jeremy,

We always have a generic rule to catch all the events but no alert (level 0). In this case:

    <rule id="80200" level="0">
        <decoded_as>json</decoded_as>
        <field name="aws.eventSource">\.+</field>
        <description>Amazon $(aws.source) alerts.</description>
    </rule>

We should change this to <field name="integration">aws</field>.

Then, we apply 2 filters: Event source and Event name. It is a way to get only the events specified by the user since AWS could be very noisy.

 
    <!-- Filter 2: Only eventSource in etc/lists/amazon/aws-sources -->

    <rule id="80201" level="0">
        <if_sid>80200</if_sid>

        <list field="aws.eventSource" lookup="match_key">etc/lists/amazon/aws-sources</list>
        <description>Amazon: $(aws.eventSource).</description>
    </rule>


    <!-- Filter 3: Only eventName in etc/lists/amazon/aws-eventnames -->
    <rule id="80202" level="3">
        <if_sid>80201</if_sid>
        <list field="aws.eventName" lookup="match_key">etc/lists/amazon/aws-eventnames</list>
        <description>Amazon: $(aws.eventSource) - $(aws.eventName).</description>
        <group>pci_dss_10.6.1,gdpr_IV_35.7.d,</group>
    </rule>

This was designed when we only pull the Cloudtrail logs. I think you are right. I created this issue: https://github.com/wazuh/wazuh-ruleset/issues/194.

Anish, can you share the raw log of your last event?.

Thanks,

To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+unsubscribe@googlegroups.com.

[Anish Mashankar]

Hello everyone,

I spend some time trying to understand how CDB lists in ossec work. According to https://documentation.wazuh.com/2.0/user-manual/ruleset/cdb-list.html, a list is only used by ossec when it is mentioned under ossec.conf's <ruleset> stanza. 

So, I added the two lists etc/lists/amazon/aws-sources and etc/lists/amazon/aws-eventnames to the configuration.

<ruleset>
    <!-- Default ruleset -->
    <decoder_dir>ruleset/decoders</decoder_dir>
    <rule_dir>ruleset/rules</rule_dir>
     <list>etc/lists/amazon/aws-sources</list>
     <list>etc/lists/amazon/aws-eventnames</list>

Then, I ran the ossec-logtest again, and was finally got it an expected result!

**Phase 3: Completed filtering (rules).

       Rule id: '80202'

       Level: '3'

       Description: 'Amazon: ec2.amazonaws.com - RunInstances.'

I couldn't find this under the documentation for AWS Wodle Plugin Configuration https://documentation.wazuh.com/current/amazon/installation.html#plugin-configuration

Just curious to know: Can the AWS Wodle startup do this automatically? Or this will be manual configuration?

Thanks

[Jesus Linares]

Hi Anish,

The <list> setting for AWS should be included in ossec.conf by default since v3.x. Anyway, we will add it to the documentation. 

Just curious to know: Can the AWS Wodle startup do this automatically? Or this will be manual configuration?

It is a manual configuration although it is in place by default.

I'm glad you fixed it. Let us know if you have more questions.

Regards,

Jesus Linares