Monitor all EC2 events via lambda ?

[Harish Navnit]

Hi,

I want to know if it's possible to use cloud custodian to monitor all EC2 events(state/tags/security group changes for eg) via lambda ? 

I'm aware of the `ec2-instance-state` mode of operation in cloud custodian and I'm able to use the same for reliably monitoring state changes. Is the same approach extensible for monitoring attribute changes via some custodian mode somewhere?

On a side note, I took a stab at the source code and I've so far been able to generate a CloudWatch rule with the following event pattern, mapped to a target lambda function.

{ "source": [ "aws.ec2" ] }

The CloudWatch rule seems to get invoked on any EC2 event as expected but I'm noticing some lambda errors in the CloudWatch logs indicating a failure.

/var/task/c7n/ipaddress.py:1116: SyntaxWarning: 'str' object is not callable; perhaps you missed a comma? raise TypeError("%s and %s are not of the same version" (a, b))

Any ideas what might be happening here, anyone?

Cheers, Harish

[Harish Navnit]

Thanks for your inputs Kapil! Just a couple more questions inline.

On 07/03/20 6:22 pm, Kapil Thangavelu wrote:

its always good to start with the why, ec2 the service and ec2 as a vm are two very different things. so you want to monitor state/tags/security changes on all vms or all changes to the security groups and tags across the entire service? I'm going to assume the former.

the ec2-instance-state is a specific event stream documented per cloud watch events as instance state change notification (starting, stopped, pending, etc).

if you want a policy to be invoked in response to api events, use a mode: cloudtrail and specify the api calls your interested in being notified on, you'll need to use the full syntax per the custodian docs (ids, event, source) per each api call.

Great, I was hoping to achieve this via the cloudtrail mode as you mention. I wasn't however able to find the relevant bits of documentation or example policies to carry this out. Can you point me to something?

an alternative is to use mode: config-rule to provision as a config rule, the latency is higher (minutes vs seconds), but its simpler configuration if your just looking for some sort of catch all changes to instances.

Likewise here, any examples ?

the ipaddress warning isn't material, I think its todo with python3.8

--
You received this message because you are subscribed to the Google Groups "cloud-custodian" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-custodi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-custodian/0fc0d274-dd83-40f6-a1bd-cdeda05ee8c4%40googlegroups.com.

[Harish Navnit]

Ok, I figured the policy should look something like below

policies:
  - name: ec2-attribute-relay
    resource: aws.ec2
    mode:
      type: cloudtrail
      role: arn:aws:iam::account:role/lambda-role
      events:
        - source: ec2.amazonaws.com
          event: CreateTags
          ids: "requestParameters.groupId"
        - source: ec2.amazonaws.com
          event: DeleteTags
          ids: "requestParameters.groupId"
    actions:
      - type: invoke-lambda
        function: my-lambda-function

Sure enough, the event gets logged in the associated cloud watch logs but the custom lambda function("my-lambda-function") doesn't seem to get invoked. I'll investigate this further.

Is there anything I'm missing in the above policy?

Cheers,
Harish

[Harish Navnit]

On Monday, 9 March 2020 13:05:21 UTC+5:30, Harish Navnit wrote:

Ok, I figured the policy should look something like below

policies:
  - name: ec2-attribute-relay
    resource: aws.ec2
    mode:
      type: cloudtrail
      role: arn:aws:iam::account:role/lambda-role
      events:
        - source: ec2.amazonaws.com
          event: CreateTags
          ids: "requestParameters.groupId"
        - source: ec2.amazonaws.com
          event: DeleteTags
          ids: "requestParameters.groupId"
    actions:
      - type: invoke-lambda
        function: my-lambda-function

Sure enough, the event gets logged in the associated cloud watch logs but the custom lambda function("my-lambda-function") doesn't seem to get invoked. I'll investigate this further.

Just filed a bug report for this. Being tracked over at https://github.com/cloud-custodian/cloud-custodian/issues/5461

To unsubscribe from this group and stop receiving emails from it, send an email to cloud-custodian+unsubscribe@googlegroups.com.