ec2_group is not idempotent

[Stephen Nelson-Smith]

I have a task which creates a security group:

- name: Create Production Security Group

  ec2_group:

    name: "Production Security Group"

    description: "Allow access on ssh and 8080"

    vpc_id: "{{ prd_vpc_id }}"

    rules:

      - proto: "tcp"

        from_port: "22"

        to_port: "22"

        cidr_ip: "{{ lookup('env', 'ATTACK_IP') }}"

      - proto: "tcp"

        from_port: "8080"

        to_port: "8080"

        cidr_ip: "{{ lookup('env', 'ATTACK_IP') }}"

  register: prd_sg

This works fine, the first time I run it.

If I run it again, I get:

TASK [vpc : Create Production Security Group] ***********************************************************************************************************************************************************************************************

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: botocore.exceptions.ClientError: An error occurred (InvalidPermission.Duplicate) when calling the AuthorizeSecurityGroupIngress operation: the specified rule "peer: 138.68.160.0/20, TCP, from port: 22, to port: 22, ALLOW" already exists

fatal: [localhost]: FAILED! => {"changed": false, "error": {"code": "InvalidPermission.Duplicate", "message": "the specified rule \"peer: 138.68.160.0/20, TCP, from port: 22, to port: 22, ALLOW\" already exists"}, "failed": true, "msg": "Unable to authorize in for ip 138.68.174.135/20 security group 'Production Security Group' - An error occurred (InvalidPermission.Duplicate) when calling the AuthorizeSecurityGroupIngress operation: the specified rule \"peer: 138.68.160.0/20, TCP, from port: 22, to port: 22, ALLOW\" already exists", "response_metadata": {"http_headers": {"connection": "close", "date": "Fri, 01 Sep 2017 18:32:20 GMT", "server": "AmazonEC2", "transfer-encoding": "chunked"}, "http_status_code": 400, "request_id": "c79c01db-78d8-43aa-b779-b204f4685052", "retry_attempts": 0}}

        to retry, use: --limit @/home/ansible/playbook.retry

I looked into this, and it seemed to be the same issue describe in:  https://github.com/ansible/ansible/issues/24476 which is fixed in https://github.com/ansible/ansible/pull/24528

With that in mind, I installed the devel branch of ansible:

$ ansible --version

ansible 2.4.0

  config file = None

  configured module search path = ['/home/ansible/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']

  ansible python module location = /usr/lib/python3.6/site-packages/ansible

  executable location = /usr/bin/ansible

  python version = 3.6.2 (default, Aug 11 2017, 11:59:59) [GCC 7.1.1 20170622 (Red Hat 7.1.1-3)]

And tried again....

But I am having the same issue.  I know that ansible is the 2.4.0 version because I added an assert especially to stop if not >=2.4.0, and the assert passes.

Am I missing something here?  Is there a simpler way to solve this?

S.

[Josh Smift]

Is this the same problem as
https://github.com/ansible/ansible-modules-core/issues/1068 ? It looks
from your output like it might be:

138.68.160.0/20, TCP, from port: 22, to port: 22, ALLOW" already exists
fatal: [localhost]: FAILED! => {"changed": false, "error": {"code":
"InvalidPermission.Duplicate", "message": "the specified rule \"peer:
138.68.160.0/20, TCP, from port: 22, to port: 22, ALLOW\" already exists"},
"failed": true, "msg": "Unable to authorize in for ip 138.68.174.135/20

When you specify 138.68.174.135/20, that gets translated to the arguably
more correct 138.68.160.0/20 when AWS creates the rule, but then Ansible
doesn't do that translation when it checks if the rule already exists --
it looks for 138.68.174.135/20, sees 138.68.160.0/20, doesn't realize
they're equivalent, tries to add the rule, and gets the error.

You might be able to work around this by just canonicalizing the ATTACK_IP
you're trying to allow, before you run Ansible, if that's easy to do.

-Josh (j...@care.com)

(apologies for the automatic corporate disclaimer that follows)
This email is intended for the person(s) to whom it is addressed and may contain information that is PRIVILEGED or CONFIDENTIAL. Any unauthorized use, distribution, copying, or disclosure by any person other than the addressee(s) is strictly prohibited. If you have received this email in error, please notify the sender immediately by return email and delete the message and any attachments from your system.

[Stephen Nelson-Smith]

Hi Josh,

On Sunday, 3 September 2017 13:30:12 UTC+1, Josh Smift wrote:

When you specify 138.68.174.135/20, that gets translated to the arguably
more correct 138.68.160.0/20 when AWS creates the rule, but then Ansible
doesn't do that translation when it checks if the rule already exists --
it looks for 138.68.174.135/20, sees 138.68.160.0/20, doesn't realize
they're equivalent, tries to add the rule, and gets the error.

Good spot - that's exactly the problem, and stems from the fact that the security group constrains access at subnet level, and has no deeper granularity.  If I want to restrict access to a single IP in a subnet, I need to do that with a NACL.

In my mind I was thinking that if I specify the IP of the machine I wish to allow, in CIDR notation, the security group would be configured to allow that IP/Netmask, without noticing that in fact it translates it to the network within which the IP sits.

Whether Ansible's behaviour is correct (I believe it isn't) is another question, but for my use case, I can just use the network in the security group and then overlay with the ACL.

Thanks!

S.