Script/How to export from ec2_group_facts and import into ec2_group? AWS

[Simon Tyler]

Dear Citizens of Ansible Project,

I may have a need to export large numbers of security groups from VPCs and import them into new VPCs; basically I want to clone all the security groups from one VPC, and import them to another. 

Naturally, I thought of doing this using Ansible. 

I have found scripts that dump ec2_group_facts or similar and migrate them to new regions for example: https://github.com/Suncatcher/aws_sg_migrate

To my mind, it would be preferable to have a copy of the security groups in ec2_group format, so that I can use Ansible to make the changes. Going forward, people who edit the Ansible ec2_group rules should commit changes to Git first, so we can track who makes changes to security rules for auditing purposes. It seems preferable to use Ansible as tool of choice, but maybe there is a better way? My initial impression is that it makes sense to maintain Ansible playbooks using ec2_group to maintain Security Groups.

There are a fairly large number of Security Groups that need to be maintained, so if there's a script out there that exports from ec2_group_facts and imports into ec2_group it would save a lot of time

Ec2_group_facts output looks like this:

===============================================

    "security_groups": [

        {

            "description": "SecGrp for DB  server", 

            "group_id": "sg-1234567a", 

            "group_name": "DBServer-SecGrp", 

            "ip_permissions": [

                {

                    "from_port": 1433, 

                    "ip_protocol": "tcp", 

                    "ip_ranges": [

                        {

                            "cidr_ip": "x.x.x.x/xx"

                        }

                    ], 

                    "ipv6_ranges": [], 

                    "prefix_list_ids": [], 

                    "to_port": 1433, 

                    "user_id_group_pairs": []

                }, 

                {

                    "from_port": 80, 

                    "ip_protocol": "tcp", 

                    "ip_ranges": [

                        {

                            "cidr_ip": "x.x.x.x/xx"

                        }, 

                        {

                            "cidr_ip": "x.x.x.x/xx"

                        }

                    ], 

                    "ipv6_ranges": [], 

                    "prefix_list_ids": [], 

                    "to_port": 80, 

                    "user_id_group_pairs": []

                }, 

                {

                    "from_port": 0, 

                    "ip_protocol": "tcp", 

                    "ip_ranges": [

                        {

                            "cidr_ip": "x.x.x.x/xx"

                        }

                    ], 

                    "ipv6_ranges": [], 

                    "prefix_list_ids": [], 

                    "to_port": 65535, 

                    "user_id_group_pairs": []

                }, 

===================================================

# tasks file for security_groups2

- name: AWS SG - New Environment

  ec2_group:

      name: DBServer-SecGrp

      description: SecGrp for DB Server

      vpc_id: "{{ vpc_id }}"

      state: present

      region: us-east-1

      rules:

         - proto: tcp

           ports:  

   - 1443

           cidr_ip: x.x.x.x/xx

- proto: tcp

   ports: 

   - 80

   cidr_ip: x.x.x.x/xx

- proto: tcp 

   ports: 

   - 80

   cidr_ip: x.x.x.x/xx

- proto: tcp

   from_port: 0

   to_port: 65535

[Karl Auer]

Hullo Simon

Did you ever figure this out?

It's a mystery to my why the writer of the ec2_group_facts module would generate output that could not be used as input for ec2_group...

Regards, K.

[Simon Tyler]

Hi Karl,

It does seem a bit odd to me. I ended up using Vim search and replace functions to make the changes, and did some of the work manually. On a related note, I discovered two things that may be useful to people faced with a similar task: you can insert multiple ports in a rule, as a list, one after another instead of having a new rule for each port; when you export the rules, it exports each port as it's own rule, which makes for a lot of unnecessary work when editing. Also, there is a clone option in AWS GUI which I didn't realize; this might also come in handy. In the end what I noticed though is that once you get familiar with the Ansible ec2_group syntax, it's often faster to make the change to the script then through the GUI specifically because it is possible to insert multiple ports in one rule, it's like you can collapse large numbers of rules into one rule with a list of ports; if you make changes to security groups through the GUI, it seems to me that you have to make a new rule for each port.  

Onwards,

--
You received this message because you are subscribed to a topic in the Google Groups "Ansible Project" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ansible-project/0YNJLxc5t8U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ansible-project+unsubscribe@googlegroups.com.
To post to this group, send email to ansible-project@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/52fb81f8-ce18-445c-8c62-a56739db8a6a%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Karl Auer]

Yes - there's a relatively new syntax in Ansible that allows source groups of ports, ranges and other groups.

I need to do my changes dynamically - I can't take code, edit it manually, then run something, I need to be able to grab what's there, add a rule and write it back.

Having looked at the two modules, it looks as if modifying ec2_group_facts to generate group-compatible output would be quite hard. It's a very simple module that essentially returns what the AWS API returns. But it looks as if it would be less difficult to make ec2_group accept that output, and it would be able to discriminate between the two formats quite easily. At the moment it validates one format; just validate that, if it fails validate the other, and if that fails, complain.

I hope someone fixes this. Neither your nor my issues can be uncommon...

Regards, K.

.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-project+unsubscribe@googlegroups.com.

To post to this group, send email to ansible-project@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CADiFY2f4k4Tsd-K46ouC1zz_cC2ruABMSAD%3DXN-3V9cj0KyFPw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Karl Auer

Email  : ka...@2pisoftware.com
Website: http://2pisoftware.com

GPG/PGP : 958A 2647 6C44 D376 3D63 86A5 FFB2 20BC 0257 5816
Previous: F0AB 6C70 A49D 1927 6E05 81E7 AD95 268F 2AB6 40EA

[flowerysong]

On Monday, June 4, 2018 at 11:22:29 AM UTC-4, Karl Auer wrote:

I need to do my changes dynamically - I can't take code, edit it manually, then run something, I need to be able to grab what's there, add a rule and write it back.

You don't need to include the existing rules in your task; if you set purge_rules to false the current rules will be preserved.

[Karl Auer]

Well - dang.

If that's all I need, you just saved me a whole heap of time. Thanks!

But I still think that a facts module should provide facts that can be used as input fo create/modify modules :-)

Regards, K.

--

You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-project+unsubscribe@googlegroups.com.
To post to this group, send email to ansible-project@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/e172a981-47b6-4c96-8364-9e08f460fec4%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.