AWS AMIs and Launch configuration UserData

[James]

Hi all

This is my first post of (hopefully) many.  I'm James, I'm an IT contractor, and I'm currently working in an infrastructure team that heavily uses Ansible, my background is mainly Java, so I'm less infrastructure and more apps/automation/aws.

So heres my problem.

My client has a load of Ansible playbooks that they use to deploy various microservices in AWS.

At the moment, the playbooks do the following:

• find a "base" AMI
• apply base config - AV, SSHD, yum updates [for the particular environment you are working with, dev, qa, sit etc]
• copy across a java application
• set up the relevant configuration on the java application [for the particular environment you are working with, dev, qa, sit etc, database URLs etc]
• create a launch config
• apply the new AMI to the launch config replacing existing instances

This works fine, *but*, I am looking to refactor this stuff in to a BUILD state and a DEPLOY state, which would go something like this:

BUILD

• find a "base" AMI
• apply AV, SSHD, yum updates WITHOUT any specific configuration
• copy across a java application

DEPLOY

• find the AMI that was created in the BUILD phase
• create a launch configuration for it
• launch the instance and let it pull in its environment specific config (dev, qa etc, database connection config, satellite service URLs etc)

On previous projects (albeit without the use of Ansible) I have achieved the DEPLOY phase using the UserData config in the AWS LaunchConfiguration.

Can anyone advise whether they have done anything similar using Ansible?

I thought I could pull the Ansible code on to the instance at boot and run Ansible to configure its specific settings, but this does sound a little messy.  It would also mean trying to work out BUILD and DEPLOY steps in the current Ansible tasks, which could be a pain.

Anyway - all thoughts and assistance gratefully received.

Thanks

James.

[Stephen Granger]

You can run Ansible from userdata. I've done this by installing Ansible locally, having the Ansible repo copied over from an s3 bucket and running Ansible locally(-c local) inside userdata. The first playbook, bootstrap.yml, inspects the tags(ec2_tag) that have been applied to the instance and then adds (add_host)the instance to a dynamic inventory host based on those tags. The site.yml is included in this boot strap playbook which then applies all of the playbooks and roles.

Only the playbooks that host patterns match the dynamic group (from the tags) are applied to the instance.

The autoscalling group launch configuration can template the userdata script to define what inventory to use.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-project+unsubscribe@googlegroups.com.
To post to this group, send email to ansible-project@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/7b055117-8031-4986-8905-885d896bbd26%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Steve

[James]

Hi Stephen

Thanks for the info.

So in your case, do you install dependencies on an AMI and then run Ansible locally to setup environment specific variables etc, or do you run the whole build on startup via the Launch config?

I'm trying to get away from having builds per environment, as currently they do a yum update every time something is built, so obviously what gets tested in QA may not be the same as what gets deployed in prod.

Cheers

James.

To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/7b055117-8031-4986-8905-885d896bbd26%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Steve

[Stephen Granger]

Sorry for the late reply, only just saw this.

It's possible to freshly install ansible via the userdata script. I do template the userdata script, typically in the launch configuration for an Auto Scaling Group, based on the inventory(i.e. environment, dev, stage, prod) to be used. My userdata template looks something like this

```
ansible-playbook -i {{ build_environment }} -c local bootstrap.yml

```

No pre-built AMI, all of the environments are handled by the inventory group_vars so it's possible to implement the same roles etc. across various platforms.

You don't need to do a yum update, I'd recommend installing ansible via pip so you can get the same version across each build. You may want to specify the versions for other software too.

This leads into other issues(conventions) of how you address overall updates for your instances. Do you update instances? Or is everything immutable? i.e. you destroy the instances each time a change taken place.

To unsubscribe from this group and stop receiving emails from it, send an email to ansible-project+unsubscribe@googlegroups.com.
To post to this group, send email to ansible-project@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/a8550bb1-8832-4d6c-920c-8d64b1a0d75c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Steve

[James]

Hi Stephen

Everything we have is immutable/throw away.  I see what you are saying re having ansible run to bootstrap the instance.

As it stands the playbooks we have use quite a lot of templates to configure the environment specific configuration.  One option would be run the specific configuration templates on startup using a userdata template similar to yours below.  I also considered generating the environment specific templates, copying them to S3 and then pulling those configs to the instance on startup.

Lots of options to try out...

James.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/a8550bb1-8832-4d6c-920c-8d64b1a0d75c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Steve