Stack update and manually terminated resources/instances

[emmanuel...@menlosecurity.com]

I have a cloudformation template with a regular instance (not in an auto-scaling group of 1).

This instance could be at any time terminated by AWS.

To my surprise, if I test this scenario and update the stack, the stack still believes that the instance is there.

(Apparently it checks for the diff between current stack and new stack and only update/create the change-set)

Is there a way to have cloudformation check for the existence of its resources?

I had this issue with manually deleted resources as well.

Is there a way to have AWS automatically recreate the  resources that have been deleted through the console (or by Amazon)

If not, how do you solve this problem?

--

Emmanuel

[Michael Barrett]

I don't believe there is a way to do this - if you want an instance to come up, no matter what happens behind the scenes, you should use an Autoscaling group.  That's the only way I've found to make this full proof.

--
You received this message because you are subscribed to the Google Groups "cloudtools-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloudtools-de...@googlegroups.com.
To post to this group, send email to cloudto...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloudtools-dev/dacb31e2-2387-49f3-9a17-fd621fccff35%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Emmanuel Mayssat]

I understand the benefit of an autoscaling group of 1.
Auto scaling groups are only valid for EC2 instance.
Now, let's assume that a health check has been deleted manually.
My stack is out of sync (still believes that the health check exists)
How can I restore this health check?

If I do it manually, the stack will reference another health check id.
Therefore the stack deletion will not remove this resource.

What I am asking for is not for something automatic.
If someone delete a DNS entry, a machine can become unreachable.
The devops team is contacted, notice the issue, ...
does a stack update to restore all the resources that have been externally deleted.

It seems that a stack update could quickly solve this issue.
(I was hoping for a just a command line flag to tell cloudformation
to check for the existence of the stack resources at update. If a few do
not exist, they are recreated according to the content of the template.)


--
Emmanuel

> > <https://groups.google.com/d/msgid/cloudtools-dev/dacb31e2-2387-49f3-9a17-fd621fccff35%40googlegroups.com?utm_medium=email&utm_source=footer>
> > .

> > For more options, visit https://groups.google.com/d/optout.
> >

--
Emmanuel
Menlo Security, Inc.
Menlo Park, CA

[Michael Barrett]

It's been a while since I've dealt with this, but in the past I think I've handled it one of three ways:

1. Deleting & recreating the entire stack (more reason to keep your stacks small and specialized)

2. Delete the resource in question in the template, roll out the stack, then add it back and roll it out once more (gets tricky when you have other resources that depend on this resource, since it  could quickly result in an invalid template)

3. Rename the resource in the template, repoint all dependent resources at the new name, and roll out the template

Again, it's been a while, but I'd likely go with #3 in the case that you're talking about.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloudtools-dev/20170129013342.GA8697%40xps13.

[Michael Osburn]

> 3. Rename the resource in the template, repoint all dependent resources at the new name, and roll out the template

This is the path that the CloudFormation development team recommends if you delete something out of band, but the correct way forward is to use an ASG with the healthcheck for the ASG be ELB not EC2.

Additionally, I have updated stacks before to add/change healthchecks into the ELB as that is not a destructive update so it can be done on the fly. CloudFormation will look for missing resources in the template and make adjustments as needed when does a physical lookup on the resource and compares your current config with the template. 

On Mon, Jan 30, 2017 at 5:23 PM, Michael Barrett <lok...@gmail.com> wrote:

It's been a while since I've dealt with this, but in the past I think I've handled it one of three ways:

1. Deleting & recreating the entire stack (more reason to keep your stacks small and specialized)

2. Delete the resource in question in the template, roll out the stack, then add it back and roll it out once more (gets tricky when you have other resources that depend on this resource, since it  could quickly result in an invalid template)

3. Rename the resource in the template, repoint all dependent resources at the new name, and roll out the template

Again, it's been a while, but I'd likely go with #3 in the case that you're talking about.

> > email to cloudtools-dev+unsubscribe@googlegroups.com.
> > To post to this group, send email to cloudtools-dev@googlegroups.com.

> > To view this discussion on the web visit
> > https://groups.google.com/d/msgid/cloudtools-dev/dacb31e2-2387-49f3-9a17-fd621fccff35%40googlegroups.com
> > <https://groups.google.com/d/msgid/cloudtools-dev/dacb31e2-2387-49f3-9a17-fd621fccff35%40googlegroups.com?utm_medium=email&utm_source=footer>
> > .
> > For more options, visit https://groups.google.com/d/optout.
> >

--
Emmanuel
Menlo Security, Inc.
Menlo Park, CA

--
You received this message because you are subscribed to the Google Groups "cloudtools-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cloudtools-dev+unsubscribe@googlegroups.com.
To post to this group, send email to cloudtools-dev@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloudtools-dev/20170129013342.GA8697%40xps13.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "cloudtools-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to cloudtools-dev+unsubscribe@googlegroups.com.
To post to this group, send email to cloudtools-dev@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloudtools-dev/CAGz0FsY9s3cStFaQXXrjd-UsxHmwf4U30sAZwydG7OaxrVrqBQ%40mail.gmail.com.