ECS Tasks not being updated if the Container changes

[Jay]

Hi,

  We've been using Terraform to do ECS updates for a while, and our standard practice has been to update images, run terraform apply, and let terraform roll a new task id and update the service. Recently, this seems to have stopped working. Terraform doesn't recognize that the image hash changed, and no longer updates the service. My suspicion is that it might have something to do with changes to the API used to describe images (https://aws.amazon.com/about-aws/whats-new/2017/01/amazon-ecr-supports-docker-image-manifest-v2-schema-2/)  Is anyone else experiencing this issue? And better yet, is there a way to force an apply to create a new task and update the service every time it's applied?

-= Jay =-

[Philip Nelson]

I can't say if this worked with terraform before, but with ecs in general it doesn't work that way. ECS will pull a new container if the service is updated with a new task definition, even if the image continues to use a :latest tag. This works with both windows and linux containers. However without a service update, and terraform would not see a change if your image hasn't changed, containers are not stopped, recreated with the image. Putting version number on your updated images, and updating your task definition would always work though. 

[Jay]

     Correct, and this is the only way currently to work around this. We have to update something in the Container definition that forces Terraform to see a difference. (We throw the jenkins build id in as a env variable in the container def) If you use the AWS console, you can just increment the task version, and it does all the updates, even without any changes to the task definition. Terraform did this at one point, where every apply would automatically increment the task, functioning exactly like the ECS consul. It no longer does this though. I guess it should be a feature request to either have a flag to force the version increment, or actually have it be smart enough to check that the image hash has changed for the underlying container and do the right thing.

-= Jay =-

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/terraform/issues
IRC: #terraform-tool on Freenode
---
You received this message because you are subscribed to a topic in the Google Groups "Terraform" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/terraform-tool/2uYz37-2xV0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to terraform-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/terraform-tool/fb393810-0a30-4480-8720-6267f5d967d5%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[gdel...@gmail.com]

Hello Jay,

If you would like to update your container task definition with the image tag updated from your build via Jenkins. You can use the python module below it will deploy your build in a blue/green manner. Make sure you're using Terraform 0.8.6+ so when you update the task definition outside of the terraform state file it gets updated on the next terraform run using the latest task definition for that service.

https://pypi.python.org/pypi/ecs-deploy

https://www.terraform.io/docs/providers/aws/r/ecs_task_definition.html

To unsubscribe from this group and all its topics, send an email to terraform-too...@googlegroups.com.

[Rafael Bernardo]

I cant figure out how you did this , for instance i deployed all from terraform , an I builded new images and i deploy a simple pulling the latest images but if i try run a plan again, they get the last from tstate file 

see the example.

  ~ module.images-ecs-service.aws_ecs_service.main

      task_definition:             "arn:aws:ecs:us-east-2:271590668009:task-definition/prod-images:11" => "arn:aws:ecs:us-east-2:271590668009:task-definition/prod-images:7"

the last active in moment is 11 but the terraform thinks there in 7.

can you help me ?

[gdel...@gmail.com]

If you run an apply does it actually overwrite the task definition with 7 or does it keep it at 11?.. Also are you using Terraform 0.8.6+..

[Rafael Bernardo]

No there really replace when i apply apply and i using the current :

Terraform v0.11.2

+ provider.aws v1.7.1

+ provider.cloudflare v0.1.0

+ provider.template v1.0.0

[Rafael Bernardo]

and if i edit the arn in tfstate all go fine