aws_ami most_recent = true doesn't works?

[setevoy]

Hi.

We have a Terrafrom job in Jenkins which must update AWS AutoScaling group's launch config with AMI ID build by Packer in previous Jenkins's stage.

The problem we have sometimes is that Terraform uses not latest AMI: usually, it works fine but sometimes it takes AMI from old (previous or even a week ago) builds.

The data source looks like:

data "aws_ami" "api_blue" {
 most_recent = true
 owners = ["${lookup(var.ami_owners, var.region)}"]
 filter {
   name = "name"
   values = ["tag-api*"]
 }
}

Jenkin's log from Packer stage:

...

[apiAmi] --> amazon-ebs: AMIs were created:
[apiAmi] 
[apiAmi] eu-west-1: ami-e628569f

...

And next is Terraform:

...

 [0m [1mmodule.api_launch_configuration_blue.aws_launch_configuration.lc: Creating... [0m
  associate_public_ip_address: "" => "false"
  ebs_block_device.#:          "" => "<computed>"
  ebs_optimized:               "" => "<computed>"
  enable_monitoring:           "" => "true"
  image_id:                    "" => "ami-78b6c901"
...

But the ID ami-78b6c901 is previous build from Jenkins - not the latest AMI from Packer == ami-e628569f.

What can be wrong here?

Thanks.

[⁞ Fernando Miguel]

Add latest to the filter 

--
Fernando Miguel

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/terraform/issues
IRC: #terraform-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Terraform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to terraform-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/terraform-tool/15bdea93-631d-449c-9d73-49cad73fb809%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[setevoy]

Thanks, Fernando.

Can I ask you please how the "latest" filter can be added? All solutions I can google are only "most_recent == true" and can't see nothing like this in Terraform's documentation.

Is it something like:

 filter { 

   name = "creationDate" 

   values = ["latest"] 

 } 

or:

 filter { 

   latest = true

 } 

?

To unsubscribe from this group and stop receiving emails from it, send an email to terraform-too...@googlegroups.com.

[⁞ Fernando Miguel]

Based on https://www.terraform.io/docs/providers/aws/d/ami.html 

data "aws_ami" "nat_ami" {
  most_recent      = true

most_recent should give you want you need.

if you follow CLI

https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-images.html

do you get the latest AMI  ? or older ones too?

-- 

Fernando Miguel

https://FernandoMiguel.net

To unsubscribe from this group and stop receiving emails from it, send an email to terraform-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/terraform-tool/96e0b258-df07-466c-a969-ce15c43e68fb%40googlegroups.com.

[setevoy]

The exact problem is is that we have newer AMI created by Packer in the same Jenkins build:

[apiAmi] eu-west-1: ami-e628569f

But Terraform takes AMI from the previous build:

image_id:                    "" => "ami-78b6c901"

Using most_recent = true.

Here they are, sorted by date:

And the AMI's names include Jenkin's build numbers (491 and 492).

And the weirdest thing is that usually, it works as expected, but sometimes - it stuck with that (bug?).

[⁞ Fernando Miguel]

can you reproduce the issue via aws cli?

-- 

Fernando Miguel

https://FernandoMiguel.net

To unsubscribe from this group and stop receiving emails from it, send an email to terraform-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/terraform-tool/08fcc563-4b64-4b44-8540-8dd4aa72ae01%40googlegroups.com.

[setevoy]

Not sure how this can be reproduced with CLI as it hasn't "SortBy" filters/queries.

$ aws ec2 --profile TAG describe-images --owner self --query 'Images[*].{ID:ImageId,Date:CreationDate}'

Will just return all AMIs. Can be sorted with jq or similar - but that's not part of AWS CLI:

$ aws ec2 --profile TAG describe-images --owner self --query 'Images[*].{ID:ImageId,Date:CreationDate}' | jq -r 'sort_by(.Date)' | tail

  },

  {

    "ID": "ami-78b6c901",

    "Date": "2018-02-20T18:19:26.000Z"

  },

  {

    "ID": "ami-e628569f",

    "Date": "2018-02-21T08:35:00.000Z"

  }

]

[Steven Nemetz]

Everything in AWS takes time. Just because a command returned, does not mean that the action is completely done and a resource is available.
It is possible, if the new image upload and the lookup for it are too close together that the image will not be available yet.

I'd try using the CLI to lookup the new image before running terraform and checking that it can be found and what state it is in

I think the default is for terraform to only return available images. But I can't verify that at the moment.

[Arseny]

Hi, Steve.

Yes - that was my idea too and finally, in the evening, I added a 1-minute sleep() before calling terraform() but that wasn't enough (maybe).

I'll try to add CLI call before terraform() today or on next week as have to switch to another project on my work.

Thanks.

[Arseny]

Yup - adding sleep(120) between packer() and terraform() calls in Jenkins job looks like solved the issue :-)

[Steven Nemetz]

The time could vary.
I'd recommend using the CLI and creating a loop to wait until the AMI is in available state