Why is TF randomly planning to recreate my aws_acm_certificate_validation resources?

[Shorn Tolley]

TF  0.11.14

AWS provider 2.11.0

But it's been happening on previous versions of TF and the AWS provider too.

When I run a plan, TF sometimes shows me that it's going to force recreate my aws_acm_certificate_validation resources, but I haven't made any changes to this stuff in ages.

TF also shows that my validation related route53 records are going to be recreated (I assume because of the recreation of the cert_validation resource).

Over the last day or two, TF plan has shown me that it's going to:

- make no changes to any cert_validation resources

- recreate one of my cert_validation resources

- recreate both of my cert_validation resources

What's going on?

The logs show this for the validation resource:

-/+ aws_acm_certificate_validation.website-prd-cloudfront-acm-certificate-validation-v2 (new resource required)
      id:                        "2018-08-23 06:20:52 +0000 UTC" => <computed> (forces new resource)
      certificate_arn:           "arn:aws:acm:us-east-1:<accountid>:certificate/6b03b32a-ea47-4d69-a669-414c2e38c9b9" => "arn:aws:acm:us-east-1:<accountid>:certificate/6b03b32a-ea47-4d69-a669-414c2e38c9b9"
      validation_record_fqdns.#: "4" => <computed> (forces new resource)

Is this some change that AWS has made that's forcing this recreation?  

I'm afraid to apply this plan because I don't what's caused it so I'm worried I don't understand what's going on and that might result in my domains not validating properly for HTTPS.

Any thoughts?  Should I just apply the plan and hope for the best?

[Shorn Tolley]

So this did end up failing when I finally tried to apply it.

There may be user error involved here, but it's still pretty annoying that the TF infrastructure appeared to be properly defined and working and then suddenly started failing and needed to be investigated and changed before it could start.

Note that I'm not saying this is TF's problem necessarily, the linked TF issue shows that AWS made some user-unfriendly changes that were causing people problems.

Anyway, in the hope that it might be useful to anyone googling around for this problem, below is my TF code that was failing and some notes on how I got my infrastructure going again.

/* The commented out code relates to fixing a problem I had where one day

TF decided to recreate my ACM cert and validations, see:

https://groups.google.com/forum/#!topic/terraform-tool/rTFz-L8RpTE

I found this github issue about an order problem, especially relating to SANs:

https://github.com/terraform-providers/terraform-provider-aws/issues/8531#issuecomment-497233510

After I saw people reporting the issue was fixed in us-east-1, TF plan was still

saying it wanted to recreate the records, so I decided to let it and ran apply.

The apply failed, complaining about being unable to create the kopi.cloud and

kopimail.net records because they already existed.

Looking in the ACM console, I noticed the records that it tells you must exist

for validation are actually identical (same record name and value).

So I decided to delete the wildcard validation records and re-run apply.

It worked for one record, but the other record failed again.

I removed the create_before_destroy attribute and re-ran apply and now

everything seems to work.

Don't know if the create_before_destroy is relevant, or if it would've fixed

the problem to just run apply again.

*/

resource "aws_acm_certificate" "website-prd-cloudfront-acm-certificate-v2" {

  tags {

    Name = "website-prd-cloudfront-acm-certificate-v2"

  }

  lifecycle {

    // I remember adding this out of paranoia back in the day, not sure if it

    // was contributing to the problem, but removed it anyway.

    // create_before_destroy = true

  }

  domain_name = "*.${local.kopi_cloud_dns_name}"

  subject_alternative_names = [

    "*.${local.kopimail_net_name}",

    "${local.kopi_cloud_dns_name}",

    "${local.kopimail_net_name}"

  ]

  validation_method = "DNS"

}

//resource "aws_route53_record" "kopi-cloud-wildcard-acm-validation-record-v2" {

//  zone_id = "${data.aws_route53_zone.kopi-cloud-root-route53-zone.id}"

//  name = "${aws_acm_certificate.website-prd-cloudfront-acm-certificate-v2.domain_validation_options.0.resource_record_name}"

//  type = "${aws_acm_certificate.website-prd-cloudfront-acm-certificate-v2.domain_validation_options.0.resource_record_type}"

//  records = ["${aws_acm_certificate.website-prd-cloudfront-acm-certificate-v2.domain_validation_options.0.resource_record_value}"]

//  ttl = 60

//}

//

//resource "aws_route53_record" "kopi-mail-wildcard-acm-validation_record-v2" {

//  zone_id = "${data.aws_route53_zone.kopimail-net-root-route53-zone.id}"

//  name = "${aws_acm_certificate.website-prd-cloudfront-acm-certificate-v2.domain_validation_options.1.resource_record_name}"

//  type = "${aws_acm_certificate.website-prd-cloudfront-acm-certificate-v2.domain_validation_options.1.resource_record_type}"

//  records = ["${aws_acm_certificate.website-prd-cloudfront-acm-certificate-v2.domain_validation_options.1.resource_record_value}"]

//  ttl = 60

//}

resource "aws_route53_record" "kopi-cloud-root-acm-validation-record-v2" {

  zone_id = "${data.aws_route53_zone.kopi-cloud-root-route53-zone.id}"

  name = "${aws_acm_certificate.website-prd-cloudfront-acm-certificate-v2.domain_validation_options.2.resource_record_name}"

  type = "${aws_acm_certificate.website-prd-cloudfront-acm-certificate-v2.domain_validation_options.2.resource_record_type}"

  records = ["${aws_acm_certificate.website-prd-cloudfront-acm-certificate-v2.domain_validation_options.2.resource_record_value}"]

  ttl = 60

}

resource "aws_route53_record" "kopi-mail-root-acm-validation_record-v2" {

  zone_id = "${data.aws_route53_zone.kopimail-net-root-route53-zone.id}"

  name = "${aws_acm_certificate.website-prd-cloudfront-acm-certificate-v2.domain_validation_options.3.resource_record_name}"

  type = "${aws_acm_certificate.website-prd-cloudfront-acm-certificate-v2.domain_validation_options.3.resource_record_type}"

  records = ["${aws_acm_certificate.website-prd-cloudfront-acm-certificate-v2.domain_validation_options.3.resource_record_value}"]

  ttl = 60

}

resource "aws_acm_certificate_validation" "website-prd-cloudfront-acm-certificate-validation-v2" {

  certificate_arn = "${aws_acm_certificate.website-prd-cloudfront-acm-certificate-v2.arn}"

  validation_record_fqdns = [

//    "${aws_route53_record.kopi-cloud-wildcard-acm-validation-record-v2.fqdn}",

//    "${aws_route53_record.kopi-mail-wildcard-acm-validation_record-v2.fqdn}"

//  ,

    "${aws_route53_record.kopi-cloud-root-acm-validation-record-v2.fqdn}",

    "${aws_route53_record.kopi-mail-root-acm-validation_record-v2.fqdn}"

  ]

}