Management of custom image families

[Georgi Sotirov]

Hello,

I’m trying to create a custom image family by starting with the public CentOS 8 image (currently centos-8-v20201014), then adding software and configurations, and finally deploying an application. I’d like to be able to upgrade the latest image in the family with the new application version and publish a new image to the family using Terraform. So far I succeed with the first part, but I’m unable to push a new image to the family. Naturally, Terraform replaces the previous image, because of the name change, but the images in a family need to have version.

I have read GCP and Terraform documentation (see References below) and searched for examples over the Internet, but I’m unable to get to the solution.

How do you mange your custom image families in GCP – with Terraform, Packer or something else?


Thanks in advance for your answers!

P.S. References:

• Images [https://cloud.google.com/compute/docs/images];
• Image management best practices [https://cloud.google.com/compute/docs/images/image-management-best-practices];
• Image families best practices [https://cloud.google.com/compute/docs/images/image-families-best-practices];
• Creating, deleting, and deprecating custom images [https://cloud.google.com/compute/docs/images/create-delete-deprecate-private-images];
• google_compute_image of Terraform’s Google provider [https://registry.terraform.io/providers/hashicorp/google/latest/docs/data-sources/compute_image].

Best regards,

--

Georgi

[Adebisi Ibirogba]

I found a documentation that might be helpful but since you are considering working with Terraform which unfortunately I'm not very vast in, you might want to share the issue in stackoverflow where it would have a wider community view

[Georgi D. Sotirov]

The link you've sent is from the article Image families best practices, which is in my references, so I'm aware of it. However, the example there is with the gcloud command. Creation of image family also works from the web interface, but I'd like to automate it with Terraform. OK, I'll post on stackoverflow as well, but I hoped someone here have already dealt with this matter.


Regards,
--
Georgi D. Sotirov

--
© 2018 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Compute Engine Discussion Google Group (gce-dis...@googlegroups.com) to participate in discussions with other members of the Google Compute Engine community and the Google Compute Engine Team.
---
You received this message because you are subscribed to the Google Groups "gce-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gce-discussio...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/3f771af1-5793-4af9-9c7f-58ce050d18c5n%40googlegroups.com.

[Georgi Sotirov]

I found no other solution to my problem (i.e. adding new image to the family without removing the previously created one) than to remove the state of the previously created image, so that Terraform creates it again. For example:

  terraform state rm google_compute_image.my_image

  terraform plan && terraform apply

This however implies that images in the family have to be managed by other means (e.g. Cloud Console or gcloud command) as they are going to accumulate over time since Terraform forgets about previously created images in the family. It is necessary anyway if you're willing to roll back and forward between image versions, because as far as I'm aware this currently cannot be done with Terraform.

[Arnau Carreras]

I have been able to reproduce this issue and tried with `lifecycle { prevent_destroy = true }` as well as `lifecycle { create_before_destroy = true }`. Nevertheless, it looks like the Terraform plan will refuse to create a new image if it can't destroy the previous one:

```

Resource google_compute_image.arnauc-famimage has lifecycle.prevent_destroy set, but the plan calls for this resource to be destroyed. To avoid this error and continue with the plan, either disable lifecycle.prevent_destroy or reduce the scope of the plan using the -target flag.

```

This appears to be part of Terraform's design as mentioned in this Stackoverflow answer based on the Infrastructure as Code definition from Terraform.

Therefore, other than the workaround that you had found, you could add the new images as independent resources in your `main.tf` file with a version suffix on both the second label for the resource and the "name" variable within it, without removing or replacing the previous ones, so they can be created without affecting the previous ones in your project. For example:
```
resource "google_compute_image" "our-custom-image-v8" {

    name = "our-custom-image-v8"

    family = "our-custom-image-family"

    ....

}

resource "google compute_image" "our-custom-image-v7" {

    name = "our-custom-image-v7"

    family = "our-custom-image-family"

    ...

}

```

[Georgi Sotirov]

On Wednesday, December 16, 2020 at 7:24:42 PM UTC+2 Arnau Carreras wrote:

I have been able to reproduce this issue and tried with `lifecycle { prevent_destroy = true }` as well as `lifecycle { create_before_destroy = true }`.

Yeah, I played with these as well, but lifecycle management settings are just not for the case I was trying to solve.

This appears to be part of Terraform's design as mentioned in this Stackoverflow answer based on the Infrastructure as Code definition from Terraform.

I understand that. However, custom image families and the possibility for automatically rolling updates to instances in managed instance groups (MIGs) are features I want to have and I would like to manage them with Terraform like the rest of the infrastructure.

Therefore, other than the workaround that you had found, you could add the new images as independent resources in your `main.tf` file with a version suffix on both the second label for the resource and the "name" variable within it, without removing or replacing the previous ones, so they can be created without affecting the previous ones in your project.

I though about this possibility as well, but it would not work (at least for me), because:

• it means I have to define new resource for every new release I have to deploy;
• it means I also have to keep the disks with these releases, so the images could be created. However, to save time I need to reuse the previously created disk with the already installed and configured software stack and just drop the new application release in it;
• once the images for the previous versions are created on the next run Terraform would fail, because these images already exists. Their Terraform state may have been removed, but the images in the cloud remain.
  

So the solution I came with was to use gcloud command to check upfront whether the image for the requested version already exists or not. It's outside Terraform, but after that it's relatively easy with a conditional data source (for when the image already exists) and conditional image resource (for when the image does not exists). This combined with removing the state of the previously created image and change of the state of previous image gave me the possibility to manage custom image family and roll backward and forward between image (application) releases. The only drawback is that old images accumulate and need to be either manually deleted or scheduled for deletion, but it is doable.

Here's an example to illustrate the whole idea (I intentionally removed unrelated code to make it shorter and readable):

...

imgexists=$(gcloud compute images list --show-deprecated --filter=name:my-custom-image-${ver} --format="csv[no-heading](name)" | wc -l)
...

terraform state rm google_compute_image.my-disk-image

gcloud compute images deprecate "my-custom-image-${prev_ver}" --state DEPRECATED

...

terraform plan|apply -var="ver=${ver}" -var="imgexists=${imgexists}" ...
...
In main.tf:

data "google_compute_image" "my-disk-image" {

     count = var.imgexst == 1 ? 1 : 0

     name = "my-custom-image-${ver}"

}

resource "google_compute_image" "my-disk-image" {

     count = var.imgexst == 1 ? 0 : 1

     family = "my-image-family"

     name = "my-custom-image-${ver}"

     source_disk = "my-disk"

}

data "google_compute_image" "my-family-image" {

    family = module.core.sgcom_image_family

    # To wait for the image creation

     depends_on = [ data.google_compute_image.my-disk-image, google_compute_image.my-disk-image]

}

resource "google_compute_instance_template" "my-instance-template" {

     name_prefix = "my-instance-template-"

...

     disk {

         source_image= data.google_compute_image.my-family-image.self_link

     }

...

     lifecycle {

         create_before_destroy = true

     }

}

...

I managed to script all this, so the operator needs a single command to deploy new version and roll backward and forward between the images and it is all properly reflected in Terraform plans. It may not be the best solution, but it works (for me).