Deployment Manager and Managed Instance Group Updates

[Jack Murphy]

I'm looking for some advice on how to manage infrastructure using deployment manager in tandem with managed instance group updates.

My setup includes a deployment manager config that stands up our managed instance groups. There's also a CI server issuing gcloud rolling-action commands to update the managed instance groups' templates. This all works fine.

However, once the CI server does it's thing and deploys a new version of our app subsequent runs of the deployment manager will attempt to revert the instance template to the original value. 

I want to decouple the infrastructure automation from the deployment automation as much as possible. It's not ideal to have to update the current app version in two places.

Terraform has an option to ignore changes on resource properties. Is there anything like that available to deployment manager users? Is it possible to query the current value of a resource's property in deployment manager?

[Navi Aujla (Google Cloud Support)]

Hello Jack 

As test, verify updating template using the deployment update in the test non-production environment instead of using gcloud beta compute instance-groups managed rolling-action,  For reference, here is an example of updating the managed instance group using the deployment manager.

I am not aware of deployment manager option to ignore resource properties, like Terraform provide. If you are looking to get the Deployment manager config of the existing resources, you can retrieve that using the the script: genconfig.py, as provided at this link. 

That being said in order for us to verify the mentioned behavior and possible feature request, please open a public issue report using this link providing detailed replication steps without the integration of the CI (just the commands in use which can be run manually). 

[Poo Bah]

Ok so I encountered this limitation too. In my project, I defined an instance group which referenced an instance template which referenced a sourceImage containing my application.

When I updated the sourceImage and tried a gcloud deployment-manager deployments update... operation which failed with a NO_METHOD_TO_UPDATE_FIELD error.

I even tried defining two instance templates with one associated with the instance group and the other unbound (unassociated with any instance group). But when I tried updating the sourceImage in the unbound instance template I got the same error.

I have included the contents of most of the relevant DM config files in a recent stackoverflow post I created:

https://stackoverflow.com/questions/52675785/deployment-manager-cannot-update-instance-templates-no-method-to-update-field

Is there an alternative way to push out updates to my application (sourceImage) while avoiding configuration drift (i.e. do the whole update via deployment manager)? I want my deployment manager configs / templates to be the "source of truth" for my "infrastructure as code".

[xchri...@google.com]

You will not be able to update the instance template through deployment manager, as it is an immutable object, & cannot be updated. If an immutable object is anywhere in a configuration, the entire configuration becomes immutable, & cannot be changed; even if you try to update a mutable resource, if an immutable resource such as a template is present in the config, it cannot be changed at all. See “Ensure that the resources you are updating are mutable.”.

Best practice is to reference the image template in the configuration with a self link, & keep the instance template separate/out of the configuration/deployment. This way you will be able to change the configuration & update the deployment without any errors. You can’t change the instance template, but you can change the reference in the config/deployment to a different instance template. This is not exactly a rolling update but will give a rolling update effect through deployment manager.

Hope this helps.

[Poo Bah]

Well less a best practice and more so a work around. The problem being that I can't fully realise infrastructure as code because the instance template lives outside of my DM scripts. Which makes deployment automation unnecessarily complex (extra shell scripting to create and manage instance templates outside of DM).

I know that in AWS that launch templates (similar to GCP instance templates) are also immutable. However with AWS CloudFormation you can describe a launch template's properties as part of an autoscale group (similar to GCP instance groups). And that allows you to update the source image referenced in the CloudFormation script and CloudFormation performs the update. In addition, AWS has now introduced mutable launch configurations, complete with versioning. A big step-up in deployment automation.

But I am resigned to working within these limitations of DM, and so yes, your suggestion does help.

Thanks,

Craig

[Patrick (Cloud Platform Support)]

The template does not have to live outside DM, it just has to live outside your primary DM template. IE you can have a master template where you make changes to all mutable resources and have other DM config files where your immutable resources are created. This still reduces the amount of shell scripting because you can still use DM and still just make small changes to a template you've already created, the difference is these DM config files won't be using the "gcloud deployment-manager deployments update" command when you want to make a change. Instead, you will use DM to create a new deployment each time for those specific resources.

The reason it is best practice is that we're simply suggesting you leave those specific configurations outside of your main template. It's not working outside of DM, it's just changing your approach to DM