What is the best way to manage env vars in App Engine (flexible environment)?

[Alex G]

Moving our apps from Heroku to App Engine has been great, but there are a few things that we miss. In particular, having a way to define config variables in Cloud Console would be great. We have tried a couple of approaches but none are optimal:

• Place config within env_variables in the app.yaml file. Not ideal to store secrets since anyone deploying the app has access to the app.yaml and those secrets.
• Place config in Cloud Storage. Hard to edit, prone to errors (e.g. breaking the JSON file that might contain them) and it doesn't trigger an app restart with the updated config (I believe Heroku restarts the app every time you change a config variable).

Are there any ideas on implementing a similar feature in App Engine?

Is there a better way of currently handling those config vars?

Thanks

[Anton Bacaj]

You could try storing them in google memcache, nothing will trigger an app to pick up new values unless you redeploy.

You can write application code that polls memcache every X interval to update the values.

[Mike Hardy]

Bumping this. We've been storing our env_variables in app.yaml but we're adding more team members and we don't want to expose these variables to everyone. Does anyone have a suggestion for storing secrets in App Engine Flexible?

[Ani Hatzis]

Hi Mike,

I believe you can use Cloud KMS (Key Management Service) to store private keys in key rings and then encrypt the secrets before they go into your files. Privileged users with the required permissions can use the private keys to locally encrypt the secrets and copy the encrypted values manually into app.yaml whenever new secrets must be added to the app. At runtime / startup the app will authenticate with KMS and get the key to decrypt the environment variables. Additional environment variables would store the information needed to retrieve the private key (project, keyring, key-name). So the source code in the repository and the one deployed to GAE production will not contain any plain secrets. And you can have fine-grain control who has access to the private keys, separately of the source code.

There is the How-to: Storing secrets that shows how to use Cloud KMS to store secrets in Cloud Storage, but the concept should work with local app.yaml or other files, too.

If you prefer to use Cloud Storage, I suggest to use a project specific bucket (restricted access of course), and put all secrets (encrypted or plain) into a path with the version ID of the app, so the app can programmatically get the secrets appropriate for the current version. Since each version will have its own folder with secrets, e.g. secret-v7-4/, you can still roll-back traffic migration safely if something goes wrong with a new version in production. If the bucket isn't used for anything else you could automate this work-flow by a Cloud Function that is triggered by new (or updated) secret folders and then for example could trigger the app to load the new variables (sending an HTTP request to the app's task handler or so). See this Cloud Storage Tutorial for such a background function. Unless of course you already use a CI/CD pipeline where you could integrate something like this.

In any case, since the encryption/decryption part is a potential cause for failure (e.g. typo in key-ring name, missing secrets files, IAM misconfiguration), the tests should cover this, too.

[Alex G]

We ended up doing something similar to what Ani suggests.

However, we are now considering this new feature (beta):
https://cloud.google.com/deployment-manager/runtime-configurator/reference/rest/v1beta1/projects.configs.variables