Managing aws credentials between staging and production

[gitted]

When using TF on AWS, how are you guys managing different access/secret key between different environments like staging and production?

Do you just hard code the values in a variables file that you don't include in your git repo?

If I also use the elb command line to push my deployments, I can't seem to see how I can easily switch environments like staging vs. prod.

Thoughts?

[David Maze]

On Thursday, January 26, 2017 at 1:52:26 PM UTC-5, gitted wrote:

When using TF on AWS, how are you guys managing different access/secret key between different environments like staging and production?

We have separate AWS accounts for staging and production, and separate VPCs and network setup.  A shell script invokes STS to get temporary IAM credentials using an MFA device, and launches a subshell with credentials in environment variables. 

In Terraform proper, the top-level files largely just say 'provider "aws" {}' with nothing in there.  For production rollouts, the AWS_* environment variables are right; for developer tests, $HOME/.aws/credentials has the relevant bits.

[Jaroslav Holub]

We keep credentials in .tfvars files, one for each environment, and don't include them in git repo.

[gitted]

Jaroslav, so you have to manually swap the .tfvar files to go from staging to production keys?

[Jaroslav Holub]

I learned that it's better to avoid managing two (or possibly more) environments from a single place (tf directory). Currently we have a git clone of our tf repo for each environment, with their own .tfvars and .tfstate files. That allows us to reproduce environments (or create new ones, or destroy them) without touching production at all, which is important. 

The obvious downside is that this way we can't version control .tfstate files. But we never really did that, since there were issues with secrets stored in .tffiles (not sure if this is already fixed).

J.

[Brian Lalor]

We're using one or more aws accounts per environment with credentials generated from Vault. A wrapper script retrieves (and optionally caches) creds for each invocation and generates an aws shared creds file with profiles for each of the accounts. The providers have an appropriate profile attribute. For a while I was passing around the actual credentials, but got bitten by the remote state having the credentials written into the state file, which (in addition to being insecure) caused future runs to fail when the short-lived credentials expired. The profiles makes for a nice layer of abstraction, too, which makes managing modules a bit easier. 

--

Brian Lalor

bla...@bravo5.org

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/terraform/issues
IRC: #terraform-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Terraform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to terraform-too...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/terraform-tool/ad758aec-e29a-4dc8-b038-2fc043b17ee5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[andy....@distilnetworks.com]

Be interested in whether you looked into Terraform aws provider profile attribute, which seems to let you have multiple sets of credentials in an external credentials file.  I imagine you could pass in a variable from the command line or from the tfvar file.

On Thursday, January 26, 2017 at 1:52:26 PM UTC-5, gitted wrote:

[David Adams]

This is an old thread, but a good chance to mention the tool Vaulted: https://github.com/miquella/vaulted

It can manage keeping encrypted AWS creds (or any secrets) stored on disk that you decrypt into memory with a passphrase. It can manage environment variables, SSH keys, provides a built-in SSH agent, can do AWS MFA and assume-role workflows. It's a really fantastic tool I use every day with Terraform.

--

This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/terraform/issues
IRC: #terraform-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Terraform" group.

To unsubscribe from this group and stop receiving emails from it, send an email to terraform-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/terraform-tool/212e2c8d-9fd7-4394-a081-c5ac280db649%40googlegroups.com.

[Philip Nelson]

Here is what we do, with two scenarios to consider. 

First all tf configuration has a line like this:

provider "aws" {
  profile = "${var.aws_profile}"
  region = "${var.aws_region}"

}

Then for a simple single profile run with the default profile, you can set the standard aws environment variables, we use jenkins credential system for that. More common though, we have to build an ~/.aws credentials file with multiple profiles in our build script. In our case we use jenkins credential system to produce env vars with the id and secret keys, and aws configure commands to created the credentials file. You could use a vault for this, though you still need on key to open a vault that you have the environment provide rather than keeping in version control. 

Hope that helps

On Thu, Aug 31, 2017 at 6:57 AM, David Adams <dave...@gmail.com> wrote:

This is an old thread, but a good chance to mention the tool Vaulted: https://github.com/miquella/vaulted

It can manage keeping encrypted AWS creds (or any secrets) stored on disk that you decrypt into memory with a passphrase. It can manage environment variables, SSH keys, provides a built-in SSH agent, can do AWS MFA and assume-role workflows. It's a really fantastic tool I use every day with Terraform.

On Thu, Aug 31, 2017 at 6:25 AM, <andy.feller@distilnetworks.com> wrote:

Be interested in whether you looked into Terraform aws provider profile attribute, which seems to let you have multiple sets of credentials in an external credentials file.  I imagine you could pass in a variable from the command line or from the tfvar file.

On Thursday, January 26, 2017 at 1:52:26 PM UTC-5, gitted wrote:

When using TF on AWS, how are you guys managing different access/secret key between different environments like staging and production?

Do you just hard code the values in a variables file that you don't include in your git repo?

If I also use the elb command line to push my deployments, I can't seem to see how I can easily switch environments like staging vs. prod.

Thoughts?

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/terraform/issues
IRC: #terraform-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Terraform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to terraform-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/terraform-tool/212e2c8d-9fd7-4394-a081-c5ac280db649%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/terraform/issues
IRC: #terraform-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Terraform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to terraform-tool+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/terraform-tool/CAN3s8zadaA3J_zJaVrNDcdrd_yhzSR2HihCyNBNmMYphkNWknQ%40mail.gmail.com.