Working with credential_source

[cedric....@d2si.io]

Hello,

I use AWS CodeBuild to launch AMI building process with Packer, in a cross-account context.

In a central account, Codebuild project is started with a specific role, then we need to make a chained assume role (short term) and let packer manage this assume-role.

But in this way, build duration is limited to ~1 hour as short term credentials can't last more than 1 hour.

Before moving to a non-cross account architecture, we want to test every possibilities provided by Packer.

One of those is using the newly integrated feature in AWS Go SDK, credential_source, but it does not seems to work with Packer.

We execute a shell script before running packer, to configure AWS profiles.

This is how the working assume role is done, but we can't renew credentials, and bypass 1 hour limitation, even if container credentials have been renew by Codebuild in metadata (~/.aws/config and ~/.aws/credentials files are created successfully after that):

AppRoleArn="arn:aws:iam::XXXXXXXXXXX:role/rol-test-dev"

curl -sqL -o aws_credentials.json http://169.254.170.2/$AWS_CONTAINER_CREDENTIALS_RELATIVE_URI > aws_credentials.json

aws configure set region "eu-west-1"

aws configure set profile.codebuild.aws_access_key_id `jq -r '.AccessKeyId' aws_credentials.json`

aws configure set profile.codebuild.aws_secret_access_key `jq -r '.SecretAccessKey' aws_credentials.json`

aws configure set profile.codebuild.aws_session_token `jq -r '.Token' aws_credentials.json`

aws configure set profile.codebuild.expiration `jq -r '.Expiration' aws_credentials.json`

aws configure set profile.packer.role_arn "${AppRoleArn}"

aws configure set profile.packer.source_profile "codebuild"

export AWS_PROFILE=packer

Here are the packer Codebuild logs when its working :

·[1;32mamazon-ebs output will be in this color.·[0m

 ·[1;32m==> amazon-ebs: Prevalidating AMI Name: XXXXXXXXXXXXXXXX·[0m

·[0;32m amazon-ebs: Found Image ID: ami-XXXXXXXXXXXXXXX·[0m

·[1;32m==> amazon-ebs: Creating temporary keypair: packer_5bd96a1d-1701-adc8-9575-70006aa0e5d2·[0m

·[1;32m==> amazon-ebs: Launching a source AWS instance...·[0m

·[1;32m==> amazon-ebs: Adding tags to source instance·[0m

·[0;32m amazon-ebs: Instance ID: i-XXXXXXXXXXXXXX·[0m

·[1;32m==> amazon-ebs: Waiting for instance (i-XXXXXXXXXXXXX) to become ready...·[0m

·[1;32m==> amazon-ebs: Using ssh communicator to connect: XXXXXXXXXXXXXX·[0m

·[1;32m==> amazon-ebs: Waiting for SSH to become available...·[0m

This is the not working assume role, the ~/.aws/config file is generated successfully:

AppRoleArn="arn:aws:iam::XXXXXXXXXXX:role/rol-test-dev"

aws configure set region "eu-west-1"

aws configure set profile.packer.role_arn "${AppRoleArn}"

aws configure set profile.packer.credential_source "EcsContainer"

export AWS_PROFILE=packer

Here are the packer Codebuild logs when its not working :

·[1;32mamazon-ebs output will be in this color.·[0m

 ·[1;32m==> amazon-ebs: Prevalidating AMI Name: XXXXXXXXXXXXX·[0m

·[0;32m amazon-ebs: Found Image ID: ami-XXXXXXXXXXXX·[0m

·[1;31m==> amazon-ebs: Describing the subnet: subnet-XXXXXXXXXXXXX returned error: InvalidSubnetID.NotFound: The subnet ID 'subnet-XXXXXXXXXXXXXX' does not exist

==> amazon-ebs: status code: 400, request id: 78cb2098-32d5-4cb6-a34f-1fc84f32257b.·[0m

·[1;31mBuild 'amazon-ebs' errored: Describing the subnet: subnet-XXXXXXXXXXXXXXX returned error: InvalidSubnetID.NotFound: The subnet ID 'subnet-XXXXXXXXXXXXX' does not exist

 status code: 400, request id: 78cb2098-32d5-4cb6-a34f-1fc84f32257b.·[0m

 ==> Some builds didn't complete successfully and had errors:

--> amazon-ebs: Describing the subnet: subnet-XXXXXXXXXXXXXXX returned error: InvalidSubnetID.NotFound: The subnet ID 'subnet-XXXXXXXXXXXXXXX' does not exist

 status code: 400, request id: 78cb2098-32d5-4cb6-a34f-1fc84f32257b.

 ==> Builds finished but no artifacts were created.

Does someone already tried credential_source or used packer in a cross account Codebuild config ?

[Rickard von Essen]

What's your packer version? You need 1.3.2, released two days ago.

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/packer/issues
IRC: #packer-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Packer" group.
To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/f36c1149-23a5-4147-ba2c-2b8923790531%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[cedric....@d2si.io]

Hello,

Absolutely, its version 1.3.2, here are download/extract commands :

- wget https://releases.hashicorp.com/packer/1.3.2/packer_1.3.2_linux_amd64.zip -nc -nv -O packer.zip && echo "==== Packer downloaded ===="

- unzip -u packer.zip && echo "==== Packer installed ===="

Here are codebuild logs :

[Container] 2018/10/30 20:50:51 Running command wget https://releases.hashicorp.com/packer/1.3.2/packer_1.3.2_linux_amd64.zip -nc -nv -O packer.zip && echo "==== Packer downloaded ===="

2018-10-30 20:50:52 URL:https://releases.hashicorp.com/packer/1.3.2/packer_1.3.2_linux_amd64.zip [27459221/27459221] -> "packer.zip" [1]

==== Packer downloaded ====

 [Container] 2018/10/30 20:50:52 Running command unzip -u packer.zip && echo "==== Packer installed ===="

Archive: packer.zip

 inflating: packer

==== Packer installed ====

[Rickard von Essen]

From just a glance it looks like there might be a problem with this.

https://github.com/aws/aws-sdk-go/pull/2201/files#diff-0553220eacb67f4665178b4ae2c89793R459

Returns an error if credential source is specified and SharedConfigEnable (which we have).

https://github.com/hashicorp/packer/blob/3315812c2c96df0d5bee903073b87b8798d51246/builder/amazon/common/access_config.go#L75

If you want you can try building from source deleting that line.

To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/8bbfefc4-722f-4cdc-a897-c5a0231ffad6%40googlegroups.com.

[cedric....@d2si.io]

Rickard,

I tested by removing that line and rebuilding from sources.

The issue still persist with the same output.

[Rickard von Essen]

Could you open a new issue with very detailed information about what you are trying to do and what steps you taken to troubleshoot this issue? I think this might not work as intended and require some deeper investigation to understand where the issue is.

To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/8e0fbb19-5e06-4793-a593-bc53a57bb34d%40googlegroups.com.