AWS Session Token expiring before Packer finishes up
3,022 views
Skip to first unread message
Ashish Bista
Aug 30, 2017, 10:43:31 PM8/30/17
to Packer
Hi All,
My packer build task is really long that takes around two hours to complete. But, AWS Session Token only lasts for an hour. So, the access keys are being expired before packer finishes the job. Do you have any workaround for this issue? I am ready to write a packer plugin that would refresh the tokens. But, would it be possible to refresh AWS keys using a plugin?
Thank you in advance!
Ashish
Ashish Bista
Aug 30, 2017, 10:45:58 PM8/30/17
to Packer
This is the error what I'm getting:23:36:29 amazon-ebs: passwd: Success 23:37:01 amazon-ebs: Final clean up 23:37:01 ==> amazon-ebs: Stopping the source instance... 23:38:45 ==> amazon-ebs: Error stopping instance: RequestExpired: Request has expired. 23:38:45 ==> amazon-ebs: status code: 400, request id: 37b5a9b7-fa46-4be8-b7e7-ff1d0972dca5 23:38:45 ==> amazon-ebs: Terminating the source AWS instance... 23:40:15 ==> amazon-ebs: Error terminating instance, may still be around: RequestExpired: Request has expired. 23:40:15 ==> amazon-ebs: status code: 400, request id: b95eaa81-6769-47d3-ae91-45e94d4c1d8e 23:40:15 ==> amazon-ebs: Cleaning up any extra volumes... 23:41:44 ==> amazon-ebs: Error describing volumes: RequestExpired: Request has expired. 23:41:44 ==> amazon-ebs: status code: 400, request id: 065e2a59-3a39-458d-a3d5-ebef321b3864 23:41:44 ==> amazon-ebs: Deleting temporary keypair... 23:43:29 ==> amazon-ebs: Error cleaning up keypair. Please delete the key manually: packer_59a5e164-638c-fe7d-ec00-88ca39b33fdb 23:43:29 Build 'amazon-ebs' errored: Error stopping instance: RequestExpired: Request has expired. 23:43:29 status code: 400, request id: 37b5a9b7-fa46-4be8-b7e7-ff1d0972dca5 23:43:29 23:43:29 ==> Some builds didn't complete successfully and had errors: 23:43:29 --> amazon-ebs: Error stopping instance: RequestExpired: Request has expired. 23:43:29 status code: 400, request id: 37b5a9b7-fa46-4be8-b7e7-ff1d0972dca5 23:43:29 23:43:29 ==> Builds finished but no artifacts were created.
Matthew Hooker
Aug 30, 2017, 11:06:19 PM8/30/17
to Packer
Hi Ashish,
This sounds like it could be a bug. First, make sure you're using the latest version of packer, and if you are, will you please create an issue at https://github.com/hashicorp/packer/issues with output from the packer log with the env var PACKER_LOG=1?
Thanks,
--Matt
--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
GitHub Issues: https://github.com/mitchellh/packer/issues
IRC: #packer-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Packer" group.
To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/3b102eb9-b605-4a90-ac79-e3b211230c7f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Rickard von Essen
Aug 31, 2017, 12:37:15 AM8/31/17
to packe...@googlegroups.com
I'm not sure that this is a bug.
If you use:
1) Access key id, Secret key, (STS) token
Then these are valid for a maximum of 1h and there is no way of renewing them with only these. I expect that this is what you are doing.
2) Are using a assume role setup with your ~/.aws/config
Example:
[profile assumer]
region = eu-west-1
# This profile has access key and secret key in ~/.aws/credentials
[profile packer-assume-role]
region = eu-west-1
source_profile = assumer
role_arn = arn:aws:iam:: 1234567890123:role/PackerRole
mfa_serial = arn:aws:iam::1234567890123:mfa/assumer
Then everything should work when packer uses "profile": "packer-assume-role". When the temporary sts credentials expires they will be automatically be renewed (by the sdk) using the source credentials.
Could you explain how you provide Packer with credentials?
To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/3b102eb9-b605-4a90-ac79-e3b211230c7f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
GitHub Issues: https://github.com/mitchellh/packer/issues
IRC: #packer-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Packer" group.
To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/CAEpfdnx12s1x1PndtsutwVQ-jYf8OU%3DNX8YKw_JX%2Bg-1SY0d4g%40mail.gmail.com.To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool+unsubscribe@googlegroups.com.
Ashish Bista
Aug 31, 2017, 11:11:52 AM8/31/17
to Packer
I'm first creating a session as below:
aws_session=$(
aws sts assume-role \
--role-arn ${AWS_ROLE_ARN} \
--role-session-name ${AWS_SESSION_NAME} \
--region ${AWS_DEFAULT_REGION} \
${AWS_ARGS}
)
Then I'm exporting environment variables:
echo AWS_DEFAULT_REGION=$AWS_DEFAULT_REGION
echo AWS_ACCESS_KEY_ID=$(echo $aws_session | jq -r '.Credentials.AccessKeyId')
echo AWS_SESSION_TOKEN=$(echo $aws_session | jq -r '.Credentials.SessionToken')
echo AWS_SECRET_ACCESS_KEY=$(echo $aws_session | jq -r '.Credentials.SecretAccessKey')
Packer relies on these environment variables to access AWS.
{
"description": "...",
"variables": {
"source_ami": "{{env `SOURCE_AMI`}}",
"enhanced_networking_type": "{{env `ENHANCED_NETWORKING_TYPE`}}",
"region": "us-east-1",
"project_name": "project1",
"os_name": "centos",
"os_version": "7"
},
"provisioners": [
{
"destination": "/tmp/packer",
"source": "./files",
"type": "file"
},
{
"type": "shell",
"execute_command": "echo 'packer' | sudo -S sh -c '{{ .Vars }} {{ .Path }}'",
"scripts": [
"scripts/0-prep.sh",
"scripts/10-bootstrap.sh",
"scripts/20-......sh",
"scripts/40-cleanup.sh"
]
}
],
"builders": [
{
"type": "amazon-ebs",
"ami_name": "{{user `os_name`}}-{{user `os_version`}}-{{user `project_name` }}-{{user `enhanced_networking_type`}}-{{isotime \"2006-01-02-030405\" | clean_ami_name}}",
"ami_description": "{{user `os_name`}} {{user `project_name`}} {{user `enhanced_networking_type`}} {{user `os_version`}} {{user `build_stage`}} image built on {{isotime \"2006-01-02\"}}",
"enhanced_networking": true,
"instance_type": "m4.2xlarge",
"region": "{{user `region`}}",
"security_group_id": "....",
"subnet_id": "...",
"ssh_username": "centos",
"source_ami": "{{ user `source_ami` }}",
"vpc_id": "...",
"iam_instance_profile": "...",
"launch_block_device_mappings": [
{
"device_name": "/dev/xvda",
"volume_size": 300,
"volume_type": "gp2",
"delete_on_termination": true
}
],
"run_tags": {
"created_by": "packer",
"environment": "packer_dev"
},
"tags": {
"Name": "{{user `os_name`}}-{{user `os_version`}}-{{user `project_name` }}-{{user `enhanced_networking_type`}}-{{isotime \"2006-01-02-030405\" | clean_ami_name}}",
"build_date": "{{isotime \"2006-01-02\"}}",
"os": "{{user `os_name`}}",
"source_ami": "{{user `source_ami`}}",
"version": "{{user `os_version`}}",
"description": "{{user `description`}}"
}
}
]
}
I couldn't find any documentation on using aws assume role profile. I'm running packer from a container. I'll follow Richard's suggestions and keep you posted.
Thank you!
Matthew Hooker
Aug 31, 2017, 3:12:08 PM8/31/17
to Packer
ahh, yeah. if you let packer manage the role for you, it should automatically refresh the session, but if you do it yourself then there's no way for it to take action if it expires.
Here's the documentation https://www.packer.io/docs/builders/amazon-ebs.html#profile which links to https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html#specifying-profiles which Rickard has documented above
--Matt
--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
GitHub Issues: https://github.com/mitchellh/packer/issues
IRC: #packer-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Packer" group.
To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/75141bdd-311d-46f9-afbd-5f3148467f1f%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages