Packer EBS volume encryption

[Thenuka Keerthibandara]

I'm using  a packer script to encrypt my EBS volumes on AWS by creating a new AMI with encrypted EBS. 

I have a created a CMK using KMS and have added the kms_key_id in packer script builder section as follows.

"variables": {

     ....................

    "kms_key_prod": "{{env `kms_key_prod`}}",

     ....................

    },

"builders": [

    {

      "type": "amazon-ebs",

      "access_key": "{{user `aws_access_key`}}",

      "secret_key": "{{user `aws_secret_key`}}",

      "vpc_id": "{{user `aws_vpc_id`}}",

      "subnet_id": "{{user `subnet_id`}}",

      "region": "{{user `region`}}",

      "kms_key_id": "{{user `kms_key_prod`}}",

      "source_ami": "{{user `base_ami`}}",

      "instance_type": "t2.micro",

      "ssh_username": "ubuntu",

      "ami_name": "{{user `name`}}-Default-{{isotime \"2006-01-02-1504\"}}",

      "ami_description": "Base AMI for Ubuntu 16.04",

      "tags": {

        "Name": "Default-Base",

        "Project": "Public Cloud",

        "Build": "{{ user `buildtime` }}"

      },

      "ami_block_device_mappings": [

        {

          "device_name": "/dev/xvdk",

          "volume_size": 25,

          "Encrypted": true,

          "volume_type": "gp2",

          "delete_on_termination": true

        }

      ],

      "launch_block_device_mappings": [

        {

          "device_name": "/dev/xvdk",

          "volume_size": 25,

          "Encrypted": true,

          "volume_type": "gp2",

          "delete_on_termination": true

        }

      ]

    }

  ],

But when packer runs, the generated AMI has an attached snapshots of the devices I have attached encrpted with the "default" CMK which is "aws/ebs" that is unique for the specific region.

I don't need my boot volume to get encrypted but the attahced EBS volumes to be encrypted with the KMS key that I have created.

Does anyone know if this a limitation in packer or if there is any other way to achieve this through packer?

Regards,

Thenuka

[Rickard von Essen]

Unfortunately that doesn't seem to be supported. See http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_EbsBlockDevice.html and http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_RunInstances.html

As a workaround you can prepare a CMK encrypted (empty) snapshot and attach that in your device mapping block. That should give you a snapshot encrypted with the KMS key you want.

// Rickard 

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/packer/issues
IRC: #packer-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Packer" group.
To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/ea5c895d-584b-45aa-9613-ec913de08231%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Thenuka Keerthibandara]

Will try that and re post if anything goes south @Rickard von Essen 

Thanks in advance. :)

Thenuka.

To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool...@googlegroups.com.

[Thenuka Keerthibandara]

Hi all 

As guided I created a new snapshot of an encrypted volume from my custom CMK and attached the snapshot id to packer.

I have created my block device mapings like this.

"ami_block_device_mappings": [

        {

          "device_name": "/dev/xvdk",

          "snapshot_id": "snap-0cb78ce26c58ecf02",

          "volume_size": 25,

          "Encrypted": true,

          "volume_type": "gp2",

          "delete_on_termination": true

        }

      ],

      "launch_block_device_mappings": [

        {

          "device_name": "/dev/xvdk",

          "snapshot_id": "snap-0cb78ce26c58ecf02",

          "volume_size": 25,

          "Encrypted": true,

          "volume_type": "gp2",

          "delete_on_termination": true

        }

      ]

But When I create the AMI I get the following error and AMI bake process terminates.

[1;31mBuild 'amazon-ebs' errored: Error waiting for instance (i-xxxxxxxxxxxxxxx) to become ready: unexpected state 'shutting-down', wanted target 'running' [0m

==> Some builds didn't complete successfully and had errors:

--> amazon-ebs: Error waiting for instance (i-xxxxxxxxxxxxxxx) to become ready: unexpected state 'shutting-down', wanted target 'running'

==> Builds finished but no artifacts were created.

Finished: FAILURE

What is the correct way to put the snapshot_id?

Thank you,

Thenuka.

[Rickard von Essen]

Full log would be great, it looks like your instance shuts down directly. 

To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/77d725be-987d-4a73-844d-f30f01601dc6%40googlegroups.com.

[Thenuka Keerthibandara]

Hi @Rickard,

Here is the full log

Started by upstream project "Check_AWS" build number 41

originally caused by:

 Started by user JenkinsUser

[EnvInject] - Loading node environment variables.

Building in workspace /mnt/.jenkins/workspace/PackerBuild

 > git rev-parse --is-inside-work-tree # timeout=10

Fetching changes from the remote Git repository

 > git config remote.origin.url https://github.com/ThenukaD/IAC.git # timeout=10

Fetching upstream changes from https://github.com/ThenukaD/IAC.git

 > git --version # timeout=10

using GIT_ASKPASS to set credentials 

 > git fetch --tags --progress https://github.com/ThenukaD/IAC.git +refs/heads/*:refs/remotes/origin/*

 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10

 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10

Checking out Revision da87fc3bde873c2de69ae342d1e19400d9dfd057 (refs/remotes/origin/master)

 > git config core.sparsecheckout # timeout=10

 > git checkout -f da87fc3bde873c2de69ae342d1e19400d9dfd057

 > git rev-list c0aa912725f366a806fdf145c9518103776f6991 # timeout=10

[PackerBuild] $ /mnt/.jenkins/tools/biz.neustar.jenkins.plugins.packer.PackerInstallation/Test_IAC/packer build /mnt/.jenkins/workspace/PackerBuild/provision/packer/aws/ubuntu/base.json

[1;32mamazon-ebs output will be in this color. [0m

[1;32m==> amazon-ebs: Prevalidating AMI Name... [0m

[0;32m    amazon-ebs: Found Image ID: ami-80861296 [0m

[1;32m==> amazon-ebs: Creating temporary keypair: packer_5a16850a-7574-3f41-b1c3-6ebde8974fe3 [0m

[1;32m==> amazon-ebs: Creating temporary security group for this instance... [0m

[1;32m==> amazon-ebs: Authorizing access to port 22 the temporary security group... [0m

[1;32m==> amazon-ebs: Launching a source AWS instance... [0m

[0;32m    amazon-ebs: Instance ID: i-027b1aaf7afb22046 [0m

[1;32m==> amazon-ebs: Waiting for instance (i-027b1aaf7afb22046) to become ready... [0m

[1;31m==> amazon-ebs: Error waiting for instance (i-027b1aaf7afb22046) to become ready: unexpected state 'shutting-down', wanted target 'running' [0m

[1;32m==> amazon-ebs: Terminating the source AWS instance... [0m

[1;32m==> amazon-ebs: No volumes to clean up, skipping [0m

[1;32m==> amazon-ebs: Deleting temporary security group... [0m

[1;32m==> amazon-ebs: Deleting temporary keypair... [0m

[1;31mBuild 'amazon-ebs' errored: Error waiting for instance (i-027b1aaf7afb22046) to become ready: unexpected state 'shutting-down', wanted target 'running' [0m

==> Some builds didn't complete successfully and had errors:

--> amazon-ebs: Error waiting for instance (i-027b1aaf7afb22046) to become ready: unexpected state 'shutting-down', wanted target 'running'

==> Builds finished but no artifacts were created.

Finished: FAILURE

Build step 'Packer' marked build as failure

Warning: you have no plugins providing access control for builds, so falling back to legacy behavior of permitting any downstream builds to be triggered

Triggering a new build of Publish Failure

Finished: FAILURE

[Rickard von Essen]

Ok, for some reason your instance directly shuts down, check the "System Log" from the instance. It can be useful to add -on-error=ask so everything is not cleaned up before you have time to investigate.

To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/89ee267c-d83f-4d01-bbac-220b25cc2a9c%40googlegroups.com.

[Thenuka Keerthibandara]

When I change my block device mapping as follows accorfing to https://github.com/hashicorp/packer/issues/2765

"ami_block_device_mappings": [

        {

          "device_name": "/dev/sdb",

          "volume_size": 25,

          "volume_type": "gp2",

          "delete_on_termination": true

        }

      ],

      "launch_block_device_mappings": [

        {

          "device_name": "/dev/sdb",

          "snapshot_id": "snap-00d5d1ebd4558da90",

          "volume_size": 25,

          "volume_type": "gp2",

          "delete_on_termination": true

        }

      ]

Then I get the following error from packer. How to exactly attach the already created snapshot to packer?

 [1;32m==> amazon-ebs: Prevalidating AMI Name... [0m
 [0;32m    amazon-ebs: Found Image ID: ami-80861296 [0m
 [1;32m==> amazon-ebs: Creating temporary keypair: packer_5a16ab0c-8b80-643f-6f53-e890a3fe3143 [0m
 [1;32m==> amazon-ebs: Creating temporary security group for this instance... [0m
 [1;32m==> amazon-ebs: Authorizing access to port 22 the temporary security group... [0m
 [1;32m==> amazon-ebs: Launching a source AWS instance... [0m
 [1;31m==> amazon-ebs: Error launching source instance: InvalidBlockDeviceMapping: snapshotId can only be modified on EBS devices
==> amazon-ebs: 	status code: 400, request id: 42e8cf61-fb92-4ab6-9953-d79a469ab87c [0m
 [1;32m==> amazon-ebs: No volumes to clean up, skipping [0m
 [1;32m==> amazon-ebs: Deleting temporary security group... [0m
 [1;32m==> amazon-ebs: Deleting temporary keypair... [0m
 [1;31mBuild 'amazon-ebs' errored: Error launching source instance: InvalidBlockDeviceMapping: snapshotId can only be modified on EBS devices
	status code: 400, request id: 42e8cf61-fb92-4ab6-9953-d79a469ab87c [0m

==> Some builds didn't complete successfully and had errors:
--> amazon-ebs: Error launching source instance: InvalidBlockDeviceMapping: snapshotId can only be modified on EBS devices
	status code: 400, request id: 42e8cf61-fb92-4ab6-9953-d79a469ab87c

==> Builds finished but no artifacts were created.
Finished: FAILURE

[vivek....@oath.com]

was there a solution to this? I seem to be running into the same error where i am trying to use packer to create an ami from a base encrypted ami which has encrypted CMK snapshots 

If i launch an instance with the base ami from the console it works fine so i know it has something to do with the configs in packer

[Rickard von Essen]

Vivek, please start a new thread and add as much info as you can and we can try and provide some advice. 

[aparna...@gmail.com]

Hello Thenuka,

wondering if you found solution for the error, I am facing same issue:

2019/04/24 14:59:43 machine readable: amazon-ebs,error []string{"Error launching source instance: InvalidBlockDeviceMapping: snapshotId cannot be modified on root device\n\tstatus code: 400, request id: ecf7e228-bf92-4e1a-9d65-57dea48c99a5"}

[Rickard von Essen]

That's not the same issue, please start a new thread. 

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/packer/issues
IRC: #packer-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Packer" group.
To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/bf028940-ddcd-4793-bfb6-fbadfb76cda5%40googlegroups.com.