kms encryption on ebs volume fails after adding additional region, eu-west-3

490 views
Skip to first unread message

johnr...@gmail.com

unread,
May 22, 2018, 8:17:31 PM5/22/18
to Packer
Hi all, 

I have added a new region, eu-west-3, for existing KMS aws ebs volume to get encrypted in this new region. However, it gets failed with two errors. However, everything works fine if I remove this new region.


[BETA: CentOS7-AWS-Dev] logger: upguard: node not found uw1-dev-jks001.csodsandbox.corp [BETA: CentOS7-AWS-Dev] logger: upguard: failed to find or create node to scan [BETA: CentOS7-AWS-Dev] logger: upguard: failed to kick off logoff node scan against uw1-dev-jks001.csodsandbox.corp [BETA: CentOS7-AWS-Ext] [1;32maws-ext output will be in this color. [0m [BETA: CentOS7-AWS-Ext] [BETA: CentOS7-AWS-Ext] 2 error(s) occurred: [BETA: CentOS7-AWS-Ext] [BETA: CentOS7-AWS-Ext] * Unknown region: eu-west-3 [BETA: CentOS7-AWS-Ext] * Region eu-west-3 is in region_kms_key_ids but not in ami_regions [BETA: CentOS7-AWS-Int] [Packer-Build-CentOS7] Running shell script

I'm on packer 1.1.0 on centos and ansible v2.5.1 running the packer build script at https://github.com/WeekendsBull/packerbuild-error/blob/master/centos7build.json (currently the actual values got updated xxxx for security reason).


I have defined the variables as below.

"variables":
{
"version" : "{{ user `version` }}",
....

"kms_key_id_int_us-west-1" : "arn:aws:kms:us-west-1:xxxxxxxxxxx:key/xxxxx-xxxx-xxxx-xxxx-xxxxx",
"kms_key_id_ext_us-west-1" : "arn:aws:kms:us-west-1:xxxxxxxxxxx:key/xxxxx-xxxx-xxxx-xxxx-xxxxx8",
"kms_key_id_ext_us-east-1" : "arn:aws:kms:us-east-1:xxxxxxxxxxx:key/xxxxx-xxxx-xxxx-xxxx-xxxxx",
"kms_key_id_ext_eu-west-2" : "arn:aws:kms:eu-west-2:xxxxxxxxxxx:key/xxxxx-xxxx-xxxx-xxxx-xxxxx",
"kms_key_id_ext_eu-central-1" : "arn:aws:kms:eu-central-1:xxxxxxxxxxx:key/xxxxx-xxxx-xxxx-xxxx-xxxxx",
"kms_key_id_ext_eu-west-3" : "arn:aws:kms:eu-west-3:xxxxxxxxxxx:key/xxxxx-xxxx-xxxx-xxxx-xxxxxx"
} ................................. ............................
{
"name" : "aws-ext",
"type" : "amazon-ebs",
.......................
"force_deregister" : true,
"kms_key_id" : "{{user `kms_key_id_ext_us-west-1`}}",
"region_kms_key_ids" : {
"us-west-1": "{{user `kms_key_id_ext_us-west-1`}}",
"us-east-1": "{{user `kms_key_id_ext_us-east-1`}}",
"eu-west-2": "{{user `kms_key_id_ext_eu-west-2`}}",
"eu-central-1": "{{user `kms_key_ext_id_eu-central-1`}}", "eu-west-3": "{{user `kms_key_id_ext_eu-west-3`}}",
},

"ami_regions" : [
"us-west-1",
"us-east-1",
"eu-west-2",
"eu-central-1", "eu-west-3"
],
Is there limit how many ebs volume I could encrypt with KMS key?
If I remove "eu-west-3": "{{user `kms_key_id_ext_eu-west-3`}}" & "eu-west-3" under ami_regions, it works fine.
These error messages are really not making any sense to me since I have defined eu-west-3 under ami_regions.

* Unknown region: eu-west-3 * Region eu-west-3 is in region_kms_key_ids but not in ami_regions


Any help or guidance will be appreciated 

John. 

Rickard von Essen

unread,
May 23, 2018, 1:07:54 AM5/23/18
to packe...@googlegroups.com
You need to upgrade to at least 1.2.0 or you can try to set skip_region_validation to true.


--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/packer/issues
IRC: #packer-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Packer" group.
To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/fd6f7553-e310-48a5-8518-db02931a431c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

John Roh

unread,
May 26, 2018, 12:27:04 AM5/26/18
to Packer
Thank you, Richard.
However, packer 1.2.0 had a bug that I wasn't able to create.
Trying with 1.2.1 and 1.2.2, there is no error about adding additional region, however, KMS encrypted ebs volume doesn't get copied to other regions.
I'm going to try with 1.2.3 tomorrow.
Is there paid version of packer? I just wonder if the KMS encryption ebs volume is really working or not. 
If anyone can share the experience, I'd like to hear from you.

John.

Rickard von Essen

unread,
May 27, 2018, 11:48:28 AM5/27/18
to packe...@googlegroups.com
To my knowledge the is no paid version of Packer, there might be enterprise support from HashiCorp. KMS is supposed to work, it's a rather new feature to allow multi region and CMS keys. But it was a well requested feature so if it was broken in the last releases I would have expected bug reports. If you still have trouble with 1.2.3 please supply your full template and we can check and test it.

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/packer/issues
IRC: #packer-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Packer" group.
To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/9de6611c-84b8-4e32-bb5f-447abce704b4%40googlegroups.com.

Rickard von Essen

unread,
May 28, 2018, 3:14:09 AM5/28/18
to packe...@googlegroups.com
I've successfully tested this template:

{
   "provisioners" : [
      {
         "type" : "shell",
         "inline" : [
            "sudo apt-get update",
            "sudo apt-get install -y apt-transport-https ca-certificates curl software-properties-common",
            "sudo apt-get update"
         ]
      }
   ],
   "builders" : [
      {
         "type" : "amazon-ebs",
         "ami_name" : "Docker EE AMI {{isotime \"2006-01-02T030406\"}}",
         "ssh_username" : "ubuntu",
         "source_ami_filter" : {
            "filters" : {
               "name" : "ubuntu/images/*ubuntu-xenial-16.04-amd64-server-*",
               "root-device-type" : "ebs",
               "virtualization-type" : "hvm"
            },
            "most_recent" : true,
            "owners" : [
               "099720109477"
            ]
         },
         "associate_public_ip_address" : true,
         "subnet_id" : "{{user `aws_subnet_id`}}",
         "instance_type" : "t2.micro",
         "encrypt_boot": true,
         "kms_key_id": "7a0bc8cd-84b5-4264-a38b-1531a1743301",
         "ami_regions": [ "eu-west-3" ],
         "region_kms_key_ids": {
           "eu-west-3": "2f2e3cca-90a8-41f7-b29f-5f11088fc9d5"
         }
      }
   ],
   "variables" : {
      "aws_subnet_id" : "{{env `AWS_SUBNET_ID`}}"
   }
}


Which correctly creates AMI's in eu-west-1 and eu-west-3 with CMK encrypted boot volumes:

$ AWS_PROFILE=packer-demo AWS_DEFAULT_REGION=eu-west-3 aws ec2 describe-snapshots --snapshot-ids snap-00371c057a56566b8                               6.6s  Mon May 28 09:09:31 2018
{
    "Snapshots": [
        {
            "Description": "Copied for DestinationAmi ami-09e23982c42c62a11 from SourceAmi ami-085c183f66a55b1d0 for SourceSnapshot snap-0cf2995a1c36fe278. Task created on 1,527,490,339,955.",
            "Encrypted": true,
            "KmsKeyId": "arn:aws:kms:eu-west-3:965990659467:key/2f2e3cca-90a8-41f7-b29f-5f11088fc9d5",
            "OwnerId": "965990659467",
            "Progress": "100%",
            "SnapshotId": "snap-00371c057a56566b8",
            "StartTime": "2018-05-28T06:52:28.000Z",
            "State": "completed",
            "VolumeId": "vol-ffffffff",
            "VolumeSize": 8
        }
    ]
}

$  AWS_PROFILE=packer-demo AWS_DEFAULT_REGION=eu-west-1 aws ec2 describe-snapshots --snapshot-ids snap-0cf2995a1c36fe278                          801ms  Mon May 28 09:10:40 2018
{
    "Snapshots": [
        {
            "Description": "Copied for DestinationAmi ami-085c183f66a55b1d0 from SourceAmi ami-0a8a95d85ce1b59f0 for SourceSnapshot snap-0b1ac50116191233b. Task created on 1,527,490,022,791.",
            "Encrypted": true,
            "KmsKeyId": "arn:aws:kms:eu-west-1:965990659467:key/7a0bc8cd-84b5-4264-a38b-1531a1743301",
            "OwnerId": "965990659467",
            "Progress": "100%",
            "SnapshotId": "snap-0cf2995a1c36fe278",
            "StartTime": "2018-05-28T06:47:06.000Z",
            "State": "completed",
            "VolumeId": "vol-ffffffff",
            "VolumeSize": 8
        }
    ]
}

John Roh

unread,
May 28, 2018, 3:58:26 AM5/28/18
to packe...@googlegroups.com
Wow, great thank you so much, Richard.
I will try it out and let you know how goes.

To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/CALz9Rt97OZ%2BF3nyLpR2_%3D-BvfMJQvm0NKbZ%2BeT24YmBjdAgN0Q%40mail.gmail.com.

John Roh

unread,
May 29, 2018, 4:42:05 PM5/29/18
to packe...@googlegroups.com
Thank you again, Richard for you help.

With Packer 1.2.3, I was able to do the custom KMS encryption in all 5 different regions (including eu-west-3), however, I had to explicitly add skip_region_validation:true that my packer build doesn't complain about the below errors. 

[BETA: CentOS7-AWS-Ext] 2 error(s) occurred: [BETA: CentOS7-AWS-Ext] [BETA: CentOS7-AWS-Ext] * Unknown region: eu-west-3 [BETA: CentOS7-AWS-Ext] * Region eu-west-3 is in region_kms_key_ids but not in ami_regions

I think my issue was also on Ansible side that we build using packer but provision each AWS instance using Ansible. 

It's time for me to work on putting the custom KMS encryption in Windows AMI and more fun to have.

John.


To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/CALz9Rt97OZ%2BF3nyLpR2_%3D-BvfMJQvm0NKbZ%2BeT24YmBjdAgN0Q%40mail.gmail.com.

Rickard von Essen

unread,
May 30, 2018, 4:15:58 AM5/30/18
to packe...@googlegroups.com

To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/CAPxBRp78034iGPuHOYvri2ErpCAD%2BOdJ6MYWArE0%3DbJX8Q-xnQ%40mail.gmail.com.

John Roh

unread,
May 31, 2018, 4:53:41 PM5/31/18
to Packer
Yes, I think the recent Ansible had some bug which caused my build gets failed.

Now I'm able to create a Windows AMI with custom KMS encryption copying all 5 different regions without having any issues.

John.
To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool...@googlegroups.com.

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/packer/issues
IRC: #packer-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Packer" group.
To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool...@googlegroups.com.

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/packer/issues
IRC: #packer-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Packer" group.
To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages