Resizing existing partitions during a Packer build

[Andrew Langhorn]

Hi,

I'm working a story at the moment where we need to take an AMI obtained from the AWS Marketplace and, amongst other things, amend the size of the disks attached to it. The AMI we obtain already has disks attached.

Here's what we have so far:

configs/base.json

{

  "builders": [

    {

      "type": "amazon-ebs",

      "ami_name": "coop-packer-{{isotime | clean_ami_name}}",

      "instance_type": "t2.medium",

      "region": "eu-west-1",

      "source_ami": "ami-0a1b2c3d",

      "ssh_username": "ec2-user",

      "launch_block_device_mappings": [

        {

          "device_name": "/dev/sda1",

          "volume_type": "gp2",

          "volume_size": "20",

          "delete_on_termination": "true"

        },

        {

          "device_name": "/dev/sdf",

          "volume_type": "gp2",

          "volume_size": "20",

          "delete_on_termination": "false"

        },

        {

          "device_name": "/dev/sdg",

          "volume_type": "gp2",

          "volume_size": "60",

          "delete_on_termination": "false"

        },

        {

          "device_name": "/dev/sdh",

          "volume_type": "gp2",

          "volume_size": "50",

          "delete_on_termination": "false"

        },

        {

          "device_name": "/dev/sdi",

          "volume_type": "gp2",

          "volume_size": "50",

          "delete_on_termination": "false"

        },

        {

          "device_name": "/dev/sdj",

          "volume_type": "gp2",

          "volume_size": "10",

          "delete_on_termination": "false"

        }

      ],

      "ami_block_device_mappings": [

        {

          "device_name": "/dev/sdf",

          "volume_type": "gp2",

          "volume_size": "20",

          "delete_on_termination": "true"

        },

        {

          "device_name": "/dev/sdg",

          "volume_type": "gp2",

          "volume_size": "60",

          "delete_on_termination": "true"

        },

        {

          "device_name": "/dev/sdh",

          "volume_type": "gp2",

          "volume_size": "50",

          "delete_on_termination": "true"

        },

        {

          "device_name": "/dev/sdi",

          "volume_type": "gp2",

          "volume_size": "50",

          "delete_on_termination": "true"

        },

        {

          "device_name": "/dev/sdj",

          "volume_type": "gp2",

          "volume_size": "10",

          "delete_on_termination": "true"

        }

      ]

    }

  ],

  "provisioners": [

    {

      "type": "shell",

      "execute_command": "sudo {{.Vars}} sh {{.Path}}",

      "script": "scripts/resize_partitions.sh"

    },

    {

      "type": "shell",

      "inline": [

        "reboot",

        "sleep 60"

      ]

    },

    {

      "type": "shell",

      "execute_command": "sudo {{.Vars}} sh {{.Path}}",

      "script": "scripts/resize_filesystems.sh"

    }

  ]

}

This works to a point - in that we get the first provisioner to at least run:

scripts/resize_partitions.sh

#!/bin/bash

set -ex

PARTITIONS=`cat /proc/partitions | awk '{ print $4 }' | awk 'length($0)==4' | grep -v 'xvda' | grep -v 'name'`

for partition in $PARTITIONS; do

  sudo fdisk /dev/$partition << EEOF

d

n

p

w

EEOF

  echo "Resized $partition successfully!"

  sleep 30

  exit 0

done

This was loosely cobbled together from various bits of experimentation and reading blog posts online, so it's not going to be perfect (iteration zero, right?!). The logic is hopefully reasonably simple: given the contents of /proc/partitions, print only the fourth column (to get rid of unnecessary stuff also included in that file), remove anything with a length less than four, and then exclude 'xvda' and 'name'. This gives me a list of partitions I want to resize. With these partitions, run fdisk on them, and provide the values inside the EEOF to fdisk.

For whatever reason, however, Packer runs this loop only once, so I get the first partition resized, but no others.

Any ideas on this one?

Thanks,

Andrew

[Bernd Naumann]

Hi andrew,

because you `{ exit 0 }` in the last line of your for-loop ;)

rubber-duck-debugging to /teh/ rescue!

greetings.

On 01/18/2016 06:50 PM, Andrew Langhorn wrote:
> Hi,
>
> I'm working a story at the moment where we need to take an AMI obtained
> from the AWS Marketplace and, amongst other things, amend the size of
> the disks attached to it. The AMI we obtain already has disks attached.
>
> Here's what we have so far:
>

> /configs/base.json/
> /
> /
> {

> /scripts/resize_partitions.sh/

> --
> This mailing list is governed under the HashiCorp Community Guidelines -
> https://www.hashicorp.com/community-guidelines.html. Behavior in
> violation of those guidelines may result in your removal from this
> mailing list.
>
> GitHub Issues: https://github.com/mitchellh/packer/issues
> IRC: #packer-tool on Freenode
> ---
> You received this message because you are subscribed to the Google
> Groups "Packer" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to packer-tool...@googlegroups.com
> <mailto:packer-tool...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/packer-tool/CAEpa1DKH6aQV1-4qe-pB8%3D-K3PGV-ZrztRBGdu4PdSqBr3R1Mw%40mail.gmail.com
> <https://groups.google.com/d/msgid/packer-tool/CAEpa1DKH6aQV1-4qe-pB8%3D-K3PGV-ZrztRBGdu4PdSqBr3R1Mw%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

[Andrew Langhorn]

Hi Bernd,

Thanks for the speedy response -- and for the rubber-duck debugging, d'oh!

It doesn't look like that did the trick, though. Here's an output from a Packer build run after removing that extra exit command:

Template validated successfully.

amazon-ebs output will be in this color.

==> amazon-ebs: Prevalidating AMI Name...

==> amazon-ebs: Inspecting the source AMI...

==> amazon-ebs: Creating temporary keypair: 

==> amazon-ebs: Creating temporary security group for this instance...

==> amazon-ebs: Authorizing access to port 22 the temporary security group...

==> amazon-ebs: Launching a source AWS instance...

    amazon-ebs: Instance ID: 

==> amazon-ebs: Waiting for instance (i-009d6eda16931fe5d) to become ready...

==> amazon-ebs: Waiting for SSH to become available...

==> amazon-ebs: Connected to SSH!

==> amazon-ebs: Provisioning with shell script: scripts/resize_partitions.sh

    amazon-ebs: + cat /proc/partitions

    amazon-ebs: + grep -v name

    amazon-ebs: + grep -v xvda

    amazon-ebs: + awk length($0)==4

    amazon-ebs: + awk { print $4 }

    amazon-ebs: + PARTITIONS=xvdj

    amazon-ebs: xvdi

    amazon-ebs: xvdf

    amazon-ebs: xvdh

    amazon-ebs: xvdg

    amazon-ebs: + sudo fdisk /dev/xvdj

    amazon-ebs:

    amazon-ebs: Command (m for help): Selected partition 1

    amazon-ebs:

    amazon-ebs: Command (m for help): Partition type:

    amazon-ebs: p   primary (0 primary, 0 extended, 4 free)

    amazon-ebs: e   extended

    amazon-ebs: Select (default p): Partition number (1-4, default 1): Using default value 1

    amazon-ebs: First sector (2048-20971519, default 2048): Using default value 2048

    amazon-ebs: Last sector, +sectors or +size{K,M,G} (2048-20971519, default 20971519): Using default value 20971519

    amazon-ebs:

    amazon-ebs: Command (m for help): The partition table has been altered!

    amazon-ebs:

    amazon-ebs: Calling ioctl() to re-read partition table.

    amazon-ebs:

    amazon-ebs: WARNING: Re-reading the partition table failed with error 16: Device or resource busy.

    amazon-ebs: The kernel still uses the old table. The new table will be used at

    amazon-ebs: the next reboot or after you run partprobe(8) or kpartx(8)

    amazon-ebs: Syncing disks.

==> amazon-ebs: Terminating the source AWS instance...

==> amazon-ebs: No AMIs to cleanup

==> amazon-ebs: Deleting temporary security group...

==> amazon-ebs: Deleting temporary keypair...

Build 'amazon-ebs' errored: Script exited with non-zero exit status: 1

==> Some builds didn't complete successfully and had errors:

--> amazon-ebs: Script exited with non-zero exit status: 1

==> Builds finished but no artifacts were created.

So, it seems that whilst $PARTITIONS is an array containing all five known partitions, only the first of these is used to pass in to fdisk. The other four are just left by the wayside, and the Packer run exits 1...

Andrew

To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/569D2EDD.50006%40spreadshirt.net.

For more options, visit https://groups.google.com/d/optout.

--

Andrew Langhorn
Senior Infrastructure Engineer
Email	andrew....@thoughtworks.com
Telephone	+44 7733 339809

[Andrew Langhorn]

This morning, we realised that this happens because the warning output from fdisk makes the command exit with 1, so Packer (correctly) exits. Crudely, we're ignoring the exit code now to get the script to continue to run through the loop using || true before the start of the heredoc.

That might help anyone else looking for inspiration, too!