by Eric Hammond, Internet Startup Technologist, @esh on Twitter

Background

Amazon EC2 currently has a limit of 1,000 GB (1 TB) for EBS volumes (Elastic Block Store).  It is possible to create file systems larger than this limit using RAID 0 across multiple EBS volumes.  Using RAID 0 can also improve the performance of the file system reducing total IO wait as demonstrated in a number of published EBS performance tests.

The following instructions walk through one way to set up RAID 0 across multiple EBS volumes. Note that there is a limit on the size of a file system on 32-bit instances, but 64-bit instances can get unreasonably large.  This test was run with 40 EBS volumes of 1,000 GB each for a total of 40,000 GB (40 TB) in the resulting file system.

Actual command line output showing the size of the RAID:

  # df /vol
  Filesystem           1K-blocks      Used Available Use% Mounted on
  /dev/md0             41942906368      1312 41942905056   1% /vol

  # df -h /vol
  Filesystem            Size  Used Avail Use% Mounted on
  /dev/md0               40T  1.3M   40T   1% /vol

These commands can run in less than 10 minutes and this could probably be reduced further by parallelizing the creation and attaching of the EBS volumes.

Note that the default limit is 20 EBS volumes per EC2 account. You can request an increase from Amazon if you need more.

Caution: 40 TB of EBS storage will cost $4,000 per month plus usage charges.

Instructions

 
# Start a 64-bit instance (say, Ubuntu 8.04 Hardy from http://alestic.com ). Use your own KEYPAIR

  ec2-run-instances                \
    --key KEYPAIR                  \
    --instance-type c1.xlarge      \
    --availability-zone us-east-1a \
    ami-005db969

# Configurable parameters (set on both local host and on EC2 instance)

  instanceid=i-XXXXXXXX
  volumes=40
  size=1000
  mountpoint=/vol

# On the local host (with EC2 API tools installed)...

# Create and attach EBS volumes

  devices=$(perl -e 'for$i("h".."k"){for$j("",1..15){print"/dev/sd$i$j\n"}}'|
             head -$volumes)
  devicearray=($devices)
  volumeids=
  i=1
  while [ $i -le $volumes ]; do
    volumeid=$(ec2-create-volume -z us-east-1a --size $size | cut -f2)
    echo "$i: created  $volumeid"
    device=${devicearray[$(($i-1))]}
    ec2-attach-volume -d $device -i $instanceid $volumeid
    volumeids="$volumeids $volumeid"
    let i=i+1
  done
  echo "volumeids='$volumeids'"

# On the EC2 instance (after setting parameters as above)...

# Install software

  sudo apt-get update &&
  sudo apt-get install -y mdadm xfsprogs

# Set up the RAID 0 device

  devices=$(perl -e 'for$i("h".."k"){for$j("",1..15){print"/dev/sd$i$j\n"}}'|
             head -$volumes)

  yes | sudo mdadm          \
    --create /dev/md0       \
    --level 0               \
    --metadata=1.1          \
    --raid-devices $volumes \
    $devices

  echo DEVICE $devices       | sudo tee    /etc/mdadm.conf
  sudo mdadm --detail --scan | sudo tee -a /etc/mdadm.conf

# Create the file system (pick your preferred file system type)

  sudo mkfs.xfs /dev/md0

# Mount

  echo "/dev/md0 $mountpoint xfs noatime 0 0" | sudo tee -a /etc/fstab
  sudo mkdir $mountpoint
  sudo mount $mountpoint

# Check it out

  df -h $mountpoint

# Tear it down

  sudo umount $mountpoint
  sudo mdadm --stop /dev/md0

# Terminate the instance

  sudo shutdown -h now

# On the local host (with EC2 API tools installed)...

# Detach and delete volumes

  for volumeid in $volumeids; do
    ec2-detach-volume $volumeid
  done

  for volumeid in $volumeids; do
    ec2-delete-volume $volumeid
  done

Credits

Thanks to M. David Peterson for the basic mdadm instructions:

http://developer.amazonwebservices.com/connect/thread.jspa?messageID=90735#90735