EBS setup

[Doron Gutman]

Hey,

I'm trying to understand about the "proper" setup for using mongodb on
ec2' ebs drives.
I've read http://groups.google.com/group/mongodb-user/browse_thread/thread/81a99bf139ea88e5?pli=1
(and some other posts in many places), and I think I've figured out
RAID10 should be my choice.
However, I'm afraid of the fact (which I'm still not sure is 100%
correct) that I can't resize it in the future.

I'm currently thinking about creating a storage of 100Gb (4X50Gb EBS
drives) which - for now - is a _lot_ more than what I need.
But I can't seem to stop thinking about the mess I'm going to be for
when I'll have to increase that to something bigger.

So:
- Is there some kind of a way to resize a RAID10 array on EBS ?
- What alternative do I have ?
- How do I back it up ?

I know some of it doesn't really relate directly to mongodb, but since
I'm using the previous link as reference...

Thank you.

[Andy O'Neill]

We're using 4 EBS volumes (4x100GB) in RAID10, as recommended.

For backups: Locking the database on a secondary and run an EBS
snapshot. This is incremental, so the first one takes a long time but
later ones are fast.
For resizing / adding capacity: it's easiest to just bring up a new
server with larger EBS volumes, because our server setup is fully
automated.

We experimented with LVM for resizing / snapshotting reasons but found
that LVM snapshots kill write performance and it wasn't worth it, so
we're using plain 'mdadm' raid setup, something like:
DEVICE=/dev/md0
MD_DRIVES="/dev/sdf /dev/sdg /dev/sdh /dev/sdi"
mdadm -C $DEVICE --chunk=256 -n 4 -l 10 -p f2 $MD_DRIVES

As an added bonus of using EBS snapshots, I can bring a new EC2
instance up with new EBS volumes that are created from snapshots, so
the mongo data is prepopulated and the instance can be online in a few
minutes. That's great for disaster recovery.

I don't want to claim that this setup is perfect. We've been trying
for months to debug intermittent locking / connection drops that may
be related to write load or EBS latency. But in terms of backups and
disaster recovery it's not keeping me awake at night.

On Feb 28, 9:37 am, Doron Gutman <gshocko...@gmail.com> wrote:
> Hey,
>
> I'm trying to understand about the "proper" setup for using mongodb on
> ec2' ebs drives.

> I've readhttp://groups.google.com/group/mongodb-user/browse_thread/thread/81a9...

[Spencer T Brody]

For backups I would recommend reading through the EC2 Backup & Restore and Backups pages of the documentation.

As for resizing, it's difficult to resize an existing RAID-10 array, but Andy's suggestion of bringing up a new instance restored from a backup will work.  You could also setup a new EBS array and attach it to your running instance, shut down mongod, copy the datafiles over to the new array, then restart mongod pointing at the new drives, then finally you can decommission the old array.