How to have mongodb span multiple disks

[Dave Dodd]

I am running version 1.41 on a FreeBSD 8.0-STABLE with master/slave
replication.

I am looking for a way to have my mongodb data files distributed
across multiple disks as my current filesystem is approaching
capacity. The disks I intend to use may be running differing
filesystems.

My first thought was to relocate the various data files (testdb.0
through testdb.20) evenly across a three disks and create appropriate
symbolic links from dbpath to real data files on the other spindles.
Alternatively I could delay my problem by using the --directoryperdb

On the surface this would work ok, however such an approach could fall
apart if a repair or rebuild operation was executed as I think the
data files are removed/recreated during a repair.

Can I control when preallocation of a new data file occurs ?

Can I pre-allocate multiple data files ahead of them being needed and
place them where I want them without normal operations destroying my
symbolic link farm in dbpath ?

Have I missed something obvious that already handles this problem for
me ?

I know I could resort to using a dynamic filesystem such as zfs and/or
use the --directoryperdb command line option to delay the filesystem
exhaustion I am heading for at the moment.

Thoughts gentlefolk ?

--Dave

[Dwight Merriman]

i think you have described the situation well

the symlinks will work but a repair would wipe them out

some form of striping is your long term best bet.  for example one data file, perhaps the most recent, could get very hot; most striping implementations would handle that well but the symlink would be to a particular drive.

--
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com.
To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.

[Alvin Richards]

As Dwight says stripping will be the way to go. The simplest way to do
this with an outage would be to move the existing dbfiles off to a
safe place. Then take the available devices and then create a stripped
array either using LVM or MD. The next step would then to be format
with EXT4 or XFS and then mount the filesystem back in your original
location. Finally copy the files back and start up mongod again.

If you need to do this without an outage, let me know.
-Alvin

> > mongodb-user...@googlegroups.com<mongodb-user%2Bunsubscribe@google groups.com>
> > .

[Markus Gattol]

I would recommend to think about where you *might* be with this setup
in, say 2 or so years down the road ... Sounds a little bit like a
nightmare to maintain/scale to me.

I think, in addition to what has been said with regards to ext4 etc.,
what you could do is put a decent hardware RAID controller in your box
and set it to write-back mode i.e. use it's cache for instant I/O into
your disk array.

This solution scales and is maintainable, the linking around and stuff
like that is what's sure possible but what only junior sysadmins do
because they never had to go and untie a system that has been "build"
like this for a few years until it simply wasn't maintainable anymore.

Have a look at http://www.markus-gattol.name/ws/hardware.html#adaptec_31205_raid_hba
so you see what I am talking about. The stack is somewhat like this

- mongod
- OS
- filesystem eg BTRFS, ext4, etc.
- hardware HBA
- bunch of disks eg in RAID 6 setup

[Dave Dodd]

Ok. It is as I thought.

Just to round out the whole picture, if I was to use --directoryperdb
and symlink
the directories for each database would normal rebuild/repair
operations destroy
the directory symlinks and recreate sub-directories in the root mongo
data directory ?

I will lay plans for a relocation of my bathtub of data :)

Thanks,

--Dave