How should hard drive space be partitioned on a node?

41 views
Skip to first unread message

Lorenzini, Joseph

unread,
Feb 26, 2013, 7:55:06 PM2/26/13
to idh-...@googlegroups.com

§  There are two drive categories:

1.    system or root drives, which is mounted on slash and contain the operating system files.

2.    data drives, where all HDFS and MapReduce data is stored. 

§  No other data aside from HDFS and MapReduce data should be stored in the data drives.

§  Each files ystem should use the ext4 file system with the options "noatime,nofail,errors=continue"

§  Turn off reserved root filesystem space on data drives with "tune2fs -m 0 /dev/sdx"

§  The root partition, which holds the root directory of the operating system, must be at least 20 GB.

§  Each data drive should be a different mount point, with a preferred location of /mnt/diskx

§  You should NOT partition data drives into multiple paritions, NOR use any RAIDing of these drives.

§  The HDFS DataNode’s directories should not use the system drive and should only use data drive

 

Reply all
Reply to author
Forward
0 new messages