Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

FIle system for maximum write speed of very large files

12 views
Skip to first unread message

Nordlöw

unread,
Mar 24, 2010, 4:27:47 AM3/24/10
to
We are building a data collection system with two striped disks.

If we want to maximize the write performance for very large (tens of
gigabytes) what file system should we use?:
ext2,3,4 reiser, xfs, jfs, btrfs?

I believe we don't want nor need journalized write operations.

Thanks in advance,
Nordlöw

Tauno Voipio

unread,
Mar 24, 2010, 11:57:52 AM3/24/10
to
> Nordl�w

How about writing directly to a raw partition?

--

Tauno Voipio
tauno voipio (at) iki fi

John Reiser

unread,
Mar 24, 2010, 12:28:59 PM3/24/10
to

A box with 128GB of RAM easily supports 64GB of tmpfs on /dev/shm.
(Or, use a solid state disk based on flash memory.)
This is about as fast as possible and might provide enough buffering
in front of a variety of file systems on ordinary rotating media.

Probably you want allocation that is based on extents (intervals
of blocks) instead of block-at-a-time. This rules out ext2 and ext3
(but not ext4.)

Probably you want to turn off journalling (or mount the file system
without enabling journalling) no matter which file system you choose.

After that, perhaps things such as long experience (btrfs is relatively new),
simplicity (reiserfs has _many_ obscure features), maintenance state
(the bug history of xfs is not pretty), or size of user community
(somewhat small for many) may matter more than you think at first.

--

0 new messages