XFS or ZFS for /NSM volume?

[Brant Hale]

I am building NSM sensors with 50+ TBs of local disk and am looking to see what the best filesystem would be.   If you have run or have an opinion on either I would like to hear your experiences.

I am planning on doing full packet capture on a 1 GB internet links.

They are dedicated SO sensors.

My servers are enterprise class with modern RAID cards, my disks are  6TB SAS.

I am looking at either running:

1. RAID5 on (12) 6TB SAS disks using the hardware RAID card and using XFS filesystems.

or

2.  ZFS managed (12) 6TB disks 

I am intrigued by the flexibility of ZFS, but I do not have real world experience running it.  Is this something I should spend time testing out or should I stick with what I know?

Thank you for your thoughts,

Brant

[Jeremy Hoel]

All I can say about ZFS is that it's a RAM hungry FS.. so with the general rule of 1GB  RAM per TB of space, that's a lot of RAM for fast file access.  I haven't done anything that big yet, but I would stick with XFS myself.

--
You received this message because you are subscribed to the Google Groups "security-onion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
To post to this group, send email to securit...@googlegroups.com.
Visit this group at http://groups.google.com/group/security-onion.
For more options, visit https://groups.google.com/d/optout.

[Brant Hale]

Thanks,  I did not even consider the RAM requirement.  I appreciate your response.

[Michał Purzyński]

Go with XFS. It is upstream, received TONS of performance improvements
around 3.10-3.13 and is rock stable. Just install the newest kernel,
you can do that easily on Ubuntu 12.04 LTS

https://wiki.ubuntu.com/Kernel/LTSEnablementStack

I have XFS with 56TB of storage (total, across two servers).

nsm1 :: ~ » hwe-support-status
Your Hardware Enablement Stack (HWE) is supported until April 2017.

nsm1 :: ~ » df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 74G 17G 57G 23% / <- SSD in RAID1
/dev/sdb1 28T 27T 552G 99% /nsm <- XFS in hardware RAID5 12+1 disks

nsm1 :: ~ » mount
/dev/sda1 on / type xfs (rw,noatime)
/dev/sdb1 on /nsm type xfs (rw,noexec,nosuid,nodev,noatime,inode64)

nsm1 :: ~ » uname -a
Linux nsm1 3.13.0-49-generic #81~precise1-Ubuntu SMP Wed Mar 25
16:32:15 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

[Ric Woodard]

I have a similar implementation as you described and I have absolutely no issues with XFS. I'm not familiar with ZFS but I can say the performance with XFS is fine.