How many disk in one pool

Albert Shih

unread,

Oct 5, 2012, 4:57:48 PM10/5/12

to

Hi all,

I'm actually running ZFS under FreeBSD. I've a question about how many
disks I �can� have in one pool.

At this moment I'm running with one server (FreeBSD 9.0) with 4 MD1200
(Dell) meaning 48 disks. I've configure with 4 raidz2 in the pool (one on
each MD1200)

On what I understand I can add more more MD1200. But if I loose one MD1200
for any reason I lost the entire pool.

In your experience what's the �limit� ? 100 disk ?

How FreeBSD manage 100 disk ? /dev/da100 ?

Regards.

JAS

--
Albert SHIH
DIO b�timent 15
Observatoire de Paris
5 Place Jules Janssen
92195 Meudon Cedex
T�l�phone : 01 45 07 76 26/06 86 69 95 71
xmpp: j...@obspm.fr
Heure local/Local time:
ven 5 oct 2012 22:52:22 CEST
_______________________________________________
freebsd-...@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questi...@freebsd.org"

Xin Li

unread,

Oct 5, 2012, 5:11:00 PM10/5/12

to

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 10/05/12 13:57, Albert Shih wrote:
> Hi all,
>
> I'm actually running ZFS under FreeBSD. I've a question about how
> many disks I �can� have in one pool.
>
> At this moment I'm running with one server (FreeBSD 9.0) with 4
> MD1200 (Dell) meaning 48 disks. I've configure with 4 raidz2 in the
> pool (one on each MD1200)
>
> On what I understand I can add more more MD1200. But if I loose one
> MD1200 for any reason I lost the entire pool.
>
> In your experience what's the �limit� ? 100 disk ?
>
> How FreeBSD manage 100 disk ? /dev/da100 ?

We have real system equipped with even more than 100 disks and yes,
the devices would be named after their numbers.

One thing that you need to keep in mind: if you don't use the
zpool.cache and there are a lot of disks, the current implementation
could take very long (like several tens of minutes) for the system to
import the pool.

Another consideration is that larger pools are more vulnerable to
damages: should one vdev fails, the whole pool is gone. So make sure
you have sufficient redundancy on every vdevs and keep in mind that
your JBOD chassis, cable, adapters are all points of failures.

Cheers,
- --
Xin LI <del...@delphij.net> https://www.delphij.net/
FreeBSD - The Power to Serve! Live free or die
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (FreeBSD)

iQEcBAEBCAAGBQJQb0zjAAoJEG80Jeu8UPuzDkIH/RZ07eG9lrlg9vDoe1MEJODu
qiVSyidrCHzw53medLOInR+kOt1CJPQX61t2eVyy3oK3MdDj+INtOCGWfO2zXukp
PLWvOCzTUoE3A/cSz2KA+1QaXI5kK/CLS4Y0NII7ibwKP7g8/17mvJqCg+UQup3K
Y3+t0+QTL9iSOtMtrgdL3ft/5DnjoU10RlKhDzjevRwNTaO+cmcr9AW5RnINTUrD
Ig1Zv3rOu+wc0hJ2pDIT8+LC6lIhjmGBi5e3iow5tf3P5VgVRbNAGufFtmVloO1q
tPVJyxK+yWK0+5FIXeeKuVeVbDGmpdtwe2+xpqbd2pnLb6Swh8jLD3ojJwAF4r4=
=HacZ
-----END PGP SIGNATURE-----

Edward Ned Harvey (opensolarisisdeadlongliveopensolaris)

unread,

Oct 5, 2012, 5:05:14 PM10/5/12

to

> From: zfs-discu...@opensolaris.org [mailto:zfs-discuss-
> bou...@opensolaris.org] On Behalf Of Albert Shih

>
> I'm actually running ZFS under FreeBSD. I've a question about how many
> disks I <can> have in one pool.
>
> At this moment I'm running with one server (FreeBSD 9.0) with 4 MD1200
> (Dell) meaning 48 disks. I've configure with 4 raidz2 in the pool (one on
> each MD1200)
>
> On what I understand I can add more more MD1200. But if I loose one
> MD1200
> for any reason I lost the entire pool.
>
> In your experience what's the <limit> ? 100 disk ?
>
> How FreeBSD manage 100 disk ? /dev/da100 ?

Correct about if you lose one storage tray you lose the pool. Ideally you would span your redundancy across trays as well as across disks - but in your situation, 12 disks in raidz2 - and 4 trays - it's just not realistic for you. You would have to significantly increase cost (not to mention rebuild pool) in order to keep the same available disk space and gain the redundancy.

Go ahead and add more trays. I've never heard of any limit of number of disks you can have in ZFS. I'm sure there is a limit, but whatever it is, you're nowhere near it.

Richard Elling

unread,

Oct 5, 2012, 11:48:20 PM10/5/12

to

On Oct 5, 2012, at 1:57 PM, Albert Shih <Alber...@obspm.fr> wrote:

> Hi all,

>
> I'm actually running ZFS under FreeBSD. I've a question about how many
> disks I «can» have in one pool.
>
> At this moment I'm running with one server (FreeBSD 9.0) with 4 MD1200
> (Dell) meaning 48 disks. I've configure with 4 raidz2 in the pool (one on
> each MD1200)
>
> On what I understand I can add more more MD1200. But if I loose one MD1200
> for any reason I lost the entire pool.
>
> In your experience what's the «limit» ? 100 disk ?

I can't speak for current FreeBSD, but I've seen more than 400
disks (HDDs) in a single pool.

-- richard

Brad Stone

unread,

Oct 8, 2012, 6:11:08 AM10/8/12

to

Here's an example of a ZFS-based product you can buy with a large
number of disks in the volume:

http://www.aberdeeninc.com/abcatg/petarack.htm
360 3T drives
A full petabyte of storage (1080TB) in a single rack, under a single
namespace or volume

On Sat, Oct 6, 2012 at 11:48 AM, Richard Elling
<richard...@gmail.com> wrote:
> On Oct 5, 2012, at 1:57 PM, Albert Shih <Alber...@obspm.fr> wrote:
>
>> Hi all,
>>
>> I'm actually running ZFS under FreeBSD. I've a question about how many
>> disks I «can» have in one pool.
>>
>> At this moment I'm running with one server (FreeBSD 9.0) with 4 MD1200
>> (Dell) meaning 48 disks. I've configure with 4 raidz2 in the pool (one on
>> each MD1200)
>>
>> On what I understand I can add more more MD1200. But if I loose one MD1200
>> for any reason I lost the entire pool.
>>
>> In your experience what's the «limit» ? 100 disk ?
>
> I can't speak for current FreeBSD, but I've seen more than 400
> disks (HDDs) in a single pool.
>
> -- richard
>
> _______________________________________________

> zfs-discuss mailing list
> zfs-d...@opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss