UFS Subdirectory limit.
David Malone
ago, where someone had an application that used about 150K
subdirectories of a single directory. They wanted to move this
application to FreeBSD, but discovered that UFS is limited to 32K
subdirectories, because UFS's link count field is a signed 16 bit
quantity. Rewriting the application wasn't an option for them.
I had a look at how hard it would be to fix this. The obvious route
of increasing the size of the link count field is trickly because
it means changing the struct stat, which has a 16 bit link count
field. This would imply ABI breakage, though it might be worth it.
I had a think about other ways to fix this problem. One way around
this limitation is to change the link count semantics so that ".."
doesn't contribute to a directories link count (this may seem silly,
but I've included a bit of a rational below). I've produced a patch
at:
http://www.maths.tcd.ie/~dwmalone/dircount_hack
which adds options to newfs (-D) and tunefs (-d) that set a new
flag in the filesystem making it use this new link counting scheme.
(If you enable the flag with tunefs you should run fsck to recalculate
the link count.) The patch also makes filesystems with this flag
be of type "wfs", so that fts knows not to use the link-count-stat
shortcut.
I'd appreciate any feedback on this patch. Please don't use it on
important filesystems, as it may chew your files! I've done some
basic testing, including making a directory with 70K subdirectories,
and it seems to work.
David.
---
Originally, I guess the link count was used to decide when you can
free the blocks belonging to a file. As directories are basically
implemented as files, they inherit link counts. However today, the
real test for "can you deallocate a directory" is "is it empty".
Thus the link count for directories isn't authoritative for
deallocation, though it does provide a shortcut (nlink > 2 => not
empty).
The link count also provides some extra consistency that fsck can
check. However, as far as I can tell, the link count doesn't provide
any extra information that can actually be used to fix inconsistencies.
The other place that directory link counts are used is as a short
cut in userland when estimating the number of subdirectories that
a directory has. This is used in the fts code to avoid stating
things. Since this shortcut only works for ufs-like filesystems,
we already have code for dealing with this.
_______________________________________________
freeb...@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "freebsd-fs-...@freebsd.org"
--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-...@muc.de
David Schultz
> There was a discussion on comp.unix.bsd.freebsd.misc about two weeks
> ago, where someone had an application that used about 150K
> subdirectories of a single directory. They wanted to move this
> application to FreeBSD, but discovered that UFS is limited to 32K
> subdirectories, because UFS's link count field is a signed 16 bit
> quantity. Rewriting the application wasn't an option for them.
>
> I had a look at how hard it would be to fix this. The obvious route
> of increasing the size of the link count field is trickly because
> it means changing the struct stat, which has a 16 bit link count
> field. This would imply ABI breakage, though it might be worth it.
Why not just...
- make a new st_nlink field that's 32 bits and put it in the spare
32-bit field in struct stat
- rename the old st_nlink to st_onlink and leave it at 16 bits
- the kernel would fill in st_onlink with max(st_nlink,SHORT_MAX)
Scott Long
> On Sat, Mar 26, 2005, David Malone wrote:
>
>>There was a discussion on comp.unix.bsd.freebsd.misc about two weeks
>>ago, where someone had an application that used about 150K
>>subdirectories of a single directory. They wanted to move this
>>application to FreeBSD, but discovered that UFS is limited to 32K
>>subdirectories, because UFS's link count field is a signed 16 bit
>>quantity. Rewriting the application wasn't an option for them.
>>
>>I had a look at how hard it would be to fix this. The obvious route
>>of increasing the size of the link count field is trickly because
>>it means changing the struct stat, which has a 16 bit link count
>>field. This would imply ABI breakage, though it might be worth it.
>
>
> Why not just...
>
> - make a new st_nlink field that's 32 bits and put it in the spare
> 32-bit field in struct stat
>
> - rename the old st_nlink to st_onlink and leave it at 16 bits
>
> - the kernel would fill in st_onlink with max(st_nlink,SHORT_MAX)
I thought that we already discussed this in the past year. There are
significant compatibility concerns here. What happens if you use an
old fsck binary on a new filesystem? Since you haven't changed the
magic, it has no way of knowing that nlink needs to be handled
differently. It would make it impossible to share a filesystem between
different versions of FreeBSD, let alone any other BSD. It's hard to
justify bumping the magic for just a change like this, since it would
basically mean that you've created UFS3. Also, the more important
concern is that large directories simply don't scale in UFS. Lookups
are a linear operation, and while DIRHASH helps, it really doesn't scale
well to 150k entries. I think the reason that there isn't more pressure
to fix the nlink size is because most people realize that it just won't
provide any real benefit. It would be much more worthwhile to introduce
a UFS3 that uses a more efficient directory layout (B-tree?) to provide
real value to increasing the nlink limitation.
Scott
David Malone
> 32-bit field in struct stat
> - rename the old st_nlink to st_onlink and leave it at 16 bits
> - the kernel would fill in st_onlink with max(st_nlink,SHORT_MAX)
Hmmm - interesting - I hadn't realised there was spare space in
struct stat. I guess we could get away with this and there's space
in both ufs1 and ufs2 inodes. I think we'd need to redefinte nlink_t,
which would need an ABI bump.
One problem I can think of might be non-obvious failures of old
programs on directories with lots of subdirectories. The hacky
scheme ends up with a link count on 2 on all directories, which
produces a reasonably obvious failure.
David.
David Malone
suggestion or mine...]
> I thought that we already discussed this in the past year. There are
> significant compatibility concerns here. What happens if you use an
> old fsck binary on a new filesystem? Since you haven't changed the
> magic, it has no way of knowing that nlink needs to be handled
> differently.
In the scheme I suggested, a flag in the superblock is used to
record when the new scheme is in use. I guess das's scheme would
do something similar. In my case, when you run an old fsck, the
link counts are just set to their traditional values by fsck (unless
you have a directory with too many links, in which case fsck will
do whatever it has always done in this situation).
(You also don't need to turn such a feature on by default. Both
schemes can also be read-only compatible with old systems too.)
> Also, the more important
> concern is that large directories simply don't scale in UFS. Lookups
> are a linear operation, and while DIRHASH helps, it really doesn't scale
> well to 150k entries.
It seems to work passably well actually, not that I've benchmarked
it carefully at this size. My junkmail maildir has 164953 entries
at the moment, and is pretty much continiously appended to without
creating any problems for the machine it lives on. Dirhash doesn't
care if the entries are subdirectories or files.
If the directory entries are largely static, the name cache should
do all the work, and it is well capable of dealing with lots of
files.
We should definitely look at what sort of filesystem features we're
likely to need in the future, but I just wanted to see if we can
offer people a sloution that doesn't mean waiting for FreeBSD 6 or
7.
(164955 files now ;-)
David.
David Schultz
> David Schultz wrote:
> >On Sat, Mar 26, 2005, David Malone wrote:
> >
> >>There was a discussion on comp.unix.bsd.freebsd.misc about two weeks
> >>ago, where someone had an application that used about 150K
> >>subdirectories of a single directory. They wanted to move this
> >>application to FreeBSD, but discovered that UFS is limited to 32K
> >>subdirectories, because UFS's link count field is a signed 16 bit
> >>quantity. Rewriting the application wasn't an option for them.
> >>
> >>I had a look at how hard it would be to fix this. The obvious route
> >>of increasing the size of the link count field is trickly because
> >>it means changing the struct stat, which has a 16 bit link count
> >>field. This would imply ABI breakage, though it might be worth it.
> >
> >
> >Why not just...
> >
> >- make a new st_nlink field that's 32 bits and put it in the spare
> > 32-bit field in struct stat
> >
> >- rename the old st_nlink to st_onlink and leave it at 16 bits
> >
> >- the kernel would fill in st_onlink with max(st_nlink,SHORT_MAX)
>
> significant compatibility concerns here. What happens if you use an
> old fsck binary on a new filesystem? Since you haven't changed the
> magic, it has no way of knowing that nlink needs to be handled
> differently. It would make it impossible to share a filesystem between
> different versions of FreeBSD, let alone any other BSD.
First of all, I was only talking about how to avoid badly breaking
the stat ABI, not about how to avoid breaking the on-disk FS
format. However, I think a similar trick could be applied to the
disk inode.
There are 24 bytes of reserved space in the UFS2 inode that
current versions of fsck ignore, and four of them could be used to
store a larger nlink field. The old nlink field would still be
kept up-to-date by newer kernels, which would provide reverse
compatibility for older kernels and versions of fsck *provided*
that no directories have more than 32767 files. Clearly there's a
fundamental limitation that older software won't be able to
properly handle large directories, but at least small directories
in the new format would be backwards compatible.
The only other problem that comes to mind is that older versions
of fsck and older kernels could cause the two nlink fields to get
out of date. However, for directories, new kernels should be able
to figure out the correct nlink value from the directory contents
when this happens, since hard links to directories are not
allowed. For regular files, it should be safe to assume the
larger nlink value is the correct one; this may leak storage, but
a new version of fsck would be able to reclaim it. Furthermore,
this benign inconsistency would only happen in bizarre situations,
such as switching from a new kernel to an old kernel, adding or
removing hard links using the older kernel, and then switching
back to the new kernel.
Poul-Henning Kamp
>On Fri, Mar 25, 2005, Scott Long wrote:
>> David Schultz wrote:
>> >On Sat, Mar 26, 2005, David Malone wrote:
>> >
>> >>There was a discussion on comp.unix.bsd.freebsd.misc about two weeks
>> >>ago, where someone had an application that used about 150K
>> >>subdirectories of a single directory. They wanted to move this
>> >>application to FreeBSD, but discovered that UFS is limited to 32K
>> >>subdirectories, because UFS's link count field is a signed 16 bit
>> >>quantity. Rewriting the application wasn't an option for them.
Has anybody here wondered how much searching a 150K directory would
suck performance wise ?
I realize that with dir-hashing and vfs-cache it is not as bad as it
used to be, but I still think it will be unpleasant performance wise.
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
p...@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
Dag-Erling Smørgrav
> It would be much more worthwhile to introduce
> a UFS3 that uses a more efficient directory layout (B-tree?) to provide
> real value to increasing the nlink limitation.
It would be even more worthwile to simply adopt an existing well-
designed and well-tested file system, such as IBM JFS.
DES
--
Dag-Erling Smørgrav - d...@des.no
Robert Watson
On Sat, 26 Mar 2005, David Malone wrote:
> > Also, the more important
> > concern is that large directories simply don't scale in UFS. Lookups
> > are a linear operation, and while DIRHASH helps, it really doesn't scale
> > well to 150k entries.
>
> It seems to work passably well actually, not that I've benchmarked it
> carefully at this size. My junkmail maildir has 164953 entries at the
> moment, and is pretty much continiously appended to without creating any
> problems for the machine it lives on. Dirhash doesn't care if the
> entries are subdirectories or files.
>
> If the directory entries are largely static, the name cache should do
> all the work, and it is well capable of dealing with lots of files.
>
> We should definitely look at what sort of filesystem features we're
> likely to need in the future, but I just wanted to see if we can offer
> people a sloution that doesn't mean waiting for FreeBSD 6 or 7.
FWIW, I regularly use directories with several hundred thousand files in
them, and it works quite well for the set of operations I perform
(typically, I only append new entries to the directory). This is with a
cyrus server hosting fairly large shared folders -- in Cyrus, a
maildir-like format is used. For example, the lists.linux.kernel
directory references 430,000 individual files. Between UFS_DIRHASH and
Cyrus's use of a cache file, opening the folder primarily consists of
mmap'ing the cache file and then doing lookups, which occur quite quickly.
My workload doesn't currently require large numbers of directories
referenced by a similar directory, but based on the results I've had with
large numbers of files, I can say it likely would work fine subject to the
ability for UFS to express it.
Robert N M Watson
Poul-Henning Kamp
writes:
>Scott Long <sco...@samsco.org> writes:
>> It would be much more worthwhile to introduce
>> a UFS3 that uses a more efficient directory layout (B-tree?) to provide
>> real value to increasing the nlink limitation.
>
>It would be even more worthwile to simply adopt an existing well-
>designed and well-tested file system, such as IBM JFS.
Even better: Do both and then on top of it encourage further
research into filesystems which take modern usage of computing
systems into account :-)
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
p...@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
Scott
Luckily, linear reads through a directory are nearly O(1) in UFS since
ufs_lookup() caches the offset to the last entry read so that a
subsequent call doesn't have to start from the beginning. I would
suspect that this, along with the namei cache, DIRHASH, and cyrus's
cache, all contribute together to make reading the spool directories
non-painful for you. I would also suspect that there is little
manual sorting going on since cyrus chooses names for new entries that
are naturally sorted. I'm still not sure I would consider these
behaviours to be representative of the normal, though. It would be
quite interesting to profile the system while cyrus is trying to append
or delete a mail file into one of your large spool directories. Would
an application that isn't as well-written as cyrus behave as well? What
about an application like Squid?
Scott
David Malone
> ufs_lookup() caches the offset to the last entry read so that a
> subsequent call doesn't have to start from the beginning.
(dirhash also has an equivelent optimisation 'cos that bit of
ufs_lookup code isn't called when dirhash is in use)
> Would
> an application that isn't as well-written as cyrus behave as well? What
> about an application like Squid?
Random lookups should be almost O(1) with dirhash when you have
many operations to amortise the cost of the hash over. You loose
out with dirhash are when you make a small number of accesses to a
large directory and all those entries live close to the beginning
of the directory (or possibly when you're thrashing against dirhash's
memory limit).
If the directory entries are actually constant (as is the case with
squid in truncate mode), then you should get ~O(1) but with a
slightly smaller constant than when the directory entries are
changing.
Just to check, I'm running a benchmark at the moment to compare
150k directories either aranged as:
1) a flat 150k subdirectories of one directory, or
2) 150k directories arranged as a two levels with ~387
subdirectories.
At the moment it looks like accessing files in either structure
performs equivelently but it is a bit slower to build/remove the
flat structure. I'll post the results once the run is complete.
David.
Julian Elischer
> In message <20050326213...@VARK.MIT.EDU>, David Schultz writes:
>
>>On Fri, Mar 25, 2005, Scott Long wrote:
>>
>>>David Schultz wrote:
>>>
>>>>On Sat, Mar 26, 2005, David Malone wrote:
>>>>
>>>>
>>>>>There was a discussion on comp.unix.bsd.freebsd.misc about two weeks
>>>>>ago, where someone had an application that used about 150K
>>>>>subdirectories of a single directory. They wanted to move this
>>>>>application to FreeBSD, but discovered that UFS is limited to 32K
>>>>>subdirectories, because UFS's link count field is a signed 16 bit
>>>>>quantity. Rewriting the application wasn't an option for them.
>
>
> Has anybody here wondered how much searching a 150K directory would
> suck performance wise ?
>
> I realize that with dir-hashing and vfs-cache it is not as bad as it
> used to be, but I still think it will be unpleasant performance wise.
We have a reason (*) to have 300000 entries in a directory..
once the dirhash cache size was made big enough, performance was acceptable.
(*) (we didn't want to but had to for "a while until it's fixed")
Don Lewis
> FWIW, I regularly use directories with several hundred thousand files in
> them, and it works quite well for the set of operations I perform
> (typically, I only append new entries to the directory). This is with a
> cyrus server hosting fairly large shared folders -- in Cyrus, a
> maildir-like format is used. For example, the lists.linux.kernel
> directory references 430,000 individual files. Between UFS_DIRHASH and
> Cyrus's use of a cache file, opening the folder primarily consists of
> mmap'ing the cache file and then doing lookups, which occur quite quickly.
I'm doing the same here and performance is OK for day to day operations,
but cloning the mail store is glacially slow. I had to move my mail
store from a dieing disk to a replacement disk, and migrating about 5GB
of data took the better part of a day. As the destination disk started
to fill up, the I/O rates dropped to very low levels and the CPU was
pegged at close to 100%, mostly system time.
The problem is that the allocation strategy of locating file inodes and
their data blocks in the same cylinder group as their parent directory
fails badly with directories of this size. Once the cylinder group
fills, the search for free inodes and blocks gets very slow.
Using bigger cylinder groups would help a lot, especially considering
disk sizes these days, but both UFS and UFS2 have inconveniently small
maximum cylinder group sizes. I think it would make sense to be able to
cluster groups of cylinder groups together for allocation purposes.
Something else that would seem to make a lot of sense would be to
implement something like the maxbpg limit for large directories that
would force the inode allocator to start allocating from another
cylinder group after a directory grows past a certain size.
> My workload doesn't currently require large numbers of directories
> referenced by a similar directory, but based on the results I've had with
> large numbers of files, I can say it likely would work fine subject to the
> ability for UFS to express it.
David Malone
I've labeled "sqrt") with a single directory with 150000 subdirectories
(which I've labeled "flat").
The benchmark is in 4 phases:
mkdir) This builds the directory structure.
write) This writes a small amount of data into 100000 files
in a pseudo random sequence of subdirectories.
read) This reads back the data from each of the 100000
files (in the same order they were written).
rm) This does an "rm -fr" of the whole tree.
I just used /usr/bin/time on each phase and synced out the data
between each phase. The results (averaged over 4 runs, see the end
of the mail for the output of ministat on the data).
real time user time sys time
mkdir write read rm | mkdir write read rm | mkdir write read rm
sqrt 499 4302 2409 1569 | 1.84 1.94 1.72 1.69 | 29.9 33.5 21.3 161.6
flat 1172 4318 2407 1717 | 1.47 1.62 1.52 1.66 | 26.1 33.5 20.6 158.1
So, it seems that while making the directory structure takes a bit
longer for the flat method, there's no significant penality in real
time for using it. The user times are pretty irrelevant (though the
flat scheme is slightly faster, probably because some of the phases
don't do sqrts ;-).
Interestingly, the system times for the flat structure are actually
*better* than the two level structure! I think this supports Don's
suggestion that the layout of data on the disk with very large
directories is not as good as it could be.
(The test was done on an amd64 machine with gobs of ram. I used my
patch to get large directories, which saves a metadata op per mkdir
and rmdir, even in the sqrt case. I upped the amount of memory
available to dirhash, though it didn't actually use more than about
2.5MB during the benchmark. Maxvnodes is set to 100000, so 150K
dirs plus 100K files should be enough to make the name cache and
vnode cache work hard.)
David.
x sqrt-real-mkdir
+ flat-real-mkdir
+--------------------------------------------------------------------------+
| + |
|x x x x + ++|
||__AM_| |A||
+--------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 4 469.16 527.83 513.14 499.2525 26.256259
+ 4 1157.18 1182.42 1176.14 1172.1225 10.737021
Difference at 95.0% confidence
672.87 +/- 34.7068
134.775% +/- 6.95175%
(Student's t, pooled s = 20.0583)
x sqrt-real-write
+ flat-real-write
+--------------------------------------------------------------------------+
|x + x x + x + + |
| |________|________A____M___________|___A________________M_____________||
+--------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 4 4286.79 4317.17 4305.92 4302.0775 12.774457
+ 4 4288.35 4339.77 4330.78 4318.195 22.606829
No difference proven at 95.0% confidence
x sqrt-real-read
+ flat-real-read
+--------------------------------------------------------------------------+
|+ + xx * +x|
| |_________________|_____________A_______A_____M_______________|| |
+--------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 4 2404.34 2417.4 2411.22 2409.3875 6.2196322
+ 4 2396.65 2417.16 2411.18 2407.08 8.9677905
No difference proven at 95.0% confidence
x sqrt-real-rm
+ flat-real-rm
+--------------------------------------------------------------------------+
| x + |
| x x x + + + |
||___AM_| |__A_M||
+--------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 4 1562.17 1578.65 1572.48 1568.9625 8.0307466
+ 4 1707.86 1722.16 1721.65 1717.3925 6.6327841
Difference at 95.0% confidence
148.43 +/- 12.7436
9.46039% +/- 0.812231%
(Student's t, pooled s = 7.36501)
x sqrt-user-mkdir
+ flat-user-mkdir
+--------------------------------------------------------------------------+
| + |
|+ + + x x x x|
| |______________AM_____________| |__AM_| |
+--------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 4 1.81 1.87 1.85 1.84 0.025819889
+ 4 1.32 1.6 1.48 1.47 0.11489125
Difference at 95.0% confidence
-0.37 +/- 0.144075
-20.1087% +/- 7.83019%
(Student's t, pooled s = 0.0832666)
x sqrt-user-write
+ flat-user-write
+--------------------------------------------------------------------------+
|+ + + + x x x x|
| |____A_M___| |_____________A__M___________| |
+--------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 4 1.82 2.06 1.95 1.935 0.099498744
+ 4 1.57 1.66 1.63 1.62 0.037416574
Difference at 95.0% confidence
-0.315 +/- 0.13006
-16.2791% +/- 6.72144%
(Student's t, pooled s = 0.0751665)
x sqrt-user-read
+ flat-user-read
+--------------------------------------------------------------------------+
|+ + x+ + x x x |
| |_______________________A_______M_____|_________|_____A______M________||
+--------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 4 1.56 1.81 1.77 1.72 0.11045361
+ 4 1.33 1.71 1.57 1.515 0.16278821
No difference proven at 95.0% confidence
x sqrt-user-rm
+ flat-user-rm
+--------------------------------------------------------------------------+
|x + + x + + x x |
| |_________|________________A____A_____M______|______M__________||
+--------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 4 1.4 1.89 1.84 1.695 0.22218611
+ 4 1.54 1.8 1.74 1.665 0.12476645
No difference proven at 95.0% confidence
x sqrt-sys-mkdir
+ flat-sys-mkdir
+--------------------------------------------------------------------------+
| ++ + + xx xx|
||_______A________| |___A__M||
+--------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 4 29.62 30.15 30.06 29.89 0.25495098
+ 4 25.7 26.84 26.07 26.1 0.5178803
Difference at 95.0% confidence
-3.79 +/- 0.706247
-12.6798% +/- 2.36282%
(Student's t, pooled s = 0.408167)
x sqrt-sys-write
+ flat-sys-write
+--------------------------------------------------------------------------+
|+ x + x x x + +|
| |________________|___________A_AM_________|__M_________________| |
+--------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 4 33.21 33.8 33.57 33.5275 0.24281337
+ 4 32.81 34.25 33.83 33.565 0.61846584
No difference proven at 95.0% confidence
x sqrt-sys-read
+ flat-sys-read
+--------------------------------------------------------------------------+
|+ + + x + x x x|
| |___________A___M_______||_______________M__A__________________| |
+--------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 4 20.94 21.97 21.27 21.33 0.44773504
+ 4 20.33 21 20.71 20.6325 0.29033027
Difference at 95.0% confidence
-0.6975 +/- 0.652893
-3.27004% +/- 3.06092%
(Student's t, pooled s = 0.377332)
x sqrt-sys-rm
+ flat-sys-rm
+--------------------------------------------------------------------------+
|x + + * x + x |
| |_______________________A_____A____M________M__|____________||
+--------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 4 138.41 175.09 168.91 161.61 16.115177
+ 4 141.94 170.84 164.4 158.175 12.513687
No difference proven at 95.0% confidence
Don Lewis
Just for grins, you might want to try a "very-flat" experiment where you
create all 100000 files in the top directory.
Traditionally directories were always allocated in another cylinder
group than their parent, which would spread them all over the disk.
This turns out to be somewhat sub-optimal because it causes an excessive
amount of seek activity when traversing large directory trees. When the
dirpref code was added, it allowed a limited number of subdirectories to
be allocated using the same cylinder group as their parent, but I
suspect that the allocations will still be fairly well distributed when
running your benchmark.