max number of directories inside a dir
Xah Lee
directory?
On our production box on NetApp with UFS
we have a directory that has about 38000 first level
subdirectories. When this is tarred up and transfered to my personal
dev box Ulta 5, and i tried to untar it, i get an error like "too many
links" after about 3 hours of untar.
I wrote a perl script that creates directoris just to see what's the
maximum, and it turns out to be 32765.
my disk has sufficient space and inode. (see below my sig)
can anyone tell me, what's the the factor that control the number of
directories one can create immediate below a given dir?
and whether this file system config can be dynamically updated or do i
have to create a new file system? (or if this is entirely somethnig
else)
Xah
x...@xahlee.org
http://xahlee.org/PageTwo_dir/more.html
dir making perl script
#!/usr/local/bin/perl
use strict;
my $path = ('/www/super_bucket/massive_files/');
chdir($path);
for (1..10) {system(mkdir $_);}
shell session that shows the maxed out dir
[xah@hypatia ~][Fri Apr 19,17:46:32]
mkdir /www/super_bucket/massive_dirs/t
mkdir: cannot make directory `/www/super_bucket/massive_dirs/t': Too
many links
[xah@hypatia ~][Fri Apr 19,17:46:47]
rmdir /www/super_bucket/massive_dirs/1
[xah@hypatia ~][Fri Apr 19,17:46:55]
mkdir /www/super_bucket/massive_dirs/t
[xah@hypatia ~][Fri Apr 19,17:46:58]
ls -d /www/super_bucket/massive_dirs/t
/www/super_bucket/massive_dirs/t
[xah@hypatia ~][Fri Apr 19,17:47:35]
mkdir /www/super_bucket/massive_dirs/1
mkdir: cannot make directory `/www/super_bucket/massive_dirs/1': Too
many links
[xah@hypatia ~][Fri Apr 19,17:48:21]
rmdir /www/super_bucket/massive_dirs/t
[xah@hypatia ~][Fri Apr 19,17:48:29]
mkdir /www/super_bucket/massive_dirs/1
[xah@hypatia ~][Fri Apr 19,17:48:35]
/usr/bin/df -k
Filesystem kbytes used avail capacity Mounted on
/dev/dsk/c0t0d0s0 15457218 8802041 6500605 58% /
/proc 0 0 0 0% /proc
fd 0 0 0 0% /dev/fd
mnttab 0 0 0 0% /etc/mnttab
swap 843096 16 843080 1% /var/run
swap 843784 704 843080 1% /tmp
/dev/dsk/c0t0d0s7 3117942 538955 2516629 18% /export/home
[xah@hypatia ~][Fri Apr 19,17:50:00]
/usr/bin/df -F ufs -o i
Filesystem iused ifree %iused Mounted on
/dev/dsk/c0t0d0s0 305639 1648281 16% /
/dev/dsk/c0t0d0s7 27314 368974 7% /export/home
[xah@hypatia ~][Fri Apr 19,17:50:27]
Darren Dunham
> what's is the maximum number of directories one can create under a
> directory?
Something close to MAXLINKS - 3. Where...
sys/param.h:#define MAXLINK 32767 /* max links */
Each directory has a hard link in all subdirectories (it's called ..) so
eventually you hit this limit.
You can't change it without changing the UFS filesystem you're using.
That would require a *lot* of word and would probably be incompatible
with other filesystems, so I've never heard of anyone doing it.
> On our production box on NetApp with UFS
Huh? Netapp has their own stuff, and it's not UFS at all...
> we have a directory that has about 38000 first level
> subdirectories. When this is tarred up and transfered to my personal
> dev box Ulta 5, and i tried to untar it, i get an error like "too many
> links" after about 3 hours of untar.
Right... I'm guessing their filesystem simply doesn't have this
specific limitation.
> I wrote a perl script that creates directoris just to see what's the
> maximum, and it turns out to be 32765.
> my disk has sufficient space and inode. (see below my sig)
> can anyone tell me, what's the the factor that control the number of
> directories one can create immediate below a given dir?
> and whether this file system config can be dynamically updated or do i
> have to create a new file system? (or if this is entirely somethnig
> else)
Nope. Never heard of anyone getting around that limit. I don't know if
VxFS has that limit or not.
--
Darren Dunham ddu...@taos.com
Unix System Administrator Taos - The SysAdmin Company
Got some Dr Pepper? San Francisco, CA bay area
< How are you gentlemen!! Take off every '.SIG'!! >
Rich Teer
> what's is the maximum number of directories one can create under a
> directory?
>
> On our production box on NetApp with UFS
> we have a directory that has about 38000 first level
> subdirectories. When this is tarred up and transfered to my personal
> dev box Ulta 5, and i tried to untar it, i get an error like "too many
> links" after about 3 hours of untar.
>
> I wrote a perl script that creates directoris just to see what's the
> maximum, and it turns out to be 32765.
> my disk has sufficient space and inode. (see below my sig)
>
> can anyone tell me, what's the the factor that control the number of
> directories one can create immediate below a given dir?
The number of links is stored in the inode in a signed short,
which is 16 bits wide (in a UFS file system). You'll need to
decrease the number of first level directories.
--
Rich Teer
President,
Rite Online Inc.
Voice: +1 (250) 979-1638
URL: http://www.rite-online.net
Darren Dunham
> Nope. Never heard of anyone getting around that limit. I don't know if
> VxFS has that limit or not.
It does (at least on solaris). I don't know if it's strictly a
filesystem limit that happens to be shared by both UFS and VxFS or if
it's more systemic than that.
Chris Thompson
Rich Teer <ri...@rite-group.com> wrote:
>On 19 Apr 2002, Xah Lee wrote:
>
>> what's is the maximum number of directories one can create under a
>> directory?
>>
>> On our production box on NetApp with UFS
>> we have a directory that has about 38000 first level
>> subdirectories. When this is tarred up and transfered to my personal
>> dev box Ulta 5, and i tried to untar it, i get an error like "too many
>> links" after about 3 hours of untar.
>>
>> I wrote a perl script that creates directoris just to see what's the
>> maximum, and it turns out to be 32765.
>> my disk has sufficient space and inode. (see below my sig)
>>
>> can anyone tell me, what's the the factor that control the number of
>> directories one can create immediate below a given dir?
>
>The number of links is stored in the inode in a signed short,
>which is 16 bits wide (in a UFS file system). You'll need to
>decrease the number of first level directories.
And it's an unsigned short, with a maximum of 65535, in the NetApp filing
system. Hence your difficulty in transferring from that to UFS.
Chris Thompson
Email: cet1 [at] cam.ac.uk