Ext4 - high non-contiguous
redskolnikow
I have a question about ext4 filesystem. Yesterday I moved my system
and /home to ext4 and after fsck it shows the 59% non-contiguous of /
home partition. This level of fragmentation is a little bit strange
because I only moved my files to new partition.
Any ideas?
Thanks,
Konrad
John Reiser
> and /home to ext4 and after fsck it shows the 59% non-contiguous of /
> home partition. This level of fragmentation is a little bit strange
> because I only moved my files to new partition.
How, _exactly_, did you move your files to the new partition?
How many files, how much total space?
Which operating system, version, etc? ("uname -a")
--
redskolnikow
I copied them with cp -a or mc (tried two times, after reformatting
partition).
It's about 6000 files, mainly music (mp3 files), about 55GB.
My system is ArchLinux - 2.6.34-ARCH.
I search around the Google and I found some similar problems, but
there is no solution or explanation.
https://bbs.archlinux.org/viewtopic.php?id=66132
https://bbs.archlinux.org/viewtopic.php?id=62831
https://bbs.archlinux.org/viewtopic.php?id=69530
http://lists.debian.org/debian-user/2009/07/msg00376.html
Pascal Hambourg
redskolnikow a écrit :
> fsck it shows the 59% non-contiguous
Does anyone know what this number means exactly ?
GangGreene
It means the files are fragmented, ie the file(s) are not in one contious
block but have pieces(blocks) all over the filesystem.
John Reiser
>>> and /home to ext4 and after fsck it shows the 59% non-contiguous of /
>>> home partition. This level of fragmentation is a little bit strange
>>> because I only moved my files to new partition.
> I copied them with cp -a or mc (tried two times, after reformatting
> partition).
> It's about 6000 files, mainly music (mp3 files), about 55GB.
> My system is ArchLinux - 2.6.34-ARCH.
So the average file size is about 55GB / 6000 which is about 9MB,
and this is not small. If a 9MB file were in two extents of
3MB and 6MB, then it would still count towards the "59% non-contiguous"
even though it was in only two large extents. The percentage
non-contiguous omits some information about the distribution
of the number of extents per file.
Here's an experiment which may give some insight:
$ strace -o strace.out cp -a file1 file2
Then look in file strace.out for any system call in the set {fallocate,
fallocate64, truncate, truncate64, ftruncate, ftruncate64}. If not,
then 'cp' gives the filesystem no hint about the ultimate size of file2,
and it becomes harder to produce exactly one extent for the whole file.
The particular ext4 filesystem should be using the features
"persistent pre-allocation" and/or "delayed allocation"
to reduce the number of extents. See the Wikipedia page for ext4.
Also, look at all the 'write(' calls in strace.out. On my recent system
the blocksize was 32KiB, or about 1/275 of 9MB. With no pre-allocation
hint then it takes work to keep all 275 chunks contiguous.
For most current harddrives, the largest amount of data that can
be read or written in one command is 255 sectors of 512 bytes each,
or 127.5KiB. Native Command Queueing (NCQ) can chain commands,
but still the drive cannot read or write continuously. Larger
contiguous allocations do reduce seek time for one file, but
reaping the benefits requires no contention with any other file
on that spindle.
--
Pascal Hambourg
Thanks but I know what fragmentation is. I asked about the meaning of
the 59% number. According to John Reiser's reply, it is the percentage
of fragmented files, and depending on the average file length it does
not necessarily have a significant impact on performance.
Doug Freyburger
It means the folks who write and maintain the EXT4 file system code are
aware of the fact that contiguous is not automatically the optimal
layout for best performance.
The technology has changed greatly since the old paper "A fast, fat file
system of UNIX" that described the old BSD 4.1 UFS, but the most basic
of its lessons remains - There is no reason to conclude that contiguous
is better.
Defrag programs help non-technical folks feel they are proactive.