Why Does the EXT2 filesystem not need defragmentation.
mike
I have been looking for detailed explainations as to why
Linux ext2 does not have a fragmentation problem.
Mike
Christopher Browne
the artist formerly known as mike said:
> I have been looking for detailed explainations as to why
>Linux ext2 does not have a fragmentation problem.
The valid statement is that it does not have a _serious_ fragmentation
problem; see:
<http://www.ntlug.org/~cbbrowne/linuxkernel.html#DEFRAG>
for some relevant links.
--
(concatenate 'string "cbbrowne" "@" "acm.org")
<http://www.ntlug.org/~cbbrowne/>
If the automobile had improved as much as the computer in the last 25
years, it would get 1,000,000 miles to the gallon, cost $1, have a top
speed of 30,000 mph, and explode at least once a year, killing all
passengers.
Floyd Davidson
>Hi,
> I have been looking for detailed explainations as to why
>Linux ext2 does not have a fragmentation problem.
>
What would it accomplish? On a multi-user/multi-tasking OS that
buffers disk reads to whatever RAM is available, there just is
no significant performance hit due to disk fragmentation.
Due to multi-tasking for many users, actual disk reads are
scattered all about the disk anyway. And since any decent
modern OS puts disk reads into RAM buffers and therefore many
(or with enough RAM, most) disk read requests actually become
memory reads instead, once again there is no added performance
hit due to disk fragmentation.
None of the above relates directly to any specific file system.
--
Floyd L. Davidson fl...@barrow.com
Ukpeagvik (Barrow, Alaska)
Floyd Davidson
>Floyd Davidson:
>>mike <mi...@escape.com> wrote:
>>>Hi,
>>> I have been looking for detailed explainations as to why
>>>Linux ext2 does not have a fragmentation problem.
>>
>>buffers disk reads to whatever RAM is available, there just is
>>no significant performance hit due to disk fragmentation.
>>
>>Due to multi-tasking for many users, actual disk reads are
>>scattered all about the disk anyway. And since any decent
>>modern OS puts disk reads into RAM buffers and therefore many
>>(or with enough RAM, most) disk read requests actually become
>>memory reads instead, once again there is no added performance
>>hit due to disk fragmentation.
>>
>>None of the above relates directly to any specific file system.
>
>question.
I do not agree. It makes the actual question raised above
simply moot.
>a) Yes, when you get multitasking into play, there is some
> _inherent_ fragmentation of accesses _regardless_ of how
> data is physically organized.
>
> But that does not establish that there is "no fragmentation
> problem," only that there _is fragmentation_.
Not fragmentation. Insteed there is the same effect, as far a
disk reads, on performance as there would be with fragmentation
whether fragmentation exists or not.
The point is that there is no "fragmentation problem", or in
another way of saying it, there is not a problem which can be
significantly corrected (i.e., from which a significant
performance gain can be had) by reducing reasonable amounts of
disk fragmentation.
>b) Yes, cacheing is good; that diminishes the ill effects of
> fragmentation as the accesses come from RAM...
>
> That means that fragmentation is a bit less important than
> it might otherwise be.
Actually, it is more than "a bit". It means that when data is
repeatedly read (for example when shell scripts invoke many
commands again and again) there is not only not a performance
hit, but in fact the "repeated" nature of it causes an increase
in performance over a one time only operation.
>The _important_ point is that ext2 only suffers _significantly_
>in performance from fragmentation when you write to a
>filesystem that is very full, in which case you likely have
>bigger problems than from performance.
I don't think that is all that important. The effect is like
saying that using your fastest disk drive for swap and putting
the swap partition in the middle of the disk will cause a
performance increase. That may be true under one particular set
of circumstances; however, it is the worst case scenario and one
which everyone trys very hard to avoid simply by buying enough
RAM to rarely ever allow the system to actually run from swap.
In other words, by the time performance is so degraded that it
actually has an effect, who cares.
>That tends to be true for _any_ reasonably modern filesystem
>design; things should be pretty quick when the FS isn't
>terribly full, and you run into the _innate_ dilemma of having
>to do a lot of work to find free space to fit in new material.
>
>MS-DOS "FAT" suffers from the problem that it doesn't have a
>good way to "coalesce" free space; once you have consumed all
>the space on the partition, it can start "choking" even though
>there is lots of free space, basically because the space
>management scheme is 15 years old, and rather naive, ignoring
>the understanding of filesystems that existed at the time.
That is true fact. But (and this is a serious question because
I don't have enough experience with Microsoft OS's to know) does
anyone actually use DOS's FAT for anything other than floppies
anymore? Granted that everything MS has done since is grossly
hampered by some form of backward compatibility, and thus also
can be described as ignoring well understood design principles
used by everyone else, but regardless those are the filesystems
to compare to in this case, not MS-DOS "FAT".
(Heh, if you think MS-DOS's filesystem ignored modern concepts,
right at about the same time Bell Northern Research (BNR) put
winchester hard disks into their DMS line of telephone
switches... and since they had 9 track tape drives using IBM
formatting, you guessed it, the hard disk used a modified form
of the same filesystem as the tape. You wanna talk about
horrible ideas...!)
Christopher Browne
>mike <mi...@escape.com> wrote:
>>Hi,
>> I have been looking for detailed explainations as to why
>>Linux ext2 does not have a fragmentation problem.
>
>What would it accomplish? On a multi-user/multi-tasking OS that
>buffers disk reads to whatever RAM is available, there just is
>no significant performance hit due to disk fragmentation.
>
>Due to multi-tasking for many users, actual disk reads are
>scattered all about the disk anyway. And since any decent
>modern OS puts disk reads into RAM buffers and therefore many
>(or with enough RAM, most) disk read requests actually become
>memory reads instead, once again there is no added performance
>hit due to disk fragmentation.
>
>None of the above relates directly to any specific file system.
... All of which is true, and is _somewhat_ orthogonal to the
question.
a) Yes, when you get multitasking into play, there is some _inherent_
fragmentation of accesses _regardless_ of how data is physically
organized.
But that does not establish that there is "no fragmentation problem,"
only that there _is fragmentation_.
b) Yes, cacheing is good; that diminishes the ill effects of fragmentation
as the accesses come from RAM...
That means that fragmentation is a bit less important than it might
otherwise be.
The _important_ point is that ext2 only suffers _significantly_ in
performance from fragmentation when you write to a filesystem that is very
full, in which case you likely have bigger problems than from performance.
That tends to be true for _any_ reasonably modern filesystem design;
things should be pretty quick when the FS isn't terribly full, and you
run into the _innate_ dilemma of having to do a lot of work to find free
space to fit in new material.
MS-DOS "FAT" suffers from the problem that it doesn't have a good way
to "coalesce" free space; once you have consumed all the space on the
partition, it can start "choking" even though there is lots of free space,
basically because the space management scheme is 15 years old, and rather
naive, ignoring the understanding of filesystems that existed at the time.
--
aa...@freenet.carleton.ca - <http://www.ntlug.org/~cbbrowne/linux.html>
The proof of a system's value is its existence.
-- Alan Perlis
[Thus implying COBOL and JCL /do/ have some value after all! Ed.]
Peter T. Breuer
: mike <mi...@escape.com> wrote:
:> I have been looking for detailed explainations as to why
:>Linux ext2 does not have a fragmentation problem.
: What would it accomplish? On a multi-user/multi-tasking OS that
: buffers disk reads to whatever RAM is available, there just is
: no significant performance hit due to disk fragmentation.
Precisely! (more or less :-).
: Due to multi-tasking for many users, actual disk reads are
: scattered all about the disk anyway. And since any decent
: modern OS puts disk reads into RAM buffers and therefore many
: (or with enough RAM, most) disk read requests actually become
: memory reads instead, once again there is no added performance
: hit due to disk fragmentation.
: None of the above relates directly to any specific file system.
Amen.
The above is also why hardly anyone is going to bother to answer this
question. On the face of it it's obvious what's going on, and if
you want to dispute the rough reckoning implicit in the reasoning you
are going to have to get very very technical about use patterns and
the characteristics of various placement algorithms.
Ext2fs does not bung down things in 123 order on the disks. Yes,
individual files will more or less be contiguous, but then there'll be
a big gap till the "next" one you ask to be written. This means that
the disk doesn't run out of space to write individual files. But
it doesn't matter ... the same thing would be achieved on a fat16
file system by erasing arbitrary files to leave lots of holes in which
more files could subsequently be written. Unforunately dos would have
filled out those holes all wrong too!
This is a linux FAQ, btw. Answered in the linux FAQ.
Peter
Johan Kullstam
> That is true fact. But (and this is a serious question because
> I don't have enough experience with Microsoft OS's to know) does
> anyone actually use DOS's FAT for anything other than floppies
> anymore?
oh yes, MS is still using FAT. windows 9[58] doesn't have any other
filesystem (unless you count cd-rom iso9660). windows 9[58] uses FAT
(aka FAT16) and the newer FAT32 (which afaik is just FAT with more
bits in various allowing for more and hence smaller disk chunks).
windows NT has FAT16 and NTFS. NT does not do FAT32.
MS-DOS does neither FAT32 nor NTFS.
> Granted that everything MS has done since is grossly
> hampered by some form of backward compatibility, and thus also
> can be described as ignoring well understood design principles
> used by everyone else, but regardless those are the filesystems
> to compare to in this case, not MS-DOS "FAT".
there's not much excuse for MS not producing a better filesystem.
FAT32 broke compatibility with either DOS or NT. they may as well
have fielded a decent FS. i guess marketing figured that a lousy FS
would drive more people into the big buy of NT.
--
J o h a n K u l l s t a m
[kull...@ne.mediaone.net]
sysengr
Quentin Christensen
>I don't think that is all that important. The effect is like
>saying that using your fastest disk drive for swap and putting
>the swap partition in the middle of the disk will cause a
>performance increase. That may be true under one particular set
>of circumstances; however, it is the worst case scenario and one
>which everyone trys very hard to avoid simply by buying enough
>RAM to rarely ever allow the system to actually run from swap.
>
This reminds me of a question, which I think I've seen a partial answer to
somewhere....
I have 256 MB of RAM on my P3 with about double the hard disk space I actually
use (8GB). I don't run any REALLY intensive programs, about the most memory
intensive I get is running the gimp with a several hundred kilobite image under
KDE. Should I have a swap partition with this much RAM? If so, how big should
it be? I don't actually have one at the moment, and things seem to be running
ok, although I haven't had that much experience with linux, so I'm not sure if
I could get better performance or not.
I remember reading a couple of docs on the issue, but they were talking about
systems with < 32 MB of RAM which is a different scenario to mine altogether :)
Another thing I'd like to mention is that yesterday I rebuilt the kernel for
the first time - and it worked! Although I did have a couple of stuff-ups
where I forgot to run lilo afterwards and it wouldn't boot... and I tried to
use the bzImage in the usr/src/linux directory rather than
/usr/src/linux/arch/i386 ... and it complained that it was too big...
How big should the kernel be (as another question :) After compiling it with
what I thought was a reasonably minimal set of specs for my system, the bzImage
is 800-something k and when I look at free (without running anything after
bootup) it says it's using 40MB RAM... Is that too big? On my system it
doesn't seem so bad, but it does seem big...
Regards
Quentin.
--
My Win9x Cursors: http://www.ozemail.com.au/~mynx/quentisl/cursors.html
Please don't send me junk leaves! (take them out before replying).
No Silicon Heaven? But where do all the calculators go? - Kryten.
Tony Lawrence
>
> In comp.os.linux.misc, on 14 Sep 2000, Floyd Davidson announced:
>
> >I don't think that is all that important. The effect is like
> >saying that using your fastest disk drive for swap and putting
> >the swap partition in the middle of the disk will cause a
> >performance increase. That may be true under one particular set
> >of circumstances; however, it is the worst case scenario and one
> >which everyone trys very hard to avoid simply by buying enough
> >RAM to rarely ever allow the system to actually run from swap.
> >
>
> This reminds me of a question, which I think I've seen a partial answer to
> somewhere....
>
> I have 256 MB of RAM on my P3 with about double the hard disk space I actually
> use (8GB). I don't run any REALLY intensive programs, about the most memory
> intensive I get is running the gimp with a several hundred kilobite image under
> KDE. Should I have a swap partition with this much RAM? If so, how big should
> it be? I don't actually have one at the moment, and things seem to be running
> ok, although I haven't had that much experience with linux, so I'm not sure if
> I could get better performance or not.
Physical memory + swap - kernel memory = virtual memory that
you can use for apps. If no app needs more than what you
have, you don't need swap. However- if your kernel
panicked, it will want to dump- and that dump couuld be a
little more than 256MB. See
http://pcunix.com/Boot/swap.html for a discussion of this on
a different Unix.
--
Tony Lawrence (to...@aplawrence.com)
SCO/Linux articles, help, book reviews, tests,
job listings and more : http://www.pcunix.com
Floyd Davidson
>Floyd Davidson announced:
>
>>I don't think that is all that important. The effect is like
>>saying that using your fastest disk drive for swap and putting
>>the swap partition in the middle of the disk will cause a
>>performance increase. That may be true under one particular
>>set of circumstances; however, it is the worst case scenario
>>and one which everyone trys very hard to avoid simply by
>>buying enough RAM to rarely ever allow the system to actually
>>run from swap.
>
>This reminds me of a question, which I think I've seen a
>partial answer to somewhere....
>
>I have 256 MB of RAM on my P3 with about double the hard disk
>space I actually use (8GB).
The number one fact to consider here is that whatever swap space
you do allocate, it is *free*. You already own it and are not
using it for anything else. Hence you can certainly afford to
be generious.
> I don't run any REALLY intensive programs, about the most
>memory intensive I get is running the gimp with a several
>hundred kilobite image under KDE. Should I have a swap
>partition with this much RAM? If so, how big should it be? I
>don't actually have one at the moment, and things seem to be
>running ok, although I haven't had that much experience with
>linux, so I'm not sure if I could get better performance or
>not.
There are some other considerations. Linux buffers disk reads,
and uses otherwise unused RAM for that purpose. Hence the more
RAM you have available for disk buffering, the faster your
programs will generally run. Linux will also swap out unused
processes, making the space in RAM they would otherwise take up
available for disk buffering. Hence you do want _some_ swap
space allocated.
I have a system similar to yours, in that it has 256Mb of RAM
and often is lightly loaded. Here is what free says right now:
total used free shared buffers cached
Mem: 257972 242560 15412 30872 62268 62000
-/+ buffers/cache: 118292 139680
Swap: 315048 15928 299120
Which indicates that 16Mb has been swapped out, and is available
for disk buffering. (You can also get an idea how much swap I
might recommend. :-)
The actual virtual memory (RAM + swap) you *must* have amounts
to "more than the need for virtual memory will ever be". And
remember that if your vm use exceeds the RAM + swap total, your
system will crash. So the question is, how much vm could your
box possibly ever use? Since for _you_, swap is free... pick
some nice large figure that boggles your mind (if you had ever
bought a computer 15-20 years ago, just idea of having half a
meg of virtual memory is worth what it is going to cost you!).
(And if you use Netscrape, at some point the amount of vm it
uses will be all there is, no matter how much you have. In that
case you want enough that you'll notice it before the crash...)
In my case I occasionally do some heavy duty image editing, and
have actually seen this system use 400Mb. Hence I have more
than 500Mb of virtual memory. It rarely ever uses more than
200Mb of vm for processes, so I have 256Mb of that vm as RAM.
Floyd
Jean-David Beyer-valinux
> There are some other considerations. Linux buffers disk reads,
> and uses otherwise unused RAM for that purpose. Hence the more
> RAM you have available for disk buffering, the faster your
> programs will generally run.
I suppose so, but it depends a lot on your access patterns. I
suspect that most of the stuff in my buffers and cache are not
useful: I am rummaging around in a large database, running a 2-hour
populate of the database. All the data coming in are new, and I
read precious little of the data that are already there. The
indices are in the cache (I hope), and that can help, but since I
am mostly inserting new records, I imagine that the cache is mostly
full of useless stuff. My disk transfer rate is about 150K
bytes/second, and I have seen the disk system can go as high as
10Megabytes per second on a regular basis, and sometimes a little
higher. Since the two hard drives have 2 Megabyte buffers and are
on an Ultra-2 SCSI, I imagine the upper limit is around 40
Megabytes per second, though I have not seen that.
> Linux will also swap out unused
> processes, making the space in RAM they would otherwise take up
> available for disk buffering. Hence you do want _some_ swap
> space allocated.
>
> I have a system similar to yours, in that it has 256Mb of RAM
> and often is lightly loaded. Here is what free says right now:
>
> total used free shared buffers cached
> Mem: 257972 242560 15412 30872 62268 62000
> -/+ buffers/cache: 118292 139680
> Swap: 315048 15928 299120
>
> Which indicates that 16Mb has been swapped out, and is available
> for disk buffering. (You can also get an idea how much swap I
> might recommend. :-)
Mine is:
total: used: free: shared: buffers:
cached:
Mem: 529330176 526209024 3121152 89923584 207638528 190623744
Swap: 279642112 17080320 262561792
This seems to support my view that no matter what the amount of RAM
(iff there is "enough"), Linux uses about 16 Megabytes of swap
space. ;-)
> The actual virtual memory (RAM + swap) you *must* have amounts
> to "more than the need for virtual memory will ever be". And
> remember that if your vm use exceeds the RAM + swap total, your
> system will crash.
Is this true of Linux? In the old days (around mid 1980s), a
UNIX system would not crash when you ran out of memory. The fork(2)
routine would just give an error return. If a shell got that, it
would just write "cannot fork", but the system would remain up. Of
course, the sysadmin would start looking around for the memory hog.
Somewhere around that time, they reserved a little memory and a
process table slot for the super-user so that something could be
done.
> So the question is, how much vm could your
> box possibly ever use? Since for _you_, swap is free... pick
> some nice large figure that boggles your mind (if you had ever
> bought a computer 15-20 years ago, just idea of having half a
> meg of virtual memory is worth what it is going to cost you!).
>
> (And if you use Netscrape, at some point the amount of vm it
> uses will be all there is, no matter how much you have. In that
> case you want enough that you'll notice it before the crash...)
On my systems, Netscape uses around 24 Megabytes, but it does not
grow unbounded, even when I open multiple windows with it. Others
report memory leaks, but I have never seen any. Does the memory
leak without appearing in the Netscape line of the top command?
--
.~. Jean-David Beyer Registered Linux User 85642.
/V\ Registered Machine 73926.
/( )\ Shrewsbury, New Jersey
^^-^^ 8:12am up 37 days, 15:39, 2 users, load average: 3.12, 3.17, 3.13
Floyd Davidson
>Floyd Davidson wrote (in part):
>
>> There are some other considerations. Linux buffers disk reads,
>> and uses otherwise unused RAM for that purpose. Hence the more
>> RAM you have available for disk buffering, the faster your
>> programs will generally run.
>
>I suppose so, but it depends a lot on your access patterns. I
>suspect that most of the stuff in my buffers and cache are not
>useful: I am rummaging around in a large database, running a 2-hour
>populate of the database. All the data coming in are new, and I
>read precious little of the data that are already there. The
>indices are in the cache (I hope), and that can help, but since I
>am mostly inserting new records, I imagine that the cache is mostly
>full of useless stuff.
Your particular userid may not, but then again it might. Shell
scripts call binaries in /bin, /usr/bin, and /usr/local/bin
often, and various data files and configuration files are also
accessed often.
Hence the quantity of re-usable data may be low, but it is also
possible even though small it might account for a much higher
percentage of disk activity than you might expect.
Additionally, the other data that you are reading might well be
available in sequential reads... which means the others being
accessed from memory speeds up access to those that are read
from disk by not mixing non-sequential disk reads in the
process.
Suffice it to say that there is no doubt at least some benefit,
and it is essentially free because it uses a resource that would
otherwise be idle.
>> Linux will also swap out unused
>> processes, making the space in RAM they would otherwise take up
>> available for disk buffering. Hence you do want _some_ swap
>> space allocated.
>>
>> I have a system similar to yours, in that it has 256Mb of RAM
>> and often is lightly loaded. Here is what free says right now:
>>
>> total used free shared buffers cached
>> Mem: 257972 242560 15412 30872 62268 62000
>> -/+ buffers/cache: 118292 139680
>> Swap: 315048 15928 299120
>>
>> Which indicates that 16Mb has been swapped out, and is available
>> for disk buffering. (You can also get an idea how much swap I
>> might recommend. :-)
>
>Mine is:
>
> total: used: free: shared: buffers:
>cached:
>Mem: 529330176 526209024 3121152 89923584 207638528 190623744
>Swap: 279642112 17080320 262561792
>
>This seems to support my view that no matter what the amount of RAM
>(iff there is "enough"), Linux uses about 16 Megabytes of swap
>space. ;-)
Not really. It does mean that you and I have just about the
same amount of memory allocated to processes that are sleeping
virtually all of the time. Right now, on my box that happens to
be kflushd, kupdate, kpiod, kswapd, bash (five instances),
agetty (two instances), rpciod, lockd, startx, xinit, gnuserv,
and pppd.
Obviously I could run either more or fewer processes that would
result in the use of swap space. :-)
>> The actual virtual memory (RAM + swap) you *must* have amounts
>> to "more than the need for virtual memory will ever be". And
>> remember that if your vm use exceeds the RAM + swap total, your
>> system will crash.
>
>Is this true of Linux? In the old days (around mid 1980s), a
>UNIX system would not crash when you ran out of memory. The
>fork(2) routine would just give an error return. If a shell got
>that, it would just write "cannot fork", but the system would
>remain up. Of course, the sysadmin would start looking around
>for the memory hog. Somewhere around that time, they reserved
>a little memory and a process table slot for the super-user so
>that something could be done.
Sure the system "remains up", but it is in a worthless state
until the sysadmin determines what has been arbitrarily killed
off and what has not. The only sensible response is to reboot
immediately after discovering which process is hogging memory.
I may be a bit more liberal about the definition of "crash" than
most, but clearly a forced reboot even if it is at my leisure, is
a "crash".
>> (And if you use Netscrape, at some point the amount of vm it
>> uses will be all there is, no matter how much you have. In that
>> case you want enough that you'll notice it before the crash...)
>
>On my systems, Netscape uses around 24 Megabytes, but it does not
>grow unbounded, even when I open multiple windows with it. Others
>report memory leaks, but I have never seen any. Does the memory
>leak without appearing in the Netscape line of the top command?
I don't remember the specifics now, but it has been positively
identified. If I remember right it has something to do with
Java applets, which means if you access only the right web pages
it will never hit you at all. I can often go for a week or two
and never have a problem. Then I'll start searching for some
odd thing and after I've accessed a couple dozen odd sites I
notice that everything is slow to start up... and what I find is
that I have 200Mb of swap in use!
Peter
Browne) wrote:
>In our last episode (Wed, 13 Sep 2000 16:32:57 -0400),
>the artist formerly known as mike said:
>> I have been looking for detailed explainations as to why
>>Linux ext2 does not have a fragmentation problem.
>
>The valid statement is that it does not have a _serious_ fragmentation
>problem; see:
> <http://www.ntlug.org/~cbbrowne/linuxkernel.html#DEFRAG>
>for some relevant links.
The explanations are great but the laws of physics do not change.
A medium quality image is 50Mb - 100Mb. Keeping only 8Kb of the image
contiguous does not help.
A decent size disk partition formatted with FAT has an allocation unit
of 32Kb or more. The 8Kb of ext2 is sounding a little fragmented.
Defragmenting disks includes a lot more than defragmenting files.
Defragmenting is a science practiced for at least 30 years. The rules
for good defragmenting have not changed for about 25 of those years.
Cheap RAM simply reduces the cost of not defragmenting disks. (The
rules changed 25 years ago because plug compatible memory
manufacturers appeared and that changed the relative cost of main
memory versus hardware buffers.)
Working with an entry level workstation (256Mb memory) and a small
local database (about 10Gb + several indexes up to 1Gb), just about
everything on disk is primitive and inadequate. Fragmentation costs
and will continue to cost until disk design is modernized or RAM size
exceeds disk size.
NTFS has some good ideas that would help but the basic hardware still
has to change.
- NTFS contains 2 file systems, one for small files and a different
one for large files. I do not claim that either is a good file system,
only that there is a value in having a file system vary based on file
size.
- NTFS lets the user set the allocation unit size by partition. If
that function were automated and available to disk management
software, the disk management software could improve disk performance
by matching the allocation unit to the files. Directories full of clip
art could be placed in a partition with a small allocation unit like
2Kb. Directories full of images could be in a partition with a larger
allocation unit like 64Kb.
In the early 80s, I publicly advocated changing disk sectors from
0.5Kb to 1Kb to keep disks up to date. Last year I advocated 4Kb as
the minimum. This year I suggest the disks manufacturers move straight
to 64Kb to directly support RAID stripes.
Once the sector matches the RAID stripe size, all operating systems
can be updated to use RAID like access. The effect of disk
geometry/hardware buffers/transfer speed/organization/RAM size would
become less important as most of the organization would be moved up to
RAM.
In the past disk and RAM was so expensive, the disk allocation unit
had to be small to conserve resources. With disk and RAM now so cheap,
the disk allocation unit can be larger than the most common file size.
There are serious reasons why RAID uses 64Kb stripes and why newer
systems are already using 128Kb.
I suggest a modern file system would let you specify the allocation
unit by partition, the minimum size would match RAID option in the
operating system and a hard disk, to be called modern, should use the
same size for sectors.
At that point there is still value in having a daily, weekly or
monthly sweep of the disk to improve performance. Linux already has a
broom sweeping the disks every night. Adding defragmentation is simply
using a better a broom.
The current broom picks up McDonalds wrappers but not the used
syringes underneath..
Peter T. Breuer
: On 13 Sep 2000 21:46:43 GMT, cbbr...@news.amrcorp.com (Christopher
: Browne) wrote:
:>In our last episode (Wed, 13 Sep 2000 16:32:57 -0400),
:>the artist formerly known as mike said:
:>> I have been looking for detailed explainations as to why
:>>Linux ext2 does not have a fragmentation problem.
:>The valid statement is that it does not have a _serious_ fragmentation
:>problem; see:
:> <http://www.ntlug.org/~cbbrowne/linuxkernel.html#DEFRAG>
: The explanations are great but the laws of physics do not change.
: A medium quality image is 50Mb - 100Mb. Keeping only 8Kb of the image
: contiguous does not help.
Yes it DOES. Go back to school. (hint: consider which objective
function you are or ought to be minimizing, and what the distribution
of access patterns looks like).
Peter