SSD mystery deepens

[root]

All speed tests mentioned here come from hdparm -t /dev/sda

In an earlier post I wondered about the decreasing performance
of my 240GB Corsair F3 SSD. Some useful suggestions were
given about changes to be made in /etc/fstab to ensure
best performance and longer life for the SSD. Among others,
these suggestions include noatime, discard, nodiratime.


I also read the Wikipedia article on SSD which mentioned
TRIM. I made the changes suggested and rebooted my system.
The Wiki article suggested that it is not good to have
your SSD run near capacity. I had partitioned the drive
into two partitions of 120GB each. So, I used gdisk
(previously I used fdisk) to wipe the partition information
and create a new 240GB partition (really 227GB).
I re-installed my OS and found a 35% speed degradation.

Here is a summary of my experience.

1. Immediately after buying the drive and installing it.
no OS, no FS speed= 475MB/s
2. After partitioning into 2 120GB partitions and installing
an ext4 system with my OS speed=318 MB/s
3. After re-partitioning with 1 240GB partition, ext4 FS,
and my OS speed=228MB/s

At this point I decided to return the drive, each install
of the OS caused about a 33% reduction in the speed

In preparation for returning the drive I nulled out
the drive with:
dd if=/dev/zero of=/dev/sdb (I had switched the drive to sdb).

4. After zeroing out the entire drive I ran another speed test
and found speed=386MB/s
5. Encouraged I put an ext4 FS but no OS: speed=479MB/s

6. Wow, I was back in business, so this time I re-installed
the OS and was back to speed=228MB/s

I am completely baffled by these results. Anyone with the
slightest idea of what is happening?

I have sent email to Corsair but I don't expect anything
from that.

[Pascal Hambourg]

Hello,

root a écrit :

>
> 1. Immediately after buying the drive and installing it.
> no OS, no FS speed= 475MB/s
> 2. After partitioning into 2 120GB partitions and installing
> an ext4 system with my OS speed=318 MB/s
> 3. After re-partitioning with 1 240GB partition, ext4 FS,
> and my OS speed=228MB/s

[...]

> In preparation for returning the drive I nulled out
> the drive with:
> dd if=/dev/zero of=/dev/sdb (I had switched the drive to sdb).
>
> 4. After zeroing out the entire drive I ran another speed test
> and found speed=386MB/s
> 5. Encouraged I put an ext4 FS but no OS: speed=479MB/s
>
> 6. Wow, I was back in business, so this time I re-installed
> the OS and was back to speed=228MB/s
>
> I am completely baffled by these results. Anyone with the
> slightest idea of what is happening?

Of course I may be wrong, but all this seems to confirm my theory that
the disk controller knows when blocks do not contain data and do not
actually read from them. So the actual read speed of actual data (which
is what matters, there is no point in reading blocks that do not contain
data) seems to be 228 MB/s.

[root]

The results are somewhat repeatable, except that I am
unable to regain the 318MB/s speed that first obtained.

The 228MB/s speed holds whether I use fdisk or gdisk
to partition the drive, and whether the drive is
partitioned as a single 240GB partition or 2 120GB
partitions.

[root]

When the drive is zeroed out the drive does contain data.
I get the fastest speed when an EXT4 file system is
on the drive. Faster then when the drive has just been
zeroed out.

[root]

I think I understand what the problem is: the hdparm test
doesn't mean anything. I just ran a real test that involved
lots of real disk i/o. The results with my sata3 spinning
disk took 15m 16s. The same test, with the same data
on the ssd took 4m 4s. So the ssd is at least 3.75
as fast. I say at least because there was considerable
output to the screen during the test.

I think this will finish me efforts with the ssd. I want
to thank everyone for contributing to the thread.

[Pascal Hambourg]

root a ï¿œcrit :

>
> I think I understand what the problem is: the hdparm test
> doesn't mean anything.

I disagree : it means something. You just need to understand what hdparm
tests and what it means. hdparms measures the sequential read speed by
sequentially reading a few blocks from the beginning of the device.

> I just ran a real test that involved
> lots of real disk i/o. The results with my sata3 spinning
> disk took 15m 16s. The same test, with the same data
> on the ssd took 4m 4s. So the ssd is at least 3.75
> as fast.

Hdparm speed test does not involve access time, which is much lower on
an SSD than on a hard disk. Any real use test will involve access time.

[crankypuss]

I don't understand how you can possibly derive accurate raw speed
numbers when working through an operating system that is running various
daemons and getting on with its usual paging activities. The only way I
can imagine actually achieving that would be to insert some monitoring
hardware between the SSD and its socket.

I'm relatively new to linux and don't understand the benefit of the ext4
filesystem over ext2, from a quick look it seems like all it adds is
journaling at the cost of some space-efficiency, and I'm not sure what
journaling is really good for unless your databases rely on it.

Seems to me that the best you're going to be able to achieve is an
apples-to-apples comparison and it sounds like you're working with
apples/oranges.

[Eef Hartman]

crankypuss <n...@email.thanks> wrote:
> I'm relatively new to linux and don't understand the benefit of the ext4
> filesystem over ext2, from a quick look it seems like all it adds is
> journaling at the cost of some space-efficiency,

No, that (journalling) and some extra options like directory indexing
is the difference between ext3 and ext2.
Ext4 is completely rewritten, finally adds extends and a LOT of other
fs capabilities (like larger then 8 TB partitions) to ext3.
--
******************************************************************
** Eef Hartman, Delft University of Technology, dept. SSC/ICT **
** e-mail: E.J.M....@tudelft.nl - phone: +31-15-27 82525 **
******************************************************************

[Richard Kettlewell]

crankypuss <n...@email.thanks> writes:
> I don't understand how you can possibly derive accurate raw speed
> numbers when working through an operating system that is running
> various daemons and getting on with its usual paging activities. The
> only way I can imagine actually achieving that would be to insert some
> monitoring hardware between the SSD and its socket.

Indeed, hdparm's documentation mentions the need to use an
otherwise-idle system.

> I'm relatively new to linux and don't understand the benefit of the
> ext4 filesystem over ext2, from a quick look it seems like all it adds
> is journaling at the cost of some space-efficiency, and I'm not sure
> what journaling is really good for unless your databases rely on it.

The advantage of journaling is that your computer doesn't spend ages
running fsck (or equivalent) to - sometimes inaccurately! - recover your
filesystem after a crash or power failure.

--
http://www.greenend.org.uk/rjk/

[root]

Pascal Hambourg <boite-...@plouf.fr.eu.org> wrote:
> root a écrit :

I don't understand how the hdparm test can vary by a factor of
2 to 3 depending upon whether the drive has been "zeroed" out,
whether it contains an ext4 file system, and whether it contains
my working slackware OS. I tested all three of these cases and
hdparm -t varied from 228MB/s to 475MB/s.

[root]

I run two servers that have 6 and 10Tb of data respectively. In case
of power failure under EXT2 those systems took many hours to
recover. With EXT4 they are back up in seconds.

[bad sector]

On 01/11/2012 06:07 AM, Richard Kettlewell wrote:
> crankypuss<n...@email.thanks> writes:
>> I don't understand how you can possibly derive accurate raw speed
>> numbers when working through an operating system that is running
>> various daemons and getting on with its usual paging activities. The
>> only way I can imagine actually achieving that would be to insert some
>> monitoring hardware between the SSD and its socket.
>
> Indeed, hdparm's documentation mentions the need to use an
> otherwise-idle system.

i would be interested only in real-life data and an idle sytem would not
do even if i could have one.

instead of various benchmarks i'd read and write some large fixed-size
folder back and forth and compare the results. This is how i came to
decide to put ssd's on hold for the foreseeable future.

I came to a similar conclusion (which i have yet to confirm for my own
satisfaction) with video cards, an old gt6600 and a new 450 something or
whaaaaatever, NO SIGNIFICANT OBSERVABLE real-life improvement.

[root]

bad sector <none@_INVALID_.net> wrote:
>
> i would be interested only in real-life data and an idle sytem would not
> do even if i could have one.
>
> instead of various benchmarks i'd read and write some large fixed-size
> folder back and forth and compare the results. This is how i came to
> decide to put ssd's on hold for the foreseeable future.
>

Reading/writing the same large file would all be done in cache
on my system, maybe yours. That would eliminate any performance
difference between ssd and spinning disks. I don't mean the cache
on the controller, I mean the system cache that is made from your
available ram. In my test case I was reading and processing several
thousand small files. When I eliminated screen i/o the speed
of the ssd was 10x that of the spinning disk.

> I came to a similar conclusion (which i have yet to confirm for my own
> satisfaction) with video cards, an old gt6600 and a new 450 something or
> whaaaaatever, NO SIGNIFICANT OBSERVABLE real-life improvement.
>

Gamers see a difference in video cards. Once you max out the resolution
of your monitor they are all the same to me.

[bad sector]

On 01/11/2012 01:42 PM, root wrote:
> bad sector<none@_INVALID_.net> wrote:
>>
>> i would be interested only in real-life data and an idle sytem would not
>> do even if i could have one.
>>
>> instead of various benchmarks i'd read and write some large fixed-size
>> folder back and forth and compare the results. This is how i came to
>> decide to put ssd's on hold for the foreseeable future.
>>
>
> Reading/writing the same large file would all be done in cache
> on my system, maybe yours. That would eliminate any performance
> difference between ssd and spinning disks. I don't mean the cache
> on the controller, I mean the system cache that is made from your
> available ram. In my test case I was reading and processing several
> thousand small files. When I eliminated screen i/o the speed
> of the ssd was 10x that of the spinning disk.

I meant a large folder with all maner of occupancy, by fixed size I
meant manipulating a known quantity and not differetnt ones. The idea
being to get a practical eye on the process. I'm sure I would have
'eyeballed' any improvement of the order of 20% or more, didn't see any,
whatever the reason. I guess another way would be to boot a system,
have it do a tb's worth of cron moves & copies and then shutdown. Repeat
the same with the ssd and tally it all up. My point is that unless you
can show a significant measurable and practical improvement then there
is no case.

>> I came to a similar conclusion (which i have yet to confirm for my own
>> satisfaction) with video cards, an old gt6600 and a new 450 something or
>> whaaaaatever, NO SIGNIFICANT OBSERVABLE real-life improvement.
>>
>
> Gamers see a difference in video cards. Once you max out the resolution
> of your monitor they are all the same to me.

the closest thing I use is tuxracer & googleearth :-)

[Martin]

root wrote:

> I think I understand what the problem is: the hdparm test
> doesn't mean anything.

well, the same is true for all benchmarks: they measure something, but the
interpretation is up to the beholder.

> I just ran a real test that involved
> lots of real disk i/o. The results with my sata3 spinning
> disk took 15m 16s. The same test, with the same data
> on the ssd took 4m 4s. So the ssd is at least 3.75
> as fast. I say at least because there was considerable
> output to the screen during the test.

That's what I meant by a more "seeky" scenario in the other thread. Bear in
mind, local memory access is in the magnitude of nanoseconds, rotational
latencies in the area of microseconds and seek times in the area of
milliseconds. When a rotational drive seeks across the whole platter the
latency is visible to the naked eye.

So the point about SSDs is not that they are so good but that rotational
drives are so bad. :)

>
> I think this will finish me efforts with the ssd. I want
> to thank everyone for contributing to the thread.

welcome

[Pascal Hambourg]

Martin a écrit :

>
> local memory access is in the magnitude of nanoseconds, rotational
> latencies in the area of microseconds and seek times in the area of
> milliseconds.

What do you call "rotational latency" ? If it is what manufacturers call
"latency time", it is in the area of milliseconds, not microseconds.

[crankypuss]

On 01/11/2012 03:54 AM, Eef Hartman wrote:
> crankypuss<n...@email.thanks> wrote:
>> I'm relatively new to linux and don't understand the benefit of the ext4
>> filesystem over ext2, from a quick look it seems like all it adds is
>> journaling at the cost of some space-efficiency,
>
> No, that (journalling) and some extra options like directory indexing
> is the difference between ext3 and ext2.
> Ext4 is completely rewritten, finally adds extends and a LOT of other
> fs capabilities (like larger then 8 TB partitions) to ext3.

I'll need to dig around one of these days to really grasp the available
filesystems. Aside from support for larger than 8TB partitions, do you
still consider ext4 the new go-to system?

[crankypuss]

Sounds like goodness to me.

[Aragorn]

On Thursday 12 January 2012 10:03, crankypuss conveyed the following to
alt.os.linux...

ext4 is certainly a very performant and reliable filesystem, but whether
it is better than anything else is open for debate, and such debate will
be highly imbued with personal experiences in all kinds of scenarios.
Benchmarks don't really say much either, because they can be interpreted
in a variety of ways.

What I do know is that there are lots of high expectations in the
GNU/Linux community - and among kernel hackers specifically - for the
upcoming btrfs, which is currently already more or less usable, but
which still lacks features and might still contain serious enough bugs
to not consider it for use on a production machine in a mission-critical
environment.

btrfs uses B-trees - whereas ext4 itself uses H-trees - and integrates a
filesystem with a volume manager, similar to how Solaris's ZFS
filesystem works. In other words, without having to create partitions
and prepare those with logical volume groups and then format individual
logical volums with filesystems, you create a pool of storage devices,
create a btrfs root filesystem on them, and then you create subvolumes
on the fly as you go along. For instance, you create a directory, and
you set it up as a subvolume. You can then also create snapshots of
subvolumes, albeit that - at least, to my knowledge - this has not been
implemented in btrfs yet at this point in time.

If you're more comfortable with a traditional filesystem, which you can
install (by way of formatting) on a traditional partition or a logical
volume created with LVM2 or EVMS, then ext4 will do excellently.
However, it doesn't /have/ to be ext4.

There are also industry-standard filesystems supported directly by the
Linux kernel via open source drivers (and thus included in the upstream
vanilla kernel as it comes from Linus & friends), and which are just as
good, if not better at some points than ext4. RedHat/CentOS are
typically so pedant that they don't let you install the system on those
or mount them anywhere in the normal file hierarchy, but other
distributions do allow you that choice. And then I'm talking of XFS,
which has been the industry-standard filesystem of the SGI IRIX
operating system since 1996, or JFS, which is the filesystem of the IBM
AIX operating system (and which has since then also been ported to the
eComStation "fork" of OS/2).

There's also reiserfs, which is version 3.6 of the Reiser filesystem,
but which, although fairly good and certainly fast - especially for
smaller files - lacks a decent toolchain, and is no longer seeing too
much active maintenance, due to its developer currently spending quite a
bit of time at the grey bar hotel after having disposed of his wife the
way his filesystem sometimes disposes of your files. And then there's
also reiser4, which was the next generation filesystem that said
developer and his team were working on just before he was convicted, and
which borrows a lot of ideas from XFS and JFS, but which has not been
included in the upstream Linux kernel yet - and probably never will be -
because it too lacks a decent toolchain and is now most certainly
unmaintained.

I have two GNU/Linux distributions on this machine here. On an older
PATA disk which came from another machine, I have a 32-bit PCLinuxOS
installation, all on reiserfs partitions, because I had had good
experiences with that in the past. However, for server machines, and
for the Mageia 1 installation from which I am typing this - which
resides on a SATA disk - I am using XFS, because that too has always
been a very decent and very fast filesystem.

I have checked out ext4, and although I prefer XFS because I know that
better, I can't say that I've found there to be anything wrong with
ext4. Compared to ext3, which is also quite robust, ext4 is a landmark
improvement in performance and features. I would certainly classify it
as a robust and reliable filesystem.

So ultimately, the choice is yours. Don't let hypes influence your
decision. Choose something that you're comfortable with and that offers
good performance. ext4, JFS and XFS are all in that category. btrfs is
promising, but not production-ready yet. Don't use it for anything
important at this stage, despite the hype around it.

There was a great deal of hype around KDE 4.0 as well, and distromakers
were pushing the move from the near-perfect KDE 3.5.10 to KDE 4.0/4.1
while the latter was barely usable. It was a hype. And it proved
detrimental to the popularity of KDE, because lots of people switched
over to Gnome, XFCE or LXDE, and KDE 3.5 was forked (by way of the
Trinity Desktop project, which has advanced KDE 3.5 to version 3.5.13
already by now).

Just my two cents. Your mileage may vary. ;-)

--
= Aragorn =
(registered GNU/Linux user #223157)

[root]

An important point is that an EXT2 fs can be converted in minutes
to EXT4 by tune2fs.

[Aragorn]

On Thursday 12 January 2012 19:08, root conveyed the following to
alt.os.linux...

> Aragorn <str...@telenet.be.invalid> wrote:
>
>> ext4 is certainly a very performant and reliable filesystem, but
>> whether it is better than anything else is open for debate, and such
>> debate will be highly imbued with personal experiences in all kinds
>> of scenarios. Benchmarks don't really say much either, because they
>> can be interpreted in a variety of ways.

>> [...]

>
> An important point is that an EXT2 fs can be converted in minutes
> to EXT4 by tune2fs.

Yes, but doing so will not bring the extended possibilities of ext4 to
your existing files. If you want those files to make use of extents
instead of the block-based algorithm of ext2, then you'll have to copy
your files to some other medium first, reformat the source partition and
then restore your files from the other medium.

As a bonus, this also makes for a good file defragmentation tactic. ;-)

[Martin]

I mean the average time you need to wait for the right sector coming along
when the head is already positioned over the right track. On average that
should be half a rotation. Let's see, on a 15000 rpm drive thats 60s / 15000
/ 2 = 2 ms.

OK, you're right, it's a small figure in the magnitude of ms. :)

[Pascal Hambourg]

Martin a écrit :

> Pascal Hambourg wrote:
>
>> Martin a écrit :
>>> local memory access is in the magnitude of nanoseconds, rotational
>>> latencies in the area of microseconds and seek times in the area of
>>> milliseconds.
>> What do you call "rotational latency" ? If it is what manufacturers call
>> "latency time", it is in the area of milliseconds, not microseconds.
>
> I mean the average time you need to wait for the right sector coming along
> when the head is already positioned over the right track.

That is indeed the latency time.

> On average that
> should be half a rotation. Let's see, on a 15000 rpm drive thats 60s / 15000
> / 2 = 2 ms.

Or ~4 ms on much more common 7200 RPM drives.

> OK, you're right, it's a small figure in the magnitude of ms. :)

Yes, about the same magnitude as the seek time.

[crankypuss]

> vanilla kernel as it comes from Linus& friends), and which are just as

Thanks for the quick overview.

[crankypuss]

The whole block architecture approach is derived from the concepts of
tracks and sectors and as such I think it is about to become history...
SSD's are just an interim compatability measure, and future filesystems
are most likely going to need the algorithms of "a big malloc in the
sky" with named pointers that have nothing to do with archaic concepts
like blocks. We are really talking about hierarchical storage now, not
"disks". Partitions are a useful concept in either case, and those who
would design "filesystems" for the future might be well advised to look
into the concept of named memory pools such as IBM has had implemented
since the late '60s or early '70s as the equivalent of partitions.

I've been off in Windows-land since the early '90s and it has always
been disgusting to me that all the existing filesystems that Windows
supports are so lame that they need a progress bar to let you know how
slowly they are deleting a directory and its subdirectories. I am
equally unimpressed by any filesystem that has to "index" files, since
the purpose of a directory is to provide that very index.

Apologies if this makes me sound opinionated, but having been around for
a long time does seem to make one opinionated; so it goes.

[Aragorn]

On Friday 13 January 2012 11:16, crankypuss conveyed the following to

alt.os.linux...

> On 01/12/2012 12:13 PM, Aragorn wrote:
>> On Thursday 12 January 2012 19:08, root conveyed the following to
>> alt.os.linux...
>>
>>> Aragorn<str...@telenet.be.invalid> wrote:
>>>
>>>> ext4 is certainly a very performant and reliable filesystem, but
>>>> whether it is better than anything else is open for debate, and
>>>> such debate will be highly imbued with personal experiences in all
>>>> kinds of scenarios. Benchmarks don't really say much either,
>>>> because they can be interpreted in a variety of ways.
>>>> [...]
>>>
>>> An important point is that an EXT2 fs can be converted in minutes
>>> to EXT4 by tune2fs.
>>
>> Yes, but doing so will not bring the extended possibilities of ext4
>> to your existing files. If you want those files to make use of
>> extents instead of the block-based algorithm of ext2, then you'll
>> have to copy your files to some other medium first, reformat the
>> source partition and then restore your files from the other medium.
>>
>> As a bonus, this also makes for a good file defragmentation tactic.
>> ;-)
>
> The whole block architecture approach is derived from the concepts of
> tracks and sectors and as such I think it is about to become
> history...

Well, historically, the ext2 and ext3 filesystems did not have their
hashed tree algorithms enabled by default, so they were essentially
index-sequential filesystems. Upon allocation, the filesystem layer in
the kernel would allocate a number of blocks as they were available.
ext4 has hashed trees enabled and uses extents, which is a form of
delayed allocation that counteracts file fragmentation.

Of course, if/when humanity switches over to a different and more
efficient kind of storage medium, certain concepts should need to be
rethought.

> SSD's are just an interim compatability measure, and future
> filesystems are most likely going to need the algorithms of "a big
> malloc in the sky" with named pointers that have nothing to do with
> archaic concepts like blocks. We are really talking about
> hierarchical storage now, not "disks".

Hierarchical storage is already quite mainstream in large storage
environments such as in the mainframe and supercomputer world, but the
cost is still too high to implement this for the domestic user.

It would seem that even the concept of storing one's own files is now
more or less being pushed out of the picture by the whole cloud hype,
and of course, cloud owners can afford hierarchical storage. Personally
I am very distrustful of corporations (and the very ostensible growth of
corporatism), so I prefer holding on to my own data. ;-)

> Partitions are a useful concept in either case, and those
> who would design "filesystems" for the future might be well advised to
> look into the concept of named memory pools such as IBM has had
> implemented since the late '60s or early '70s as the equivalent of
> partitions.

Like I said, it'll all be a matter of technology, depending on what the
ultimate replacement of the Winchester-style fixed disk will turn out to
be.

> I've been off in Windows-land since the early '90s and it has always
> been disgusting to me that all the existing filesystems that Windows
> supports are so lame that they need a progress bar to let you know how
> slowly they are deleting a directory and its subdirectories.

Well, you said the "W" word, and to me that's synonymous with low-grade,
unreliable, cheap Hollywood-fake-boobs style. :p

> I am equally unimpressed by any filesystem that has to "index" files,
> since the purpose of a directory is to provide that very index.

Well, I'm not sure whether I completely understand you at this point,
because there is ambiguity in that statement. You always have to have
some form of indexing, whether it's at the filesystem level - and if so,
whether it's just a generic FAT-style index or a tree algorithm - or
whether it's somewhere else. The best approach would of course be to do
it at the filesystem level.

> Apologies if this makes me sound opinionated, but having been around
> for a long time does seem to make one opinionated; so it goes.

I find the word "opinionated" to be ambiguous too. Is having an opinion
the same as "being opinionated"? If it is, then I'd rather be
opinionated than to dwell among the statistics of the uninformed, grey,
unwashed masses. ;-)

[crankypuss]

On 01/13/2012 05:01 AM, Aragorn wrote:

> Hierarchical storage is already quite mainstream in large storage
> environments such as in the mainframe and supercomputer world, but the
> cost is still too high to implement this for the domestic user.

What cost are you talking about that is too high?

> It would seem that even the concept of storing one's own files is now
> more or less being pushed out of the picture by the whole cloud hype,
> and of course, cloud owners can afford hierarchical storage. Personally
> I am very distrustful of corporations (and the very ostensible growth of
> corporatism), so I prefer holding on to my own data. ;-)

I too am distrustful of corporations, I've worked for too many and
personally known too many executives who habitually trash quality for
profit. I was distrustful when RedHat first appeared and I'm
distrustful about Canonical. People start out with altruistic goals and
then get sucked into the "more money!" thing and pretty soon the free
market has disappeared and you'll damn well take what you're given.

>> > Partitions are a useful concept in either case, and those
>> > who would design "filesystems" for the future might be well advised to
>> > look into the concept of named memory pools such as IBM has had
>> > implemented since the late '60s or early '70s as the equivalent of
>> > partitions.
> Like I said, it'll all be a matter of technology, depending on what the
> ultimate replacement of the Winchester-style fixed disk will turn out to
> be.

I don't see how it can get more "ultimate" than a big ol' chunk of
byte-addressable storage, and that's what SSD's amount to, that plus a
compatability layer to make them look like disk drives.

> I find the word "opinionated" to be ambiguous too. Is having an opinion
> the same as "being opinionated"? If it is, then I'd rather be
> opinionated than to dwell among the statistics of the uninformed, grey,
> unwashed masses. ;-)

"Opinionated" is when you have an opinion and you've been around long
enough to know you're right and why. <g>

[Aragorn]

On Friday 13 January 2012 17:43, crankypuss conveyed the following to
alt.os.linux...

> On 01/13/2012 05:01 AM, Aragorn wrote:
>
>> Hierarchical storage is already quite mainstream in large storage
>> environments such as in the mainframe and supercomputer world, but
>> the cost is still too high to implement this for the domestic user.
>
> What cost are you talking about that is too high?

Well, the concept of hierarchical storage I'm familiar with is the one
where you use faster but more space-limited storage for most used data
and slower but more spacious storage for less used data. So in the
simplest scenario, that would signify at least two hard disks, with one
of them being a SAS disk or possibly an SSD.

But then again, I don't like the very limited life expectancy of SSDs at
this point in time. And as with anything in this world, if you want it
to be better, then you're going to have to pay (a lot) more.

>> It would seem that even the concept of storing one's own files is now
>> more or less being pushed out of the picture by the whole cloud hype,
>> and of course, cloud owners can afford hierarchical storage.
>> Personally I am very distrustful of corporations (and the very
>> ostensible growth of corporatism), so I prefer holding on to my own
>> data. ;-)
>
> I too am distrustful of corporations, I've worked for too many and
> personally known too many executives who habitually trash quality for
> profit.

Ultimately that's what they all do, because in this day and age, all
sorts of scenarios are in an accelerated phase. It's like a chess game
near its end. It's all becoming more drastic, more aggressive. You
barely have time to blink anymore.

Just look at the patent wars going on these days. Microsoft is
demanding license fees - read: extortion money - from Google, Samsung
and everyone involved with Android over alleged but unproven Microsoft
patents in Android. Barnes & Noble have now called Microsoft's bluff.
Samsung and Apple are also involved in a patent war, started by Apple.

Microsoft partly funded the SCO versus Novell litigation over ownership
of the AT&T Unix patents. SCO lost that litigation and Novell was
declared the rightful owner of the patents. A short while later, Novell
gets casually sold to Attachmate, but its patent portfolio is sold to
LLC Holdings, a consortium of corporations led by... Microsoft.

People think from news item to news item, but nobody stops and connects
the dots, while the plot is unfolding. It's a chess game, and the
pieces are being strategically positioned. The noose is being
tightened. And this is what "doing business" is all about these days.
It's not about development and evolution. It's about upscaling,
monopolizing and strangling the competition. No mercy. And it's all
happening a lot faster now than it used to.

> I was distrustful when RedHat first appeared and I'm distrustful about
> Canonical.

I don't like Canonical either, but I consider RedHat more of a threat.
Not in financial-economic terms, but because of the fact that some of
their newer developers have started to venture along the wrong paths
(because they themselves have been miseducated by their prolonged use of
Microsoft Windows) and are beginning to break standards in favor of
their own "new standards", which they know they can push because RedHat
has the power to do so.

Just in case you're wondering what I'm talking about, take a closer look
at the latest developments on account of udev and systemd, which breaks
certain GNU/Linux installation scenarios because they both now require
that "/usr" be mounted before either of them runs. The philosophy of
these particular developers is that "/{bin,sbin,lib}" should be moved to
below the "/usr" hierarchy. This goes against the FHS 2.3, but RedHat
has the power to influence the creation of a new FHS - 3.0 is in the
making, I believe.

So it's a matter of throwing away the sane standards in favor of the
not-so-sane standards that you're going to be pushing yourself. And
that's exactly what Microsoft has been doing all along.

> People start out with altruistic goals and then get sucked into the
> "more money!" thing and pretty soon the free market has disappeared
> and you'll damn well take what you're given.

... And pay up for it bigtime first, of course, or you're not getting
any of it.

>>> Partitions are a useful concept in either case, and those
>>> who would design "filesystems" for the future might be well
>>> advised to look into the concept of named memory pools such as
>>> IBM has had implemented since the late '60s or early '70s as the
>>> equivalent of partitions.
>>
>> Like I said, it'll all be a matter of technology, depending on what
>> the ultimate replacement of the Winchester-style fixed disk will turn
>> out to be.
>
> I don't see how it can get more "ultimate" than a big ol' chunk of
> byte-addressable storage, and that's what SSD's amount to, that plus a
> compatability layer to make them look like disk drives.

Yes, but that is not what I meant. SSDs still have only limited
writeability. The enterprise-grade specimens are better than the
consumer-grade ones, but at this point in time, SSDs are still way more
expensive per unit of storage than a Winchester-style hard disk.

So whether SSDs will be successful or not will depend both on their
durability/usability and on their market value, and it's not unthinkable
that a new variety of SSD would be invented somewhere in the near
future, with a higher reliability, better performance, greater storage
capacity and possibly even a far lower pricetag.

At least, if the companies can drag themselves away from the patent wars
long enough, that is... <grin>

>> I find the word "opinionated" to be ambiguous too. Is having an
>> opinion the same as "being opinionated"? If it is, then I'd rather
>> be opinionated than to dwell among the statistics of the uninformed,
>> grey, unwashed masses. ;-)
>
> "Opinionated" is when you have an opinion and you've been around long
> enough to know you're right and why. <g>

In that case, I'm opinionated. :-)

[Martin]

Pascal Hambourg wrote:


>> OK, you're right, it's a small figure in the magnitude of ms. :)
>
> Yes, about the same magnitude as the seek time.

Granted, although even today seek time should be higher than rotational
latency when seeking across multiple tracks.

The point being made originally was that SSDs do not have any of those
mechanical delays (even though access to flash memory is still slower than
access to RAM).

[Martin]

crankypuss wrote:

> I don't see how it can get more "ultimate" than a big ol' chunk of
> byte-addressable storage, and that's what SSD's amount to, that plus a
> compatability layer to make them look like disk drives.

Isn't the adressing via sequential block number (and offset within a block)
already of that nature? This abstraction from cylinder-head-sector was one
the great achievments of SCSI early on.

[David W. Hodgins]

On Fri, 13 Jan 2012 05:16:21 -0500, crankypuss <n...@email.thanks> wrote:

> The whole block architecture approach is derived from the concepts of
> tracks and sectors and as such I think it is about to become history...

Due to the need to keep block writes aligned with the Erase Block Size
of ssd drives, partitions are normally aligned at 1MB "tracks", so
the concept will stay, just using a different size and reason.

Regards, Dave Hodgins

--
Change nomail.afraid.org to ody.ca to reply by email.
(nomail.afraid.org has been set up specifically for
use in usenet. Feel free to use it yourself.)

[Richard Kettlewell]

"David W. Hodgins" <dwho...@nomail.afraid.org> writes:
> crankypuss <n...@email.thanks> wrote:

>> The whole block architecture approach is derived from the concepts of
>> tracks and sectors and as such I think it is about to become
>> history...
>
> Due to the need to keep block writes aligned with the Erase Block Size
> of ssd drives, partitions are normally aligned at 1MB "tracks", so the
> concept will stay, just using a different size and reason.

Agreed. Tracks may have disappeared into history but sectors
(i.e. blocks) are certainly not; they can be found in all kinds of
media, including RAM, where the blocks are cache lines.

And I don't think the reasons are so different after all, fundamentally
- you trade a bit of latency for increased throughput and, in many
cases, reduced cost.

Byte addressability is a higher-level construct, at any rate.

--
http://www.greenend.org.uk/rjk/

[crankypuss]

I suppose that depends on what you consider "natural". I don't find it
very natural to complicate things by basing abstraction on some
particular instance of a more general class, but that's just my view of
things.

[Jasen Betts]

On 2012-01-13, crankypuss <n...@email.thanks> wrote:

> I don't see how it can get more "ultimate" than a big ol' chunk of
> byte-addressable storage, and that's what SSD's amount to, that plus a
> compatability layer to make them look like disk drives.

ssds are flash, and flash often needs to be erased in blocks,
so they may not be byte addressible for write.

(not sure why flash needs to be erased in blocks, maybe it's cheaper to
make it that way.)

--
⚂⚃ 100% natural