very poor JFS write I/O performance compared to HPFS (JFS 20x slower)

[J P]

I have been observing that the performance of my JFS file system was
extremely slow. The only reason I keep a JFS volume at all is for
backups of DVD images, which are >2GB and can't be written to HPFS in
one shot. The problem was that my hard drive couldn't keep up with the
I/O rate of the DVD digital extraction. We are talking about nearly an
hour to extract and write a 8.5GB dual-layer disk.

My hard drives isn't supposed to be slow - it's a Seagate 36GB 15,000
rpm Ultra160 drive, onnected to an LSI Logic 21040 Ultra160 SCSI
controller with a 53C1010 chipset.

A full description of the machine is at
http://www.madbrain.com/computer.html . This is the first computer from
the top.

So, yesterday, I wrote a little benchmark program that uses the OS/2 Dos
calls to measure the file system write performance only (without any DVD
extraction in the picture). The program can be found at
http://www.madbrain.com/software/fill.zip, including source . The
program's only purpose is to create a file of the name and size
specified, and then fill it with the character 'a'.

I used the same partition for my test - the script reformatted it to JFS
or to HPFS before creating the files, like this :

format z: /FS:HPFS <yes
fill z:\test1 64
fill z:\test2 128
fill z:\test3 256
fill z:\test4 512
fill z:\test5 1024
fill z:\test6 2047

format z: /FS:JFS <yes
fill z:\test1 64
fill z:\test2 128
fill z:\test3 256
fill z:\test4 512
fill z:\test5 1024
fill z:\test6 2047
fill z:\test7 4096 (this size can only be achieved on a JFS)

The results were :

For HPFS

Writing file z:\test1 of size 64 MB.
Time elapsed : 2 seconds.
Writing file z:\test2 of size 128 MB.
Time elapsed : 4 seconds.
Writing file z:\test3 of size 256 MB.
Time elapsed : 7 seconds.
Writing file z:\test4 of size 512 MB.
Time elapsed : 14 seconds.
Writing file z:\test5 of size 1024 MB.
Time elapsed : 28 seconds.
Writing file z:\test6 of size 2047 MB.
Time elapsed : 56 seconds.

For JFS :

10233373 kilobytes total disk space.
10198708 kilobytes are available.
Writing file z:\test1 of size 64 MB.
Time elapsed : 30 seconds.
Writing file z:\test2 of size 128 MB.
Time elapsed : 46 seconds.
Writing file z:\test3 of size 256 MB.
Time elapsed : 159 seconds.
Writing file z:\test4 of size 512 MB.
Time elapsed : 274 seconds.
Writing file z:\test5 of size 1024 MB.
Time elapsed : 573 seconds.
Writing file z:\test6 of size 2047 MB.
Time elapsed : 1108 seconds.
Writing file z:\test7 of size 4096 MB.
Time elapsed : 2218 seconds.

As you can see, the JFS is 15x to 22x slower than HPFS. And in this
test, I was running regular HPFS, with a 2MB cache - not HPFS386 . JFS
had a 64MB cache. CACHEJFS says that lazy writes are enabled. But the
results sure don't agree with that - if the cache really was enabled,
then the 64MB file test should be instant with the JFS, since there is a
64MB cache. But it takes 30 seconds !

I have the following in CONFIG.SYS :
IFS=D:\OS2\JFS.IFS /AUTOCHECK:* /CACHE:65536
IFS=D:\OS2\HPFS.IFS /CACHE:2048 /CRECL:64 /AUTOCHECK:DE

This was under WSEB ACP3, with the SMP kernel, since the box is a Dual
athlon 2800. Many patches were also applied over the years.

Suspecting some issue with the software and patches, I went to test the
same program on a different machine, also running OS/2 WSEB ACP3, but
that was one was reinstalled a couple of months ago only, and had few if
any patches.

The other machine is single-processor, and much slower box
(K6-III+/500), but otherwise is fairly similar : same video card, same
ethernet controller, Ultra160 SCSI controller with the same LSI 53C1010
chipset (except it's the 32-bit PCI version vs 64-bit PCI). The disks
are also of a slightly older generation. This is the second box
http://www.madbrain.com/computer.html .

I tested on the fastest of the 3 disks that machine has, the Quantum
Atlas 10,000 rpm Ultra160. I didn't see anywhere near the difference I
saw on this box. HPFS still won over JFS, but JFS was not 20x slower.
Here are the results from that K6 machine with the Quantum 10k disk :

HPFS :

Writing file e:\test1 of size 64 MB.
Time elapsed : 4 seconds.
Writing file e:\test2 of size 128 MB.
Time elapsed : 7 seconds.
Writing file e:\test3 of size 256 MB.
Time elapsed : 15 seconds.
Writing file e:\test4 of size 512 MB.
Time elapsed : 30 seconds.
Writing file e:\test5 of size 1024 MB.
Time elapsed : 61 seconds.
Writing file e:\test6 of size 2047 MB.
Time elapsed : 126 seconds.

JFS :

Writing file e:\test1 of size 64 MB.
Time elapsed : 4 seconds.
Writing file e:\test2 of size 128 MB.
Time elapsed : 11 seconds.
Writing file e:\test3 of size 256 MB.
Time elapsed : 20 seconds.
Writing file e:\test4 of size 512 MB.
Time elapsed : 40 seconds.
Writing file e:\test5 of size 1024 MB.
Time elapsed : 90 seconds.
Writing file e:\test6 of size 2047 MB.
Time elapsed : 176 seconds.
Writing file e:\test7 of size 4096 MB.
Time elapsed : 368 seconds.

These show that JFS only took 50% longer for files of the same size.

I wasn't about to give up on this problem, since the dual athlon is my
primary system, and I don't want to backup DVDs on the K6 box - since
its memory and PCI bus are not enough to sustain the I/O rate of both
the hard drive and the DVD extraction - but the Dual athlon should be
quite sufficient . So, I created an emergency partition on the dual
athlon, and made a brand new OS/2 install from the WSEB ACP3 CD-ROMs. I
did a minimal install - everything unchecked, no networking, no SMP
kernel, only JFS selected in the selective install.

Then I reran my I/O test. JFS still came out dogslow like it did on the
other boot parttion ...

I really don't get it. Can anyone guess why the JFS is so slow on this
Dual athlon machine, while HPFS works fine ?
I'm willing to try any JFS, OS/2 kernel patches or configuration tricks
to improve this, even if they may cause loss of data. I have backups.

Thanks !

[William L. Hartzell]

Sir:

J P wrote:
> I have been observing that the performance of my JFS file system was
> extremely slow. The only reason I keep a JFS volume at all is for
> backups of DVD images, which are >2GB and can't be written to HPFS in
> one shot. The problem was that my hard drive couldn't keep up with the
> I/O rate of the DVD digital extraction. We are talking about nearly an
> hour to extract and write a 8.5GB dual-layer disk.
>
> My hard drives isn't supposed to be slow - it's a Seagate 36GB 15,000
> rpm Ultra160 drive, onnected to an LSI Logic 21040 Ultra160 SCSI
> controller with a 53C1010 chipset.
>
> A full description of the machine is at
> http://www.madbrain.com/computer.html . This is the first computer from
> the top.
>
> So, yesterday, I wrote a little benchmark program that uses the OS/2 Dos
> calls to measure the file system write performance only (without any DVD
> extraction in the picture). The program can be found at
> http://www.madbrain.com/software/fill.zip, including source . The
> program's only purpose is to create a file of the name and size
> specified, and then fill it with the character 'a'.
>

<snip>

> I wasn't about to give up on this problem, since the dual athlon is my
> primary system, and I don't want to backup DVDs on the K6 box - since
> its memory and PCI bus are not enough to sustain the I/O rate of both
> the hard drive and the DVD extraction - but the Dual athlon should be
> quite sufficient . So, I created an emergency partition on the dual
> athlon, and made a brand new OS/2 install from the WSEB ACP3 CD-ROMs. I
> did a minimal install - everything unchecked, no networking, no SMP
> kernel, only JFS selected in the selective install.
>
> Then I reran my I/O test. JFS still came out dogslow like it did on the
> other boot parttion ...
>
> I really don't get it. Can anyone guess why the JFS is so slow on this
> Dual athlon machine, while HPFS works fine ?
> I'm willing to try any JFS, OS/2 kernel patches or configuration tricks
> to improve this, even if they may cause loss of data. I have backups.
>
> Thanks !

IF you want make a fast test, install the UNI kernel beforehand. SMP is
slower than UNI because of the extra code need for the reintrancy of the
code. Second reformat the disks using the last JFS code released to
testcase by Scott in August of last year
<http://www.os2site.com/sw/drivers/filesystem/jfs/index.html>. Then run
the freeJFS tools on the volume to fix any format bugs (should not be
any, but never hurts to check. Note some tests require that the JFS IFS
not be loaded beforehand, eg. remove it from config.sys (REM the line) &
reboot)
<http://www.os2site.com/sw/drivers/filesystem/openjfs/index.html>. Then
double check the the controller & disks has the latest firmware update.
While you are at this, check the main board firmware for updates.
Then check the DVD for firmware updates. Blueberry, you might as well
check the other cards for firmware updates while at this as you won't
later. Be sure that the system is on a UPS as the failure chance is
greatest without a sure power supply. Finally check the cables for firm
connections and that the termination is okay.

BTW, are you forcing a flush of the caches by doing a read after each write?
--
Bill
Thanks a Million!

[Julien Pierre]

Bill,

William L. Hartzell wrote:
>> quite sufficient . So, I created an emergency partition on the dual
>> athlon, and made a brand new OS/2 install from the WSEB ACP3 CD-ROMs.
>> I did a minimal install - everything unchecked, no networking, no SMP
>> kernel, only JFS selected in the selective install.
>>
>> Then I reran my I/O test. JFS still came out dogslow like it did on
>> the other boot parttion ...
>

> IF you want make a fast test, install the UNI kernel beforehand. SMP is
> slower than UNI because of the extra code need for the reintrancy of the
> code.

As I wrote above, I already did that - uni kernel made no difference,
unfortunately.

>Second reformat the disks using the last JFS code released to
> testcase by Scott in August of last year
> <http://www.os2site.com/sw/drivers/filesystem/jfs/index.html>.

I did that too after my original post. No difference.

> Then run
> the freeJFS tools on the volume to fix any format bugs (should not be
> any, but never hurts to check. Note some tests require that the JFS IFS
> not be loaded beforehand, eg. remove it from config.sys (REM the line) &
> reboot)
> <http://www.os2site.com/sw/drivers/filesystem/openjfs/index.html>. Then
> double check the the controller & disks has the latest firmware update.

The controller & disks are up-to-date, and work fine with all other file
systems and operating systems. To me this doesn't look like a hardware
problem. The problem happens on multiple hard drives on the same dual
athlon box, so that rules out the drive.

> While you are at this, check the main board firmware for updates.

It's up-to-date.

> Then
> check the DVD for firmware updates.

The benchmark I wrote no longer involves the DVD - it merely writes a
file to disk, so that's out of the picture. But the DVD firmware is
up-to-date, anyway.

> BTW, are you forcing a flush of the caches by doing a read after each
> write?

You can check the source code for my test - it is at
http://www.madbrain.com/software/fill.zip . It is a very short program.
The essence of it is : DosOpenL, loop of DosWrite of 1 MB, then DosClose.

[Andreas Lerch]

>>>>>>>>>>>>>>>>>> Ursprüngliche Nachricht <<<<<<<<<<<<<<<<<<

Am 21.11.05, 04:49:07, schrieb J P <use...@madbrain.com> zum Thema very
poor JFS write I/O performance compared to HPFS (JFS 20x slower):

<big snip>

hello j p

have you tried to open the file with the size you total want to write.
I think os/2 is using the technique to preallocate the space at
file_open. If you extend in a small file, there will be allocation of
new space.

My experiments with Changi: i copy my news-files with os/2 to another
volume and back. The expire-prozess take half the time before copying

will you test it?

Einen schoenen Tag
Andreas Lerch

[Julien Pierre]

Andreas Lerch wrote:
>
>>>>>>>>>>>>>>>>>>>Ursprüngliche Nachricht <<<<<<<<<<<<<<<<<<
>
>
> Am 21.11.05, 04:49:07, schrieb J P <use...@madbrain.com> zum Thema very
> poor JFS write I/O performance compared to HPFS (JFS 20x slower):
>
> <big snip>
>
> hello j p
>
> have you tried to open the file with the size you total want to write.

Thanks for the suggestion, but I already set the cbFile argument of
DosOpenL to the total size I'm going to DosWrite next. It's all in the
source at http://www.madbrain.com/software/fill.zip

[Marty]

Julien Pierre wrote:
> You can check the source code for my test - it is at
> http://www.madbrain.com/software/fill.zip . It is a very short program.
> The essence of it is : DosOpenL, loop of DosWrite of 1 MB, then DosClose.

For curiosity's sake, have you run an experiment with lazy write off? I
found on my system that for large files (> cache size) lazy write slows
down the process significantly, in addition to dragging the rest of the
system performance down with it (music hiccups, etc.). With lazy write
off, this doesn't happen.

--
[Reverse the parts of the e-mail address to reply.]

[Neil Waldhauer]

Just for comparison, I've got a cheap Celeron 2 GHz with an equally
cheap IDE 120 GB hard drive.

for JFS
64 MB in 1 second
1024 MB in 25 seconds

for HPFS
64 MB in 2 seconds
1024 MB in 31 seconds

I admit to having more JFS cache, whatever the default is with 768 MB
RAM.
ACP w/ FP 4.
--
Expert Consulting for OS/2 and eComStation
http://www.blondeguy.com

[Lars Erdmann]

J P schrieb:

Are these settings the same on both boxes you have listed ?
Have you tried other settings for JFS.IFS like /LAZY ?

JFS.IFS Switches in CONFIG.SYS

Syntax of switches:
IFS=x:\OS2\JFS.IFS [/SWITCH:VALUE] [/SWITCH:VALUE]

where "x:" Specifies the drive letter of the partition from which the
OS/2 program is started.

for example,
IFS=X:\OS2\JFS.IFS /AUTOCHECK:DRIVES /CACHE:SIZE
/LAZY:[Off|synctime,maxage,bufferidle]

Allowed aliases for switches:

/AUTOCHECK /A
/CACHE /C
/LAZY /L, /LW, /LAZYWRITE

For example, "/AUTOCHECK:*" and "/A:*" are equivalent.
/AUTOCHECK

Specifies a list of drive letters to be automatically verified by the
CHKDSK command at system startup if JFS detects that the drive may have
been shutdown unexpectedly.

The drivelist value is a string of letters indicating the JFS drives in
the workstation. The drive letters are not separated by commas or
spaces. If any drive specified is not a valid JFS drive, the drive is
not checked.

Example: /A:DEW - will check volumes D:, E: and W: /A:* - will check all
found JFS-volumes
/CACHE

Specifies the amount of memory, in kilobytes, to use for the JFS cache.

Example:
/C:8192 - set JFS cache to 8 MBytes of RAM
/LAZY
enables or disables write cache.

/LAZY:OFF

disables write cache. It forces asynchonous writes to be immediately
initiated (synctime=1, maxage=0, bufferidle=0)

/LAZY:synctime,maxage,bufferidle

enables write cache with the following parameters:
synctime the interval at which the sync thread runs (in seconds)
default value - 16
maxage is the longest time in seconds that a frequently modified file
is kept in cache. default = synctime * 4
bufferidle is the time in seconds indicating a "recent" change. Changes
newer than this value are not written unless the last write was older
than maxage. default = MIN(1,synctime/8)
Examples
IFS=C:\OS2\JFS.IFS /L:32
synctime = 32, maxage = 128, bufferidle = 4

IFS=C:\OS2\JFS.IFS /LW:5,20,4
synctime = 5, maxage = 20, bufferidle = 4

Lars

[Trevor Hemsley]

On Tue, 22 Nov 2005 22:13:36 UTC in comp.os.os2.bugs, "Neil Waldhauer"
<SendM...@Spam.Org> wrote:

> Just for comparison, I've got a cheap Celeron 2 GHz with an equally
> cheap IDE 120 GB hard drive.
>
> for JFS
> 64 MB in 1 second
> 1024 MB in 25 seconds
>
> for HPFS
> 64 MB in 2 seconds
> 1024 MB in 31 seconds
>
> I admit to having more JFS cache, whatever the default is with 768 MB
> RAM.
> ACP w/ FP 4.

I've downloaded it and tested it here. My results are the opposite, all for a
1GB file.

Drive Elapsed-time
d: (HPFS) 25
e: (HPFS) 50
f: (JFS) 19
g: (JFS) 22

I think this has to do with the fragmentation of the volume - at least mine all
seem to reflect the amount of usage the various partitions have had over the
years since they were all formatted and the drive was new. D: is rarely written
to but only has 1GB free (I could only write a 950MB file), E: is used all the
time, lots of writes and deletes. F: is almost empty, G: is used but mainly
write once and then keep.

Filesystem MB-blocks Used Available Capacity Mounted on
1537 561 976 36% D:
51207 33341 17866 65% E:
5094 1079 4015 21% F:
data_g 93899 29533 64366 31% G:

Both filesystems set to use 128MB cache each.

--
Trevor Hemsley, Brighton, UK.
Trevor-Hemsley at dsl dot pipex dot com

[Heiko Nitzsche]

Just for comparison, Athlon64 3000+, 200GB SATA hard drive,
eCS 1.1 (FP5), DaniS506.ADD):

IFS=C:\OS2\JFS.IFS /AUTOCHECK:* /CACHE:120000 /L:5,20,4
IFS=C:\OS2\HPFS.IFS /CACHE:2048 /CRECL:64 /AUTOCHECK:CDJ

for JFS
512 MB in 8 seconds
1024 MB in 16 seconds
1536 MB in 26 seconds
2047 MB in 36 seconds

for HPFS
512 MB in 12 seconds
1024 MB in 26 seconds
1536 MB in 37 seconds
2047 MB ... sorry not that much space left on HPFS volumes
(only used for OS boot)

> The results were :
>
> For HPFS
>

> Writing file z:\test4 of size 512 MB.
> Time elapsed : 14 seconds.
> Writing file z:\test5 of size 1024 MB.
> Time elapsed : 28 seconds.

Looking at your hard drive this seems to be slow.
My disk is a 7,200 rpm Maxtor SATA1. Your's should
much faster.

Try DISKIO
http://hobbes.nmsu.edu/pub/os2/util/benchmark/diskio118z.zip
and measure the bus/cache transfer rate and the disk transfer rate.

Here are the results of my hard drive:
...
DISKIO - Fixed Disk Benchmark, Version 1.18z
...
Dhrystone benchmark for this CPU: 5587309 runs/sec
...
Hard disk 2: 255 sides, 24792 cylinders, 63 sectors per track = 194474 MB
Drive cache/bus transfer rate: 104558 k/sec
Data transfer rate on cylinder 0 : 63050 k/sec
Data transfer rate on cylinder 24790: 30936 k/sec
CPU usage by full speed disk transfers: 16%
Average data access time: 14.5 ms
Multithreaded disk I/O (4 threads): 15257 k/sec, 10% CPU usage

> For JFS :

>
> Writing file z:\test4 of size 512 MB.
> Time elapsed : 274 seconds.
> Writing file z:\test5 of size 1024 MB.
> Time elapsed : 573 seconds.

That's strange.

> I have the following in CONFIG.SYS :
> IFS=D:\OS2\JFS.IFS /AUTOCHECK:* /CACHE:65536

Try with reduced settings for disk sync. Scott G. recommended
/L:5,20,4. Otherwise flushing the full cache takes much more time
and can lead to temporary deadlocks of the system.

> I tested on the fastest of the 3 disks that machine has, the Quantum
> Atlas 10,000 rpm Ultra160. I didn't see anywhere near the difference I
> saw on this box. HPFS still won over JFS, but JFS was not 20x slower.
> Here are the results from that K6 machine with the Quantum 10k disk :
>
> HPFS :
>

> Writing file e:\test4 of size 512 MB.
> Time elapsed : 30 seconds.
> Writing file e:\test5 of size 1024 MB.
> Time elapsed : 61 seconds.
>

> JFS :

>
> Writing file e:\test4 of size 512 MB.
> Time elapsed : 40 seconds.
> Writing file e:\test5 of size 1024 MB.
> Time elapsed : 90 seconds.
>

> These show that JFS only took 50% longer for files of the same size.

Did you already try after a long format of the JFS volume?

Otherwise I'd guess it's an issue with the controller,
driver or hardware. I'd check:
- SCSI bus termination correct?
- Is the controller using the fastest transfer mode for the disk?
- IRQ conflict with other PCI cards or even the AGP slot?
Many SCSI controller drivers don't like this.
- try another driver revision

[Lars Erdmann]

another way to influence this is to change the blocksize when formatting
the partition:

format.com /FS:JFS /BS:4096 (for a 4096 byte Blocksize, compatible with
Linux).
Also the /S switch could make a difference:

/S Specifies that files on the device will be sparse files. JFS has two
file allocation schemes, these being "sparse" and "dense." Sparse
allocation is the amount of space allocated to a random-access file when
a block written may not be within the earliest portion of a file. Dense
allocation allocates all the space prior to the block being written when
the block is written. For instance, if a random-access file has 5120
bytes (ten 512 byte blocks), and the fifth block is written to, the
resulting usage with sparse allocation is 512 bytes. With dense, the
resulting usage will be 2550 bytes, even though the first 2048 bytes are
not yet being used. The sparse method is server efficient; only the
space used is allocated to the file but this would presumably lead to
fragmentation, since when another block is written, the sparse
allocation will cause the data to not be in sequentially ordered sectors.

Therefore, it looks like it is advantageous to format WITHOUT specifying
option /S for maximum performance but higher waste of disk space.

Lars

[Kevin K]

Many years ago (before JFS on OS/2), I ran tests comparing FAT and
HPFS.

At the time, the main speed advantage of HPFS was on tests with very
large numbers of files in a directory, well in excess of 1000. So, in
general, FAT was faster in artificial usage patterns.

Of course, FAT also had the advantage in that the cache could be
considerably larger than the HPFS cache.

So, speed differences between HPFS and JFS probably depend on usage
patterns.

Of course, between FAT and HPFS, there were other issues, such as
better reliability in HPFS, larger partition size, long filenames, an
EA scheme that wasn't a kludge. Also, HPFS was probably noticably
better in low memory configurations common in OS/2 1.X where you
couldn't throw 12 meg of RAM just for the FAT cache.

JFS has other advantages other than pure speed over HPFS, such as
larger partition size, larger file support, journelling when
improperly shutdown.

[Julien Pierre]

Hi,

I just tried that (CACHEJFS /LAZY:OFF) . Unfortunately, it didn't help.
It doesn't seem to have made things worse either though, on the short
run I did (100 MB, 32s).

[Julien Pierre]

Trevor,

Trevor Hemsley wrote:
> I've downloaded it and tested it here. My results are the opposite, all for a
> 1GB file.
>
> Drive Elapsed-time
> d: (HPFS) 25
> e: (HPFS) 50
> f: (JFS) 19
> g: (JFS) 22

Thanks for running the test !

> I think this has to do with the fragmentation of the volume - at least mine all
> seem to reflect the amount of usage the various partitions have had over the
> years since they were all formatted and the drive was new. D: is rarely written
> to but only has 1GB free (I could only write a 950MB file), E: is used all the
> time, lots of writes and deletes. F: is almost empty, G: is used but mainly
> write once and then keep.

The fragmentation couldn't have been the problem for me. The first step
of my test, run from a CMD file, was to format the partition fresh, with
either JFS or HPFS . So I had an empty, totally unfragmented volume, in
both cases.

[Julien Pierre]

Lars,

I originally ran the format command with just /FS:JFS, no /BS or /S
flags. CHKDSK shows the block size selected was 4096 bytes. I just
reformatted with a 512 bytes blocksize, and it did not help. I also
tried /S . That made it significantly worse - time jumped from 32 to 46
seconds for a 100 MB file - more than an order of magnitude more than
what it should be in both cases compared to HPFS.

[Julien Pierre]

Heiko,

> Looking at your hard drive this seems to be slow.

It's definitely not as fast as I would expect, even on HPFS.

> My disk is a 7,200 rpm Maxtor SATA1. Your's should
> much faster.

Even though mine is a 15,000 rpm drive Ultra160 SCSI, it's an older
generation (5 year olds or so), and only has a 36GB capacity, which
means not as many platters as yours, which is SATA and therefore must be
fairly recent. I'm not entirely surprised that your larger drive with
slower rotation speed ends up with the faster throughput.

> Try DISKIO
> http://hobbes.nmsu.edu/pub/os2/util/benchmark/diskio118z.zip
> and measure the bus/cache transfer rate and the disk transfer rate.
>
> Here are the results of my hard drive:
> ...
> DISKIO - Fixed Disk Benchmark, Version 1.18z
> ...
> Dhrystone benchmark for this CPU: 5587309 runs/sec
> ...
> Hard disk 2: 255 sides, 24792 cylinders, 63 sectors per track = 194474 MB
> Drive cache/bus transfer rate: 104558 k/sec
> Data transfer rate on cylinder 0 : 63050 k/sec
> Data transfer rate on cylinder 24790: 30936 k/sec
> CPU usage by full speed disk transfers: 16%
> Average data access time: 14.5 ms
> Multithreaded disk I/O (4 threads): 15257 k/sec, 10% CPU usage

Here are my results.
The CPU usage percentages are probably useless because mine is a dual
CPU machine (Athlon MP 2800).

Dhrystone 2.1 C benchmark routines (C) 1988 Reinhold P. Weicker
Dhrystone benchmark for this CPU: 5194250 runs/sec

Hard disk 2: 255 sides, 4462 cylinders, 63 sectors per track = 35000 MB
Drive cache/bus transfer rate: 79876 k/sec
Data transfer rate on cylinder 0 : 59087 k/sec
Data transfer rate on cylinder 4460: 42383 k/sec
CPU usage by full speed disk transfers: 6%
Average data access time: 5.8 ms
Multithreaded disk I/O (4 threads): 14782 k/sec, 2% CPU usage

I disabled the second CPU with MPCPUMON, and reran the test to make it a
fairer comparison to a single CPU box :

Dhrystone 2.1 C benchmark routines (C) 1988 Reinhold P. Weicker
Dhrystone benchmark for this CPU: 5191957 runs/sec

Hard disk 2: 255 sides, 4462 cylinders, 63 sectors per track = 35000 MB
Drive cache/bus transfer rate: 82231 k/sec
Data transfer rate on cylinder 0 : 59115 k/sec
Data transfer rate on cylinder 4460: 42386 k/sec
CPU usage by full speed disk transfers: 4%
Average data access time: 5.8 ms
Multithreaded disk I/O (4 threads): 14773 k/sec, 2% CPU usage

The above results are unintuitive : the percentages went down where I
would have expected them to go up, and throughput went up slightly, by
disabling the second CPU ...

Regardless of this anomaly, it looks like Ultra160 SCSI still has a big
advantage for CPU usage, even over your SATA1.

I have the best of both worlds in my machine because I hooked up a 200
GB Ultra ATA drive to an Ultra160 SCSI/ATA adapter, on my SCSI
controller. It performs like this :

Hard disk 1: 255 sides, 24321 cylinders, 63 sectors per track = 190779 MB
Drive cache/bus transfer rate: 42035 k/sec
Data transfer rate on cylinder 0 : 33797 k/sec
Data transfer rate on cylinder 24319: 28308 k/sec
CPU usage by full speed disk transfers: 2%
Average data access time: 13.4 ms
Multithreaded disk I/O (4 threads): 13630 k/sec, 2% CPU usage

As a result of being connected to the SCSI controller, this UltraATA
drive benefits from its low CPU overhead.

I didn't do this for performance - my motherboard only has two IDE
channels and I have two DVD devices. And having two IDE/ATAP devices on
one channel made performance plummet to 1987 MFM controller levels ;).

[Julien Pierre]

Peter Weilbacher wrote:
> bldlevels are:
> c:\os2\jfs.ifs 14.100
> c:\os2\HPFS.IFS 14.103

I'm running the same level of JFS.IFS you have ...
My HPFS.IFS is 14.93 .

> Perhaps you have a several performance drop at the position of the disk
> where the journal is located?! :-)

No, that can't be it ;) I tried the test on freespace on both of the
hard drives that are in my dual athlon machine. I got the same 20x perf
decrease on JFS .

Is there an OPENJFS.IFS I could try to see if it doesn't perform as
poorly as the IBM one does on this box ?

[Julien Pierre]

Kevin,

I'm aware of that and I'm perfectly willing to pay some reasonable
performance penalty for the benefits of both JFS over HPFS, and HPFS
over FAT. But 20x is a 2000% penalty and I don't consider that to be
reasonable under any circumstance.

I know of no other filesystem for OS/2 than JFS that supports >2GB
files. This is the main reason I'm trying to use JFS at all. Is there
another filesystem that I can use if this JFS problem can't be fixed ?

[Trevor Hemsley]

On Wed, 23 Nov 2005 05:16:06 UTC in comp.os.os2.bugs, Julien Pierre
<use...@madbrain.com> wrote:

> The fragmentation couldn't have been the problem for me. The first step
> of my test, run from a CMD file, was to format the partition fresh, with
> either JFS or HPFS . So I had an empty, totally unfragmented volume, in
> both cases.

Something werd going on on my results. I formatted E: again and retried and
still got twice the timings that I do on D:. Both are HPFS and both those and
the two JFS partitions are on the same physical disk. Only E: gives those high
write timings. In an effort to speed it up I tried altering the open flags to
denyreadwrite, writeonly and open_flags_sequential but these didn't seem to
affect it. I also added a

if (i%100 == 99)
rc = DosResetBuffer(hfFileHandle);

to flush the buffers every 100MB and this does seem to speed up the access on E:
(times dropped from 95 seconds for 2GB to 75 but all other partitions write the
same file in ~40 seconds). This didn't seem to affect the results on JFS at all.

[Mike Ruskai]

On or about Wed, 23 Nov 2005 07:14:43 -0600 did "Trevor Hemsley"
<Trevor-...@mytrousers.dsl.pipex.com> dribble thusly:

>Something werd going on on my results. I formatted E: again and retried and
>still got twice the timings that I do on D:. Both are HPFS and both those and
>the two JFS partitions are on the same physical disk. Only E: gives those high

As you're no doubt aware, hard drives are not equally fast across the
platter. You didn't say where on the drive this partition was, but if
it was at the end of the logical drive space, it'd be the slowest area
of the drive.
--
- Mike

Ignore the Python in me to send e-mail.

[Trevor Hemsley]

These partitions are all on the same drive, D: is fast, E: is slow, F: and G:
are the same speed as D: (approx). Partition layout follows the drive lettering
(I haven't done any relettering with LVM) so E: is nearer the front of the drive
than F: and G: so should be faster.

About the only difference I can think of is the size of the HPFS drive - the
slow one is 52GB, the fast one 1.5GB. Unless the partition has a bunch of bad
sectors on it that are not showing up - it's been reformatted but still is
slower than the rest.

Oh, and one other thing to add into the mix, when I use my own utility which
writes multiple 2GB files, it works at the same speed on all drives (42MB/s) -
the main difference is that mine uses the CRT i/o functions and not
DosOpenL/DosWrite.

[Lars Erdmann]

Have you updated the firmware of the controller itself ?

http://www.lsilogic.com/downloads/downloads.do?product=2149&download_type=bios

Lars

"J P" <use...@madbrain.com> schrieb im Newsbeitrag
news:ea5f9$438151c2$c690ceb7$16...@TSOFT.COM...

[Trevor Hemsley]

On Wed, 23 Nov 2005 22:02:44 UTC in comp.os.os2.bugs, "Peter Weilbacher"
<news...@weilbacher.org> wrote:

> To my surprise cstats.exe tells me
> that my cachesize would be 16 MiB while I have 64 MiB set in
> Config.sys?!

Just seems to be out by a factor of 4 - mine says 32MB when it's really 128MB
(as reported by cachejfs).

[Julien Pierre]

Well, I made some progress today - I think I solved my problem, in fact.
I reduced the JFS cached from 65536 KB to 2048 KB - which matches the
HPFS cache size.

Here are the results, on the Seagate 15K drive. The JFS results are
first, HPFS second.

This time JFS is much, much faster - significantly faster in fact than
HPFS even - 22 seconds to write 1 GB on JFS, vs 29 seconds on HPFS.

I don't understand why the larger JFS cache hurts so much and causes a
2000% degradation. It's not like the RAM in my system is slow - it's
DDR266 which is many orders of magnitude faster than any hard drive and
controller (measured in GB/s ...).

Format completed successfully.

12281661 kilobytes total disk space.
12246684 kilobytes are available.

Writing file z:\test1 of size 64 MB.

Time elapsed : 2 seconds.
Writing file z:\test2 of size 128 MB.
Time elapsed : 2 seconds.

Writing file z:\test3 of size 256 MB.

Time elapsed : 6 seconds.

Writing file z:\test4 of size 512 MB.

Time elapsed : 10 seconds.

Writing file z:\test5 of size 1024 MB.

Time elapsed : 22 seconds.

Writing file z:\test6 of size 2047 MB.

Time elapsed : 42 seconds.

Writing file z:\test7 of size 4096 MB.

Time elapsed : 92 seconds.
The type of file system for the disk is JFS.
Warning! All data on hard disk Z: will be lost!
Proceed with FORMAT (Y/N)? Y

Formatting...

100% of the disk has been formatted.

Format completed successfully.

Enter up to 11 characters for the volume label
or press Enter for no volume label.
12281661 kilobytes total disk space.
12268492 kilobytes are available.

Writing file z:\test1 of size 64 MB.

Time elapsed : 2 seconds.
Writing file z:\test2 of size 128 MB.
Time elapsed : 4 seconds.
Writing file z:\test3 of size 256 MB.

Time elapsed : 8 seconds.

Writing file z:\test4 of size 512 MB.
Time elapsed : 14 seconds.
Writing file z:\test5 of size 1024 MB.

Time elapsed : 29 seconds.

Writing file z:\test6 of size 2047 MB.

Time elapsed : 57 seconds.

Peter Weilbacher wrote:

> On Wed, 23 Nov 2005 23:30:06 UTC, "Trevor Hemsley" wrote:
>
>
>>On Wed, 23 Nov 2005 22:02:44 UTC in comp.os.os2.bugs, "Peter Weilbacher"
>><news...@weilbacher.org> wrote:
>>
>>
>>>To my surprise cstats.exe tells me
>>>that my cachesize would be 16 MiB while I have 64 MiB set in
>>>Config.sys?!
>>
>>Just seems to be out by a factor of 4 - mine says 32MB when it's really 128MB
>>(as reported by cachejfs).
>
>

> Ah, OK. It doesn't display the units but with this hint I boldly deduce
> the units are blocks (4096 bytes in my case). :-)

[Julien Pierre]

Lars Erdmann wrote:
> Have you updated the firmware of the controller itself ?
>
> http://www.lsilogic.com/downloads/downloads.do?product=2149&download_type=bios

Yes, I have been running the 4.19 Symbios/LSI firmware ever since it got
released.

[Heiko Nitzsche]

> Well, I made some progress today - I think I solved my problem, in fact.
> I reduced the JFS cached from 65536 KB to 2048 KB - which matches the
> HPFS cache size.

Did you try my proposed settings for JFS.IFS (/L:5,20,4)
together with the large cache?

I rephrase:

"Try with reduced settings for disk sync. Scott G. recommended
/L:5,20,4. Otherwise flushing the full cache takes much more time
and can lead to temporary deadlocks of the system."

The defaults are much higher. I use 120MB cache size together
with the proposed parameters and it works very well.

> Format completed successfully.
>
> 12281661 kilobytes total disk space.
> 12246684 kilobytes are available.
> Writing file z:\test1 of size 64 MB.
> Time elapsed : 2 seconds.
> Writing file z:\test2 of size 128 MB.
> Time elapsed : 2 seconds.
> Writing file z:\test3 of size 256 MB.
> Time elapsed : 6 seconds.
> Writing file z:\test4 of size 512 MB.
> Time elapsed : 10 seconds.
> Writing file z:\test5 of size 1024 MB.
> Time elapsed : 22 seconds.

This comes close now to my results.

[Julien Pierre]

Heiko Nitzsche wrote:
>> Well, I made some progress today - I think I solved my problem, in fact.
>> I reduced the JFS cached from 65536 KB to 2048 KB - which matches the
>> HPFS cache size.
>
>
> Did you try my proposed settings for JFS.IFS (/L:5,20,4)
> together with the large cache?
>
> I rephrase:
>
> "Try with reduced settings for disk sync. Scott G. recommended
> /L:5,20,4. Otherwise flushing the full cache takes much more time
> and can lead to temporary deadlocks of the system."
>
> The defaults are much higher. I use 120MB cache size together
> with the proposed parameters and it works very well.

I tried

IFS=D:\OS2\JFS.IFS /AUTOCHECK:* /CACHE:120000 /L:5,20,4

The above produced the same very slow results as I had originally.

The folowing provides good results for me, but the /L settings you
suggested did not make a visible difference.

IFS=D:\OS2\JFS.IFS /AUTOCHECK:* /CACHE:2048 /L:5,20,4

[Ilya Zakharevich]

[A complimentary Cc of this posting was sent to
Julien Pierre
<use...@madbrain.com>], who wrote in article <4387C4F0...@madbrain.com>:

> IFS=D:\OS2\JFS.IFS /AUTOCHECK:* /CACHE:120000 /L:5,20,4
>
> The above produced the same very slow results as I had originally.

These are probably stupid questions, but in the last century one
should have been concerned with matters like this...

Is not there a "slow spot" in your memory? How much memory is cached?
How much memory your OS/2 system thinks it has? One (incredible!)
scenario is that some memory is not seen, and the cache is swapped
out... ;-) But such incredible results require incredible
explanations...

Hope this helps,
Ilya

[Rich Wonneberger]

Julien Pierre wrote:
>
> I tried
>
> IFS=D:\OS2\JFS.IFS /AUTOCHECK:* /CACHE:120000 /L:5,20,4
>
> The above produced the same very slow results as I had originally.
>
> The folowing provides good results for me, but the /L settings you
> suggested did not make a visible difference.
>
> IFS=D:\OS2\JFS.IFS /AUTOCHECK:* /CACHE:2048 /L:5,20,4

Can someone with a fast running system try the /cache:120000 setting?
It would seem to me the system would not write the cache until it hit
the 120 meg point (or is it 120 K??) or a timeout point was reached. At
this point the disk i/o is the bottleneck??
Ilya's comment made me think about this one.

Maybe??
Rich W.

[Jim Moe]

Rich Wonneberger wrote:
>
> Can someone with a fast running system try the /cache:120000 setting?
>

I have tried caches larger than 64MB (the default max jfs allows). It
has no advantage and in fact is slower. I believe the cache management
overhead negates any advantage a cache larger than 64MB provides.
The only advantage of using /cache is to force a larger cache than jfs
would use by default (12.5% of available RAM). And for experimenting with
small caches, of course, like the /cache:2048 (2MB) mentioned earlier.

--
jmm (hyphen) list (at) sohnen-moe (dot) com
(Remove .AXSPAMGN for email)

[Michael Lueck]

Test results here, just to add to the mix...

IBM 300PL 6892 (Intel 440 BX, 350MHz, 512MB ECC)
JFS 64MB file, 3 sec
HPFS 64MB file, 6 sec

Intel 915GEV (2.4GHz w/ 2MB Cache, 1GB)
JFS 64MB blip
HPFS 64MB blip
.... thus, bigger file
JFS 100MB 2 sec
HPFS 100MB 3 sec

I have the JFS settings per the Scott suggestion, eCS 1.2MR ships that way.

Both systems grab 64MB cache for JFS, 2MB for HPFS.

Now, this is "IDE" in both cases... 33MB IDE in the IBM box, SATA in the Intel box. Danis driver in both cases.

I did see someone mention that forcing the cache larger than the JFS driver cares to grab does not guarantee better performance... guess I would have to tour d'thread one more time to review all of
your responses... guess I would be interested to see performance if you don't specify cache settings. My appologies if that is stated, the thread is getting a tad long at this point.

--
Michael Lueck
Lueck Data Systems
http://www.lueckdatasystems.com/

Remove the upper case letters NOSPAM to contact me directly.

[Scott]

I'm glad you solved your problem (by reducing the cache size). Having a
too-large cache probably caused JFS to spend lots of time caching and then
flushing, in your case.

I wish that IBM had decided to make the OS/2 JFS open source. Since it
didn't, you're going to be stuck with jfs.ifs as it stands, unfortunately.
And, no, AFAIK there is no other OS/2 IFS aside from UDF that supports > 2gb
files. Sigh.

-Scott

[Julien Pierre]

Ilya,

Ilya Zakharevich wrote:
>
> These are probably stupid questions, but in the last century one
> should have been concerned with matters like this...
>
> Is not there a "slow spot" in your memory? How much memory is cached?
> How much memory your OS/2 system thinks it has? One (incredible!)
> scenario is that some memory is not seen, and the cache is swapped
> out... ;-) But such incredible results require incredible
> explanations...

There is no limitation on memory system for specific area that I am
aware of.

The motherboard is an Asus A7M266-D , with an AMD 760 MPX chipset for
dual Athlon . It's the original version of the board that had a USB bug
and therefore no USB ports on-board (it came with a separate USB PCI
card). The memory is DDR266 PC2100 . I have three 256 MB sticks of ECC
PC2100 memory . I tried to remove two sticks, which left just 256MB of
system memory. I still got the very poor results with the large JFS
cache, and the fast results with the small cache.

OS/2 sees the entire amount of memory in my system. The 4OS2 "memory"
command reports the following.

804,761,600 bytes total physical RAM
3,156,750,336 bytes total resident RAM

22,732,800 bytes largest free block

268,435,456 bytes total swap file

12,288 bytes total environment
5,336 bytes free

4,096 bytes total alias
4,095 bytes free

1,024 bytes total history

[Julien Pierre]

Scott,

Scott wrote:
> I'm glad you solved your problem (by reducing the cache size). Having a
> too-large cache probably caused JFS to spend lots of time caching and then
> flushing, in your case.

I wish I could understand the circumstances that lead to that situation,
since I have two nearly identical machines except for CPU and
motherboard, with fresh OS/2 installation and identical settings, and
only one exhibiting this problem at the large JFS cache size.

> I wish that IBM had decided to make the OS/2 JFS open source. Since it
> didn't, you're going to be stuck with jfs.ifs as it stands, unfortunately.

Is all official support for OS/2 100% over at this point in time, or are
there still large companies with support contracts that could get that
sort of problem investigated/resolved if they so needed ?

> And, no, AFAIK there is no other OS/2 IFS aside from UDF that supports > 2gb
> files. Sigh.

But I can't format my hard drive partition as UDF, can I ?

[Heiko Nitzsche]

> Can someone with a fast running system try the /cache:120000 setting?
> It would seem to me the system would not write the cache until it hit
> the 120 meg point (or is it 120 K??) or a timeout point was reached. At

That's why the reduced cache sync settings are recommended.

> this point the disk i/o is the bottleneck??
> Ilya's comment made me think about this one.

On my Athlon64 3000+, 1GB RAM DDR400, 200GB SATA hard drive,
eCS 1.1 (FP5), DaniS506.ADD I get:
(socket 754, so only single channel RAM access,
socket 939 with dual channel support is much faster)

IFS=C:\OS2\JFS.IFS /AUTOCHECK:* /CACHE:120000 /L:5,20,4
IFS=C:\OS2\HPFS.IFS /CACHE:2048 /CRECL:64 /AUTOCHECK:CDJ

for JFS
512 MB in 8 seconds
1024 MB in 16 seconds
1536 MB in 26 seconds
2048 MB in 36 seconds

for HPFS
512 MB in 12 seconds
1024 MB in 26 seconds
1536 MB in 37 seconds
2048 MB ... sorry not that much space left on HPFS volumes
(only used for OS boot)

IFS=C:\OS2\JFS.IFS /AUTOCHECK:* /L:5,20,4
(cache size reported: 65536 kb)

for JFS
512 MB in 8 seconds
1024 MB in 16 seconds
1536 MB in 26 seconds
2048 MB in 36 seconds

So there seems to be no benefit.

The question is whether this test is appropriate for
general file system performance testing. As we just write
a single file, of course writing without a cache would
probably generate even better results as no overhead for
cache management is involved. But this is a rare use case.
The file system has to be tuned also for other, more common
use cases.

If one for instance uses JFS on a web server with lots
of files on different disks and they are all read randomly
under high network load (2 GBit LAN cards in parallel or so),
I guess then you would see the benefit of a large cache.

I also think that if you do a lot of parallel I/O on
different disks, the system might be more responsive.

Also if you start larger programs very often (batches),
the large read cache is beneficial. You can see it if
you start for instance Mozilla, close it, restart it.
The first call takes on my sytem around 5sec the second
call takes only 1sec or less.

For the normal user, which just works sequentially, the
larger cache size have no real benefit.
Though if you have lots of RAM and a fast core system,
why don't spend some cache memory for those cases it
would make the system more snappy? Of course, this only
makes sense if you don't use it for other purposes.

[Jim Moe]

Scott wrote:
>
> I wish that IBM had decided to make the OS/2 JFS open source. Since it
> didn't, you're going to be stuck with jfs.ifs as it stands, unfortunately.
> And, no, AFAIK there is no other OS/2 IFS aside from UDF that supports > 2gb
> files. Sigh.
>

IBM did, in a way. There was (is?) an alphaworks project whose exact
name I cannot recall (JFS for Linux?) started in 2000 or 2001. The base
code for the project was os/2's jfs source. Of course, the code has
diverged since then.
There is freejfs on hobbes
<http://hobbes.nmsu.edu/cgi-bin/h-search?key=freejfs> by some russian
programmers. I believe they got the code from the alphaworks project.
Netlabs has an openjfs project <http://openjfs.netlabs.org/>. Not much
happening there but the source code is available.

[Veit Kannegieser]

Julien Pierre wrote:

> > Is not there a "slow spot" in your memory? How much memory is cached?
> > How much memory your OS/2 system thinks it has? One (incredible!)
> > scenario is that some memory is not seen, and the cache is swapped
> > out... ;-) But such incredible results require incredible
> > explanations...

> There is no limitation on memory system for specific area that I am
> aware of.

Verify that, for example using memtest!
There exist boards that have limits how many memory they can cache
in several cache size configurations...

--
Veit Kannegieser

[Lorne Tyndale]

Just out of curiosity, has anyone run comparison tests beween JFS and
HPFS386, setting equal cache sizes? I can recall hearing one time that
HPFS386 was still quicker then JFS, but I havn't seen any actual data to
back that up, and since I don't have either JFS or HPFS386 on my system
(I've stayed with plain old HPFS all these years), I can't run such a test.

----== Posted via Newsfeeds.Com - Unlimited-Unrestricted-Secure Usenet News==----
http://www.newsfeeds.com The #1 Newsgroup Service in the World! 120,000+ Newsgroups
----= East and West-Coast Server Farms - Total Privacy via Encryption =----

[Scott G.]

Lorne Tyndale wrote:
> Just out of curiosity, has anyone run comparison tests beween JFS and
> HPFS386, setting equal cache sizes? I can recall hearing one time that
> HPFS386 was still quicker then JFS, but I havn't seen any actual data to
> back that up, and since I don't have either JFS or HPFS386 on my system
> (I've stayed with plain old HPFS all these years), I can't run such a test.

Results will vary greatly, I think, based on the i/o mix. Also, the key
advantage of hpfs386 is that it re-implements a lot of lanserver functionality
at ring 0, so it is esp. advantageous for heavy fileserver activity. FWIW.

[MMI]

Lorne Tyndale wrote:
> Just out of curiosity, has anyone run comparison tests beween JFS and
> HPFS386, setting equal cache sizes? I can recall hearing one time that
> HPFS386 was still quicker then JFS, but I havn't seen any actual data to
> back that up, and since I don't have either JFS or HPFS386 on my system
> (I've stayed with plain old HPFS all these years), I can't run such a test.

Read it at

http://pages.prodigy.net/michaln/os2/os2fsperf.html

Cheers,
Martin

[William L. Hartzell]

Sir:

Scott G. wrote:
<snip>
What is IBM policy of posting of its publicly available device drivers
and fixpacks to Hobbes? I've seen them posted there in the past. I am
wondering about this as IBM will be closing down the OS/2 support sites
soon. Any chance of getting (someone at) IBM to do the posting?

--
Bill
Thanks a Million!

[Scott G.]

> http://pages.prodigy.net/michaln/os2/os2fsperf.html

With respect to Michal, I think he missed some important
- HPFS386 was NEVER included with the base OS -- you had to buy Warp Server
Advanced to get it. This is not because of lack of development ability(a
completely bogus asserion) but because it required royalty payments for each
copy shipped. So, one reason for not using HPFS386 is that very few
installations have legitimate licenses for it (for those who care).
- The hpfs386/jfs performance balance shifts greatly if you're using an
hpfs386-based fileserver. That is, the test numbers would be very different if
done to the same machine as a server, with reads/writes done to hpfs386 vs. jfs
(vs fat/hpfs). Other lanserver remote services are similarly enhanced.
- JFS is, unfortunately, less stable, IMO. I hope that the remaining OS/2
community developers someday do something cool with the openjfs project.
- On the plus side for JFS, he does not mention that HPFS/HPFS386 volumes
are (IIRC) limited to 64gb, where JFS is not.
- Also in JFS' favor, the journaling feature is very nice.

-seg

[MMI]

Scott G. wrote:
>> http://pages.prodigy.net/michaln/os2/os2fsperf.html
>
>
> With respect to Michal, I think he missed some important
> - HPFS386 was NEVER included with the base OS -- you had to buy Warp
> Server Advanced to get it.

Hi Scott,

this is barely mentioned in the first paragraph, all the server-and-cost
stuff.

> This is not because of lack of development
> ability(a completely bogus asserion) but because it required royalty
> payments for each copy shipped. So, one reason for not using HPFS386 is
> that very few installations have legitimate licenses for it (for those
> who care).

I think IBM itself is a bit to blame for this. I know there was HPFS386
available even on OS/2 1.3 Lan Manager, that's about 1989, so for more
than 10 years (up to first Merlin Convenience Package) client OS/2
wasn't given a decent main filesystem driver. In the 1993, when I
started with OS/2 (2.1 at that time), the slow nature of HPFS.IFS wasn't
easily spotted, we had small RAMs and slow disks, but still I remember
how quiet and faster my system started to be when I hacked that 386
beast inside without making cache for 386 any larger than for HPFS.IFS.

I also remember how Doug Azzarito developed a semi-dynamic cache for
HPFS.IFS, that was able to expand up to 64M. That would breathe some
life into the old HPFS.IFS, wouldn't it? But it was never delivered,
although promised for Merlin.

Until the advent of JFS for OS/2 client, OS/2 user was in an unlucky
position. He would either had to suffer the slow throughput (2M of cache
is ridiculously small on then-standard machines with 128+ M of RAM), or
fsck the licensing to have a decent throughput on disk operations.

Putting the JFS into the convenience packages was a great thing. It
removed the hard decisions I described above, and also gave us a good
IFS to work with big post-2000 disks (HPFS386 wasn't suitable for that
either).

> - The hpfs386/jfs performance balance shifts greatly if you're using
> an hpfs386-based fileserver. That is, the test numbers would be very
> different if done to the same machine as a server, with reads/writes
> done to hpfs386 vs. jfs (vs fat/hpfs). Other lanserver remote services
> are similarly enhanced.

I talked with Michal about this, and I got an answer that this test was
about local disk operations, not remote. And I think he was right when
he said that it would need some gigabit or faster network to see the
difference between HPFS386 and JFS in a big way on remote operations,
since every other slower type of network would reduce the speed of
transfer below the IFS driver capabilities.

> - JFS is, unfortunately, less stable, IMO.

Unfortunately, it is less battle-proven. I estimate the HPFS.IFS
codebase is with us since OS/2 1.2.

> I hope that the remaining
> OS/2 community developers someday do something cool with the openjfs
> project.
> - On the plus side for JFS, he does not mention that HPFS/HPFS386
> volumes are (IIRC) limited to 64gb, where JFS is not.
> - Also in JFS' favor, the journaling feature is very nice.

IMHO everyone deciding about IFS to use puts these features into the
decision. I can't imagine using HPFS or HPFS386 again on anything else
than boot disk (yet). I don't want those lengthy autochkdsks on startup
again ;-)

Cheers,
Martin

>
> -seg

[Alex Taylor]

On Sun, 04 Dec 2005 12:55:42 +0100, MMI <m...@nautimail.com> wrote:
>> - JFS is, unfortunately, less stable, IMO.
>
> Unfortunately, it is less battle-proven. I estimate the HPFS.IFS
> codebase is with us since OS/2 1.2.
>

> IMHO everyone deciding about IFS to use puts these features into the
> decision. I can't imagine using HPFS or HPFS386 again on anything else
> than boot disk (yet). I don't want those lengthy autochkdsks on startup
> again ;-)

I use HPFS386 on my boot drive, and on my main data volume (containing
files I do NOT want to risk anything Bad happening to). I use JFS on most
other volumes, especially my application volume; and also my 'misc' volume
which is where I keep my CVS sandboxes, my MP3s, video clips, and image
collections (cute kittens and sunsets, yup) which is all theoretically
re-downloadable in the event of catastrophic data loss.

One odd thing I've noticed about HPFS386 -- I don't think I've ever
succeeded in using a cache larger than 64MB. Weird things happen if I go
above that amount.

In particular, on my current box (AMD Athlon 64), when I tried to
configure a larger HPFS386 cache (128 MB) I got a boot-time error saying
there wasn't enough memory. (The box has 1 GB of RAM.) ISTR having
similar issues on an old P-II box with 512 MB.

Another weirdness on this AMD box -- if I didn't start CACHE386.EXE from
CONFIG.SYS, my system trapped nastily right after the desktop came up.
Once I added CALL=...\CACHE386.EXE to CONFIG.SYS, everything came up
sunshine and roses. Strange.

I've never been entirely sure whether CACHE386.EXE is supposed to be
required or not. The HPFS386 installer doesn't add it, and the
documentation doesn't even mention whether you're supposed to start
it or not.

--
Alex Taylor
http://www.cs-club.org/~alex

Remove hat to reply (reply-to address).

[MMI]

Alex Taylor wrote:
> On Sun, 04 Dec 2005 12:55:42 +0100, MMI <m...@nautimail.com> wrote:
>
>>> - JFS is, unfortunately, less stable, IMO.
>>
>>Unfortunately, it is less battle-proven. I estimate the HPFS.IFS
>>codebase is with us since OS/2 1.2.
>>
>>IMHO everyone deciding about IFS to use puts these features into the
>>decision. I can't imagine using HPFS or HPFS386 again on anything else
>>than boot disk (yet). I don't want those lengthy autochkdsks on startup
>>again ;-)
>
>
> I use HPFS386 on my boot drive, and on my main data volume (containing
> files I do NOT want to risk anything Bad happening to).

Well, yes, for sensitive unbacked up data I'd use HPFS, too.

> I use JFS on most
> other volumes, especially my application volume; and also my 'misc' volume
> which is where I keep my CVS sandboxes, my MP3s, video clips, and image
> collections (cute kittens and sunsets, yup) which is all theoretically
> re-downloadable in the event of catastrophic data loss.
>
> One odd thing I've noticed about HPFS386 -- I don't think I've ever
> succeeded in using a cache larger than 64MB. Weird things happen if I go
> above that amount.

I did. On a box with about 256M RAM. All worked well except for the same
thing that hits JFS with big cache - HPFS386 spent a lot of time by
managing cache after big transfers - lazy writicg could bring the box to
the temporary halt for several seconds.

> In particular, on my current box (AMD Athlon 64), when I tried to
> configure a larger HPFS386 cache (128 MB) I got a boot-time error saying
> there wasn't enough memory. (The box has 1 GB of RAM.) ISTR having
> similar issues on an old P-II box with 512 MB.
>
> Another weirdness on this AMD box -- if I didn't start CACHE386.EXE from
> CONFIG.SYS, my system trapped nastily right after the desktop came up.
> Once I added CALL=...\CACHE386.EXE to CONFIG.SYS, everything came up
> sunshine and roses. Strange.

Since I always used HPFS386 for effective and fast caching, I never
tried this :-)

> I've never been entirely sure whether CACHE386.EXE is supposed to be
> required or not. The HPFS386 installer doesn't add it, and the
> documentation doesn't even mention whether you're supposed to start
> it or not.

I think on Lanserver the server should complain that it's not running...

Cheers,
martin

[Trevor Hemsley]

On Mon, 5 Dec 2005 16:33:05 UTC in comp.os.os2.bugs, Alex Taylor
<mai...@reply.to.address> wrote:

> when I tried to
> configure a larger HPFS386 cache (128 MB) I got a boot-time error saying
> there wasn't enough memory. (The box has 1 GB of RAM.) ISTR having
> similar issues on an old P-II box with 512 MB.

I seem to remember an APAR about this that said that you had to code maxheap= in
hpfs386.ini to a value less than something that I've now forgotten.
Unfortunately the PSP APAR database now gives a Lotus Notes exception 'file does
not exist' when you try to go there so I can't look it up.

[Alex Taylor]

On Mon, 05 Dec 2005 13:19:50 -0600, Trevor Hemsley
<Trevor-...@mytrousers.dsl.pipex.com> wrote:
>> when I tried to
>> configure a larger HPFS386 cache (128 MB) I got a boot-time error saying
>> there wasn't enough memory. (The box has 1 GB of RAM.) ISTR having
>> similar issues on an old P-II box with 512 MB.
>
> I seem to remember an APAR about this that said that you had to code maxheap= in
> hpfs386.ini to a value less than something that I've now forgotten.
> Unfortunately the PSP APAR database now gives a Lotus Notes exception 'file does
> not exist' when you try to go there so I can't look it up.

Somebody one one of these newsgroups (maybe this very one) once posted,
IIRC, that to avoid some bug you should set maxheap to the same or less
than the cache size parameter. But that never made much sense to me.

OTOH, since I don't actually know what the 'heap' really _is_ WRT a
filesystem, I can't judge the sensibility of this advice -- so I stay with
the documentation's instructions, which say to leave it unset.

I haven't played around any further, since I don't particularly want to
make my box trap again.

--
Alex Taylor

[Bob Martin]

Did anyone determine if JFS.IFS version 14.104 is freely available?

[Hendrik Schmieder]

Bob Martin schrieb:

This is a private version, so you have to wait for the next fixpak
(, if there will be one).

Hendrik

[Julien Pierre]

Peter Weilbacher wrote:

> On Sun, 27 Nov 2005 02:27:33 UTC, Julien Pierre wrote:
>
>
>>But I can't format my hard drive partition as UDF, can I ?
>
>

> Even if you could, I doubt that would solve your speed problems. UDF
> doesn't even reach the speed limits of DVDs. ;-)

Well, JFS in my tests was performing at barely over 1x DVD speed.
Reading 1024 MB in 573 seconds is only 1.78MB/s . I think DVD is about
1.4MB /s . Of course that's only the base throughput for DVD video
playback. Most DVD readers/writers perform at multiples of that speed.