Poor VMS file create performance compared to Linux/FreeBSD
Simon Clubley
trying to perform the same task - unpacking a GCC tar archive - on these
operating systems. A table of results is below.
I've gone into a lot of detail about my test environments, so that hopefully
if I've missed something it will be noticed. I am also interested in seeing
the results on other platforms if anyone is so inclined.
The tar archive in question is GCC 3.4.0 which contains 22735 files/dirs and
is 191,467,520 bytes long uncompressed; as VMSTAR does not support compressed
archives I uncompressed all archives first[1]. The times below are for the untar
operation only. The archive is available from the usual GCC mirrors. I used:
ftp://ftp.mirror.ac.uk/sites/sources.redhat.com/pub/gcc/releases/gcc-3.4.0/
The archive is: gcc-3.4.0.tar.bz2 (25.9MB) or gcc-3.4.0.tar.gz (34.3MB)
I turned off Linux deferred writes on the filesystem in question by
remounting the filesystem with "-o sync" on the mount command; I have
included the Linux deferred write performance for comparison.
In all cases, the input tar file was on the same disk as the extracted files.
Machines:
Avg Avg
Machine/OS Filesystem Disk/Interface RPM Seek(ms) Latency(ms)
AlphaStation 200 4/166, VMS 7.3-1
ODS-5 IBM DORS-32160, SCSI 5400 8.5 5.56
Testdrive Alpha DS20, 500MHz, VMS 7.3-2
ODS-5 RZ2DD-LS, SCSI 10000 5.4 2.99
Pentium 233Mhz, Redhat Linux 6.2
ext2 ST320410A, IDE 5400 8.9 5.55
Pentium 233Mhz, FreeBSD 4.5
FreeBSD ST34321A, IDE 5400 11.5 5.6
native
I choose the PC above so that the CPU performance would be as close as
possible to my AlphaStation.
In all cases, the command line was: [tar_program] -vxf [archive_name].tar
Results:
Machine OS Unpack time Notes
===================================================================
AS 200 4/166 VMS 7.3-1 1 hour 46 mins Extract didn't complete [2]
Pentium 233 Redhat 6.2 2 mins 10 secs Deferred writes ON
Pentium 233 Redhat 6.2 4 mins 22 secs Deferred writes OFF
Pentium 233 FreeBSD 4.5 15 mins 55 secs [3]
Testdrive VMS 7.3-2 55 mins 48 secs Extract didn't complete [2,4]
For VMS, I used VMSTAR V3.4-1, obtained from Hunter's site and linked on the
target machine. All VMS measurements were done using this version. [5]
Does anyone have any ideas on how I can increase VMS file create performance
to bring it up to Linux/FreeBSD performance levels ?
Simon.
[1] Interestingly, the uncompress operation completed in roughly the same time
(allowing for hardware differences) on VMS as it did on Linux. This would
seem to suggest that raw I/O write performance on VMS is ok, and it's the
actual creating of files that appears to be the performance problem.
[2] The untar failed towards the end, because of a full path that was longer
than 100 bytes. VMSTAR does not support LongLink tar blocks, which means that
VMSTAR will not fully unpack any tar archive with a full path of more than
100 bytes. (I've already notified Hunter; I encountered the same problem doing
some early testing with a GCC preview kit.) The last file successfully
extracted was:
Jan 4 23:18:59 2004 2316 [C3.pvg.libstdc^+^+-v3.testsuite.27_io.basic_istre
am.extractors_arithmetic.char]13.cc;
[C3.^.]
tar: unexpected EOF on tar file.
This is sufficiently close to the end of the archive that VMSTAR shouldn't have
taken more than a few minutes to finish extracting the files.
[3] The FreeBSD filesystem metadata is written using sync I/O; file data is
written using async I/O. Even allowing for the slower disk that FreeBSD is on
(you can't place FreeBSD on an extended partition), the native FreeBSD
filesystem appears to be slower than ext2 at writing large numbers of files.
Any FreeBSD experts care to comment ?
[4] I made sure that VMSTAR was the only process doing any significant I/O
by using MONITOR at regular intervals.
[5] The VMSTAR binaries supplied as part of the GNV kits were slower than
the VMSTAR that I linked. I do not know why.
--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
SCO: Proudly pushing Microsoft down to #2 on the list of most disliked companies
Hoff Hoffman
:I am seeing very poor VMS I/O performance compared to Linux/FreeBSD when
:trying to perform the same task - unpacking a GCC tar archive - on these
:operating systems. A table of results is below.
Search for "mybenchmark"; a set of tools made available by David Mathog.
Around the discussions of the tools, you'll also find various previous
discussions of the I/O performance.
:Does anyone have any ideas on how I can increase VMS file create performance
:to bring it up to Linux/FreeBSD performance levels ?
Given differences in the file systems of UNIX and OpenVMS, there are and
will be differences in performance. That written, the OpenVMS file
system, XFC caching, and file system I/O performance are all areas that
are all under investigation within OpenVMS Engineering; there is typically
on-going profiling and performance-related work within engineering.
We have made various improvements in recent releases, and have been
actively working to increase the aggregate I/O performance -- this
includes changes made in V7.3-2, as well as work targeting future
OpenVMS releases.
--
Y'all can get a hobbyist license for the HP C compiler, too.
---------------------------- #include <rtfaq.h> -----------------------------
For additional, please see the OpenVMS FAQ -- www.hp.com/go/openvms/faq
--------------------------- pure personal opinion ---------------------------
Hoff (Stephen) Hoffman OpenVMS Engineering hoff[at]hp.com
Simon Clubley
>
> Search for "mybenchmark"; a set of tools made available by David Mathog.
> Around the discussions of the tools, you'll also find various previous
> discussions of the I/O performance.
>
Thanks for the pointer; I will take a look.
>
> Y'all can get a hobbyist license for the HP C compiler, too.
>
Perfectly true, and I have one installed. Unfortunately, my interest is
because of Ada...
Simon.
John Santos
Do you know the total # of I/Os (Direct and buffered), the compute time,
and the system overhead (% CPU usage in kernel, exec, super and interrupt
stack?) (I think SUPER mode time should be charged back to the process,
but inner modes may not be.)
The reason I ask is because everyone's immediate assumption is
probably that it is I/O-bound, and the slowness is due to VMS file
system overhead (extending directories, flushing attributes to disk,
etc.) But what if something horrible is going on in some library
routine, and it is actually compute bound? Maybe using different
compiler options or a different compiler would help enormously.
Someone in another followup mentioned using the DEC/Compaq/HP C
compiler as something that might help. I don't know what compiler
was used for the tar on Hunter's site, but I would guess that the
GNV stuff all uses the latest and greatest DEC C, and you said
that's even slower...
Assuming it is I/O bound, maybe VMS needs a "$ MOUNT/CACHE=DANGEROUS"
(with a synonymous "/CACHE=UNIX_STYLE") where *nothing* is flushed except
when it runs out of buffers or when the volume is dismounted (or maybe on
an explicit "set volume/cache=safe" (aka "/cache=vms_style"), which would
write everything out.) You could use this on scratch volumes or when
loading software, when you are willing to delete and/or re-init everything
and start over in the event of an untimely system crash. (If at mount
time, a volume is found to have the "dangerous" and "mounted" bits set
(indicating it wasn't cleanly dismounted), it tells you so and either
write-locks it or forces you to use a /OVERRIDE=possibly_messed_up
qualifier.)
(Sorry for all the nested parens... too much LISP in college...)
Maybe this flag should be on a directory basis (with default
inherited when the directory is created from its parent) rather
than on a volume basis. You would have to delete the directory
and ANALYZE/DISK/REPAIR to recover after a crash, but the rest
of the disk should be fine.
--
John Santos
Evans Griffiths & Hart, Inc.
781-861-0670 ext 539
David J. Dachtera
>
> I am seeing very poor VMS I/O performance compared to Linux/FreeBSD when
> trying to perform the same task - unpacking a GCC tar archive - on these
> operating systems. A table of results is below.
>
>
> [5] The VMSTAR binaries supplied as part of the GNV kits were slower than
> the VMSTAR that I linked. I do not know why.
I think there's a primary clue. tar, being a UN*X "baby", is not
optimized for VMS.
In general, I find the performance of most UN*X-ware to be rather
disappointing on VMS, and I don't consider it for comparison to "native"
VMS code.
--
David J. Dachtera
dba DJE Systems
http://www.djesys.com/
Unofficial Affordable OpenVMS Home Page:
http://www.djesys.com/vms/soho/
Barry Treahy, Jr.
>Simon Clubley wrote:
>
>
>>I am seeing very poor VMS I/O performance compared to Linux/FreeBSD when
>>trying to perform the same task - unpacking a GCC tar archive - on these
>>operating systems. A table of results is below.
>>
>>[snip]
>>
>>[5] The VMSTAR binaries supplied as part of the GNV kits were slower than
>>the VMSTAR that I linked. I do not know why.
>>
>>
>
>I think there's a primary clue. tar, being a UN*X "baby", is not
>optimized for VMS.
>
>In general, I find the performance of most UN*X-ware to be rather
>disappointing on VMS, and I don't consider it for comparison to "native"
>VMS code.
>
>
"Native" probably isn't the best word as anything that is compiled,
linked, and runs on VMS is "native" to the VMS OS. The real issue is
that the API libraries that allow Un*x code to run on VMS, basically
emulating a Un*x environment, fumbles the proverbial performance ball
therefore I would say that has less to do with the apps, otherwise they
would also run poorly on Un*x, which isn't the case...
IMNHO, VMS needs to do a better job of handling the Un*x code because
Un*x predates VMS, and though many of us thrashed Un*x during the
VMS/Un*x wars, I suspect that Un*x will outlive VMS and it is something
we need to learned to live with... Frankly, I'd take a good Un*x system
over Microcrap anytime...
Barry
--
Barry Treahy, Jr E-mail: Tre...@MMaz.com
Midwest Microwave Phone: 480/314-1320
Vice President & CIO FAX: 480/661-7028
Paul Sture
> In article <urMoc.1392$q53...@news.cpqcorp.net>, ho...@hp.nospam (Hoff Hoffman) writes:
>
>> Search for "mybenchmark"; a set of tools made available by David Mathog.
>> Around the discussions of the tools, you'll also find various previous
>> discussions of the I/O performance.
>>
>
>
> Thanks for the pointer; I will take a look.
>
IIRC, one of the things which came out of David Mathog's research or one
of the related discussions was that of default RMS extend size. IIRC
(again), Hunter Goatley incorporated a larger value for this in HGFTP,
resulting in significant performance gains.
Patrick MOREAU, CENA Athis, Tel: 01.69.57.68.40
[...]
>
> IIRC, one of the things which came out of David Mathog's research or one
> of the related discussions was that of default RMS extend size. IIRC
> (again), Hunter Goatley incorporated a larger value for this in HGFTP,
> resulting in significant performance gains.
RMS default parameters are too conservative. I use some modified parameters:
$ set rms/seq/block=127/buff=8
$ set rms/ind/buf=20
$ set rms/extent=2048
and performance is not bad at all (however untar is always rather slow).
Greater extent size are even useful when creating big backup savesets or
big zip archives.
Patrick
--
===============================================================================
pmo...@ath.cena.fr (CENA) ______ ___ _ (Patrick MOREAU)
more...@decus.fr (DECUS) / / / / /| /|
CENA/Athis-Mons FRANCE / /___/ / / | / | __ __ __ __
BP 205 / / / / |/ | | | |__| |__ |__| | |
94542 ORLY AEROGARE CEDEX / / :: / / | |__| | \ |__ | | |__|
http://www.ath.cena.fr/~pmoreau/ http://www.multimania.com/pmoreau/
===============================================================================
Main, Kerry
As Patrick mentioned, there are a number of areas where the default
OpenVMS parameters should be adjusted if one wants to compare apples to
apples performance with other platforms.
A few quick thoughts:
1. Ensure all drives have the default "highwater marking" attribute
removed - especially for write performance. This is a security feature
that will zero any unallocated blocks before it performs the IO
requested by the user. Many Customers choose not to have this default
attribute on all of their drives. It can be removed by:
$ set volume/nohigh $1$dgaxx (do not need to dismount drives)
$ pipe show dev d/full |search sys$pipe high (to check all drives to see
if any drives have highwater marking still enabled)
2. With VMS V7.3-1 and later, you can set the application to use RMS
write back in a similar manner as the default on a UNIX system.
Reference:
http://h71000.www7.hp.com/openvms/os/v731features.html (scroll down to
RMS)
- Increase in default I/O transfer size
- Increase in default autoextend size
- RMS writebehind performance option (RMS_SEQFILE_WBH is dynamic sysgen
parameter)
- New readahead hint passed by RMS to XFC to prefetch disk blocks
Reference the following for more details on the RMS new features:
http://h71000.www7.hp.com/doc/731FINAL/6657/6657pro_007.html (section
5.11.3 for the write behind feature that allows OpenVMS to perform in a
nature similar to UNIX default i.e. write-back.)
3. I would use VMS V7.3-2 as it has numerous other performance
enhancement features as well over V7.3-1.
Regards
Kerry Main
Senior Consultant
HP Services Canada
Voice: 613-592-4660
Fax: 613-591-4477
Email: kerryDOTmainAThpDOTcom
(remove the DOT's and AT for email address)
Bob Koehler
>
> IMNHO, VMS needs to do a better job of handling the Un*x code because
> Un*x predates VMS, and though many of us thrashed Un*x during the
> VMS/Un*x wars, I suspect that Un*x will outlive VMS and it is something
> we need to learned to live with... Frankly, I'd take a good Un*x system
> over Microcrap anytime...
>
Nonsense. We all know the ENTERP:: will run on VMS, long after UNIX
literally runs out of time.
Andrew Rycroft
Gee its fast, but boy can it corrupt your data
"Barry Treahy, Jr." <Tre...@MMaz.com> wrote in message news:<40A44ABC...@MMaz.com>...
Bill Todd
"Simon Clubley" <clubley@remove_me.eisner.decus.org-Earth.UFP> wrote in
message news:ACNLDA...@eisner.encompasserve.org...
> I am seeing very poor VMS I/O performance compared to Linux/FreeBSD when
> trying to perform the same task - unpacking a GCC tar archive - on these
> operating systems. A table of results is below.
>
> I've gone into a lot of detail about my test environments, so that
hopefully
> if I've missed something it will be noticed. I am also interested in
seeing
> the results on other platforms if anyone is so inclined.
>
> The tar archive in question is GCC 3.4.0 which contains 22735 files/dirs
and
> is 191,467,520 bytes long uncompressed; as VMSTAR does not support
compressed
> archives I uncompressed all archives first[1].
This in itself should be a rather glaring clue: according to the numbers
you provided later, on the Red Hat systems you seem to be creating output
files at a rate of close to 90 per second with 'deferred writing' disabled,
and close to twice that when it's enabled.
You'd have difficulty doing that with truly synchronous disk access even if
each file creation/population took only a single disk access: clearly, -o
sync is not doing what you think it's doing - quite likely, it may be
controlling deferring of writes to the files, but isn't affecting deferrals
of creation (directory) operations at all.
20 years ago (about the most recently I've used VMS), there was no way to
defer directory operations at all - and for a file creation, you've got at
least a single write to enter the name, plus file header allocation, plus
the header update for the creation, plus the header update after the data is
written, plus the data write itself (and that's if there's no other
bookkeeping such as the high-water-mark maintenance already mentioned).
Only the last of these is likely affected by specifying -o sync to Red Hat:
everything else is likely asynchronous, and if you're dumping many files
into the same directories a lot of those asynchronous writes will also be
coalesced.
David Mathog's tests point out differences like this, and others as well.
VMS started out with a significantly different attitude toward on-disk
consistency than Unix did, and clustering only intensified it (because with
Unix, you can try to fix up a potentially inconsistent file system on
restart, at the cost of a potentially lengthy analysis, whereas with VMS the
file system must be left in an internally consistent state all the time,
since other nodes may continue processing it even if one node fails in the
middle of an operation).
It's possible that VMS has changed enough lately to make the above
observations inaccurate, so take them for what they're worth.
- bill
Simon Clubley
> trying to perform the same task - unpacking a GCC tar archive - on these
> operating systems. A table of results is below.
>
Thanks to everyone for the comments. When I get around to looking at this
again, I will try the various suggestions.
BTW, the comment that Patrick made about finding that untar was slow
was interesting. Is anyone working on making unzip ODS-5 capable ?
Simon.