All email replies thankfully accepted.
Now, of course, DEC no longer rates their products in VUP's; they
are publishing TPC-A benchmarks in the SOC. Oh well....
=====================================
George S. Chapman
Director, Technology Planning
The Midwest Stock Exchange, Inc.
Chicago, Illinois; Bangkok Thailand
mithr...@isvnet.enet.dec.com
=====================================
>DATE: Thu, 29 Oct 92 22:44:40 +0100
>FROM: Ronald Becker Williams <r...@bwunltd.wciu.edu>
MIPS, however, is a much looser and less well defined term, and much less
closely equivalent. While a VAX 11/780 is often called roughly a one mip machine,
in fact, it executes around 1/2 million VAX instructions/second; but each instruction
on a VAX tends to do much more than most other machines (accomplishing around twice
as much as on most other machines). RISC technology has shown, however, that
this tradeoff in the end, with current technology, was a poor trade
(An Alpha, manufactured in the identical process technology as an NVAX chip,
runs real programs at least twice the speed as an NVAX)
One of the IBM 360 or 370's was the original "one MIP machine",
but VAXen seemed to become much more widespread.
In anycase, MIPS is best thought of as "Meaningless Indicator of Performance"; stick
to useful benchmark programs to actually measure speed for your application. For commonplace
integer and floating point applications, the integer and floating SPECmarks are
reasonable starting points when comparing machines.
- Jim Gettys
--
Digital Equipment Corporation
Cambridge Research Laboratory
[stuff deleted]
>
>MIPS, however, is a much looser and less well defined term, and much less
[more stuff deleted]
>In anycase, MIPS is best thought of as "Meaningless Indicator of Performance"; stick
> - Jim Gettys
>
>--
>Digital Equipment Corporation
>Cambridge Research Laboratory
I have an acronym program. Here is its output for "MIPS" :
MIPS Massively Inconclusive Performance Suite
MIPS Meaningless Indicator of Processor Speed
MIPS Meaningless Information Propagated by Salesmen
MIPS Million Instructions Per Second
MIPS Millions of floating-point Instructions Per Second
The last one is probably wrong. The others may be correct. ;-)
Fred W. Bach , Operations Group | Internet: mu...@erich.triumf.ca
TRIUMF (TRI-University Meson Facility) | Voice: 604-222-1047 loc 278/419
4004 WESBROOK MALL, UBC CAMPUS | FAX: 604-222-1074
University of British Columbia, Vancouver, B.C., CANADA V6T 2A3
These are my opinions, which should ONLY make you read, think, and question.
They do NOT necessarily reflect the views of my employer or fellow workers.
Folks,
The use of the VUPS measurement (VAX Unit of Performance) was an early move
by Digital to provide some standard comparison of machine performance by some
means other than MIPS.
Vendor MIP ratings had varied so much (and some of the processors were being
generated with fun instructions that were specifically targeted at increassing
the Dhrystone rating) that it seemed some standard was necessary. We needed one
even in describing our own machines.
A year or so later, the SPEC rating was established. DEC continued to provide
VUP ratings along WITH the SPEC ratings in order to provide a transition for
the customers who had gotten accustomed to VUPS. I can even remember being
taken to task by a customer at DECUS New Orleans in 1990 because he believed
that DEC had created the SPEC rating (he didn't realize it was a multi-company
effort) as a new way to confuse people.
SPECmarks seemed a reasonable effort (an independent body to control both the
test code and the reporting format). It's based on a VAX 11/780 because "THE"
known-base-machine had to be guaranteed to be supported for a long time (read,
forever) and DEC was the only participant willing to do so. I had actually
remembered that it was the SPECmark rating that used new compilers and the VUP
testing that had a fixed compiler revision level; but Jim's memory is probably
better than mine.
The benchmarks will continue to evolve. They will have to, to keep up with
the new machines (you can do some fun things with benchmarks when you can run
the entire thing in a 1MB cache). They are provided as a roughest approximate
yard stick. Predominately for those people who only look at an overall chart of
machines.
The best way of judging a systems performance is still to run your own set of
applications on it, with the hardware configured the way you want it. Sometimes
that isn't possible, and that's when folks default to MIPS/SPECs/VUPS as a way
to judge.
All of the above is just my own silly opinion, of course. Still, I hope it
provides some prespective.
Thanx,
AJ
ps. Now, if I could just get everybody to agree to stop those silly "Entry
System Configuration" routines where every company quotes a configuration
you wouldn't wish on your worst enemy. Unfortunately, that's one of those
"I'll put my gun down if you put your's down. You go first..." type of
discussions. Sigh...
**********************************************************************
* AJ Casamento "The question is not whether or *
* Digital's TRI/ADD Program not the opinions are mine; but *
* 529 Bryant Ave. PAG-2 rather, which of my personalities *
* Palo Alto, CA 94301-1616 do they belong to?" *
* 415.617.3460 *
* a...@pa.dec.com *
**********************************************************************
Not true. 1 VUP is about 1.4-1.5 MIPS. But 1 VUP = 1 Specmark.
There are 3 metrics that use the VAX-11/780 as the base:
1. Dhrystone MIPS. A system's Dhrystones/sec is divided by 1757 (this was the
VAX-11/780's Dhrystone number at one time).
2. VUPS. We used the central tendency (geometric mean) of 99 diverse benchmarks
to determine the relative performance compared to a VAX-11/780, both using
current compilers.
3. SPECmarks. The VAX-11/780 was used to generate reference numbers (most using
VAX/VMS, a couple using Ultrix) and then used to compute SPECratios, SPECmark
etc. Reference numbers were not updated as VAX compilers, preprocessors etc
improved/emerged. There was a very strong correlation between VUPS and
SPECmarks for VAX systems until the matrix300 benchmark was cracked.
Our competition has often used VUPS or MIPS to mean "times VAX-11/780" for any
benchmarks using any (not always the best) compiler on VAX.
At this stage, I would toss all of these out the window and look at SPECint92
and SPECfp92. Of course your mileage will vary and individual SPECratios may
be closer to your application.
Dileep
greg pavlov
pav...@fstrf.org
To the clear thinking among us, it was a marketing move aimed at the
simpletons who think that "MIPS" is the only real comparison between
processors because "VUPS only apply to VAXes." When DEC started selling
RISC machines, they were moving into another market, one which they didn't
shape, and they had to conform with standards already in place. That meant
using MIPS.
It's funny, because MIPS are meaningful only within an architecture, whereas
VUPS are an actual unit of power that can be applied across multiple
architectures. However, it was common practice in the RISC world to use MIPS
because they sounded so much better.
Even the DOS people caught on. I can remember someone telling me (back in
1988) that the 386 was running at 21 MIPS. I believe that if you ran it at
50MHz (unsupported, didn't last long), and had some realistic instruction
sequence like "add ax to ax, branch to that instruction", you could get a
rating like that. Companies like Sun, HP, IBM, and DEC rarely stoop so low,
but MIPS are meaningless anyway.
For a good comparison among machines, look for the SpecMark comparisons.
For an understanding of how machines compare on all levels, the TPC benchmarks
are excellent, and they are governed by a third party, so there should be
no bullsh*t.
Darrin
--
mdchaney@iubacs mdch...@bronze.ucs.indiana.edu mdch...@rose.ucs.indiana.edu
"I want- I need- to live, to see it all..."
Well, the last one should be
FLOP FLoating-Point OPeration
Which reminds me of a conference (on parallel processing) where somebody
claimed that "megaflops are bunk", and somebody else later suggested
BUNK Binary or Unary Normalized Konstruct
and commented this would leed to Megabunks, which would probably be a flop.
Thomas
--
*** This is the operative statement, all previous statements are inoperative.
* email: cmaae47 @ ic.ac.uk (Thomas Sippel - Dau) (uk.ac.ic on Janet)
* voice: +44 71 589 5111 x4937 or 4934 (day), or +44 71 823 9497 (fax)
* snail: Imperial College of Science, Technology and Medicine
* The Center for Computing Services, Kensington SW7 2BX, Great Britain
Cui-Qing Yang
Dept. of Computer Science
University of Norht Texas
cqy...@ponder.csci.unt.edu
>Absolutely. That was what the original equivalence standard was.
>I've been using it for years when comparing to other vendors.
The problem with this is that, once you jump machine architectures, this
number becomes mostly (if not completely) useless. The reason that DEC
stopped quoting MIPS was that they were beating up IBM a few years back
(circa 1985 ?) about how their price per MIP beat the pants off of IBM
by something like a factor of 2 to 1. Then IBM came back with some benchmarks
and proved that you could get twice as many 'transactions' processed with
an IBM MIP than you could with a DEC MIP. To which DEC responded that IBM's
numbers were invalid because they represented batched transactions and not
online, to which IBM responded that large systems are measured in transactions
and not interactive access, to which ...
VUPS does make a lot of sense in comparing a VAXen, since the VAX
architecture is still somewhat the same (although the implementation has
varied quite a bit from processor line to processor line). However, DEC
generally gives VUP's in ranges (particularly on their larger systems)
because a VUP started out as an 11/780 (which they used to call a MIP)
and there have been enough changes in the VAX implementation since
then that your rating will vary depend on specifically what you are
trying to doing. Case in point is that some of the old 8xxx series
could easily beat some of the earlier 6xxx series machines which had
a much higher VUP rating, primarily because of some instructions which
were emulated (the older 6xxx series actually used the uVAX chip set,
although I am not sure if the 64xx/65xx still do).
An of course, I/O intensive applications will run much better on an XMI
attached RF-series disk than a Unibus attached RA-series disk, regardless
of the VUP rating !
IMHO, VUP/MIP ratings between completely different machine architectures
are relatively useless. VUP/MIP ratings between the same architecture
machines gives a close ballpark for comparison, but are still not
concrete, unless you take other differences in the machines implementation
into effect (i.e., I/O bus, memory bus, emulated instructions, etc.)
Regards,
Dave.
--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
= David E. Filip UUCP : dfi...@colornet.com =
= ColorNet Information Systems CIS : 76430,3111 =
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
= Standards are wonderful 'cause there are so many to choose from ! =
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=