Humans win Sixth Harvard Cup, 23.5-12.5
Christopher F. Chabris
Here are the final results of all nine rounds of the Sixth Harvard Cup
Human Versus Computer Chess Challenge, just concluded at the Manhattan
Conference Center in New York. A complete crosstable will appear
at H3 Online (http://www.h3.org/h3) tomorrow. Complete game scores will
follow in a subsequent article.
Round 1:
Socrates 95 1/2-1/2 Patrick Wolff
WChess 1/2-1/2 Boris Gulko
Ilya Gurevich 0-1 Chessmaster 4000 for Windows 95
Gregory Kaidanov 0-1 M-Chess Pro
Round 2:
Socrates 95 0-1 Ilya Gurevich
Junior 0-1 Gregory Kaidanov
WChess 0-1 Michael Rohde
Joel Benjamin 0-1 Chessmaster 4000 for Windows 95
Round 3:
Gregory Kaidanov 0-1 WChess
Patrick Wolff 1-0 Junior
Boris Gulko 0-1 Virtual Chess
M-Chess Pro 0-1 Ilya Gurevich
Round 4:
Ilya Gurevich 0-1 Junior
Virtual Chess 1/2-1/2 Gregory Kaidanov
Socrates 95 0-1 Joel Benjamin
Chessmaster 4000 for Windows 95 0-1 Michael Rohde
Round 5:
Michael Rohde 1-0 Socrates 95
Joel Benjamin 1-0 M-Chess Pro
WChess 0-1 Patrick Wolff
Chessmaster 4000 for Windows 95 0-1 Boris Gulko
Round 6:
Michael Rohde 0-1 Virtual Chess
Junior 0-1 Boris Gulko
Joel Benjamin 1-0 WChess
M-Chess Pro 1-0 Patrick Wolff
Round 7:
Boris Gulko 1/2-1/2 Socrates 95
Virtual Chess 1/2-1/2 Ilya Gurevich
Chessmaster 40000 for Windows 95 0-1 Gregory Kaidanov
Michael Rohde 1-0 Junior
Round 8:
Patrick Wolff 1-0 Virtual Chess
Boris Gulko 1-0 M-Chess Pro
Junior 0-1 Joel Benjamin
Ilya Gurevich 1-0 WChess
Round 9:
Gregory Kaidanov 1-0 Socrates 95
M-Chess Pro 1/2-1/2 Michael Rohde
Virtual Chess 1/2-1/2 Joel Benjamin
Patrick Wolff 0-1 Chessmaster 4000 for Windows 95
Final Scores (out of 6 games) and performance ratings:
Joel Benjamin: 4.5 / 2694
Michael Rohde: 4.5 / 2656
Boris Gulko: 4.0 / 2597
Gregory Kaidanov: 3.5 / 2511
Patrick Wolff: 3.5 / 2510
Ilya Gurevich: 3.5 / 2508
Virtual Chess: 3.5 / 2644
Chessmaster 4000 for Windows 95: 3.0 / 2581
M-Chess Pro: 2.5 / 2514
WChess: 1.5 / 2358
Socrates 95: 1.0 / 2268
Junior: 1.0 / 2254
Overall total: Humans 23.5, Computers 12.5 (35%)
--
======================
Christopher F. Chabris (c...@wjh.harvard.edu)
P.O. Box 382967, Harvard Square Station, Cambridge, MA 02238-2967 USA
GivenRandy
Like last year, I do not give much credit to a computer tournament without
the top-rated computer software (Chess Genius). Oh well.
Randy Given <Given...@aol.com>
Ed Seedhouse
>Final Scores (out of 6 games) and performance ratings:
>Joel Benjamin: 4.5 / 2694
>Michael Rohde: 4.5 / 2656
>Boris Gulko: 4.0 / 2597
>Gregory Kaidanov: 3.5 / 2511
>Patrick Wolff: 3.5 / 2510
>Ilya Gurevich: 3.5 / 2508
>Virtual Chess: 3.5 / 2644
>Chessmaster 4000 for Windows 95: 3.0 / 2581
>M-Chess Pro: 2.5 / 2514
>WChess: 1.5 / 2358
>Socrates 95: 1.0 / 2268
>Junior: 1.0 / 2254
>Overall total: Humans 23.5, Computers 12.5 (35%)
Looks like the silicon was comprehensivly whupped. The best score for
a program was equal to the worst score for the humans.
And this at a time rate that was "unfair" for the humans.
Still, two of the computers were able to post performance rating
higher than some of the humans. And these were *grandmaster* humans.
Ed Seedhouse
President, Victoria Chess Club.
CFC Rating: 2040
jlu...@hoflink.com
On 29 Dec 1995, GivenRandy wrote:
> What does this say about Windows 95?!
>
Very little. The only program that was run under Windows 95 was
Chessmaster 4000 for Windows 95. The others ran under DOS. And
Chessmaster 4000 for Windows 95 had a decent result.
[deletia]
> Randy Given <Given...@aol.com>
.
Jason Luchan
Kenneth Sloan
Luis A. Dissett <ldis...@qew.cs.toronto.edu> wrote:
>...
>
>Wait, wait, wait! Wait a minute! In order to compute performance ratings for
>player X, some (or all) of its opponents must be rated.
Not necessarily. In theory, you need only one rated player in the event
in order to come up with a reasonable PR for every player in the event.
>... I can understand
>then how we can compute the PR of Junior, WChess and all the silicon gang in
>this tournament. What I can't understand is how do we compute the human's
>PR: as far as I know, none of these programs has received a FIDE (USCF,
>whatever) rating.
Several of the programs have USCF ratings. Check under "COMPX,...".
Oh...what the...here are all of the COMPX rated programs that I know of:
[format is: NAME;USCF ID;expiration date;STATE;OTB rating;Quick rating]
Compx,86;12557303;19920430;AL;2050*A5;
COMPX,B P;12490909;19960531;IL;2313*m9;
Compx,Blzx;12561117;19910831;CA;1624/16;
COMPX,EVALUATOR;12563195;19950630;MA;1805*B5;
COMPX,FIDELITY MACH 3;12626398;19940831;MD;2100*X0;2136/15
COMPX,KASPAROV'S GAMBIT X;12643909;19960331;CA;2531/07;
COMPX,M CHESS;12581892;19931031;CA;2331/04;
COMPX,NOVAG DIAMOND;12629182;19951031;AL;1881/05;
COMPX,NOVAG H;12580988;19950228;AL;2090/17;
COMPX,NOVAG M;12580989;19931031;AL;1970/10;
COMPX,R25;12595184;19940430;AL;2253/07;
COMPX,SH8;12591181;19940430;AL;1914/04;
Compx,Slig;12559002;19920630;NY;1456/09;
COMPX,SPECTOR;12516851;19960531;MA;1848*B9;
Compx,Testnx;12561572;19920930;AL;1948*a2;
COMPX,TURBO X;12463699;19900731;AL;2176*x0;
COMPX,WCHESS;12625473;19950731;AL;2093/08;
COMPX,ZK25;12556160;19950331;CO;2275*X7;
> In light of this, I would not take the PRs for humans too
>seriously
PR's should *never* be "taken seriously!
... in particular, I wouldn't dare to make a comparison with the
>computers' PRs.
I would.
--
Kenneth Sloan sl...@cis.uab.edu
Computer and Information Sciences (205) 934-2213
University of Alabama at Birmingham FAX (205) 934-5473
Birmingham, AL 35294-1170 http://www.cis.uab.edu/info/faculty/sloan/
Luis A. Dissett
Wait, wait, wait! Wait a minute! In order to compute performance ratings for
player X, some (or all) of its opponents must be rated. I can understand
then how we can compute the PR of Junior, WChess and all the silicon gang in
this tournament. What I can't understand is how do we compute the human's
PR: as far as I know, none of these programs has received a FIDE (USCF,
whatever) rating. In light of this, I would not take the PRs for humans too
seriously ... in particular, I wouldn't dare to make a comparison with the
computers' PRs.
> Ed Seedhouse
> President, Victoria Chess Club.
> CFC Rating: 2040
Luis Dissett
Jean-Christophe Weill
>
>> What does this say about Windows 95?!
>>
>
>Very little. The only program that was run under Windows 95 was
>Chessmaster 4000 for Windows 95. The others ran under DOS. And
>
>
Virtual Chess was definitively running the WIndows 95 version....
Even, if it exists a DOS and a windows 3.1 version.
But, it is true that it says nothing about Windows 95. Plus, we
can try to know what does this say about Pentium Pro ? etc...
Jean-Christophe Weill
Jean-Christophe Weill
>>Ilya Gurevich: 3.5 / 2508
>
>>Virtual Chess: 3.5 / 2644
..
>>Socrates 95: 1.0 / 2268
>>Junior: 1.0 / 2254
>
>a program was equal to the worst score for the humans.
Yup, this is definitively not a signifiant remark since it was 2 distincts
group and that humans did not play humans and computers did not play computers.
To understand where is the error, let's suppose that we remove some of the
humans and replace them by beginner (or someone like me, the computers
would have done 100% on those) and perhaps Virtual Chess and
ChessMaster would be upper.
What we can remark is that :
- some of the computers did not very good (Socrates and Junior),
but this is not really a great surprise (at least in my eyes).
- some did a complete underperformance. Wchess....
- some did not convince us that special openings books prepared
against computers which did well in computers championship and
Swedish list help against human oposition...
- some did better. (But was lucky of course !)
>And this at a time rate that was "unfair" for the humans.
30' is enough for good and prepared humans...
>
>Still, two of the computers were able to post performance rating
>higher than some of the humans. And these were *grandmaster* humans.
>
yes, and it is all ready not so bad... I'd like to have 3.5 out of 6
against those grandmasters. BTW, Virtual Chess had the 3rd place
at the Megeve PCA tournament (also 30' games) this year, and realized
there a performance of 2605 (8 points out of 11). This result is compatible
with the Harvard Cup result...
Jean-Christophe Weill <j...@cnam.fr>
Christopher F. Chabris
>On 29 Dec 1995, GivenRandy wrote:
>
>> What does this say about Windows 95?!
>
>Very little. The only program that was run under Windows 95 was
>Chessmaster 4000 for Windows 95. The others ran under DOS. And
>Chessmaster 4000 for Windows 95 had a decent result.
Just a small correction. In the Sixth Harvard Cup, Chessmaster 4000 for
Windows 95, Virtual Chess, and M-Chess Pro ran under Windows 95. WChess,
Socrates 95, and Junior ran under DOS. All of this was either as required
by the programs or as requested by the people who entered them.
glen...@delphi.com
1) In terms of "chess-mips" , what was the effect of the P-6 chip ??
How much stronger were the programs on the 150 MHz Pentium Pro
as opposed to a regular 133MHz P-5 chip ??
2) Was this years version of W-Chess the same as last years ??
That's a pretty steep drop off , from scoring 5/6 last year
on a 90 MHz P-5 to 1.5/6 this year on a 150 MHz P-6 ...
Is this the effect of a small sampling ?? .... or ....
3) How much practice have the humans had against the programs ??
I could easily imagine that any of these GM's with a few weeks
work would readily increase their performance margin....
4) ChessMaster 4000 / WIN95 ...... ??
a) What happened to CM5000 ??
b) Is the WIN95 version the same engine as the original ??
c) What was the effect of the P-6 chip on WIN 95 ??
All the computer rags claim WIN 95 on a 150MHz P-6 would
be no faster than a 133MHz P-5...
Is this true for chess programs ??
.................................... Thanks......... Tom Glenn
Halibut
: A few questions regarding the Harvard Cup ???
: 1) In terms of "chess-mips" , what was the effect of the P-6 chip ??
: How much stronger were the programs on the 150 MHz Pentium Pro
: as opposed to a regular 133MHz P-5 chip ??
Originally, they said 150Mhz Pentium Pro. Then, they "corrected" it
to 133Mhz Pentium Pro. When I asked whether they were even making 133Mhz
Pentium Pros, I received no response. Perhaps you will.
: 2) Was this years version of W-Chess the same as last years ??
: That's a pretty steep drop off , from scoring 5/6 last year
: on a 90 MHz P-5 to 1.5/6 this year on a 150 MHz P-6 ...
: Is this the effect of a small sampling ?? .... or ....
The 150Mhz Pentium Pro runs 16 bit applications at around the
same speed as the 100Mhz Pentium. So...
: 3) How much practice have the humans had against the programs ??
: I could easily imagine that any of these GM's with a few weeks
: work would readily increase their performance margin....
I don't think that the Grandmasters are being paid to play
BETTER, not with all the ChessMaster 4000 promos and sponsorships around
this Harvard Cup. I imagine that Intel would like nothing better than for
the GM's to show up very very tired.
: All the computer rags claim WIN 95 on a 150MHz P-6 would
: be no faster than a 133MHz P-5...
: Is this true for chess programs ??
Closer to a 100Mhz Pentium, perhaps a bit faster. For chess
programs not optimized for the Pentium Pro, why should they be any
different?
Trimm Industries
>[format is: NAME;USCF ID;expiration date;STATE;OTB rating;Quick rating]
>COMPX,NOVAG H;12580988;19950228;AL;2090/17;
>COMPX,NOVAG M;12580989;19931031;AL;1970/10;
I broke open my $10 hand-held chess computer I bought at K-Mart to
see what kind of chip it used, and it said:
NOVAG 873A <== the software name and version code, I guess
47C220AF - 4174 <== the microprocessor (I don't recognize the p/n
and a look through a directory yields nothing)
Japan 9250EBI <== when and where the part was mask programmed.
--
Gary M. Watson
Electronic Engineer Internet: tr...@netcom.com
Sigma-Trimm Technologies, A Division of Robroy Industries
350 Pilot Road
Las Vegas, NV 89119 Phone: (800) 423-2024 x2115
** Enclosures for SCSI, RAID, Fibre Channel, and SSA **
Halibut
: Just a small correction. In the Sixth Harvard Cup, Chessmaster 4000 for
: Windows 95, Virtual Chess, and M-Chess Pro ran under Windows 95. WChess,
: Socrates 95, and Junior ran under DOS. All of this was either as required
^^^^^^^^^^^^
: by the programs or as requested by the people who entered them.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mchess Pro 5 ran under Windows 95?????????
Mchess Pro 5 is a DOS program. Its maximum strength occurs when
it is booted under clean DOS. I cannot imagine why anyone would prefer to
have it run under the DOS GUI called Win95, any more than you would
"prefer" to have your chess program run at the same time as you are
running a word processing program.
And what does "as requested by the people who entered them" mean?
If a program runs best under plain DOS, shouldn't that be the way it is
run? This sounds very much like yet another promotional angle.
Don Getkey
(Christopher F. Chabris) writes:
>>On 29 Dec 1995, GivenRandy wrote:
>>
>>> What does this say about Windows 95?!
>>
>>Very little. The only program that was run under Windows 95 was
>>Chessmaster 4000 for Windows 95. The others ran under DOS. And
>>Chessmaster 4000 for Windows 95 had a decent result.
>
>Just a small correction. In the Sixth Harvard Cup, Chessmaster 4000 for
>Windows 95, Virtual Chess, and M-Chess Pro ran under Windows 95. WChess,
>Socrates 95, and Junior ran under DOS. All of this was either as required
>by the programs or as requested by the people who entered them.
>
>
>
Why would whoever entered MCP5 run under normal Win95 mode? It plainly
states in the MCP5 manual that "mode X provides the best possible
performance on 80386-based systems and above (including Pentium systems)
that have extended memory." Mode X is designed as a DOS optimized
operating mode. My experience is that the difference is minimal but why
not optimize MCP5 to play its very best? Maybe thats why Marty Hirsch and
others don't endorse this tournament to any great degree?
in chess
Don
Ramsey MN USA
Christopher F. Chabris
> Can anyone confirm that they did actually use the Pentium
>"Pro" 133 and not the P5-133? I was under the impression that, at
>least for something like CM4000 for Win95, a P-5 133 would perform
>much better...
Yes, I can confirm it. The systems used were Pentium Pro 133 MHz systems
with 32 megabytes RAM, as supplied to us by Intel.
Evan
"Pro" 133 and not the P5-133? I was under the impression that, at
least for something like CM4000 for Win95, a P-5 133 would perform
much better...
--Evan
Halibut
: >"Pro" 133 and not the P5-133? I was under the impression that, at
: >least for something like CM4000 for Win95, a P-5 133 would perform
: >much better...
: Yes, I can confirm it. The systems used were Pentium Pro 133 MHz systems
Why didn't Intel supply you with Pentium Pro 200Mhz machines,
which are now being advertised in the markeplace? Or, at the very least,
the original Pentium Pro 150mhz machines you promoted?
Christopher F. Chabris
>Why would whoever entered MCP5 run under normal Win95 mode? It plainly
>states in the MCP5 manual that "mode X provides the best possible
>performance on 80386-based systems and above (including Pentium systems)
>that have extended memory." Mode X is designed as a DOS optimized
>operating mode. My experience is that the difference is minimal but why
>not optimize MCP5 to play its very best? Maybe thats why Marty Hirsch and
>others don't endorse this tournament to any great degree?
I'm sorry, my earlier article was in error. I have checked with Marty
Hirsch (who entered M-Chess Pro in the Harvard Cup) and it was indeed run
under DOS, not Windows 95.
I don't understand the source of the "don't endorse this tournament to
any great degree" comment, however. Marty Hirsch entered the tournament,
as did several other developers. We don't ask for endorsements, just
entries. And to my knowledge Marty was satisfied with the event.
Halibut
: I don't understand the source of the "don't endorse this tournament to
: any great degree" comment, however. Marty Hirsch entered the tournament,
: as did several other developers. We don't ask for endorsements, just
: entries. And to my knowledge Marty was satisfied with the event.
To what extent is ChessMaster going to use your tournament in
their future advertising?
Halibut
: Can anyone confirm that they did actually use the Pentium
: "Pro" 133 and not the P5-133? I was under the impression that, at
: least for something like CM4000 for Win95, a P-5 133 would perform
: much better...
Correct.
BYTE magazine, and other magazines, have reported that the
Pentium (P5) 133 runs software that has not been optimized for the
Pentium Pro (which applies to both Win95 and to most chess programs)
faster than the Pentium Pro does. In fact, so much faster, that a Pentium
133 was significantly faster than a Pentium Pro 150 in running such
software.
Which makes Intel's decision to supply Pentium Pro 133 machines
to the Harvard Cup even more curious. They certainly have their own chips
and motherboards available. They could have easily supplied Pentium Pro
200mhz machines. Heck, they probably could have even sent the successor
chip to the Pentium Pro in 300Mhz format to play.
But sending a Pentium Pro 133mhz machine means that it will run
Win95 and its programs *much* slower than the regular Pentium 133 that
has been out for a long time now.
Christopher F. Chabris
>In article <4c9g08$s...@decaxp.harvard.edu>, c...@wjh.harvard.edu
>(Christopher F. Chabris) writes:
>
>>I don't understand the source of the "don't endorse this tournament to
>>any great degree" comment, however. Marty Hirsch entered the tournament,
>>as did several other developers. We don't ask for endorsements, just
>>entries. And to my knowledge Marty was satisfied with the event.
>
>I said this because there was no specific mention of the names behind the
>controls. It seems to me that it wouldn't hurt to let everyone know that
>it was the programs designer who was at the controls or present at the
>tournament. Nothing that I have read plainly indicated anything other
>than anonymous people running the programs.
I take your point that we ought to give more recognition to the people
who operate the computers at the Harvard Cup. This year Larry Kaufman
operated his program Socrates 95, Aviv Bushinsky operated his brother
Shay's program Junior, Roger LaFlair operated M-Chess Pro (for the third
consecutive year), Mark Glickman operated Chessmaster 4000 for Windows 95
(for the second year, with Joel Salman filling in for the Harvard Cup
Preview match), and Tim Mirabile operated WChess. Virtual Chess was
operated by Roger, Mark, or Tim depending on who was available for duty
each round. The latter four gentlemen are master-strength or higher
players with plenty of experience with computers and chess software --
the credentials we insist upon for operators.
I should also point out that whether or not a program developer appears
in person at an event like the Harvard Cup is probably no indication of
whether or not he or she "endorses" the event, since issues of
scheduling and travel expenses will be important. Some developers will
not participate unless they operate their own program, some merely prefer to
operate their own program, some will send a close associate, some do not
care who operates their program (as long as the job is done properly),
and some actually prefer NOT to be the operator and ask us to supply one
even if they could be present. And I'm sure they all have pretty good
reasons. Our policy has always been to provide a competent operator if
requested by the developer.
Don Getkey
>I don't understand the source of the "don't endorse this tournament to
>any great degree" comment, however. Marty Hirsch entered the tournament,
>as did several other developers. We don't ask for endorsements, just
>entries. And to my knowledge Marty was satisfied with the event.
>
>
>
I said this because there was no specific mention of the names behind the
controls. It seems to me that it wouldn't hurt to let everyone know that
it was the programs designer who was at the controls or present at the
tournament. Nothing that I have read plainly indicated anything other
than anonymous people running the programs.
in chess
Don
Don Getkey
(Christopher F. Chabris) writes:
Thanks for the explanation and insites Chris.
yours in chess
Don
Ramsey MN USA