The MCHESS5 computer killer book...
Ed Schröder
The MCHESS5 computer killer book...
A few weeks ago Marty Hirsch (author of Mchess5) wrote:
"Opening preparation against commercial opponents matters somewhat, but
not as much as one might expect, because an SSDF rating is based on
hundreds of games against at least twenty opponents."
I replied that at matters AT LEAST 100 ELO points on SSDF.
I have posted my reply also to Marty's email address so he couldn't miss
my comments. Till now I have not received any reply from Marty. Not here
and not in RGCC.
In the between time I have started several matches between Mchess5 and a
few chess programs to find out the REAL impact of killer books.
I find the results very shocking!
Here are the results against:
Mchess5 - Genius3 (currently no. 1 on SSDF ELO 2420) 7.5 - 2.5
Mchess5 - Rebel6 (currently no. 3 on SSDF ELO 2415) 13.0 - 1.0
Mchess5 - Hiarcs3 (currently no. 9 on SSDF ELO 2380) 19.0 - 0.0
According to the HIGH ratings of Genius3, Rebel6 and Hiarcs3 these
results are IMPOSSIBLE in normal play (without book traps)
I think you all now can see the impact of killer lines and maybe you
understand my feelings better and my aversion against cooked books.
To make it more clear I have made a statistic where you can see the
move where Mchess5 left the book, but take special care about the first
score. Mchess5 comes most often (if not all the time) with a TOTAL WON
position out of book! ^^^^^ ^^^
At Aegon 1994 Sandro Necchi (the Mchess5 book editor) very openly stated:
"With Mchess5 we will book out all programs and we will be the new
no.1 on SSDF".
Sandro also explained to Jeroen Noomen (the Rebel book editor) how he
prepares the book cooking on concurrent computer opponents like Genius,
Hiarcs and Rebel the main competitors for a first place on SSDF.
"A specific opening line is chosen, Sandro is watching the play of
Genius3, Rebel etc. When Genius or Rebel makes a mistake Sandro makes
advantage of that and put this WON line in the Mchess5 book".
One of the MANY examples of Sandro's work is the following game
between Mchess5 and Genius3:
1.e4 e5 2.Bc4 Nf6 3.d4 exd4 4.Nf3 Nxe4 5.Qxd4 Nf6 6.Bg5
Be7 7.Nc3 c6 8.O-O-O d5 9.Qh4 Be6 10.Rhe1 h6 11. Bd3 O-O
12.Bxh6 Ne4 13.Qh5 g6 14. Qe5 Bf6 15.Qf4 Nxc3 16.Rxe6 fxe6
17.Qg4 g5 18.Nxg5 Kh8 19.Qh5 Nxa2+ 20.Kb1 Nc3+ 21.bxc3 Qb6+
22.Kc1 Qb2+ 23. Kxb2 Bxc3+ 24.Kb3 Nd7 25.Bxf8+ Kg8 26.Qf7+
Kh8 27.Qh7# 1-0
The error is of course 11..0-0
After that move black is lost.
It's obvious that 11..0-0 is no theory at all.
Still the Mchess5 book continues till move 19 and Mchess even announces
a mate!! Sandro found a weak point in Genius and added the 11.. 0-0 ??
trap to the Mchess book.
The trap also works on other chess programs.
Here are the statistics:
Please note that the games are played in the same way SSDF does so you
will see many duplicates.
--------------------------------------------------------------------------
--
Match Mchess5 - Genius3
Level 40 in 2:00
Machine 2 x P90 (identical)
Move = The move number where Mchess5 left the book.
Score = The score of the first Mchess5 move after leaving the book.
Game Move Score Result
---- ---- ----- ------
1 25 + 3.21 1-0
2 25 + 3.21 1-0
3 27 + 0.55 draw
4 46! + 3.26 draw
5 24 +11.53! 1-0
6 18 +11.04! 1-0
7 18 + 0.14 draw
8 18 +11.04 1-0
9 23 + 0.66 0-1
10 19 + Mat9!!! 1-0
Mchess5 - Genius3 7.5 - 2.5
-------------------------------------------------------------
Match Mchess5 - Hiarcs3
Level 40 in 2:00
Machine 2 x P90 (identical)
Move = The move number where Mchess5 left the book.
Score = The score of the first Mchess5 move after leaving the book.
Game Move Score Result
---- ---- ----- ------
1 28 +14.32 1-0
2 25 + 8.53 1-0
3 24 + 8.80 1-0
4 18 + 1.01 1-0
5 24 + 8.53 1-0
6 25 + 2.20 1-0
7 29 + 2.20 1-0
8 29 + 7.33 1-0
9 35 + 7.33 1-0
10 35 + 7.33 1-0
11 25 + 8.53 1-0
12 29 + 2.20 1-0
13 35 + 7.33 1-0
14 35 + 7.33 1-0
15 35 + 7.33 1-0
16 35 + 7.33 1-0
17 25 + 8.53 1-0
18 18 + 1.01 1-0
19 25 + 8.53 1-0
Mchess5 - Hiarcs3 19 - 0 I find this unacceptable.
-------------------------------------------------------------------
Match Mchess5 - Rebel6
Level 40 in 2:00
Machine 2 x P90 (identical)
Move = The move number where Mchess5 left the book.
Score = The score of the first Mchess5 move after leaving the book.
Game Move Score Result
---- ---- ----- ------
1 16 + 0.33 1-0
2 16 + 1.29 1-0
3 19 + Mat8!!! 1-0
4 16 + 7.45!! 1-0
5 28 + 1.49 1-0
6 19 + Mat8!!! 1-0
7 17 + 0.90 1-0
8 19 + Mat8!!! 1-0
9 13 + 0.00 draw
10 19 + Mat8!!! 1-0
11 16 + 1.29 1-0
12 19 + Mat8!!! 1-0
13 17 + 0.88 1-0
14 28 + 1.39 draw
Mchess5 - Rebel6 13 - 1 Also unacceptable
--------------------------------------------------------------------------
I have posted all the games in PGN format on a new subject here in RGCC
in case anybody wants to check them. Just replay the games with Mchess5
and watch the cooked book lines.
Coming to the GOAL of this posting:
- Is this the future of computer chess?
- Spending months of our time on cooked books to get a good rating on
SSDF?
- Should the programmers of Genius, Hiarcs and Rebel do the same?
I obvious prefer to spend my time on improving the chess engine of Rebel
rather than spending months of my time looking for weak points in other
chess programs and add total won lines to the Rebel opening book!
Personally I find this behavior disgusting since it hides the truth of the
real playing strength of a chess program.
But I really wonder if I have any choice left!
What to do?
Comments are *VERY* welcome because I want to know what you all think
about this subject.
I mean if nobody really cares why should I care any longer?
Just confused and worried.
- Ed Schroder -
chessman
Ed Schröder <rebc...@xs4all.nl> wrote in article
<53ting$n...@news.xs4all.nl>...
Personally, I believe this is a problem ...not only for consumers, but
programmers and the SSDF. I like to plays computers vs computers ..but
with killer books...that's not my idea of fun..bascially what you are doing
is just documenting the fact that the killer book author has found a
weakness in another program's opening book...and the SSDF will count the
duplicate gamnes over and over....I think the SSDF needs to get away from
this autoplaying....start picking positions at random from GM games ..and
have the programs play each other from both sides against each other with
books turned off fromthis random start ...otherwise the SSDF is just
wasting it's time contriving ratings that have absolutely no
credibility...unless they are interested in telling the world which program
has the best *cooked* book...*cooking* books will not take computer chess
to the next level...in fact, the end result will be wasted resources
...once an author know one of his lines is cooked..he will simple modify
the line and we'll be back to ground zero in improving chess
programs....it's really up to the SSDF to take the initiative to discourage
the book *cooking* ..and they can by changing the method on how they will
play the programs...but I doubt if you will see any changes....because my
perception is that they are not truly independent from all the programmers
....if they were independent they would recognize that this is a problem
and do something about it...we shall see...I hope my perception is wrong
Chris Whittington
"Ed Schröder" <rebc...@xs4all.nl> wrote:
>
> The MCHESS5 computer killer book...
>
> A few weeks ago Marty Hirsch (author of Mchess5) wrote:
>
> "Opening preparation against commercial opponents matters somewhat, but
> not as much as one might expect, because an SSDF rating is based on
> hundreds of games against at least twenty opponents."
>
> I replied that at matters AT LEAST 100 ELO points on SSDF.
>
> I have posted my reply also to Marty's email address so he couldn't miss
> my comments. Till now I have not received any reply from Marty. Not here
> and not in RGCC.
>
> In the between time I have started several matches between Mchess5 and a
> few chess programs to find out the REAL impact of killer books.
>
> I find the results very shocking!
>
> Here are the results against:
> Mchess5 - Genius3 (currently no. 1 on SSDF ELO 2420) 7.5 - 2.5
> Mchess5 - Rebel6 (currently no. 3 on SSDF ELO 2415) 13.0 - 1.0
> Mchess5 - Hiarcs3 (currently no. 9 on SSDF ELO 2380) 19.0 - 0.0
>
> According to the HIGH ratings of Genius3, Rebel6 and Hiarcs3 these
> results are IMPOSSIBLE in normal play (without book traps)
>
The answer is simple.
Apply a learning function to your program so that it avoids
opening lines that it loses.
Weight the function so that avoidance is stronger the more
recently the game was played.
So the 'cooked' program will win some games where your book is
busted, but your learning function will 'unbust' your book; and
you'll be back to playing random games again.
IMHO this feature is a chess strength negative for Mchess.
For it to work, Mchess has to have a very narrow and specific book.
Anrrow and specific books are very easy to counter attack - the
counter does not require a large amount of human intervention.
Secondly the 'cooked' lines are often bad. Its just that they
place a *computer* in difficulties. Think about it, if they
were good, they'ld be common human-human book lines, except
they aren't.
I've had great fun autoplaying Mchess with CST (both programs
learning function on). Mchess plays these obscure lines (Urusov
gambit is a favourite, or Volga Gambit). CST loses, self-mods
its book, finds a way out of the line that Mchess continually
throws at it, and, then, and here's the joke, CST wins 5 or
6 games in succession, until Mchess finds a way round or, more
usually, gives up on that particular opening.
Simple learning function is the answer. Its an arms race, get
racing.
Chris Whittington
Ian Harris
In article: <53ting$n...@news.xs4all.nl> "Ed Schr?der" <rebc...@xs4all.nl>
writes:
> <snip>
>
> Personally I find this behavior disgusting since it hides the truth of the
> real playing strength of a chess program.
>
> But I really wonder if I have any choice left!
>
> What to do?
>
> Comments are *VERY* welcome because I want to know what you all think
> about this subject.
>
> I mean if nobody really cares why should I care any longer?
>
> Just confused and worried.
>
> - Ed Schroder -
>
I, for one, am in full agreement with your view, Ed.
The problem is in getting ALL progammers to agree amongst themselves that
"killer moves" should not be placed into the book - if this cannot be done
then I'm afraid you are left with having to join in.
--
---------------------------------------------------------------------------
Ian Harris EMail i...@iharris.demon.co.uk or CompuServe 70374,3166
PGP 2.6.3i public key available on request
---------------------------------------------------------------------------
Peter Mckone
Ed Schröder wrote:
>
> The MCHESS5 computer killer book...
> In the between time I have started several matches between Mchess5 and a
> few chess programs to find out the REAL impact of killer books.
>
> I find the results very shocking!
>
> Here are the results against:
> Mchess5 - Genius3 (currently no. 1 on SSDF ELO 2420) 7.5 - 2.5
> Mchess5 - Rebel6 (currently no. 3 on SSDF ELO 2415) 13.0 - 1.0
> Mchess5 - Hiarcs3 (currently no. 9 on SSDF ELO 2380) 19.0 - 0.0
>
> According to the HIGH ratings of Genius3, Rebel6 and Hiarcs3 these
> results are IMPOSSIBLE in normal play (without book traps)
>
> I think you all now can see the impact of killer lines and maybe you
> understand my feelings better and my aversion against cooked books.
> What to do?
>
> Comments are *VERY* welcome because I want to know what you all think
> about this subject.
>
> I mean if nobody really cares why should I care any longer?
>
> Just confused and worried.
>
> - Ed Schroder -
One of the things I like about Rebel Decade is that it never plays the
same opening twice, and that it plays different variations when it
does repeat an opening. Could this be the way to defeat the
killer books? When GMs play a match they also try to find flaws
in their opponent's openings, and when they suspect that they are
approaching a prepared line, they play something that they haven't
played previously. Computers don't neet to always play the move
which they evaluate highest. Randomly playing second or third best,
when the scores are close, would do a lot to dodge the killer books
and to make the program more fun to play against. Altering the
opening book a little for each tournamant also seems prudent.
Putting some bad moves in your commercial book for Mchess to find
and then removing them for tournaments might be amusing.
Finding flaws in your opponents' openings, and dodging prepared traps
is an interesting part of chess, don't you think?
... Peter McKone
Robert Hyatt
Chris Whittington (chr...@cpsoft.demon.co.uk) wrote:
: "Ed Schröder" <rebc...@xs4all.nl> wrote:
: >
: > The MCHESS5 computer killer book...
: >
: > A few weeks ago Marty Hirsch (author of Mchess5) wrote:
: >
: > "Opening preparation against commercial opponents matters somewhat, but
: > not as much as one might expect, because an SSDF rating is based on
: > hundreds of games against at least twenty opponents."
: >
: > I replied that at matters AT LEAST 100 ELO points on SSDF.
: >
: > I have posted my reply also to Marty's email address so he couldn't miss
: > my comments. Till now I have not received any reply from Marty. Not here
: > and not in RGCC.
: >
: > In the between time I have started several matches between Mchess5 and a
: > few chess programs to find out the REAL impact of killer books.
: >
: > I find the results very shocking!
: >
: > Here are the results against:
: > Mchess5 - Genius3 (currently no. 1 on SSDF ELO 2420) 7.5 - 2.5
: > Mchess5 - Rebel6 (currently no. 3 on SSDF ELO 2415) 13.0 - 1.0
: > Mchess5 - Hiarcs3 (currently no. 9 on SSDF ELO 2380) 19.0 - 0.0
: >
: > According to the HIGH ratings of Genius3, Rebel6 and Hiarcs3 these
: > results are IMPOSSIBLE in normal play (without book traps)
:
: The answer is simple.
: Apply a learning function to your program so that it avoids
: opening lines that it loses.
:
: Weight the function so that avoidance is stronger the more
: recently the game was played.
:
: So the 'cooked' program will win some games where your book is
: busted, but your learning function will 'unbust' your book; and
: you'll be back to playing random games again.
:
: IMHO this feature is a chess strength negative for Mchess.
: For it to work, Mchess has to have a very narrow and specific book.
: Anrrow and specific books are very easy to counter attack - the
: counter does not require a large amount of human intervention.
Not necessarily. Marty could just hook up Rebel and Mchess (or
any other program for that matter) and let his learning function
simply "learn" how to beat the other program by culling book lines
that result in losses and keeping lines that win. It's not elegant,
but it is simple because it does not take a lot of human time to
find these oddball lines...
:
: Secondly the 'cooked' lines are often bad. Its just that they
: place a *computer* in difficulties. Think about it, if they
: were good, they'ld be common human-human book lines, except
: they aren't.
A possible "learning disability" too. This is an issue I'm looking
at in Crafty now, but learning to beat weak players might convince
you that 1. e4 2. Bc4 and 3. Qf3 are good moves if if results in
lots of wins. The strength of the opponent has to be factored in,
where wins over opponents that are too weak simply don't affect the
learning at all.
More ideas for discussion as I get further into trying to implement
something here. As always I'll explain what I'm trying and see what
kind of discussion ensues...
:
: I've had great fun autoplaying Mchess with CST (both programs
: learning function on). Mchess plays these obscure lines (Urusov
: gambit is a favourite, or Volga Gambit). CST loses, self-mods
: its book, finds a way out of the line that Mchess continually
: throws at it, and, then, and here's the joke, CST wins 5 or
: 6 games in succession, until Mchess finds a way round or, more
: usually, gives up on that particular opening.
I see the same thing as Crafty plays WchessX, except that crafty
doesn't learn as of yet. However, WchessX keeps going back to
the same well over and over and the only thing that keeps crafty
in the match is that I give it some random freedom with a really
huge book so that it doesn't repeat too often...
:
: Simple learning function is the answer. Its an arms race, get
: racing.
:
: Chris Whittington
:
Robert Hyatt
Peter Mckone (pet...@loc3.tandem.com) wrote:
: Ed Schröder wrote:
: >
: > The MCHESS5 computer killer book...
:
: --- text deleted ---
:
: > In the between time I have started several matches between Mchess5 and a
: > few chess programs to find out the REAL impact of killer books.
: >
: > I find the results very shocking!
: >
: > Here are the results against:
: > Mchess5 - Genius3 (currently no. 1 on SSDF ELO 2420) 7.5 - 2.5
: > Mchess5 - Rebel6 (currently no. 3 on SSDF ELO 2415) 13.0 - 1.0
: > Mchess5 - Hiarcs3 (currently no. 9 on SSDF ELO 2380) 19.0 - 0.0
: >
: > According to the HIGH ratings of Genius3, Rebel6 and Hiarcs3 these
: > results are IMPOSSIBLE in normal play (without book traps)
: >
: > I think you all now can see the impact of killer lines and maybe you
: > understand my feelings better and my aversion against cooked books.
: --- Games deleted ---
:
: > What to do?
: >
: > Comments are *VERY* welcome because I want to know what you all think
: > about this subject.
: >
: > I mean if nobody really cares why should I care any longer?
: >
: > Just confused and worried.
: >
: > - Ed Schroder -
: One of the things I like about Rebel Decade is that it never plays the
: same opening twice, and that it plays different variations when it
: does repeat an opening. Could this be the way to defeat the
: killer books? When GMs play a match they also try to find flaws
: in their opponent's openings, and when they suspect that they are
: approaching a prepared line, they play something that they haven't
: played previously. Computers don't neet to always play the move
: which they evaluate highest. Randomly playing second or third best,
: when the scores are close, would do a lot to dodge the killer books
: and to make the program more fun to play against. Altering the
: opening book a little for each tournamant also seems prudent.
: Putting some bad moves in your commercial book for Mchess to find
: and then removing them for tournaments might be amusing.
: Finding flaws in your opponents' openings, and dodging prepared traps
: is an interesting part of chess, don't you think?
:
: ... Peter McKone
Never is a long time. If you put it on a server, and play 20,000 games a
year, you'll likely see just how often it will repeat, because Crafty's
book is way bigger, and it still repeats openings too often for my liking...
Garby Leon
In <53ting$n...@news.xs4all.nl> "Ed Schröder" <rebc...@xs4all.nl>
writes:
>The MCHESS5 computer killer book...
>A few weeks ago Marty Hirsch (author of Mchess5) wrote:
> "Opening preparation against commercial opponents matters somewhat,
but not as much as one might expect, because an SSDF rating is based on
hundreds of games against at least twenty opponents."
>I replied that at matters AT LEAST 100 ELO points on SSDF.
>I have posted my reply also to Marty's email address so he couldn't
miss my comments. Till now I have not received any reply from Marty.
Not here and not in RGCC.
>In the between time I have started several matches between Mchess5 and
a few chess programs to find out the REAL impact of killer books.
>I find the results very shocking!
(SNIP - games and statistics that amply demonstrate Ed's point)
>Coming to the GOAL of this posting:
>
>- Is this the future of computer chess?
>- Spending months of our time on cooked books to get a good rating on
>SSDF? Should the programmers of Genius, Hiarcs and Rebel do the same?
>I obvious prefer to spend my time on improving the chess engine of
Rebel rather than spending months of my time looking for weak points in
other chess programs and add total won lines to the Rebel opening book!
>Personally I find this behavior disgusting since it hides the truth of
the real playing strength of a chess program.
>But I really wonder if I have any choice left!
>What to do?
>Comments are *VERY* welcome because I want to know what you all think
>about this subject.
>I mean if nobody really cares why should I care any longer?
>Just confused and worried.
>- Ed Schroder -
Dear Ed --
Here's one person's opinion, for what it's worth.
As an ecstatic new owner of Rebel 8.0 -- OF COURSE I'd like to see you
spend 100% of your development time making Rebel 9.0 even stronger as a
chess-playing engine.
That's your strength -- your strongest game, if you will. You
shouldn't waste your time trying to do things which are tangential to
your main arena of artistry and specialization. Others can do that;
you shouldn't have to.
One wouldn't require or expect a concert pianist to tune the piano.
Even if bad tuning had negative effects on her or his "results."
The real-world problem for a commercial developer, if I understand it,
is that probably many people look at the SSDF list (or other tournament
results -- but NOT the WMCCC in Jakarta this year!!) and use those
results to make their purchase decision. I.e., they buy the program
which beats the others.
Now that seems logical and unassailable -- if we were putting together
a chess team for the Olympiad, we'd take the highest-rated players we
could find.
But what your dilemma suggests is that perhaps computer chess is
DIFFERENT from OTB play, with other considerations -- and very
important ones.
In short, if I understand you correctly, you're saying that
***with 'cooked' books, tournament play is not reflective of engine
strength.***
This is a really important point. In fact, for computer chess, it
tends to negate the value of tournaments at all! What a concept.
In fact, using 'cooked books' takes computer chess competition success
COMPLETELY OUTSIDE of the computer arena entirely: it makes tournament
results dependent on a purely human factor -- who has the sharpest IM
or GM to prepare the lines the computer will play. Or worse, which
company is able to pay the most for such services!
So, the resulting competition has nothing to do with computer chess at
all.
It's human chess, with computers pushing the pawns.
(Here, thanks to this newsgroup: this is something I would never have
been aware of without reading the posts of yourself and others in this
newsgroup).
OTHER PURCHASE CRITERIA (a personal aside):
In my own purchase decision (and I own two other commercial programs
besides Rebel 8.0), tournament results were NOT primary. I wanted the
strongest engine. Tournament results weren't a factor. Testing the
STRENGTH of the program WAS a factor.
Why did I look for engine strength? Well, I saw that my Chessica
program (a Fritz 3 version, I've heard -- which was hyped as being
"World Champion" when I bought it) often did stupid things. Beating it
ceased to interest me. It doesn't fear a passed pawn, for example, and
it rarely tries to queen a pawn on its own. In this and other
respects, I began to doubt its underlying chessplaying ability.
In contrast, Rebel 8.0 is strong enough to hammer me off the board with
alarming ease. It's actually kind of scary playing it -- a very
dangerous feeling, watching it move. And, it's exciting and inspiring
to have this kind of horsepower available on my desktop P-133.
Most important, engine strength addresses my primary interest in
computer chess -- the testing and learning of opening theory, and
subjecting one's own ideas to analysis -- which can only be aided by
having the strongest engine possible. That's what I use Rebel 8.0 for,
so that was my chief determinant in choosing which program to buy. For
this use, tournament results are irrelevant.
* * *
But I can see the commercial quandary in the underlying issue you
raise:
From a programmer's perspective: a 'cooked' book CONCEALS THE PLAYING
WEAKNESS of a program -- just that quality which you (and many others)
have spent so long trying to develop... and doing so brilliantly, I
might add.
So what is the answer here?
Does every professional program developer or company need its own IM or
GM -- or, forseeably even a TEAM of GM's like the old Soviet system --
to 'cook' its books for tournaments?
It seems like we're headed exactly in that direction. Especially if
and as more money pours into the field.
This, of course, may be what 'real' chessplayers do, as part of their
training and preparation. In OTB play, it isn't extrinsic to the fair
and open competition of chess -- just the opposite. It's part of the
game. If you don't want to prepare your openings, you won't get very
far. But this analogy to OTB play can be VERY MISLEADING.
Computer chess is fundamentally different: you don't make your living
by winning tournaments, but by selling programs. Winning isn't really
your goal, except as a means to maintain the commercial viability of
your product. Instead, you're a brilliant programmer and developer.
For you, winning a tournament, while gratifying, is secondary to
creating a monster chess engine. That's your gift, talent, strength,
and maybe, destiny.
A MODEST PROPOSAL
-- Maybe we're looking at the development of two distinct types of
computer chess competitions: a "database" competition, where opening
books are unlimited, and an "engine power" competition, where books
would be carefully limited -- or even STANDARDIZED, WITH ALL
COMPETITORS USING THE SAME BOOK -- to provide a level playing field.
Of course, all other *hardware* sports routinely create such mechanical
restrictions, and even "classes", to keep competition fair and
interesting. Formula One racing, America's cup and all sailboat
competition -- in fact all the *hardware* competitions I can think of,
have developed very sophisticated formulas for keeping competition in
carefully-defined boundaries.
Computer chess might well consider doing the same.
Why? All these sports have recognized long ago that only close
competition is interesting. Without such rules, every event becomes a
vigil for a coup d'etat of some kind: you wait to learn which
competitor found the best "cheat" that blows everyone else away.
I think computer chess may be in that very situation now.
If that happens, as your posting of MChess results show, the
competition is no fun at all, and meaningless. Even worse, it could
have a dangerously weakening effect on progress towards the central
issue and problem of computer chess: developing the most powerful
engine possible.
That's the unique arena of computer chess, and the one it should follow
to the exclusion of other tangential concerns.
It's an important goal, and should be protected -- especially by the
participants themselves -- by building safeguards into competitions
that discourage achievement OUTSIDE the computer chess arena that could
come to dominate computer chess as a whole.
The worst result I can see from the present situation is that it could
tend to keep you and others from following your own path and destiny --
that of creating the program which will play the strongest, deepest,
smartest chess that our CPU's are capable of. It would be something of
a tragedy if you and others were deflected from pursuing that
fascinating and exciting goal.
Sorry for the rambling answer -- but I think this may be a very
important issue indeed...
-- garb leon
PS congratulations on Rebel 8 -- I'll be first in line as soon as Rebel
9 is available... gl
Chris Whittington
hy...@crafty.cis.uab.edu (Robert Hyatt) wrote:
>
> Chris Whittington (chr...@cpsoft.demon.co.uk) wrote:
> : "Ed Schröder" <rebc...@xs4all.nl> wrote:
> : >
> : >
> : > A few weeks ago Marty Hirsch (author of Mchess5) wrote:
> : >
> : > "Opening preparation against commercial opponents matters somewhat, but
> : > not as much as one might expect, because an SSDF rating is based on
> : > hundreds of games against at least twenty opponents."
> : >
> : > I replied that at matters AT LEAST 100 ELO points on SSDF.
> : >
> : > I have posted my reply also to Marty's email address so he couldn't miss
> : > my comments. Till now I have not received any reply from Marty. Not here
> : > and not in RGCC.
> : >
> : > In the between time I have started several matches between Mchess5 and a
> : > few chess programs to find out the REAL impact of killer books.
> : >
> : > I find the results very shocking!
> : >
> : > Here are the results against:
> : > Mchess5 - Genius3 (currently no. 1 on SSDF ELO 2420) 7.5 - 2.5
> : > Mchess5 - Rebel6 (currently no. 3 on SSDF ELO 2415) 13.0 - 1.0
> : > Mchess5 - Hiarcs3 (currently no. 9 on SSDF ELO 2380) 19.0 - 0.0
> : >
> : > According to the HIGH ratings of Genius3, Rebel6 and Hiarcs3 these
> : > results are IMPOSSIBLE in normal play (without book traps)
> :
> : The answer is simple.
> :
> : Apply a learning function to your program so that it avoids
> : opening lines that it loses.
> :
> : Weight the function so that avoidance is stronger the more
> : recently the game was played.
> :
> : So the 'cooked' program will win some games where your book is
> : busted, but your learning function will 'unbust' your book; and
> : you'll be back to playing random games again.
> :
> : IMHO this feature is a chess strength negative for Mchess.
> : For it to work, Mchess has to have a very narrow and specific book.
> : Anrrow and specific books are very easy to counter attack - the
> : counter does not require a large amount of human intervention.
>
> Not necessarily. Marty could just hook up Rebel and Mchess (or
> any other program for that matter) and let his learning function
> simply "learn" how to beat the other program by culling book lines
> that result in losses and keeping lines that win. It's not elegant,
> but it is simple because it does not take a lot of human time to
> find these oddball lines...
>
Sorry, I didn't explain myself well enough.
That is sort of what it is being alleged he does.
Or, it is alleged that, by hand, he looks for errors in opponent
programs and then programs in (narrow) book lines to try and seek
out these errors.
Conversely, it can be assumed that lines bad for Mchess are avoided.
This, it is alleged, is done by hand prior to release.
I am arguing that this avoidance/reward process can be automated
by the opponent program *after* release by learning and unlearning
book lines as they are played at SSDF testing.
If Rebel (say) loses a SSDF game, it can avoid losing that line
again by diverging on the next SSDF game or later occasion.
So, Rebel can do what Mchess does on-line. Its not so easy for
Mchess to deal with this behaviour *after* release.
Believe me, CST has played long sequences of games with Mchess
with variance (on both sides, since both learn). Never the same
game gets played. Its quite a battle. Either side can get the
better of it, I've seen strings of different wins in the same
basic opening as the opponent program wriggles more each game,
eventually finding the refutation.
It works, this learning. Nobody can book up for a series
of games played this way.
Chris Whittington
tries
Mark Uniacke
Hi,
In article <53ting$n...@news.xs4all.nl>, =?iso-8859-1?q?Ed_Schr=F6der?=
<rebc...@xs4all.nl> writes
>The MCHESS5 computer killer book...
>
>A few weeks ago Marty Hirsch (author of Mchess5) wrote:
>
> "Opening preparation against commercial opponents matters somewhat, but
> not as much as one might expect, because an SSDF rating is based on
> hundreds of games against at least twenty opponents."
>
>I replied that at matters AT LEAST 100 ELO points on SSDF.
Having seen the results I have to agree with you Ed.
>
>I have posted my reply also to Marty's email address so he couldn't miss
>my comments. Till now I have not received any reply from Marty. Not here
>and not in RGCC.
>
>In the between time I have started several matches between Mchess5 and a
>few chess programs to find out the REAL impact of killer books.
>
>I find the results very shocking!
>
>Here are the results against:
>Mchess5 - Genius3 (currently no. 1 on SSDF ELO 2420) 7.5 - 2.5
>Mchess5 - Rebel6 (currently no. 3 on SSDF ELO 2415) 13.0 - 1.0
>Mchess5 - Hiarcs3 (currently no. 9 on SSDF ELO 2380) 19.0 - 0.0
>
>According to the HIGH ratings of Genius3, Rebel6 and Hiarcs3 these
>results are IMPOSSIBLE in normal play (without book traps)
These results seem to fit in with the SSDF results. For example,
MChess5 P90 - Hiarcs3 P90 16.5 - 3.5
Remember, that contains 10 white and 10 black games.
It is interesting that Hiarcs4 with a larger and significantly varied
book (with absolutely no "cooks") scores:
MChess5 P90 - Hiarcs4 P90 6.5 - 13.5
While Hiarcs4 is stronger than Hiarcs3, it certainly isn't 396 Elo
(267+129) stronger as indicated by the respective match scores!!
It is obvious from my testing too that MChess5 has a heavily "cooked"
book for Genius2/3, Rebel6 and Hiarcs3. Which incidentally were MChess'
main opposition when it was released.
This means there are at least 7 SSDF matches of 20 games each which are
influenced by the killer lines and NOT the relative engines strengths.
There is no doubt in my opinion that killer lines in a cooked book on
this scale will severely affect the SSDF rating of MChess5.
The manor in which these results were achieved is quite shocking.
>
>I have posted all the games in PGN format on a new subject here in RGCC
>in case anybody wants to check them. Just replay the games with Mchess5
>and watch the cooked book lines.
>
>Coming to the GOAL of this posting:
>
>- Is this the future of computer chess?
>- Spending months of our time on cooked books to get a good rating on
>SSDF?
>- Should the programmers of Genius, Hiarcs and Rebel do the same?
>I obvious prefer to spend my time on improving the chess engine of Rebel
>rather than spending months of my time looking for weak points in other
>chess programs and add total won lines to the Rebel opening book!
I have never put killer lines in Hiarcs' opening book for computer
opponents. What limited time I have I prefer to devote to work on the
chess engine.
I belive chess programs should be developed for the users/customers who
are willing to purchase them. It seems some chess programs are being
developed to beat other chess programs as a main priority. Surely this
cannot be right?
>
>Personally I find this behavior disgusting since it hides the truth of the
>real playing strength of a chess program.
>
>But I really wonder if I have any choice left!
>
>What to do?
I think Chris mentioned about learning and this may be the only way
forward for us all. However, it leaves a serious problem with the rating
lists like the SSDF whose accuracy is surely being severely affected,
particularly when new programs released now and in the future get to
play "old" programs like Genius2/3, Hiarcs3 and Rebel6.
I believe such a large number of possible "cooked matches" gives
programs like MChess5 an inflated rating.
>
>Comments are *VERY* welcome because I want to know what you all think
>about this subject.
>
>I mean if nobody really cares why should I care any longer?
>
Ed, you are not alone.
>Just confused and worried.
>
>- Ed Schroder -
>
>
Regards,
Mark
Author of Hiarcs3, Hiarcs4 and soon Hiarcs5!
Roger Davis
Perhaps the solution to cooked books in tournaments is some adaptation of
Fischer Random Chess... :) Maybe the idea is absurd, I haven't read the
rules Fischer was proposing, but his intention was basically to modify the
rules and bypass the "book" so that the best "engine" could emerge
victorious, right?
---------------
Roger Davis
Robert HYATT
I've seen this happen. In several test suites, positions are found
in the large GM database Crafty has. I have modified the "test"
function to disable the book before it runs a test suite to keep
from fudging the results... One position I remember came from an
old Cray Blitz vs Belle game, (ACM 1981 in fact) where the solution
is Bxh6 which leads to a draw. It's in the database. :)
Bob
On Mon, 14 Oct 1996, Pete Nielsen wrote:
--> I think you may have to factor in time pressure as well.
-->
--> I don't know if you're familiar with the Internet Chess Academy, but you
--> get a puzzle followed shortly by a lecture on the solution and how that
--> solution should have been found.
-->
--> Often after finding my solution, I give the position to Crafty to see
--what
--> it thinks. Recently, It gave one answer (Book). It turned out in the
--> lecture that the possition was from a GM vs GM game, and that the reason
--> that the move was good, was because of time pressure.
-->
Komputer Korner
Ed Schr=F6der wrote:
> snipped
> Coming to the GOAL of this posting:
> - Is this the future of computer chess?
> - Spending months of our time on cooked books to get a good rating on
> SSDF?
> - Should the programmers of Genius, Hiarcs and Rebel do the same?
> =
> I obvious prefer to spend my time on improving the chess engine of Rebel
> rather than spending months of my time looking for weak points in other
> chess programs and add total won lines to the Rebel opening book!
> =
> Personally I find this behavior disgusting since it hides the truth of th=
e
> real playing strength of a chess program.
> =
> But I really wonder if I have any choice left!
> =
> What to do?
> =
> Comments are *VERY* welcome because I want to know what you all think
> about this subject.
> =
> I mean if nobody really cares why should I care any longer?
> =
> Just confused and worried.
> =
> - Ed Schroder -
I have already stated that if the computer chess programmers would sell
updated cooked medium rare "killer books on a regular basis, that these
would actually sell. Don't underestimate the consumer's capacity to want
to have the latest killer book for his program. But maybe my ICU is =
fried by now and my circuits are overheating. SSSSSSSSSSSSSSSSSSSSSS
OH OH, My circuits ARE overheating caused by the thought of having to
absorb all those new killer books. =
-- =
Komputer Korner
Chris Whittington
Peter Mckone <pet...@loc3.tandem.com> wrote:
>
> Ed Schröder wrote:
> >
> > The MCHESS5 computer killer book...
>
>
> > In the between time I have started several matches between Mchess5 and a
> > few chess programs to find out the REAL impact of killer books.
> >
> > I find the results very shocking!
> >
> > Here are the results against:
> > Mchess5 - Genius3 (currently no. 1 on SSDF ELO 2420) 7.5 - 2.5
> > Mchess5 - Rebel6 (currently no. 3 on SSDF ELO 2415) 13.0 - 1.0
> > Mchess5 - Hiarcs3 (currently no. 9 on SSDF ELO 2380) 19.0 - 0.0
> >
> > According to the HIGH ratings of Genius3, Rebel6 and Hiarcs3 these
> > results are IMPOSSIBLE in normal play (without book traps)
> >
> > I think you all now can see the impact of killer lines and maybe you
> > understand my feelings better and my aversion against cooked books.
>
>
> > What to do?
> >
> > Comments are *VERY* welcome because I want to know what you all think
> > about this subject.
> >
> > I mean if nobody really cares why should I care any longer?
> >
> > Just confused and worried.
> >
> > - Ed Schroder -
>
> same opening twice, and that it plays different variations when it
> does repeat an opening.
Don't agree. I've played autoplayer sequences against Rebel (in -A)
mode, where the same game gets repeated over and over. I've seen
9 identical sequential draws often.
In normal autoplayer mode, rebel will just throw out a repeat
if its an identical on book exit. For obvious reasons the SSDF can't use
this mode since they can't allow Rebel to just abort games.
The problem is a technical one.
Ed wants to deal with repeats, So he aborts if the game repeats
out of *his* opening book. Fine against a non-learner, but not
fine against a learner. Ed knows about this problem, and its up to
us programmers to deal with it. I think its best dealt with by us
all using learning.
Then its an arms race which will lead back to programs being
unable to rely on killer lines.
Really, if we all use learning, then iuts a non-problem.
Chris Whittington
> Could this be the way to defeat the
> killer books? When GMs play a match they also try to find flaws
> in their opponent's openings, and when they suspect that they are
> approaching a prepared line, they play something that they haven't
> played previously. Computers don't neet to always play the move
> which they evaluate highest. Randomly playing second or third best,
> when the scores are close, would do a lot to dodge the killer books
> and to make the program more fun to play against. Altering the
> opening book a little for each tournamant also seems prudent.
> Putting some bad moves in your commercial book for Mchess to find
> and then removing them for tournaments might be amusing.
> Finding flaws in your opponents' openings, and dodging prepared traps
> is an interesting part of chess, don't you think?
>
> .... Peter McKone
Alexander Fuchs
Ed Schröder <rebc...@xs4all.nl> schrieb im Beitrag
<53ting$n...@news.xs4all.nl>...
> The MCHESS5 computer killer book...
>
> A few weeks ago Marty Hirsch (author of Mchess5) wrote:
>
> "Opening preparation against commercial opponents matters somewhat, but
> not as much as one might expect, because an SSDF rating is based on
> hundreds of games against at least twenty opponents."
>
> I replied that at matters AT LEAST 100 ELO points on SSDF.
>
> I have posted my reply also to Marty's email address so he couldn't miss
> my comments. Till now I have not received any reply from Marty. Not here
> and not in RGCC.
>
> In the between time I have started several matches between Mchess5 and a
> few chess programs to find out the REAL impact of killer books.
>
> I find the results very shocking!
>
> Here are the results against:
> Mchess5 - Genius3 (currently no. 1 on SSDF ELO 2420) 7.5 - 2.5
> Mchess5 - Rebel6 (currently no. 3 on SSDF ELO 2415) 13.0 - 1.0
> Mchess5 - Hiarcs3 (currently no. 9 on SSDF ELO 2380) 19.0 - 0.0
>
> According to the HIGH ratings of Genius3, Rebel6 and Hiarcs3 these
> results are IMPOSSIBLE in normal play (without book traps)
>
> I think you all now can see the impact of killer lines and maybe you
> understand my feelings better and my aversion against cooked books.
[large cut]
Thanks a lot for this nice illustration of the impact of killer books.
I agree 100% with you.
I love testing chess programs, but these killer books really make me lose
interest in computer chess. It has nothing to do with chess.
It is necessary that a) the SSDF changes their testing methods. They should
have a large set of opening positions (maybe positions after 7-10 moves)
with open and closed positions in the right proportion. They should play
each position twice and switch the colours after the first game. Opening
books should be turned off. These positions should be kept secret so that
there will be no return of booking.
b) ICCA should modify their rules for their championships. Ed Schroeder has
clearly expressed his reasons why he did not participate in Jarkata. Maybe
Richard Lang had similar reasons. - Genius is often considered as a
reference and their might be a lot of cooks against the Genius book.
The fact that a top program like Rebel does not compete should really make
the ICCA reflect about their rules.
Again: Cooking books does not make computer chess advance although it may
be very tempting because it means easy SSDF points and good sales.
Alexander Fuchs
Kevin James Begley
Mark Uniacke (ma...@acc-ltd.demon.co.uk) wrote:
: I have never put killer lines in Hiarcs' opening book for computer
: opponents. What limited time I have I prefer to devote to work on the
: chess engine.
: I belive chess programs should be developed for the users/customers who
: are willing to purchase them. It seems some chess programs are being
: developed to beat other chess programs as a main priority. Surely this
: cannot be right?
I happen to think that killer-lines should be encouraged. They only
demonstrate how sensitive computer's are to opening traps. Human's
have long had to contend with preperation, and with databases so
widespread, it's only getting worse. The side-effect of all this is
that to be good at chess, one must be extremely well prepared.
Some of you programmers are starting to sound like you'd like to
switch to Fischer's-Random-Chess to level the playing field. Kind
of funny that you're all upset that you've been dupped by your
own invention. Face facts, if you "turn-off" your opening-books,
your machines are very likely to stumble into some very, very
bad positions. So, live by the sword, die by the sword.
As for the issue of consumer's being fooled, I suggest that they be
educated about some standardized benchmarks (I'll let y'all agree
on which bench-mark should be used).
In competition, anything legal is fair. There's no good reason why
someone should refuse to take advantage of their opponent's weakness
in the opening. Rather than crying about how unfair it is to be
defeated by an inferior opponent, I suggest you start recognising
that a weakness of yours has been revealed (be thankful that this
was an easily identifiable weakness that your opponent preyed upon!).
Of course, we all realize that such a weakness is difficult to repair,
and most will confess that it leads only to an arms-race approach to
opening theory. When the majority of tournament players are willing to
concede that this kind of preperation is no longer enjoyable, I (for one)
will gladly follow them over to Fischer's variation of chess, but I doubt
very much that such a migration will occur in the near future -- it is
against every strong player's interest to do so (save maybe RJF).
Kevin.
Chris Whittington
da...@laraby.tiac.net (James Garner) wrote:
>
> Ed Schröder (rebc...@xs4all.nl) wrote:
>
> : I have posted my reply also to Marty's email address so he couldn't miss
> : my comments. Till now I have not received any reply from Marty. Not here
> : and not in RGCC.
>
> doesn't even respond to posts that ask for information, such as the one I
> posted about the 10MB Hash table limit. Apparently, he, like Steve at
> ICD, figures that the less said, the sooner the thread will drop.
>
> Wrong.
This is completely unreasonable.
Nobody *has* to respond to anything.
There are many programmers whose products get mentioned on rgcc
or who get emailed and who don't reply. That's just their way.
Its the height of arrogance to imagine rgcc is some kind of
god-forum where anyone and everyone has to explain themselves.
Chris Whittington
Ed Schröder
Kevin,
You are missing the point of the discussion!
The issue is that SSDF allows to test "book cooking" chess programs
against older chess programs who CAN NOT defend themselves.
These "book cooking" programs therefore gain a LOT of ELO points on the
SSDF rating list!! and therefore the rating of "book cooking" programs
ARE NOT RELIABLE.
At the moment the only "book cooking" program is Mchess5.
I have explained this in detail in my previous posting including a lot
of examples and complete games.
I am just worried about this new development, we *ALL* want a SSDF list
with the STRONGEST chess program on TOP. No?
As I producer myself I surely hope that this will be my own program, no
doubt about that, but I prefer a reliable no.1 on SSDF and I don't care
if that is one of my concurrents as long as it is reliable!
For years this was the ChessMachine The King;
For years this was Genius3;
Both programs were the strongest at that moment!
I have NO problems with Mchess, Genius, Hiarcs, Crafty, Schredder on TOP
of SSDF as long as the rating and the no.1 position is earned by the
STRONGEST chess engine.
Unfortunately this has been not been the case in the past year and
this is the main reason for our discussion.
Please feel free to correct me if I am wrong.
- Ed Schroder -
From: kjbe...@chimi.engr.ucdavis.edu (Kevin James Begley)
Mark Uniacke (ma...@acc-ltd.demon.co.uk) wrote:
: I have never put killer lines in Hiarcs' opening book for computer
: opponents. What limited time I have I prefer to devote to work on the
: chess engine.
: I belive chess programs should be developed for the users/customers who
: are willing to purchase them. It seems some chess programs are being
: developed to beat other chess programs as a main priority. Surely this
: cannot be right?
>I happen to think that killer-lines should be encouraged. They only
Alastair Scott
On 14 Oct 1996 20:39:44 GMT, ga...@ix.netcom.com(Garby Leon) wrote:
[snip]
>A MODEST PROPOSAL
>
>-- Maybe we're looking at the development of two distinct types of
>computer chess competitions: a "database" competition, where opening
>books are unlimited, and an "engine power" competition, where books
>would be carefully limited -- or even STANDARDIZED, WITH ALL
>COMPETITORS USING THE SAME BOOK -- to provide a level playing field.
[snip]
There are some excellent points here.
The fundamental problem is that, no matter what tests are used, somebody
will find some way of biassing them. I remember (from an old Selective
Search) a test position where a program found the solution instantly yet,
when a completely irrelevant pawn was moved from h2 to h3 and the program
retested, it failed. Ditto standardised book lines - what would stop someone
adding special behaviour to their program if they discovered a weakness in
the way a prospective opponent handled one of those lines?
I suggest a draconian (anti-sleaze?) solution: that computer versus
computer, or computer-only tests, are disallowed for rating purposes (by
SSDF, Eric Hallsworth and so on), and that human versus computer games only
are considered. Remember that _people_ operate computers, write programs and
buy them.
Alastair
Alastair Scott
That would solve all the problems - except that people buy chess programs to
play chess, not to play Fischerandom.
It's a bit like deciding the English Premiership by giving Newcastle United
and Manchester United a bag of golf clubs each and asking them to play
eighteen holes! (Apologies for the football/soccer analogy).
Alastair
Robert Hyatt
Ed Schröder (rebc...@xs4all.nl) wrote:
: From: Mark Uniacke <ma...@acc-ltd.demon.co.uk>
:
:
: :
: : "Opening preparation against commercial opponents matters somewhat, but
: : not as much as one might expect, because an SSDF rating is based on
: : hundreds of games against at least twenty opponents."
: :
: :I replied that at matters AT LEAST 100 ELO points on SSDF.
:
: >Having seen the results I have to agree with you Ed.
: Thank you for backing me up Mark, at least I know that I am not
: alone now.
:
: [ snip ]
:
: >It is obvious from my testing too that MChess5 has a heavily "cooked"
: >book for Genius2/3, Rebel6 and Hiarcs3. Which incidentally were MChess'
: >main opposition when it was released.
: >This means there are at least 7 SSDF matches of 20 games each which are
: >influenced by the killer lines and NOT the relative engines strengths.
: >There is no doubt in my opinion that killer lines in a cooked book on
: >this scale will severely affect the SSDF rating of MChess5.
: Unfortunately I have to agree.
: I have the same results and the same conclusions as Mark.
:
: [ snip ]
:
: :Coming to the GOAL of this posting:
: :
: :- Is this the future of computer chess?
: :- Spending months of our time on cooked books to get a good rating on
: :SSDF?
: :- Should the programmers of Genius, Hiarcs and Rebel do the same?
:
: :I obvious prefer to spend my time on improving the chess engine of Rebel
: :rather than spending months of my time looking for weak points in other
: :chess programs and add total won lines to the Rebel opening book!
:
: >I have never put killer lines in Hiarcs' opening book for computer
: >opponents. What limited time I have I prefer to devote to work on the
: >chess engine.
: >I belive chess programs should be developed for the users/customers who
: >are willing to purchase them. It seems some chess programs are being
: >developed to beat other chess programs as a main priority. Surely this
: >cannot be right?
: Corect, I have never seen any cooked book lines in Hiarcs3/4, neither
: did I found any cooked lines in Genius2/3/4. Just Like Rebel, Hiarcs
: and Genius opening books are written for humans and I am in favor to
: keep it that way!!
:
: :What to do?
:
: >I think Chris mentioned about learning and this may be the only way
: >forward for us all. However, it leaves a serious problem with the rating
: >lists like the SSDF whose accuracy is surely being severely affected,
: >particularly when new programs released now and in the future get to
: >play "old" programs like Genius2/3, Hiarcs3 and Rebel6.
: >I believe such a large number of possible "cooked matches" gives
: >programs like MChess5 an inflated rating.
: I agree, with or without learning, book cooking gains a lot of ELO points
: especially when SSDF test older versions which does not have a learning
: system. These older versions:
: - Genius 2.0
: - Genius 3.0
: - Hiarcs 3.0
: - Rebel 6.0
I think this is a problem that (a) has been here a long time; and (b) will
be around for a long time. I remember a "meeting" years ago with Ken Thompson,
Mike Valvo and myself, in a tournament hall somewhere, where ken showed us an
opening that would trap most any computer. He sprang it on NuChess and crushed
it in an ACM event. Do I like getting trapped? no. Do I like trapping
someone else? yes. :)
In any case, a standard starting position would be a start, although it would
certainly weaken some engines like Genius that play certain openings very
well and others poorly. That's about the only down-side to randomly choosing
the starting position. If you pick one that one program doesn't understand
or like, it could also be unfair...
Needs thought...
Bob
Ed Schröder
From: Mark Uniacke <ma...@acc-ltd.demon.co.uk>
:A few weeks ago Marty Hirsch (author of Mchess5) wrote:
:
: "Opening preparation against commercial opponents matters somewhat, but
: not as much as one might expect, because an SSDF rating is based on
: hundreds of games against at least twenty opponents."
:
:I replied that at matters AT LEAST 100 ELO points on SSDF.
>Having seen the results I have to agree with you Ed.
Thank you for backing me up Mark, at least I know that I am not
alone now.
[ snip ]
>It is obvious from my testing too that MChess5 has a heavily "cooked"
>book for Genius2/3, Rebel6 and Hiarcs3. Which incidentally were MChess'
>main opposition when it was released.
>This means there are at least 7 SSDF matches of 20 games each which are
>influenced by the killer lines and NOT the relative engines strengths.
>There is no doubt in my opinion that killer lines in a cooked book on
>this scale will severely affect the SSDF rating of MChess5.
Unfortunately I have to agree.
I have the same results and the same conclusions as Mark.
[ snip ]
:Coming to the GOAL of this posting:
:
:- Is this the future of computer chess?
:- Spending months of our time on cooked books to get a good rating on
:SSDF?
:- Should the programmers of Genius, Hiarcs and Rebel do the same?
:I obvious prefer to spend my time on improving the chess engine of Rebel
:rather than spending months of my time looking for weak points in other
:chess programs and add total won lines to the Rebel opening book!
>I have never put killer lines in Hiarcs' opening book for computer
>opponents. What limited time I have I prefer to devote to work on the
>chess engine.
>I belive chess programs should be developed for the users/customers who
>are willing to purchase them. It seems some chess programs are being
>developed to beat other chess programs as a main priority. Surely this
>cannot be right?
Corect, I have never seen any cooked book lines in Hiarcs3/4, neither
did I found any cooked lines in Genius2/3/4. Just Like Rebel, Hiarcs
and Genius opening books are written for humans and I am in favor to
keep it that way!!
:What to do?
>I think Chris mentioned about learning and this may be the only way
>forward for us all. However, it leaves a serious problem with the rating
>lists like the SSDF whose accuracy is surely being severely affected,
>particularly when new programs released now and in the future get to
>play "old" programs like Genius2/3, Hiarcs3 and Rebel6.
>I believe such a large number of possible "cooked matches" gives
>programs like MChess5 an inflated rating.
I agree, with or without learning, book cooking gains a lot of ELO points
especially when SSDF test older versions which does not have a learning
system. These older versions:
- Genius 2.0
- Genius 3.0
- Hiarcs 3.0
- Rebel 6.0
and soon
- Genius 4.0
- Hiarcs 4.0
- Rebel 7.0
are easy victims for "cooked lines".
No defense possible!
:Comments are *VERY* welcome because I want to know what you all think
:about this subject.
:I mean if nobody really cares why should I care any longer?
>Ed, you are not alone.
Great!
>Regards,
> Mark
>Author of Hiarcs3, Hiarcs4 and soon Hiarcs5!
- Ed Schroder -
Author of Rebel8
graham_laight
Everybody in Europe always seems to want protectionist measures at the first
sniff of competition. I wish we could be more like the USA.
How do you expect consumers around the world to judge programs except by looking
at tables?
Is everybody expected to make deep studies every time they buy a USD100 product?
My opinion is that, if you can't compete, and you stand and complain noisily
about it, you make yourself look weak. I personally think that it is not good
to present yourself to the world with the word "victim" stamped upon your
forehead.
In article <53ting$n...@news.xs4all.nl>, "Ed says...
Don Fong
In article <53urm3$n...@news-central.tiac.net>,|: I have posted my reply also to Marty's email address so he couldn't miss
|: my comments. Till now I have not received any reply from Marty. Not here
|: and not in RGCC.
|
|doesn't even respond to posts that ask for information, such as the one I
|posted about the 10MB Hash table limit.
maybe he's just too busy. or maybe he's on vacation.
or maybe he just doesn't think your questions are worth
responding to. believe it or not, the world does not revolve
around RGCC.
|Apparently, he, like Steve at
|ICD, figures that the less said, the sooner the thread will drop.
|
| Wrong.
i can't speak for Hirsch, but if it were me, i would not
dignify that kind of insulting drivel with -any- kind of response,
just on principle.
--
--- don fong ``i still want the peace dividend''
--
john quill taylor
da...@laraby.tiac.net (James Garner) wrote:
>Ed Schröder (rebc...@xs4all.nl) wrote:
>: I have posted my reply also to Marty's email address so he couldn't miss
>: my comments. Till now I have not received any reply from Marty. Not here
>: and not in RGCC.
> Marty Hirsch does not respond to posts that he doesn't like. He
>doesn't even respond to posts that ask for information, such as the one I
>posted about the 10MB Hash table limit. Apparently, he, like Steve at
>ICD, figures that the less said, the sooner the thread will drop.
> Wrong.
I'm just a lowly "D" player out here in Idaho, yet Marty has answered
virtually all of my e-mail inquiries. It looks _so_ unprofessional to
see him attacked in this newsgroup. Can't we just face it that all
"book" knowledge and opening theory is an admission of our weakness,
whether we're a human or a computer program? If we extol a program
because it has the ability to find an elusive draw, we must also
give credit to the program that has a better book. Rather than
"steaming" here about how bad it is, I would politely ask that the
programmers go and remove those "cookable" weaknesses from your books!
Your book is only as strong as its weakest link, and M-Chess has
proven to be a very tough chain to crack. Do you want to have a
tournament with "all books off," or play the game as we currently
know it?
__
john quill taylor / /\
writer at large / / \
Hewlett-Packard, Storage Systems Division __ /_/ /\ \
Boise, Idaho U.S.A. /_/\ __\ \ \_\ \
e-mail: jqta...@hpdmd48.boi.hp.com \ \ \/ /\\ \ \/ /
Telephone: (208) 396-2328 (MDT = GMT - 6) \ \ \/ \\ \ /
Snail Mail: Hewlett-Packard \ \ /\ \\ \ \
11413 Chinden Blvd \ \ \ \ \\ \ \
Boise, Idaho 83714 \ \ \_\/ \ \ \
Mailstop 852 \ \ \ \_\/
\_\/
"When in doubt, do as doubters do." - jqt -
haiti, rwanda, cuba, bosnia, ... we have a list,
where is our schindler?
Robert Hyatt
john quill taylor (jqta...@hpdmd48.boi.hp.com) wrote:: >: I have posted my reply also to Marty's email address so he couldn't miss
: >: my comments. Till now I have not received any reply from Marty. Not here
: >: and not in RGCC.
: > Marty Hirsch does not respond to posts that he doesn't like. He
: >posted about the 10MB Hash table limit. Apparently, he, like Steve at
: >ICD, figures that the less said, the sooner the thread will drop.
:
: > Wrong.
:
: I'm just a lowly "D" player out here in Idaho, yet Marty has answered
: virtually all of my e-mail inquiries. It looks _so_ unprofessional to
: see him attacked in this newsgroup. Can't we just face it that all
: "book" knowledge and opening theory is an admission of our weakness,
: whether we're a human or a computer program? If we extol a program
: because it has the ability to find an elusive draw, we must also
: give credit to the program that has a better book. Rather than
: "steaming" here about how bad it is, I would politely ask that the
: programmers go and remove those "cookable" weaknesses from your books!
:
: Your book is only as strong as its weakest link, and M-Chess has
: proven to be a very tough chain to crack. Do you want to have a
: tournament with "all books off," or play the game as we currently
: know it?
:
: __
I agree.
Between the hyperbole and personal attacks, there's not much room left
for useful information here. :)
Bob
Enrique Irazoqui
> Kevin,
> From: kjbe...@chimi.engr.ucdavis.edu (Kevin James Begley)
> Mark Uniacke (ma...@acc-ltd.demon.co.uk) wrote:
>
1- Mchess 5 has a killer-book that makes it crush Genius 3, Rebel 6 and
Hiarcs 3.
2- Because of this killer book Mchess's rating in the SSDF list has been or
is quite a bit higher than its chess engine would have achieved otherwise.
3- To the extent that they reflect the results of games, sometimes even
double games, won by cooked opening lines, the ratings given by SSDF do
not reflect the real strength of a chess engine, and therefore in some
specific cases they are misleading consumers that pay attention to this
rating list before making a buying decision and they are giving top honors
where top honors are not due.
4- A different rating procedure should be followed if we want to know and
let people know the relative strength of chess engines.
Whether or not the inclusion of cooked lines in the opening book is a fair
approach, it may morally be a matter of opinion. In practical terms what I
find not arguable is that "killer books" give a false idea of the real
strength of a chess engine.
Enrique
Alexander Fuchs
john quill taylor <jqta...@hpdmd48.boi.hp.com> schrieb im Beitrag
<540fqj$l...@hpbs2500.boi.hp.com>...
> da...@laraby.tiac.net (James Garner) wrote:
>
> >Ed Schröder (rebc...@xs4all.nl) wrote:
>
>
> Your book is only as strong as its weakest link, and M-Chess has
> proven to be a very tough chain to crack.
The book of M-Chess Pro can be cooked like any other book.
Do you want to have a
> tournament with "all books off," or play the game as we currently
> know it?
I'd prefer that rather than see computer chess reduced to cooking books
Alexander Fuchs
Joe Stella
Maybe someone in this newsgroup presented themselves this way but I
certainly don't remember it. I do remember the question being asked
was "What should I do next to improve my program; killer book or
stronger chess engine?"
A stronger chess engine is what most customers really want to buy,
but the trouble is, killer books can make an engine look stronger than
it really is.
Maybe a good learning algorithm that can develop such books during
auto-play mode is one answer. That way, authors can still devote
most of their time to improving the chess engine.
I don't know if this is the best answer or not, or even if there
is a good answer; but in any case, let's get the question straight!
Joe Stella
> Graham Laight wrote:
>[...]
Kevin James Begley
Ed Schröder (rebc...@xs4all.nl) wrote:
<snip : many very good points>
Ed,
I agree w/ most everything you say,
except I fail to see how you can get
rid of killer-books.
As already pointed out, Fischerandom is
not an adequate solution, nor is having
the opening books turned off (there is
virtually NO WAY to assure that the evaluation-
function does not assure a "cooked" opening
anyway).
I suggest the consumer rely upon a benchmark
rather than the results of competition if they
are interested in purchasing the "strongest"
chess program (rather than the "best in competition").
Meanwhile, I maintain that MChess5 has done nothing
unethical -- this aspect of competition has been
too long neglected (though certainly it has always
been present, if only in a more subtle form). I
think it's about time that the anti-computer computer
emerges. I'd have tried to equip it with a way to
take advantage of the opponent's horizon-effect.
Victory need not always go to the strongest, and I
happen to admire the MChess5 team for taking Lasker's
advice (PLAY THE MAN!). There's a certain amount of
irony and justice to it all -- like when lawyers first
started suing other lawyers.
Kevin.
-Live by the sword, die by the sword.
Dirk Frickenschmidt
Hi Kevin,
I don't want to comment your taste of chess programs. If you like having a
program which gets many, many points in the Swedish list - where they
aren't even willing to do such a primitive thing as killing doubles - you
will of course buy the program which is so nice in "cooking" just according
to your kind of taste in chess and simlpy has a "cooked" rating as well
according to the facts.
For me it's something completely else: My only interest is that in the
playing style and playing strength of a program (and as long as strength is
ok - not necesarily best - I'm looking at style in first place).
Mchess would be an attractive program without killer booking as well,
although I doubt it ever was strong enough for the first place in the
Swedish list. It was simply perfect for the Swedish way of testing :-)))
But the only reason why I bought it is the remaining playing strength and
style after diregarding the whole cooking. It's okay for me if a
programmer's team adopts the book to the playing style: not anything
more!!!
So I'm definitely not interested in this kind of programming and if it goes
on like this it will sooner or later have influence on my decision: simply
not to buy this kind of programs any more.
By the way: the learning function regularly produces more stupid stuff than
interesting games.
If you ever saw the variations on one variation produced by two computers,
the results are in my eyes in 80% of the games boring, in 15% ridiculous
and in 5% interesting. I'm simply not interested in this kind of opening
études either.
All in all: I'm happy to see the real playing strength and style of
programs after
a) switching off learning functions immediately
b) using variety books instead of tornament book
c) playing tournaments with certain openings for black and white and then
switching off beoth books.
Yours Dirk!
--
Yours Dirk
Kevin James Begley <kjbe...@chimi.engr.ucdavis.edu> schrieb im Beitrag
<53un7s$r...@mark.ucdavis.edu>...
> Mark Uniacke (ma...@acc-ltd.demon.co.uk) wrote:
> : I have never put killer lines in Hiarcs' opening book for computer
> : opponents. What limited time I have I prefer to devote to work on the
> : chess engine.
> : I belive chess programs should be developed for the users/customers who
> : are willing to purchase them. It seems some chess programs are being
> : developed to beat other chess programs as a main priority. Surely this
> : cannot be right?
>
chessman
I made a comment before about this topic...I just want to add that Marty is
perfectly within his rights to design a chess program as he see fits...with
or without killer books....it's up to the SSDF whether or not they want to
continue testing in the same manner ... perhaps there should be two lists
..one with book on and one with book off/or from a random start positions
(not the Fischer start ,...but chosing postions at random from GM games...
have the programs play from both sides.. as it stands now the current
systems is subject to abuse...maybe misuse is the word...oh well..it's up
to the SSDF ..not Marty ....he's not doing anything wrong
Eran
Mark Rawlings wrote:
>
> Ian Harris <i...@iharris.demon.co.uk> wrote:
>
> >In article: <53ting$n...@news.xs4all.nl> "Ed Schr?der" <rebc...@xs4all.nl>
> writes:
>
> >> <snip>
> >>
> >> Personally I find this behavior disgusting since it hides the truth of the
> >> real playing strength of a chess program.
> >>
> >> But I really wonder if I have any choice left!
> >>
> >> What to do?
> >>
> >> Comments are *VERY* welcome because I want to know what you all think
> >> about this subject.
> >>
> >> I mean if nobody really cares why should I care any longer?
> >>
> >> Just confused and worried.
> >>
> >> - Ed Schroder -
> >>
>
>
> >The problem is in getting ALL progammers to agree amongst themselves that
> >"killer moves" should not be placed into the book - if this cannot be done
> >then I'm afraid you are left with having to join in.
>
> >--
> >---------------------------------------------------------------------------
> > Ian Harris EMail i...@iharris.demon.co.uk or CompuServe 70374,3166
> > PGP 2.6.3i public key available on request
> >---------------------------------------------------------------------------
>
> Somehow this doesn't make sense to me. Why is it wrong to put a book
> move in a program which wins the game? Sure, it hides the true
> strength of the chess engine, but there are other ways to test just
> the engine. (test suites, etc.)
>
> Maybe shortly after MChess6 comes out, Ed should come out with a
> revised Rebel Book that refutes all of the MChess6 killer moves... ;)
>
> Mark
That's a great idea! Ed should be motivated to improve Rebel Book to
refute all of the Mchess6 killer moves. :)
Eran
Eran
Ed Schröder wrote:
>
Hi,
I suggest that maybe all of you, chess programmers who don't like
killer-book, establish an honest, international organization where
computer chess tests can be made most accurate and reliable results. For
example, a name for the new international organization can be
"International Computer Chess Standard." It is where all computer chess
tests must meet strict standard requirements where killer-book is not
allowed, for instance. All chess programmers can be members of that
organization and can vote equally what yes and what not allow to include
anything such as killer-book in strict standard. So, I hope this make
all chess programmers satisfied and happy. Furthermore, creating newer
chess programs should meet "International Computer Chess Standard"
requirements, because this also helps customers all over the world to
buy correct chess software happily without confusion. I understand that
SSDF may be no longer reliable, because the computer chess tests are in
poor and unfair condition; maybe this condition is too liberal and not
strict enough. Therefore, I think establishing a new strict and honest
organization is a very good idea, it will solve many problems in both
computer chess tests and chess software and clear any confusion among
chess programmers and customers alike.
Eran
> From: Mark Uniacke <ma...@acc-ltd.demon.co.uk>
>
> :A few weeks ago Marty Hirsch (author of Mchess5) wrote:
> :
> : "Opening preparation against commercial opponents matters somewhat, but
> : not as much as one might expect, because an SSDF rating is based on
> : hundreds of games against at least twenty opponents."
> :
> :I replied that at matters AT LEAST 100 ELO points on SSDF.
>
> >Having seen the results I have to agree with you Ed.
>
> Thank you for backing me up Mark, at least I know that I am not
> alone now.
>
> [ snip ]
>
> >It is obvious from my testing too that MChess5 has a heavily "cooked"
> >book for Genius2/3, Rebel6 and Hiarcs3. Which incidentally were MChess'
> >main opposition when it was released.
> >This means there are at least 7 SSDF matches of 20 games each which are
> >influenced by the killer lines and NOT the relative engines strengths.
> >There is no doubt in my opinion that killer lines in a cooked book on
> >this scale will severely affect the SSDF rating of MChess5.
>
> Unfortunately I have to agree.
> I have the same results and the same conclusions as Mark.
>
> [ snip ]
>
> :Coming to the GOAL of this posting:
> :
> :- Is this the future of computer chess?
> :- Spending months of our time on cooked books to get a good rating on
> :SSDF?
> :- Should the programmers of Genius, Hiarcs and Rebel do the same?
>
> :I obvious prefer to spend my time on improving the chess engine of Rebel
> :rather than spending months of my time looking for weak points in other
> :chess programs and add total won lines to the Rebel opening book!
>
> >I have never put killer lines in Hiarcs' opening book for computer
> >opponents. What limited time I have I prefer to devote to work on the
> >chess engine.
> >I belive chess programs should be developed for the users/customers who
> >are willing to purchase them. It seems some chess programs are being
> >developed to beat other chess programs as a main priority. Surely this
> >cannot be right?
>
> Corect, I have never seen any cooked book lines in Hiarcs3/4, neither
> did I found any cooked lines in Genius2/3/4. Just Like Rebel, Hiarcs
> and Genius opening books are written for humans and I am in favor to
> keep it that way!!
>
> :What to do?
>
> >I think Chris mentioned about learning and this may be the only way
> >forward for us all. However, it leaves a serious problem with the rating
> >lists like the SSDF whose accuracy is surely being severely affected,
> >particularly when new programs released now and in the future get to
> >play "old" programs like Genius2/3, Hiarcs3 and Rebel6.
> >I believe such a large number of possible "cooked matches" gives
> >programs like MChess5 an inflated rating.
>
> I agree, with or without learning, book cooking gains a lot of ELO points
> especially when SSDF test older versions which does not have a learning
> system. These older versions:
> - Genius 2.0
> - Genius 3.0
> - Hiarcs 3.0
> - Rebel 6.0
>
> and soon
>
> - Genius 4.0
> - Hiarcs 4.0
> - Rebel 7.0
>
> are easy victims for "cooked lines".
> No defense possible!
>
> :Comments are *VERY* welcome because I want to know what you all think
> :about this subject.
> :I mean if nobody really cares why should I care any longer?
>
> >Ed, you are not alone.
>
> Great!
>
> >Regards,
> > Mark
>
> >Author of Hiarcs3, Hiarcs4 and soon Hiarcs5!
>
A.Mader
Komputer Korner wrote:
>
> Ed Schröder wrote:
> > snipped
> > Coming to the GOAL of this posting:
> >
> > - Is this the future of computer chess?
> > - Spending months of our time on cooked books to get a good rating on
> > SSDF?
> > - Should the programmers of Genius, Hiarcs and Rebel do the same?
> >
> > I obvious prefer to spend my time on improving the chess engine of Rebel
> > rather than spending months of my time looking for weak points in other
> > chess programs and add total won lines to the Rebel opening book!
> >
> > real playing strength of a chess program.
> >
> > But I really wonder if I have any choice left!
> >
> > What to do?
> >
> > about this subject.
> >
> > I mean if nobody really cares why should I care any longer?
> >
> >
> > - Ed Schroder -
>
> updated cooked medium rare "killer books on a regular basis, that these
> would actually sell. Don't underestimate the consumer's capacity to want
> to have the latest killer book for his program. But maybe my ICU is
> OH OH, My circuits ARE overheating caused by the thought of having to
> absorb all those new killer books.
> Komputer Korner
I think we have to be careful. The opening is a very important part of
the game and therefore is a good opening book a very important part of a
chess program. Everybody is trying to make programs more "human like"
and I often read attacks against the bad bad brute forcers. It IS human
like to study openings, learn them by heart and trying to fool your
opponent with opening tricks. Sure it is disgusting to study an
opponents opening book and use cooked books, but what's wrong about a
GOOD opening book without any cooking? Is it also disgusting to use
endgame databases? At what point is an opening a "killer variant"?
I can understand Ed's feelings, but I also understand e.g. Alex Kures
feelings. He (and some others) worked for 2 years to make a good opening
book for Nimzo (without any cooked variations). His work would have been
in vain if only "pure" chess engines are allowed in further
competitions.
Just a few thoughts of my own....
Andreas Mader
Mark Rawlings
Ian Harris <i...@iharris.demon.co.uk> wrote:
>In article: <53ting$n...@news.xs4all.nl> "Ed Schr?der" <rebc...@xs4all.nl>
writes:
>> <snip>
>>
>> Personally I find this behavior disgusting since it hides the truth of the
>> real playing strength of a chess program.
>>
>> But I really wonder if I have any choice left!
>>
>> What to do?
>>
>> Comments are *VERY* welcome because I want to know what you all think
>> about this subject.
>>
>> I mean if nobody really cares why should I care any longer?
>>
>> Just confused and worried.
>>
>> - Ed Schroder -
>>
>I, for one, am in full agreement with your view, Ed.
Robert Hyatt
Dirk Frickenschmidt (di...@jimknopf.wupper.de) wrote:
: Hi Kevin,
:
: I don't want to comment your taste of chess programs. If you like having a
: program which gets many, many points in the Swedish list - where they
: aren't even willing to do such a primitive thing as killing doubles - you
: will of course buy the program which is so nice in "cooking" just according
: to your kind of taste in chess and simlpy has a "cooked" rating as well
: according to the facts.
While we are on this subject, I want to "shift gears" a second. If you
are trying to figure out which program is best, you are using the wrong
metric if by "best" you mean "the program most likely to perform best
against a strong human opponent."
If you want the program that can produce the highest ICC/FICS/whatever
rating, by *only* playing other computers, the SSDF rating is ideal and
this is exactly what it shows... how the programs will stack up against
each other.
If you want a program that produces the highest quality chess against a
human opponent, that's another matter, and has to be measured in a different
way, namely by real USCF or FIDE ratings. Since there are very few USCF
events for computers to participate in, and since a FIDE computer membership
is astronomically expensive, you don't have that option. Your only real
choice is to take advantage of the computers on the servers, play them, ask
the operators what program they are using, and then make up your mind as to
which program you like. You'll likely find that each has different
characteristics that you may like/dislike.
If you look at your needs honestly, you'll likely decide that the "engine"
is not the whole package, otherwise you could not sell a Saturn automobile.
The GUI is also important and will make the program either more enjoyable
or a miserable opponent, depending on how well you like the GUI, how easy
it is to use, and whether it supports the things you want to use it for.
In short, the SSDF is but one data point you should use in selecting the
right program for you. treat it like buying a car. Certainly you'd test-
drive before making up your mind I'd hope? Ditto for a hess program...
:
: For me it's something completely else: My only interest is that in the
: playing style and playing strength of a program (and as long as strength is
: ok - not necesarily best - I'm looking at style in first place).
Good point. One of my old favorites was Dave Kittingers SuperConstellation
that would play speculative attacking sacrifices. You had to watch h7 with
a careful eye. :) Most programs nowadays don't behave like this. I did
have a pretty marvelous version of Crafty a few months back that was attacking
left and right. Lost too many games however, but the "style" was quite
flamboyant... almost Tal-like.
:
: Mchess would be an attractive program without killer booking as well,
: although I doubt it ever was strong enough for the first place in the
: Swedish list. It was simply perfect for the Swedish way of testing :-)))
: But the only reason why I bought it is the remaining playing strength and
: style after diregarding the whole cooking. It's okay for me if a
: programmer's team adopts the book to the playing style: not anything
: more!!!
:
It's still hellishly strong. There's little difference between Rebel,
Mchess Pro, Genius, and others. Not enough that you could really tell
the difference by looking at the games, until you get to know each program
well enough to understand each one's unique differences. I'm beginning to
develop a "feel" for Rebel 8, ChessMaster 5000, Genius 4 and Fritz 4, based
on hundreds of games played by Lonnie against Crafty. Each one has its
own set of strengths and weaknesses, and these even change from version to
version...
: So I'm definitely not interested in this kind of programming and if it goes
: on like this it will sooner or later have influence on my decision: simply
: not to buy this kind of programs any more.
:
: By the way: the learning function regularly produces more stupid stuff than
: interesting games.
: If you ever saw the variations on one variation produced by two computers,
: the results are in my eyes in 80% of the games boring, in 15% ridiculous
: and in 5% interesting. I'm simply not interested in this kind of opening
: études either.
:
: All in all: I'm happy to see the real playing strength and style of
: programs after
: a) switching off learning functions immediately
: b) using variety books instead of tornament book
: c) playing tournaments with certain openings for black and white and then
: switching off beoth books.
:
: Yours Dirk!
:
You'd love Crafty then. Unfortunately, it's book is *so* wide, it gets
into more trouble than Dennis the Menace. Makes for interesting games,
and for interesting losses too.. :)
Garby Leon
In <540fqj$l...@hpbs2500.boi.hp.com> jqta...@hpdmd48.boi.hp.com (john quill taylor)
writes:
(snip Hirsch stuff)
>Can't we just face it that all
>"book" knowledge and opening theory is an admission of our weakness,
>whether we're a human or a computer program? If we extol a program
>because it has the ability to find an elusive draw, we must also
>give credit to the program that has a better book.
>Rather than
>"steaming" here about how bad it is, I would politely ask that the
>programmers go and remove those "cookable" weaknesses from your books!
>Your book is only as strong as its weakest link, and M-Chess has
>proven to be a very tough chain to crack. Do you want to have a
>tournament with "all books off," or play the game as we currently
>know it?
yeah, okay, but it's not quite that simple.
ed's question is: should he spend his time (or maybe, waste his time) countering
'killer' lines in other programs that are specifically targeted to make his program
look bad? Or would he better devote that time to developing the strongest chess
engine he possibly can?
despite the seductive parallel, human play and computer chess aren't exactly the same.
Humans, of course, play to gain a tactical advantage and win. Computer programs have
a slightly different mission: the number of victories they can score in the short run
is perhaps less important than success in the overall project -- that of creating the
strongest chess-playing engine possible, over the course of time.
in that sense computer chess is somewhat out of the realm of pure sport -- pure
winning and losing -- and instead moves more closely to the realm of art, the quest to
reach an ideal.
***to me, in order to encourage and ultimately achieve that lofty goal, computer chess
competition should maximize the importance of the programmer's skill and the engine's
strength, while minimizing all other factors.***
which it already does, to some degree -- running programs on identical machines to
make the contest 'fair,' for example
now you might say, well, playing a game is playing a game, whether by human or
machine.
but killer lines aren't invented by machines -- they're invented by humans, special
humans whose superior positional skill and experience enable them to see through a
machine's weaknesses, especially in opening play. and, they're static rather than
dynamic -- they're not part of the 'thinking' of the program, only a guide to that
thinking with an outside assist.
so killer lines can be seen as kind of human-created crutches (or brass knuckles),
unbalancing the contest and enabling a given program to perform beyond its inherent
strength and capacity. they turn computer chess back into human chess.
that's because 'cooked' books, made by people, and then externally appended to the
engine itself, are not, strictly speaking, making a contribution to the larger mission
of computer chess itself. developing them is a different challenge than that of
developing the strongest chess playing engine. while it might be fun once in a while
to spring such a 'surprise,' when you see two strong programs come out of competition
at 19-2, you know something is not quite right.
ed's worry -- that cooked books minimize the importance of the chess engine, the
programmer's crown jewel and masterpiece -- seems reasonable and even a bit alarming.
if a killer line can actually conceal the playing weakness of a program, it makes any
tournament nearly meaningles.
ed's question is actually very practical: what should he do? To me, cooked books
distract from and undermine the unique and ancient dream of creating a chess-playing
automaton. I really don't want ed, or any other programmer, wasting his time trying
to find and build in lines which will beat Genius, or MChess, or Hiarcs, or any other
program, especially if it's at the expense of program development.
if one is interested in seeing the development of the ultimate chess-playing engine,
it tends to follow that engaging in these short-run tactical skirmishes using cooked
books -- say, to win the top rating on the SSDF list -- distracts from the central,
unique mission of programmers.
of course all this has a commercial ramification as well: but knocking off the
competition in the SSDF list with a program which may be inferior in playing strength
also seems to defeat the purpose of that list itself. the list then becomes very
misleading, and could lead astray people who are thinking of buying a program, and who
look to the SSDF to find the one playing the strongest overall game. it almost
becomes a problem of truth in advertising, or something.
this discussion actually reaches into a broader area as well, the general problem of
benchmarking computer chess programs as accurately as possible. of course humans do
that for themselves in OTB play, but for computers -- still at a very early stage of
technology -- the problem is slightly different.
Perhaps this discussion will lead to a better, more fair way to test program playing
strength.
My own feeling is that a true test would limit both programs to the same, standardized
book -- just as testers use two identical CPU's and hardware systems when running a
fair contest between programs. Otherwise the result has very little meaning --
exactly for the same reason that a contest between programs on unequal computers
doesn't reveal very much.
Such mechanical 'rules' -- like limiting competitors to a standardized opening book,
are part and parcel of every mature sport, whether limiting sail area, engine size,
ball size, take your pick. Baseball is played on a standard diamond; it certainly
hasn't hurt the game. Examples are too numerous to mention. to make a regatta really
exciting, you limit the amount of sail area any boat can use and make dozens of other
strict rules to enforce a 'one design' craft: that tends to highlight the competitive
skill and strategic savvy of each skipper and crew, makes for a more exciting race,
and tells us to a much greater degree which crew performed best -- which wouldn't
happen in a race between between big boats, little boats, yawls, catamarans,
windsurfers and so on. (although all major yacht racing uses complex formulas to
handicap racing results between differing craft as well...).
something like that seems like a reasonable strategy for computer chess to consider.
to me the central problem with killer lines is that they tend to
undermine any attempt to measure strength accurately in
computer-vs-computer chess contests. they also move computer chess
contests away from the crucial arena of the programmer's skill and the
engine's strength, and replace it with a much more mundane and
short-sighted activity.
-- garb leon
Goran Grottling
In article <53vqco$3...@news.xs4all.nl>, rebc...@xs4all.nl says...
>
At the moment the only "book cooking" program is Mchess5.
>I have explained this in detail in my previous posting including a lot
>of examples and complete games.
>
Dear Ed,
MChess 5 is not the only "book cooking" program on the Swedish Rating List.
Some programmers have put in special killer lines against their most important
competitors for more than ten years! It has caused some irritation to me from
time to time, but it is very difficult to solve this problem in a way, that
everybody could agree upon.
For example the Mach II (or was it Mach III?) from Fidelity had several
killerlines against some version of the Novag Expert. I was then worried about how
that would affect the rating figure. But when whe had played more than 500 games,
it only mattered about 5-10 points.
The conclusion was that it is important to play as many games as possible against
as many opponents as possible. Then a biased result will drown in the flood of games.
One can say that the team behind MChess (Marty Hirsch/Sandro Necchi) has made this
problem bigger, because MChess has killer lines against several opponents. And many
of them!
The killer library (right or wrong) does of course have an affect on the rating, but
not as much as you (Ed) believe. I wanted to give you the correct proportion on this
"problem", so I ran some tests with our ratingprogram. It is easy to just remove some
results and then make a new list.
So I removed all results between MChess 5.0 and Rebel6, Hiarcs3, Genius2 and Genius3
- programs that MCPro5 is told to be cooked against. All together 170 games.
First: Here is the top of the official rating list from the 11th of September.
THE SSDF RATING LIST 1996-09-11 50990 games played by 156 computers
Rating + - Games Won Oppo
------ --- --- ----- --- ----
1 Genius 3.0 Pentium 90 MHz 2420 29 -28 626 64% 2320
2 MChess Pro 5.0 Pentium 90 MHz * 2418 28 -27 699 65% 2313
3 Rebel 6.0 Pentium 90 MHz 2415 31 -31 520 60% 2339
4 Rebel 7.0 Pentium 90 MHz 2412 28 -27 671 61% 2330
5 Genius 4.0 Pentium 90 MHz 2409 27 -26 705 65% 2298
6 Hiarcs 4.0 Pentium 90 MHz 2392 30 -30 545 57% 2341
7 Genius 4.0 486/50-66 MHz 2391 31 -31 516 60% 2319
8 Nimzo 3.0 Pentium 90 MHz 2388 30 -29 577 60% 2314
9 Hiarcs 3.0 Pentium 90 MHz 2380 31 -30 525 57% 2333
10 MChess Pro 4.0 Pentium 90 MHz 2367 30 -30 538 54% 2341
11 Genius 3.0 486/50-66 MHz 2362 24 -24 870 63% 2265
12 Fritz 3.0 Pentium 90 MHz 2361 29 -29 593 55% 2324
13 R30 v. 2.5 2356 52 -48 215 68% 2226
2 MChess Pro 5.0 Pentium 90 MHz, 2418
Genius 3 P90 13-7 Rebel 6.0 P90 16-4 Rebel 7.0 P90 8.5-11.5
Genius 4 P90 10-10 Hiarcs 4 P90 6.5-13.5 Geniu4 486/66 9.5-10.5
Nimzo 3.0 P90 6-14 Hiarcs 3 P90 16.5-3.5 MCPro 4.0 P90 9-11
Geniu3 486/66 11.5-8.5 Fritz 3.0 P90 12.5-7.5 R30 v. 2.5 8-12
MCPro5 486/66 10-10 Rebel7 486/66 11.5-8.5 Geniu2 486/66 15-5
Kallis198 P90 11-9 WChess P90 6-14 MCPr40 486/66 12-8
Fritz 4.0 P90 2-4 WChess 486/66 14.5-5.5 Hiarc3 486/66 16-4
Rebel6 486/66 16-4 Genius 68 030 7.5-2.5 CM30 King 2.0 21-8
ChGen1 486/66 22-8 MCPr35 486/66 15.5-4.5 Decade P90 13-7
Fritz3 486/66 12-8 Lyon 68030 15-5 Comet32 P90 14.5-5.5
Kallis 486/66 33-7 SPARC 20 MHz 14.5-5.5 Meph. RISC 18.5-1.5
Chess M. King 4-0 Sapphire 19.5-0.5
And here is the same list without 170 games for MCPro5:
SAME LIST - GAMES REMOVED! 50820 games played by 156 computers
Rating + - Games Won Oppo
------ --- --- ----- --- ----
1 Genius 3.0 Pentium 90 MHz 2425 30 -28 606 65% 2318
2 Rebel 6.0 Pentium 90 MHz 2424 32 -31 500 62% 2337
3 Rebel 7.0 Pentium 90 MHz 2412 28 -27 671 61% 2330
4 Genius 4.0 Pentium 90 MHz 2408 27 -26 705 65% 2298
5 Hiarcs 4.0 Pentium 90 MHz 2392 30 -30 545 57% 2341
6 Genius 4.0 486/50-66 MHz 2391 31 -31 516 60% 2319
7 Hiarcs 3.0 Pentium 90 MHz 2389 32 -31 505 58% 2330
8 Nimzo 3.0 Pentium 90 MHz 2388 30 -29 577 60% 2314
9 MChess Pro 5.0 Pentium 90 MHz * 2386 31 -30 529 62% 2302
10 MChess Pro 4.0 Pentium 90 MHz 2367 30 -30 538 54% 2341
11 Genius 3.0 486/50-66 MHz 2363 25 -24 850 64% 2263
12 Fritz 3.0 Pentium 90 MHz 2361 29 -29 593 55% 2324
13 R30 v. 2.5 2353 52 -48 215 68% 2223
9 MChess Pro 5.0 Pentium 90 MHz, 2386
Rebel 7.0 P90 8.5-11.5 Genius 4 P90 10-10 Hiarcs 4 P90 6.5-13.5
Geniu4 486/66 9.5-10.5 Nimzo 3.0 P90 6-14 MCPro 4.0 P90 9-11
Fritz 3.0 P90 12.5-7.5 R30 v. 2.5 8-12 Rebel7 486/66 11.5-8.5
MCPro5 486/66 10-10 Kallis198 P90 11-9 WChess P90 6-14
MCPr40 486/66 12-8 Fritz 4.0 P90 2-4 WChess 486/66 14.5-5.5
CM30 King 2.0 21-8 ChGen1 486/66 22-8 MCPr35 486/66 15.5-4.5
Fritz3 486/66 12-8 Lyon 68030 15-5 Comet32 P90 14.5-5.5
Kallis 486/66 33-7 SPARC 20 MHz 14.5-5.5 Meph. RISC 18.5-1.5
Chess M. King 4-0 Sapphire 19.5-0.5
The "cooking" has this far given MCPro5 32 ratingpoints, (and not more than 100!)
32 points is not much, but of course it looks much better to be No 2 than No 9!
Well, I will follow this thread hoping that anybody will have an acceptable
solution to take out the plus effect of killer books! Of course I also am inte-
rested in "the real" playing strength of the programs. But please remember, that
the opening library must be one part of this strength.
I think that only the programmers can do anything about this, I don't really
think that the SSDF can prevent it. I agree with Chris Whittington, that learning
functions can help out. And I will also remind, that a program with many, many
variations in it's library and with a good, random play will be more difficult to
trap with killer lines.
At the same time it would be nice for us humans to play against programs with
a wide variety of play.
Goran Grottling (who once started the SSDF rating list...)
PS. BTW, I can confirm your news about Rebel8 on the next rating list. After 511
games it has a rating of 2479 and is indeed the new Number One! Here are our results
so far:
1 Rebel 8.0 Pentium 90 MHz, 2479
Genius 3 P90 11.5-8.5 MCPro 5.0 P90 13.5-6.5 Rebel 6.0 P90 10.5-9.5
Rebel 7.0 P90 12.5-7.5 Genius 4 P90 9.5-10.5 CM5000 P90 1.5-0.5
Hiarcs 4 P90 11.5-8.5 Geniu4 486/66 11.5-8.5 Nimzo 3.0 P90 13.5-6.5
Hiarcs 3 P90 10.5-9.5 MCPro 4.0 P90 15-5 Geniu3 486/66 15-5
Fritz 3.0 P90 15.5-4.5 Rebel7 486/66 15-5 MCPro5 486/66 19-1
Geniu2 486/66 11.5-8.5 Kallis198 P90 15-5 WChess P90 11-9
Hiarc3 486/66 15-5 Rebel6 486/66 14-6 ChGen1 486/66 9.5-1.5
Decade P90 12-8 MCPr35 486/66 15.5-4.5 Fritz3 486/66 17-3
Lyon 68030 14-6 Kallis 486/66 20.5-3.5 SPARC 20 MHz 10-3
Meph. RISC 1-0
Next official list will appear October 23.
Ed Schröder
Dear Goran,
Thank you very much for your reply!
Your test (removing several matches) looks very convincing, but I think
it is in error. Your conclusion of 32 is too low since the book cooking
works on all programs.
You have removed the results of just 4 opponents giving already a loss
of 32 ELO points! I find that a lot!
It also means a free gain of 32 ELO against these 4 older versions (they
can not defend themselves!) on future releases if the book in question
remains unchanged on these killer lines.
I agree with you that the situation is difficult but I think there are
several solutions. One of them is to simply not allow double games.
I know this is an old subject we disagree on but at least it will prevent
this silly results I have published a few days ago:
Mchess5 (ELO 2418) - Hiarcs3 (ELO 2380) 19.0 - 0.0
with > 90% double games.
Without counting double games the results would look like:
Mchess5 (ELO 2418) - Hiarcs3 (ELO 2380) 2 - 0
which certainly looks more in balance looking at the final elo difference
of 38 points between these 2 programs.
I know this has other disadvantages:
- Less games on computer opponents with (very) small books.
- A lot more work for SSDF to get the needed number of games!
I know the whole subject is a pain.
- Ed Schroder -
Robert Hyatt
Moritz Berger (ber...@athene.informatik.uni-bonn.de) wrote:
: Graham Laight wrote:
: > Everybody in Europe always seems to want protectionist measures at the
: first
: > sniff of competition. I wish we could be more like the USA.
: very interesting. You are against protectionism? You are suggesting fair
: play? I think that book cooking is comparable with the worst kinds of
: protectionism (i.e. it doesn't favour the best but instead influences
: competition by means the don't really belong to the real issue, e.g.
: monetary subsidizes, trade customs or cooked books).
:
: > How do you expect consumers around the world to judge programs except by
: looking
: > at tables?
: >
: > Is everybody expected to make deep studies every time they buy a USD100
: product?
:
: Maybe you (the customer) would be pissed off if you bought a product in the
: good faith that it had a very strong chess engine (rated #1 on the SSDF
: list), only to find out that against humans its playing strength is more
: than 100 SSDF/ELO points below its advertised strength?
This is a problem anyway. I haven't seen any of the programs that are
rated at >2400 ("Elo") perform at that level in real games against real
FIDE IM's and GM's... So from this perspective, the issue is not so
important because you'll be disappointed killer books or not. A preview
was posted this morning with Rebel 8 at 2475 or so on the next list.
I have a lot of respect for all the commercial programs, but have not
seen a one that I think really plays at nearly a FIDE 2500 level. They
simply don't "know enough." They play tactically wonderful, positionally
o.k., but long-range planning is the pits, yet that's exactly what a 2500
player excels at, "where do I want my pieces and pawns 20 moves from now?"
Not "where do I want my pieces *now*?" and then two moves later ask the
same question and get a slightly different answer. :)
In any case, take the SSDF as computer vs computer performance and nothing
more. The programs at the top will do better against humans than the ones
at the bottom because they are way stronger... however, whether #1 is better
against Kasparov than #2 is anybody's guess...
: These killer
: openings are often quite bad against humans and mean an overall worse book
: quality! MChess 5 often plays the French Defense, good against other
: computers (computers don't play the French very well), bad against humans.
:
: BTW: By avoiding the opening traps of the MChess 5 book and using a few
: cooked lines specifically against MChess itself, Rebel 8 will be very
: likely the top program on the next list by a margin of maybe 50-60 SSDF/ELO
: point (just to prove the point). At least the intermediate results posted
: here seem to suggest this. This shows that Ed could play the game very well
: if he wanted, he could even post monthly killer updates on his web page
: where he currently provides outstanding customer support.
:
: > My opinion is that, if you can't compete, and you stand and complain
: noisily
: > about it, you make yourself look weak. I personally think that it is not
: good
: > to present yourself to the world with the word "victim" stamped upon your
: > forehead.
:
: Consider the case of Hiarcs: Hiarcs is one of the strongest programs
: around, especially against humans. It plays a very interesting style and
: has much positional and endgame knowledge. A real winner. IMHO mostly due
: to Mchess book-cooking the program isn't recognized as one of the very
: strongest (although despite of the cooked results against Mchess, it's
: rated quite high in the SSDF list). The author doesn't get the commercial
: rewards for his ingenious programming and the further development of Hiarcs
: is at risk. Mark Uniacke a victim? Yes, I would certainly say so. Doping is
: considered unethical in all kind of sports, since it hides the real
: strength and commitment of athletes and deprives the fair players of the
: recognition they deserve.
:
: BUY HIARCS! You won't regret it if you're after one of the very best chess
: engines!
:
: I'm simply not interested in buying a program that only performs well
: against other programs by using a cooked book. Chess programming is a very
: intelligent task, book cooking is FOUL PLAY.
:
: About learning functions: T he SSDF usually plays 20-40 games against one
: opponent, maybe even on different machines. Book learning could only work
: in advance, not for such a limited ammount of games.
Depends. Genius usually uses a pretty narrow book that is really nicely
tailored to its style of play. A narrow book, playing someone that learns,
is a recipe for disaster. In this case it wouldn't be cooking, it is just
that the "learner" learns how to bust the opponent's book by playing... If
the opponent doesn't also learn, someone's going to lose. *big*. 40 games
may be enough with a narrow book. 10 might be enough at times to see the
last 4-5 settled in dramatic style, and the last 4 might be the same games
twice with each color. :)
:
: The non-response of the MChess team (doesn't need to be Marty Hirsch
: himself) to the facts revealed here in this newsgroup is very interesting.
: I fear that they are just too busy adjusting the MChess 6 book against
: other opponents ...
:
: If the MChess development continues in this questionable direction, I am
: proposing to ignore all of its games in the SSDF list and simply calculate
: a new list based on all other games (excepting recent MChess versions).
: This would only improve the margin of error in all ratings ...
it would also distort things, because there's lots of book-cooking by most
everyone there at one time or another. Maybe the "scale" is different for
Mchess vs Genius 4, but both have cooked lines...
:
: Disclaimer: I like Mchess and would certainly support it if not for the
: killer book issue. If Marty Hirsch decides to optionally disable the killer
: variants, I will be the first to applaude and buy the new version. The
: Mchess concept is very interesting and Mchess has been one of the first
: really good PC programs. Marty Hirsch deserves much respect for his
: excellent programming and I hope that he will invest the time he will save
: in the future by not cooking his books in many new and successful chess
: engines.
:
: --
Meik Weber
On 14 Oct 1996 14:29:04 GMT, "Ed Schröder" <rebc...@xs4all.nl> wrote:
>The MCHESS5 computer killer book...
>
Hi all you chess computer and chess programs fans here.
I want to add my personal opinion here about these "killer books".
Ed, you state that the MChess programmers modified their opening books
during the tournament (or testing phase) and added a killer line.
If this is allowed, you are allowed to do the same.
I don't think it is necessary to add killer lines to REBEL to defeat
other programs, but I think it is necessary to avoid being trapped
this easy way.
If your program looses a game, then you should modify your opening
books in a way that this particular loss will not occur again.
(blanking out the line, correcting it, whatever).
You will have to do the same with all other lost games in the
tournament, to get trapped the same way another program got trapped.
If you do not do modify your program or opening book, to avoid this
trap, it is your fault, if you loose again the same game.
If you look at human chess players, it is the same. They get trapped
and after that they learn and avoid this particular variant or learn
the correct way.
I don't know what you are allowed to do to your program during a
tournament or the rating phase. I believe it is better to improve the
engine in a way that this loss will never occur again, but the easier
and faster way is modifying the opening book.
If your program does not learn, you better not wonder why you get
trapped the same way again and again or if another player/program gets
profit out of this situation.
I will buy a computer chess program around christmas. My decision
which program to take is only partially based on the published ELO
ratings. I am not going to buy a program which ranks around No 20. The
top ten will be in my basket. I will then take the program which best
fits my need for usabilty.
1) Copy protection
The less, the better (I defrag my HDD every 3 days and I already have
3 hardlocks on my parallel port and there is no more place to stick
another one onto that). This criterium is something as a k.o.
criterium. I already returned 2 programs to the manufacturer with
money-back as they had hardlocks and did not mention them in
advertising (these were not chess programs).
I have no problem with showing the program a CD or a disk every week
or month.
2) Analysis of played games:
in a way that the program notes the line it would play in each
position with rating. It can do more, but this is the basics.
(BTW: What is Fritz4 doing here. My friend is not capable of getting
more out of the program than "xxx is not better.", Fritz2 was much
easier/better in this topic)
An enhanced feature could be a line that the program did not choose. I
mean sometimes I see DECADE thinking on a move and then taking another
and I don't know why. In a moment the program finds a trap in the move
it should show it. Sometimes I do a move just to see why DECADE did
not choose that move and often after long thinking it shows me why.
(Do you understand what I want?)
3) Ease of use
I hate sticking around with hotkeys. My input medium is the mouse and
nothing else.
(BTW: do you have a move-suggestion feature? I mean if I am to move,
does your program mark the most reasonable target field if I click a
piece? In that way I think I can avoid this dragging most of the time
or save a click. I do not mean this "Give me a tip" feature, all
programs have).
4) Database functions
I have no idea what I want there, so I think this is not the main part
of my decision.
REBEL comes to mind, as I already have this freeware version (DECADE)
and I don't like to buy something I don't know. Thank you for this
freeware program.
Kind Regards and I hope this is something useful to this group
Meikel Weber
m.w...@public.ndh.com
Make sure you visit my homepage
http://ourworld.compuserve.com/Homepages/meikel/mew.htm
Herbert Pohlai
|> Well, I will follow this thread hoping that anybody will have an acceptable
|> solution to take out the plus effect of killer books! Of course I also am inte-
|> rested in "the real" playing strength of the programs. But please remember, that
|> the opening library must be one part of this strength.
What about the following: Take the position where the SECOND (the last) program
leaves it's book and play another game with changed colors starting there.
A program with a well tuned book (a book that leads to positions the program understands)
will understand the position from both sides and therefore have an advantage.
A program with a killer book will hurt itself.
Herbert
Don Fong
In article <01bbbb7c$535c0920$Loca...@ibm.rhrz.uni-bonn.de>,
Moritz Berger <ber...@athene.informatik.uni-bonn.de> wrote:
>Maybe you (the customer) would be pissed off if you bought a product in the
>good faith that it had a very strong chess engine (rated #1 on the SSDF
>list), only to find out that against humans its playing strength is more
>openings are often quite bad against humans and mean an overall worse book
>quality! MChess 5 often plays the French Defense, good against other
>computers (computers don't play the French very well), bad against humans.
PMFJI. it seems to me that the core problem here is not with
MCHESS or with SSDF. the problem is that the human consumer is
making an incorrect -assumption- that the best program in a computer
competition will also be the best program in a human competition.
IMHO that assumption is no more reasonable than -assuming- that
the best racing bike will also be the best mountain bike. or that
the best commuter car will also be a good snowmobile.
perhaps SSDF could alter their methods to eliminate the killer
book anomalies, or perhaps all the programs could join the "arms race".
but you would still have the same fundamental problem:
testing against computers is not the same as testing against humans.
i don't think SSDF ever claimed to determine which is the strongest
program against humans. if you really want to know that, then the
SSDF way is the wrong way to do it.
if you think the "killer book" approach gives an inflated
estimate of strength, then prove it by beating it.
SSDF may not be perfect, but it is at least objective. when you
say that MCHESS (or some other program's) rating is inflated, that
presumes some other objective system of measurement. define it,
implement it, and popularize it.
Stephen B Streater
In article <01bbb9e8$7cd67000$2e06...@omni2.voicenet.com>, chessman
<URL:mailto:ches...@voicenet.com> wrote:
>=20
>=20
>=20
> Ed Schr=F6der <rebc...@xs4all.nl> wrote in article
> <53ting$n...@news.xs4all.nl>...
> Personally, I believe this is a problem ...not only for consumers, but
> programmers and the SSDF.
Don't human players have the same problem as well? Some GMs learn
masses of theory and examine opponent's moves while preparing for a
match. I think the solution is to add some randomness to the computer's
moves, so that the chance of a match is low.
--=20
Stephen B Streater
Tord Kallqvist Romstad
Robert Hyatt (hy...@crafty.cis.uab.edu) wrote:
: Depends. Genius usually uses a pretty narrow book that is really nicely
: tailored to its style of play. A narrow book, playing someone that learns,
: is a recipe for disaster. In this case it wouldn't be cooking, it is just
: that the "learner" learns how to bust the opponent's book by playing... If
: the opponent doesn't also learn, someone's going to lose. *big*. 40 games
: may be enough with a narrow book. 10 might be enough at times to see the
: last 4-5 settled in dramatic style, and the last 4 might be the same games
: twice with each color. :)
M-Chess would be even more vulnerable than Genius, I think. The book is
much narrower than Genius' book, and contains lots of unsound lines.
Tord
Stephen B Streater
In article <32644E...@earthlink.net>, Eran
<URL:mailto:mr...@earthlink.net> wrote:
>=20
> Ed Schr=F6der wrote:
> >=20
>=20
> Hi,
>=20
> I suggest that maybe all of you, chess programmers who don't like
> killer-book, establish an honest, international organization where
> computer chess tests can be made most accurate and reliable results. For
> example, a name for the new international organization can be
> "International Computer Chess Standard." It is where all computer chess
> tests must meet strict standard requirements where killer-book is not
> allowed, for instance. All chess programmers can be members of that
> organization and can vote equally what yes and what not allow to include
> anything such as killer-book in strict standard.
I think killer books should be allowed. How can you ban it without
coming across vast grey areas on how programs work? I think that
in a chess competition, the chess program should be treated like
a black box, and the internal structure should be the sole preserve
of the programmer.
--=20
Stephen B Streater
Stephen B Streater
In article <541o4g$h...@dfw-ixnews12.ix.netcom.com>, Garby Leon
<URL:mailto:ga...@ix.netcom.com> wrote:
> but killer lines aren't invented by machines -- they're invented by humans, special
> humans whose superior positional skill and experience enable them to see through a
> machine's weaknesses, especially in opening play. and, they're static rather than
> dynamic -- they're not part of the 'thinking' of the program, only a guide to that
> thinking with an outside assist.
>
> so killer lines can be seen as kind of human-created crutches (or brass knuckles),
> unbalancing the contest and enabling a given program to perform beyond its inherent
> strength and capacity. they turn computer chess back into human chess.
I agree. That's why I don't use any human-created crutches in my chess program
(C_897d for the StrongARM micro-processor from Digital Semiconductor). It doesn't
rely on my tweaking parameters in the chess engine, as this would be "cheating".
Instead, I've spent my time optimizing the exhaustive search, and it now reaches
its current maximum depth of 12 ply (there is an additional quiescent search of
up to 15 ply) quite early in the game. I'll increase it from 12 to 15 ply when
I have time ie soon, though I regret only allowing 4 bits in the position to
specify the depth, not realising how deep it would go.
It currently generates 750,000 positions per second, though as the SA is quite a
new processor, I expect to reach 1,000,000 positions per second before long. I
also have a 5-processor upgrade for my computer, so could be up to 5 million
nodes/sec soon as exhaustive search is easy to parallelise :-)
PS The StrongARM is only $49, so if anyone is making a dedicated chess computer,
perhaps they should consider using it.
--
Stephen B Streater
Moritz Berger
Graham Laight wrote:
> Everybody in Europe always seems to want protectionist measures at the
first
> sniff of competition. I wish we could be more like the USA.
>
very interesting. You are against protectionism? You are suggesting fair
play? I think that book cooking is comparable with the worst kinds of
protectionism (i.e. it doesn't favour the best but instead influences
competition by means the don't really belong to the real issue, e.g.
monetary subsidizes, trade customs or cooked books).
> How do you expect consumers around the world to judge programs except by
looking
> at tables?
>
> Is everybody expected to make deep studies every time they buy a USD100
product?
>
Maybe you (the customer) would be pissed off if you bought a product in the
good faith that it had a very strong chess engine (rated #1 on the SSDF
list), only to find out that against humans its playing strength is more
than 100 SSDF/ELO points below its advertised strength? These killer
openings are often quite bad against humans and mean an overall worse book
quality! MChess 5 often plays the French Defense, good against other
computers (computers don't play the French very well), bad against humans.
BTW: By avoiding the opening traps of the MChess 5 book and using a few
The non-response of the MChess team (doesn't need to be Marty Hirsch
himself) to the facts revealed here in this newsgroup is very interesting.
I fear that they are just too busy adjusting the MChess 6 book against
other opponents ...
If the MChess development continues in this questionable direction, I am
proposing to ignore all of its games in the SSDF list and simply calculate
a new list based on all other games (excepting recent MChess versions).
This would only improve the margin of error in all ratings ...
Disclaimer: I like Mchess and would certainly support it if not for the
killer book issue. If Marty Hirsch decides to optionally disable the killer
variants, I will be the first to applaude and buy the new version. The
Mchess concept is very interesting and Mchess has been one of the first
really good PC programs. Marty Hirsch deserves much respect for his
excellent programming and I hope that he will invest the time he will save
in the future by not cooking his books in many new and successful chess
engines.
--
---------
Moritz Berger
ber...@zeus.informatik.uni-bonn.de
Komputer Korner
Eran wrote:
> =
> Ed Schr=F6der wrote:
> >
> =
> Hi,
> =
> I suggest that maybe all of you, chess programmers who don't like
> killer-book, establish an honest, international organization where
> computer chess tests can be made most accurate and reliable results. For
> example, a name for the new international organization can be
> "International Computer Chess Standard." It is where all computer chess
> tests must meet strict standard requirements where killer-book is not
> allowed, for instance. All chess programmers can be members of that
> organization and can vote equally what yes and what not allow to include
> anything such as killer-book in strict standard. So, I hope this make
> all chess programmers satisfied and happy. Furthermore, creating newer
> chess programs should meet "International Computer Chess Standard"
> requirements, because this also helps customers all over the world to
> buy correct chess software happily without confusion. I understand that
> SSDF may be no longer reliable, because the computer chess tests are in
> poor and unfair condition; maybe this condition is too liberal and not
> strict enough. Therefore, I think establishing a new strict and honest
> organization is a very good idea, it will solve many problems in both
> computer chess tests and chess software and clear any confusion among
> chess programmers and customers alike.
And how would they decide exactly what opening moves are allowed?
And how would they make sure that there aren't any hidden killer books?
Dream on!!!!
-- =
Komputer Korner
Eran
Kevin,
I agree with you. :-)
Eran
James E. Hughes
> In article <01bbbb7c$535c0920$Loca...@ibm.rhrz.uni-bonn.de>,
> Moritz Berger <ber...@athene.informatik.uni-bonn.de> wrote:
> >Maybe you (the customer) would be pissed off if you bought a product in the
> >good faith that it had a very strong chess engine (rated #1 on the SSDF
> >list), only to find out that against humans its playing strength is more
> >than 100 SSDF/ELO points below its advertised strength? These killer
> >openings are often quite bad against humans and mean an overall worse book
> >quality! MChess 5 often plays the French Defense, good against other
> >computers (computers don't play the French very well), bad against humans.
>
> PMFJI. it seems to me that the core problem here is not with
> MCHESS or with SSDF. the problem is that the human consumer is
> making an incorrect -assumption- that the best program in a computer
> competition will also be the best program in a human competition.
> IMHO that assumption is no more reasonable than -assuming- that
> the best racing bike will also be the best mountain bike. or that
> the best commuter car will also be a good snowmobile.
>
> perhaps SSDF could alter their methods to eliminate the killer
> book anomalies, or perhaps all the programs could join the "arms race".
> but you would still have the same fundamental problem:
> testing against computers is not the same as testing against humans.
> i don't think SSDF ever claimed to determine which is the strongest
> program against humans. if you really want to know that, then the
> SSDF way is the wrong way to do it.
>
> if you think the "killer book" approach gives an inflated
> estimate of strength, then prove it by beating it.
>
> SSDF may not be perfect, but it is at least objective. when you
> say that MCHESS (or some other program's) rating is inflated, that
> presumes some other objective system of measurement. define it,
> implement it, and popularize it.
>
> --
> don fong ``i still want the peace dividend''
The fact that an individual with an obviously strong moral sense can argue
so passionately about Indonesia and the WMCCC yet not express any
reservations about the ethics of using killer books reinforces my view
that we all need to be more tolerant of people whose ethical judgements
differ from our own.
My apologies if this posts twice. My first attempt generated an error msg.
Dirk Frickenschmidt
Hi Don
Don Fong <df...@cse.ucsc.edu> schrieb im Beitrag
<540fgc$r...@darkstar.ucsc.edu>...
> In article <53urm3$n...@news-central.tiac.net>,
> James Garner <da...@laraby.tiac.net> wrote:
> |Ed Schröder (rebc...@xs4all.nl) wrote:
>
> maybe he's just too busy. or maybe he's on vacation.
> or maybe he just doesn't think your questions are worth
> responding to. believe it or not, the world does not revolve
> around RGCC.
You are right in this point: one should never talk about phantasies why
someone might be doing something or not.
But
> |Apparently, he, like Steve at
> |ICD, figures that the less said, the sooner the thread will drop.
> |
> | Wrong.
>
> i can't speak for Hirsch, but if it were me, i would not
> dignify that kind of insulting drivel with -any- kind of response,
> just on principle.
Here I think everybody should return lots of clear facts Ed has presented.
And Ed presented them after Marty made an absolutely unacceptable about
Mchess cookbooks right here in RGCC (so at least he is revolving around
RGCC sometimes).
How would you feel if your (and other's) programs would have been outbooked
to such an extreme extent?
So you might try to keep fair in judging Ed and perhaps spare some of your
ethics for judging nice little killer books ;-)
> --- don fong ``i still want the peace dividend''
> --
>
Sheila Popcorn
On 15 Oct 1996 10:52:08 GMT, "Ed Schröder" <rebc...@xs4all.nl> wrote:
>Kevin,
>
>You are missing the point of the discussion!
>
>The issue is that SSDF allows to test "book cooking" chess programs
>against older chess programs who CAN NOT defend themselves.
>
>These "book cooking" programs therefore gain a LOT of ELO points on the
>SSDF rating list!! and therefore the rating of "book cooking" programs
>ARE NOT RELIABLE.
>
>At the moment the only "book cooking" program is Mchess5.
>I have explained this in detail in my previous posting including a lot
>of examples and complete games.
>
>I am just worried about this new development, we *ALL* want a SSDF list
>with the STRONGEST chess program on TOP. No?
>
>As I producer myself I surely hope that this will be my own program, no
>doubt about that, but I prefer a reliable no.1 on SSDF and I don't care
>if that is one of my concurrents as long as it is reliable!
>
>For years this was the ChessMachine The King;
>For years this was Genius3;
>Both programs were the strongest at that moment!
>
>I have NO problems with Mchess, Genius, Hiarcs, Crafty, Schredder on TOP
>of SSDF as long as the rating and the no.1 position is earned by the
>STRONGEST chess engine.
>
>Unfortunately this has been not been the case in the past year and
>this is the main reason for our discussion.
>
>Please feel free to correct me if I am wrong.
>
>- Ed Schroder -
>
>
>From: kjbe...@chimi.engr.ucdavis.edu (Kevin James Begley)
>Mark Uniacke (ma...@acc-ltd.demon.co.uk) wrote:
>
>: I have never put killer lines in Hiarcs' opening book for computer
>: opponents. What limited time I have I prefer to devote to work on the
>: chess engine.
>: I belive chess programs should be developed for the users/customers who
>: are willing to purchase them. It seems some chess programs are being
>: developed to beat other chess programs as a main priority. Surely this
>: cannot be right?
>
I agree...In business..superior products should win out. If live chess
players have teams of people help them to prepare to play a specific
opponent, then why not machines also. This will force the quality of
computer opening preparation to a higher level, which in turn will
have computers play at a higher level.
Sheila Popstein
>
>
>
Goran Grottling
In article <54325v$m...@news.xs4all.nl>, rebc...@xs4all.nl says...
>
>Dear Goran,
>
>Thank you very much for your reply!
>
>Your test (removing several matches) looks very convincing, but I think
>it is in error. Your conclusion of 32 is too low since the book cooking
>works on all programs.
>
>You have removed the results of just 4 opponents giving already a loss
>of 32 ELO points! I find that a lot!
>
I obviously don't pay as much attention to 32 ELO points as you do! It is
almost within the margin of error.
I removed the opponents you and Eric pointed out. And I just wanted to show
that it is not a question of hundreds of points. I just feel that you are
overreacting about this.
In my last letter I presented the current results for Rebel8. I am sure of that
you can agree with me that some of them are good and some of them are worse for
your program. The result against MCPro5/486 (19-1) is indeed a very good one,
while for instance the result against your own Decade (12-8) is below the average
level.
1 Rebel 8.0 Pentium 90 MHz, 2479
Genius 3 P90 11.5-8.5 MCPro 5.0 P90 13.5-6.5 Rebel 6.0 P90 10.5-9.5
Rebel 7.0 P90 12.5-7.5 Genius 4 P90 9.5-10.5 CM5000 P90 1.5-0.5
Hiarcs 4 P90 11.5-8.5 Geniu4 486/66 11.5-8.5 Nimzo 3.0 P90 13.5-6.5
Hiarcs 3 P90 10.5-9.5 MCPro 4.0 P90 15-5 Geniu3 486/66 15-5
Fritz 3.0 P90 15.5-4.5 Rebel7 486/66 15-5 MCPro5 486/66 19-1
Geniu2 486/66 11.5-8.5 Kallis198 P90 15-5 WChess P90 11-9
Hiarc3 486/66 15-5 Rebel6 486/66 14-6 ChGen1 486/66 9.5-1.5
Decade P90 12-8 MCPr35 486/66 15.5-4.5 Fritz3 486/66 17-3
Lyon 68030 14-6 Kallis 486/66 20.5-3.5 SPARC 20 MHz 10-3
Meph. RISC 1-0
I can divide the different results in two groups (good and "bad") and calculate
the ratings. You must be aware of that you will get quite different rating figures
for the two groups. (I don't say that this is about "cooking" - it may be just chance)
>I agree with you that the situation is difficult but I think there are
>several solutions. One of them is to simply not allow double games.
>
Yes, we disagree about double games. But we can agree that the situation is diffi-
cult to solve.
>I know this is an old subject we disagree on but at least it will prevent
>this silly results I have published a few days ago:
>Mchess5 (ELO 2418) - Hiarcs3 (ELO 2380) 19.0 - 0.0
>with > 90% double games.
>
Let me just put in, that 19-0 is not the result we received. Our result was 16.5-3.5,
which of course also is a very good result for MCPro5. Did you only get two different
games out of 19? Do you mean one win as white and one win as black for MChess? Or did
you only let MChess play from the white side?
>Without counting double games the results would look like:
>Mchess5 (ELO 2418) - Hiarcs3 (ELO 2380) 2 - 0
>which certainly looks more in balance looking at the final elo difference
>of 38 points between these 2 programs.
>
If we hadn't counted double games we had had a different result than 16.5-3.5 (I can
check that out) but it hadn't been 2-0...
>I know this has other disadvantages:
>- Less games on computer opponents with (very) small books.
>- A lot more work for SSDF to get the needed number of games!
>
>I know the whole subject is a pain.
>
>- Ed Schroder -
>
Yes, it is a pain! But your suggestion doesn't solve this at all. You seem to forget,
that also the losses of MChess can be repeated. Or do you have any indication of that
MChess avoids repeating it's losses or draws and only repeats it's wins?
The number of repeats is due to how narrow and monotonous the program's library is. If you play
20 games (10 as white, 10 as black), it is of course possible to get 20 different games
with MChess against an opponent who plays with a big variety.
Let's assume that 50 % of all games with MChess are repetitions, which you want to be
removed. If the result after 20 games was 12-8, the new result would be 6-4. And if it was 16-4,
the new result would be 8-2.
The current line for MCPro5 is:
2 MChess Pro 5.0 Pentium 90 MHz 2418 +28 -27 699 65% 2313
If we removed all repeated games, the line should be:
2 MChess Pro 5.0 Pentium 90 MHz 2418 +45 -45 350 65% 2313
It is true, that MChess takes advantage of weaknesses in other program's openinglibrarys
and early mistakes just after the opening. This hadn't been possible, if the other programs
had played more openinglines and also had played more randomely during the game.
I once asked one of the top-programmers, why his program only played so few lines activ in spite
of that his program "knew" so much theory (which only was shown, if the opponent played those lines).
He answered, that he thought that his program would do much worse against humans - and computers as
well - if it played lines which would lead to a more open play.
But this decision has also made his program easy to book out (and that is what has happened).
You can't eat the cake and still have it...
I could just add, that also human players take advantage of a certain program's lack of variety.
And I don't think that anybody can judge this as unfair!
Chris Whittington
jhu...@indirect.com (James E. Hughes) wrote:
>
> In article <54397q$l...@darkstar.ucsc.edu>, df...@cse.ucsc.edu (Don Fong) wrote:
>
> > In article <01bbbb7c$535c0920$Loca...@ibm.rhrz.uni-bonn.de>,
> > Moritz Berger <ber...@athene.informatik.uni-bonn.de> wrote:
> > >good faith that it had a very strong chess engine (rated #1 on the SSDF
> > >list), only to find out that against humans its playing strength is more
> > >than 100 SSDF/ELO points below its advertised strength? These killer
> > >openings are often quite bad against humans and mean an overall worse book
> > >quality! MChess 5 often plays the French Defense, good against other
> > >computers (computers don't play the French very well), bad against humans.
> >
> > MCHESS or with SSDF. the problem is that the human consumer is
> > making an incorrect -assumption- that the best program in a computer
> > competition will also be the best program in a human competition.
> > IMHO that assumption is no more reasonable than -assuming- that
> > the best racing bike will also be the best mountain bike. or that
> > the best commuter car will also be a good snowmobile.
> >
> > perhaps SSDF could alter their methods to eliminate the killer
> > book anomalies, or perhaps all the programs could join the "arms race".
> > but you would still have the same fundamental problem:
> > testing against computers is not the same as testing against humans.
> > i don't think SSDF ever claimed to determine which is the strongest
> > program against humans. if you really want to know that, then the
> > SSDF way is the wrong way to do it.
> >
> > if you think the "killer book" approach gives an inflated
> > estimate of strength, then prove it by beating it.
> >
> > SSDF may not be perfect, but it is at least objective. when you
> > say that MCHESS (or some other program's) rating is inflated, that
> > presumes some other objective system of measurement. define it,
> > implement it, and popularize it.
> >
> > --
>
> so passionately about Indonesia and the WMCCC yet not express any
> reservations about the ethics of using killer books reinforces my view
> that we all need to be more tolerant of people whose ethical judgements
> differ from our own.
Huh ?
What does this mean ?
Seems to make no sense at all.
The issues are unconnected.
The conclusion is a non-sequitur.
Chris Whittington
Dirk Frickenschmidt
Hi Bob,
you know things too good to agree with someone who clearly shows he has not
even understood the problem (cooking against *past* versions of others
without any possible defence for them and getting points which do not have
much in common with the prgrams playing strength). It is a massive
misunderstanding to treat the whole thing as a nice little competition
between programs and programmers!!
Please note: for me as a computer programs user there is simply *nothing*
attractive or even acceptable in such an extensive use of killer
variations. Of course I can't force anyone to respect my opinion, but I can
descide which programs I will buy and use...
So if enough users will express their disgust for this kind of programming,
programmers perhaps will slowly learn to see this as a hint to concentrate
on real playing strength: and Marty certainly belongs to those who have to
offer enough here.
I really don't have anything against Marty or his program: but quite a lot
against this kind of cooking!
It's enough to give a program the type of openings it plays better than
others.
This is simply what I as one of the users wants from Mchess just as from
*any* chess program: good play, nice book, no cooks!
And if Mchess6 or any other program continues with this kind of thing, I
will simply not buy it and not recommend it to anyone.
Active disinterest, if you understand what I mean :-)
Yours Dirk
Robert Hyatt <hy...@crafty.cis.uab.edu> schrieb im Beitrag
<540r6q$4...@juniper.cis.uab.edu>...
> john quill taylor (jqta...@hpdmd48.boi.hp.com) wrote:
> : da...@laraby.tiac.net (James Garner) wrote:
> :
> : >Ed Schröder (rebc...@xs4all.nl) wrote:
> :
> : >: I have posted my reply also to Marty's email address so he couldn't
miss
> : >: my comments. Till now I have not received any reply from Marty. Not
here
> : >: and not in RGCC.
> :
> : > Marty Hirsch does not respond to posts that he doesn't like. He
> : >doesn't even respond to posts that ask for information, such as the
one I
> : >posted about the 10MB Hash table limit. Apparently, he, like Steve at
> : >ICD, figures that the less said, the sooner the thread will drop.
> :
> : > Wrong.
> :
> : I'm just a lowly "D" player out here in Idaho, yet Marty has answered
> : virtually all of my e-mail inquiries. It looks _so_ unprofessional to
> : see him attacked in this newsgroup. Can't we just face it that all
> : "book" knowledge and opening theory is an admission of our weakness,
> : whether we're a human or a computer program? If we extol a program
> : because it has the ability to find an elusive draw, we must also
> : give credit to the program that has a better book. Rather than
> : "steaming" here about how bad it is, I would politely ask that the
> : programmers go and remove those "cookable" weaknesses from your books!
> :
> : Your book is only as strong as its weakest link, and M-Chess has
> : proven to be a very tough chain to crack. Do you want to have a
> : tournament with "all books off," or play the game as we currently
> : know it?
> :
> : __
>
> I agree.
>
> Between the hyperbole and personal attacks, there's not much room left
> for useful information here. :)
>
> Bob
>
>
Dirk Frickenschmidt
I really can't imagine something more unneccesary than forcing something
which I regard as absolutely stupid upon anybody else.
I could rather decide to ignore all programs that use lots of killer
variations...
--
Yours Dirk
Mark Rawlings <raw...@erols.com> schrieb im Beitrag
<541eij$l...@boursy.news.erols.com>...
Robert Hyatt
Dirk Frickenschmidt (di...@jimknopf.wupper.de) wrote:
:
: Here I think everybody should return lots of clear facts Ed has presented.
: And Ed presented them after Marty made an absolutely unacceptable about
: Mchess cookbooks right here in RGCC (so at least he is revolving around
: RGCC sometimes).
:
: How would you feel if your (and other's) programs would have been outbooked
: to such an extreme extent?
: So you might try to keep fair in judging Ed and perhaps spare some of your
: ethics for judging nice little killer books ;-)
One thing I'd like to point out here. When you are on "top" of the heap,
you are *very* easy for everyone to see. I was there with Cray Blitz for
many years, and had lots of traps "sprang" on me. Fortunately, CB was
fast enough that the traps usually were busted tactically, but it was a
constant struggle for us. Since the ACM and WCCC events were only 4 or 5
rounds, we usually went prepared with 5 new openings we'd never played
before. That stopped a lot of funny business, but not all.
I think that in one respect, this "comes with the territory." Just like
being a politician or anything else. As your "height" increases, so does
your visibility. And you become a better target.
This is not to say that I like this particular aspect of computer chess,
I don't. I've spent a lot of time in Crafty to stop "killer" book lines,
by giving it hundreds of thousands of games to choose its openigs from,
and the flexibility to choose randomly (and hopefully, most of the time,
wisely.) I'm continuing to work at this, and am studying ways to have
it learn to avoid bad opening lines without learning to become too
predictable, which won't work on a large server like ICC. I've had lots
of problems with 1. e4 e5 2. c4 of late. It's a *very* narrow set of
book lines that gives humans a chance to cook it. I've taken evasive
action, however. :) Just want crafty to start evading "on its own"
so I can go back to the engine again.
Robert Hyatt
Dirk Frickenschmidt (di...@jimknopf.wupper.de) wrote:
: Hi Bob,
:
: you know things too good to agree with someone who clearly shows he has not
: even understood the problem (cooking against *past* versions of others
: without any possible defence for them and getting points which do not have
: much in common with the prgrams playing strength). It is a massive
: misunderstanding to treat the whole thing as a nice little competition
: between programs and programmers!!
I agree. there are two distinct issues: (1) winning tournaments for the
sake of winning tournaments, which is what I did with CB, and what I hope to
do with Crafty; (2) distort my SSDF rating in a favorable way to make my
program look better, resulting in better sales.
(1) happens all the time; (2) is/can be a real problem. However, part of
the problem is caused by the "trust" placed in the SSDF ratings. I'm reminded
of several software engineering texts that talk about measuring productivity.
Lines of Code is one standard measure, but it is a *very* poor one. It produces
large (unnecessarily large) codes since every line makes you look better, rather
than trying to make the code smaller and more efficient. No matter what "metric"
you use to measure productivity, these clever "humans" will find a way to turn
that to their advantage in a way that will negatively affect the project, but
positively affects their pocketbook. Ditto for ratings like the SSDF. It is
an interesting exercise, but has likely gotten so distorted that the rankings
don't have much to do with program strength, except that those near the top are
good, those near the bottom are not so good. However, to take two numbers like
2395 and 2402 and say the 2402 player is better is actually pretty funny when
you think about it. And that's the resolution we are seeing at the top, and
everyone takes that 7 point spread as significant, when it isn't. A 50 point
spread seems to not mean much in light of repeated games and book cooks. Maybe
even 100 points doesn't mean anything now. And yet the numbers are treated as
absolute measurements, accurate to the nearest "Elo point" of *exactly* how
two programs compare.
The rating system can't be fixed easily, everyone simply has to become familiar
with what the SSDF ratings show, what they don't show, and act accordingly. I'm
already pissed that I can't walk away from my lawn mower because some idiot in
Washington passed a law that supposedly prevents that mower from backing up over
me and amputating my feet. If I let go on a hill, I ought to get run over.
Education works better than legislation every time. Don't use the SSDF to
figure out which is really best, unless you want to know "out of that pool of
programs which wins the most games among themselves." Don't extrapolate to
answer "which would play the best game against Kasparov?" The data isn't
there...
:
: Please note: for me as a computer programs user there is simply *nothing*
Ed Schröder
: I obviously don't pay as much attention to 32 ELO points as you do!
: I removed the opponents you and Eric pointed out. And I just wanted to
:show that it is not a question of hundreds of points. I just feel that
:you are overreacting about this.
I said "100 ELO points and maybe more", not hundreds :)
Lets keep it on 100 than :)
Since the book cooking works on EVERY program I find this quite
acceptable.
: In my last letter I presented the current results for Rebel8. I am sure
:of that you can agree with me that some of them are good and some of them
:are worse for your program. The result against MCPro5/486 (19-1) is
indeed
:a very good one, while for instance the result against your own
:Decade (12-8) is below the average level.
Yes both results are crazy!
It sometimes happens also on my own testings.
That's why I like your policy so much to play hundreds of games!
It is needed to exclude statistical errors.
Very good.
:Let me just put in, that 19-0 is not the result we received. Our result
:was 16.5-3.5, which of course also is a very good result for MCPro5.
:Did you only get two different games out of 19? Do you mean one win as
:white and one win as black for MChess? Or did you only let MChess play
:from the white side?
The 19-0 is only the Mchess5 (white) part which finally was 2-0 after
removing the (17) doubles.
>I know the whole subject is a pain.
>- Ed Schroder -
: Yes, it is a pain! But your suggestion doesn't solve this at all. You
:seem to forget, that also the losses of MChess can be repeated.
Correct, and if you remove these doubles (won, draw or lost) you will
exclude that part, INCLUDING the cooks! Remember a cooked line can only
gain just *ONE* win and not *FIVE* , *SEVEN* or more.
:Or do you have any indication of that MChess avoids repeating it's
:losses or draws and only repeats it's wins?
I wouldn't know.
: It is true, that MChess takes advantage of weaknesses in other
program's
:openinglibrarys and early mistakes just after the opening. This hadn't
been
:possible, if the other programs had played more openinglines and also had
:played more randomely during the game.
I do not think "the other programs" are to blame, they did not start
the cooking :)
: I could just add, that also human players take advantage of a certain
:program's lack of variety.
:And I don't think that anybody can judge this as unfair!
I agree but there is a HUGE difference:
^^^^
Humans can defend themselves, old versions NOT!!
They will be losing their games OVER and OVER on these cookings!!
Should old programs versions like Genius2/3, Hiarcs3, Rebel6 etc. etc.
a) not be used for testing on SSDF at all?
b) not be used for testing on SSDF against all new released versions?
It's obvious that these *OLD* versions can't defend themselves against
book cooking, resulting in new crazy scores and gain a NEW FREE 32
ELO POINTS against the 4 above mentioned programs if the book in question
remains unchanged on these killer lines.
I am not saying that "removing double games" is a PERFECT solution!
Neither I am saying that excluding old versions is THE solution.
Just like you I am searching for a solution to deal with this new
development and to KEEP SSDF the best list in the world!
You guys (40 if I remember well!) have my highest admiration for all the
work you do now and did in the past! You always have given us a list that
comes very very close to the real truth of the playing strength of chess
programs. Thanks a lot for that!
- Ed Schroder -
MCHESS PRO
Reply by Sandro Necchi to Shroeder's "The MCHESS5 computer killer
book...":
1. Ed did not make a fair article, because if the opening preparation
could bring the results he wrote M-Chess 5.0 would have been much higher
in the SSDF list.
2. It is not true that I told someone, in Aegon, how to out-book other
programs and that M-Chess 5.0 would be no. 1 on the list due to that. The
last time I was in Aegon was April-May 1994, some months before the
release of M-Chess 4.0!!
I told Jeroen Noomen (the Rebel opening book editor) that I believed
M-Chess 4.0 would be better than 3.5, also to book improvements, as tests
showed. I think the difference is quite high.
3. Ed did not say that opening book preparation against other programs is
made by all opening book editors, not only by me. Did anyone forget how
well scoring was Hiarcs 3.0 against M-Chess 4.0 due to specific and long
opening book moves preparation? Not to say about Genius etc...
Rebel 8 is no exception, so since he is doing the same why is he
criticizing M-Chess!!!!! THIS IS FAR MORE DISGUSTING!!!!!
4. We did not make any protest or criticism about other people's opening
preparation, even when we were faced with specific opening books like in
Paderborn by the Genius team, because this is allowed by chess!!
5. Opening books have disadvantages and advantages, like everything, but I
believe they are welcome by the chess program owners because they help the
program to play more human -like and from my point of view more fun.
6. Anyone can play chess the way they like and this does not give the
right to criticize other people, even to a great programmer like Ed.
7. Going back to opening books, since this preparation against other
programs is made by everyone it is a lie to make other people believe that
this has given to M-Chess 5.0 100 points advantage. Sorry, but nobody can
do it!!
8. Also, to say that I look at other programs play and when they make a
mistake I add moves to the book is not true at all. I have another system
and I prepare the book thinking more about human players, than computers!!
This poor explanation of how I improve the M-Chess opening book, by Ed,
simply means he does not know M-Chess 5.0 enough, but I am not willing to
explain my secrets.
9. Ed, Marty is spending all year to improve the M-Chess program and you
will see it with M-Chess 6.0 wich is killing Rebel 8.0, quite often, in
all phases of the game!
10. Ed, Opening books are becoming more important due to strength
increase, like it or not!!
-Sandro Necchi
\
MCHESS PRO
Reply by Marty Hirsch to Shroeder's "The MCHESS5 computer killer book...":
1. M-Chess is not the only program that prepares against other programs.
2. We do not prepare traps, we only prepare lines we believe to be
objectively strong. We DO NOT hunt for mistakes by other programs.
3. Contrary to what I've seen in this newsgroup, the Urusoff Gambit is a
marvelous sacrifice. If you are a serious chess player I strongly
recommend you to study the M-Chess repertoire: 1) to learn a very exciting
variation which you might well enjoy playing and winning with! and 2) to
consider how you might defend against this should you encounter it in a
tournament. An important goal of the M-Chess Opening Book is to assist
you, the chess enthusiast, to develop a winning repertoire of your own.
4. Having a successful opening book should not be considered a
disadvantage. How can it be that strong openings by MChess are to be
termed "book cooks" and removed from consideration, while the opening
preparations of other programs such as Rebel 8.0 are to be accepted
without comment?
5. With regard to duplicate games: There are no chess tournaments where
you are forbidden to repeat an opening, or to vary your play once your
opponent is out-of-book. According to Mr. Schroeder, it is somehow unfair
for MChess to do either of these things. It seems to me that the learning
features make MChess more fun and more interesting.
6. MChess 5.0 won the championship in Paderborn against a field of mostly
amateurs who had prepared against MChess, not the other way around.
7. MChess 5.0 won the playoff in Paderborn against a "killer book"
prepared by the Genius team. M-Chess was out of book on move 11, Genius
on move 22, but M-Chess 5.0 still won the game and the title.
8. MChess Pro 5.0 won first place (decisively) over Genius, Rebel, and
Hiarcs in two out of two round-robin matches against the Finnish National
chess team. Clearly, its strength does not depend on opening preparation
against known opponents.
-Marty Hirsch
graham_laight
Eric Hallsworth's list should, in theory, work against programs with cooked
books, because it also takes games against humans into account. Cooked books
would be expected to get worse results against humans.
I think I'm right in saying that Mchess never quite got to the top of Eric's
list. Speaking as a subscriber to "Selective Search", and since I trust Eric's
honesty, I see Eric's list as a better list than the Swedish one.
If I had a chess computer in the market, I would offer cooked lines against
other programs as an option that could be toggled on or off. This would help
to educate the wider public about the issue. I gather that the cooked lines
cannot be turned off in Mchess, and that is bad (for the consumer of the
product). Even so, if a list (like the Swedish one) contains nothing but
information about computers playing each other, the computers SHOULD be allowed
to use their cooked lines. To not do so would result in the cooks being put in
by stealth!
In article <541o4g$h...@dfw-ixnews12.ix.netcom.com>, ga...@ix.netcom.com says...
{snipped by Graham}
>
>yeah, okay, but it's not quite that simple.
>
>ed's question is: should he spend his time (or maybe, waste his time) countering
>'killer' lines in other programs that are specifically targeted to make his program
>look bad? Or would he better devote that time to developing the strongest chess
>engine he possibly can?
>
>despite the seductive parallel, human play and computer chess aren't exactly the same.
>Humans, of course, play to gain a tactical advantage and win. Computer programs have
>a slightly different mission: the number of victories they can score in the short run
>is perhaps less important than success in the overall project -- that of creating the
>strongest chess-playing engine possible, over the course of time.
>
>in that sense computer chess is somewhat out of the realm of pure sport -- pure
>winning and losing -- and instead moves more closely to the realm of art, the quest to
>reach an ideal.
>
>***to me, in order to encourage and ultimately achieve that lofty goal, computer chess
>competition should maximize the importance of the programmer's skill and the engine's
>strength, while minimizing all other factors.***
>
>which it already does, to some degree -- running programs on identical machines to
>make the contest 'fair,' for example
>
>now you might say, well, playing a game is playing a game, whether by human or
>machine.
>
>but killer lines aren't invented by machines -- they're invented by humans, special
>humans whose superior positional skill and experience enable them to see through a
>machine's weaknesses, especially in opening play. and, they're static rather than
>dynamic -- they're not part of the 'thinking' of the program, only a guide to that
>thinking with an outside assist.
>
>so killer lines can be seen as kind of human-created crutches (or brass knuckles),
>unbalancing the contest and enabling a given program to perform beyond its inherent
>strength and capacity. they turn computer chess back into human chess.
>
>that's because 'cooked' books, made by people, and then externally appended to the
>engine itself, are not, strictly speaking, making a contribution to the larger mission
>of computer chess itself. developing them is a different challenge than that of
>developing the strongest chess playing engine. while it might be fun once in a while
>to spring such a 'surprise,' when you see two strong programs come out of competition
>at 19-2, you know something is not quite right.
>
>ed's worry -- that cooked books minimize the importance of the chess engine, the
>programmer's crown jewel and masterpiece -- seems reasonable and even a bit alarming.
>if a killer line can actually conceal the playing weakness of a program, it makes any
>tournament nearly meaningles.
>
>ed's question is actually very practical: what should he do? To me, cooked books
>distract from and undermine the unique and ancient dream of creating a chess-playing
>automaton. I really don't want ed, or any other programmer, wasting his time trying
>to find and build in lines which will beat Genius, or MChess, or Hiarcs, or any other
>program, especially if it's at the expense of program development.
>
>if one is interested in seeing the development of the ultimate chess-playing engine,
>it tends to follow that engaging in these short-run tactical skirmishes using cooked
>books -- say, to win the top rating on the SSDF list -- distracts from the central,
>unique mission of programmers.
>
>of course all this has a commercial ramification as well: but knocking off the
>competition in the SSDF list with a program which may be inferior in playing strength
>also seems to defeat the purpose of that list itself. the list then becomes very
>misleading, and could lead astray people who are thinking of buying a program, and who
>look to the SSDF to find the one playing the strongest overall game. it almost
>becomes a problem of truth in advertising, or something.
>
>this discussion actually reaches into a broader area as well, the general problem of
>benchmarking computer chess programs as accurately as possible. of course humans do
>that for themselves in OTB play, but for computers -- still at a very early stage of
>technology -- the problem is slightly different.
>
>Perhaps this discussion will lead to a better, more fair way to test program playing
>strength.
>
>My own feeling is that a true test would limit both programs to the same, standardized
>book -- just as testers use two identical CPU's and hardware systems when running a
>fair contest between programs. Otherwise the result has very little meaning --
>exactly for the same reason that a contest between programs on unequal computers
>doesn't reveal very much.
>
>Such mechanical 'rules' -- like limiting competitors to a standardized opening book,
>are part and parcel of every mature sport, whether limiting sail area, engine size,
>ball size, take your pick. Baseball is played on a standard diamond; it certainly
>hasn't hurt the game. Examples are too numerous to mention. to make a regatta really
>exciting, you limit the amount of sail area any boat can use and make dozens of other
>strict rules to enforce a 'one design' craft: that tends to highlight the competitive
>skill and strategic savvy of each skipper and crew, makes for a more exciting race,
>and tells us to a much greater degree which crew performed best -- which wouldn't
>happen in a race between between big boats, little boats, yawls, catamarans,
>windsurfers and so on. (although all major yacht racing uses complex formulas to
>handicap racing results between differing craft as well...).
>
>something like that seems like a reasonable strategy for computer chess to consider.
>
>to me the central problem with killer lines is that they tend to
>undermine any attempt to measure strength accurately in
>computer-vs-computer chess contests. they also move computer chess
>contests away from the crucial arena of the programmer's skill and the
>engine's strength, and replace it with a much more mundane and
>short-sighted activity.
>
>
>
>-- garb leon
>
>
>
> __
>>john quill taylor / /\
>
brucemo
Ed Schröder wrote:
>
>[snip]
> I have posted all the games in PGN format on a new subject here in RGCC
> in case anybody wants to check them. Just replay the games with Mchess5
> and watch the cooked book lines.
>
> Coming to the GOAL of this posting:
>
> - Is this the future of computer chess?
> - Spending months of our time on cooked books to get a good rating on
> SSDF?
> - Should the programmers of Genius, Hiarcs and Rebel do the same?
>
> I obvious prefer to spend my time on improving the chess engine of Rebel
> rather than spending months of my time looking for weak points in other
> chess programs and add total won lines to the Rebel opening book!
>
> Personally I find this behavior disgusting since it hides the truth of the
> real playing strength of a chess program.
>
> But I really wonder if I have any choice left!
>
> What to do?
>
> Comments are *VERY* welcome because I want to know what you all think
> about this subject.
>
> I mean if nobody really cares why should I care any longer?
>
> Just confused and worried.
>
> - Ed Schroder -
I agree with you, if the data presented is correct, it's a bad trend. It is
apparently an attempt to falsify matches between MChess and other programs, in
order to gain an artificially inflated ELO rating, in order to mislead consumers
as to how strong MChess will play against THEM.
I don't know what to do about it, other than to ignore the Swedish list.
bruce
brucemo
> On 14 Oct 1996 20:39:44 GMT, ga...@ix.netcom.com(Garby Leon) wrote:
>
> [snip]
>
> >A MODEST PROPOSAL
> >
> >-- Maybe we're looking at the development of two distinct types of
> >computer chess competitions: a "database" competition, where opening
> >books are unlimited, and an "engine power" competition, where books
> >would be carefully limited -- or even STANDARDIZED, WITH ALL
> >COMPETITORS USING THE SAME BOOK -- to provide a level playing field.
I think that when the Swedish Rating List folks are ready to start up another iteration
of their tests, they should send out a list of perhaps fifty (if fifty is too many,
pick another number, I don't care) PGN game fragments to the testers. The final
position in each of these fragments doesn't have to be dead level, although it shouldn't
be a foregone conclusion that one side is going to win, there should still be a lot of
play left in each position.
The tester would then run each game twice, each program involved in the test gets to
play the position once as white and once as black.
May the best program win, and the best program probably would win.
The next time they do an iteration, they would choose a different set of start
positions, of course.
bruce
john quill taylor
The more I think about chess book openings and their history,
I come back to the same central idea: If a line can be "cooked,"
it is a bad line, and does not belong in memory, computer or
human. Period. Once you pre-program your software or your
head to play 1. b3 (for example), you'd better have a solid,
defensible line to follow it up. Otherwise that move does
not belong in your repertoire. Whether a human or a machine
"kills" your inferior line is academic. It might be better
to think of a solid book as one that is bug-free, not just
toward other machines, but to chess in general.
As far as consuming the programmers' time to find the "killer"
book lines--it shouldn't be too difficult to discover these
weaknesses AFTER your competitors' programs are commercially
available. It probably does boil down more to reputation than
anything else. If a programmer works just on cooking books,
eventually the program will not keep pace with the others.
I doubt that most programmers would "share" their entire
books before a tournament, but in a sense they do share it
when the program is sold commercially. Now, if the commercial
program is not identical to the one used in the tournament,
that is another matter. How can we tell when this is the case?
Do we WANT to buy the "basic" program, or the one that was
"tuned" to beat a specific player or machine?
__
john quill taylor / /\
writer at large / / \
Hewlett-Packard, Storage Systems Division __ /_/ /\ \
Boise, Idaho U.S.A. /_/\ __\ \ \_\ \
e-mail: jqta...@hpdmd48.boi.hp.com \ \ \/ /\\ \ \/ /
Telephone: (208) 396-2328 (MDT = GMT - 6) \ \ \/ \\ \ /
Snail Mail: Hewlett-Packard \ \ /\ \\ \ \
11413 Chinden Blvd \ \ \ \ \\ \ \
Boise, Idaho 83714 \ \ \_\/ \ \ \
Mailstop 852 \ \ \ \_\/
\_\/
"When in doubt, do as doubters do." - jqt -
haiti, rwanda, cuba, bosnia, ... we have a list,
where is our schindler?
Dirk Frickenschmidt
Hi Goran,
thanks for the information you supplied!
The only real solution I see is an agreement between the nowadays most
important programmers.
Perhaps the whole discussion was good for this?
As far as I can see, Ed and Mark Uniacke have already expressed that they
don't want or need outbooking.
And Richard Lang will shurely agree as well as Chrilly Donninger!
So why shouldn't Marty Hirsch - who has written excellent programs in the
past and will write more in the future - agree as well and tell Sandro he
wants another kind of job from him than outbook lots of other programs.
My trust in the integrity of Marty is that he will agree to make a new
beginning.
Won't you, Marty?
If so, we all will have much more fun from computer games than from the
outbooking-learning-book-combination.
What do you think, Ed Schroeder, Mark Uniacke, Richard Lang, Chrilly
Donninger, Franz Morsch ... and others?
--
Yours Dirk
Goran Grottling <goran.g...@mailbox.swipnet.se> schrieb im Beitrag
<542mg7$h...@mn5.swip.net>...
> In article <53vqco$3...@news.xs4all.nl>, rebc...@xs4all.nl says...
> >
> At the moment the only "book cooking" program is Mchess5.
> >I have explained this in detail in my previous posting including a lot
> >of examples and complete games.
> >
>
> Dear Ed,
>
> MChess 5 is not the only "book cooking" program on the Swedish Rating
List.
> Some programmers have put in special killer lines against their most
important
> competitors for more than ten years! It has caused some irritation to me
from
> time to time, but it is very difficult to solve this problem in a way,
that
> everybody could agree upon.
> For example the Mach II (or was it Mach III?) from Fidelity had several
> killerlines against some version of the Novag Expert. I was then worried
about how
> that would affect the rating figure. But when whe had played more than
500 games,
> it only mattered about 5-10 points.
> The conclusion was that it is important to play as many games as
possible against
> as many opponents as possible. Then a biased result will drown in the
flood of games.
> One can say that the team behind MChess (Marty Hirsch/Sandro Necchi)
has made this
> problem bigger, because MChess has killer lines against several
opponents. And many
> of them!
...
> And here is the same list without 170 games for MCPro5:
>
> SAME LIST - GAMES REMOVED! 50820 games played by 156 computers
> Rating + - Games Won
Oppo
> ------ --- --- ----- ---
----
> 1 Genius 3.0 Pentium 90 MHz 2425 30 -28 606 65%
2318
> 2 Rebel 6.0 Pentium 90 MHz 2424 32 -31 500 62%
2337
> 3 Rebel 7.0 Pentium 90 MHz 2412 28 -27 671 61%
2330
> 4 Genius 4.0 Pentium 90 MHz 2408 27 -26 705 65%
2298
> 5 Hiarcs 4.0 Pentium 90 MHz 2392 30 -30 545 57%
2341
> 6 Genius 4.0 486/50-66 MHz 2391 31 -31 516 60%
2319
> 7 Hiarcs 3.0 Pentium 90 MHz 2389 32 -31 505 58%
2330
> 8 Nimzo 3.0 Pentium 90 MHz 2388 30 -29 577 60%
2314
> 9 MChess Pro 5.0 Pentium 90 MHz * 2386 31 -30 529 62%
2302
> 10 MChess Pro 4.0 Pentium 90 MHz 2367 30 -30 538 54%
2341
> 11 Genius 3.0 486/50-66 MHz 2363 25 -24 850 64%
2263
> 12 Fritz 3.0 Pentium 90 MHz 2361 29 -29 593 55%
2324
> 13 R30 v. 2.5 2353 52 -48 215 68%
2223
>
> 9 MChess Pro 5.0 Pentium 90 MHz, 2386
> Rebel 7.0 P90 8.5-11.5 Genius 4 P90 10-10 Hiarcs 4 P90
6.5-13.5
> Geniu4 486/66 9.5-10.5 Nimzo 3.0 P90 6-14 MCPro 4.0 P90
9-11
> Fritz 3.0 P90 12.5-7.5 R30 v. 2.5 8-12 Rebel7 486/66
11.5-8.5
> MCPro5 486/66 10-10 Kallis198 P90 11-9 WChess P90
6-14
> MCPr40 486/66 12-8 Fritz 4.0 P90 2-4 WChess 486/66
14.5-5.5
> CM30 King 2.0 21-8 ChGen1 486/66 22-8 MCPr35 486/66
15.5-4.5
> Fritz3 486/66 12-8 Lyon 68030 15-5 Comet32 P90
14.5-5.5
> Kallis 486/66 33-7 SPARC 20 MHz 14.5-5.5 Meph. RISC
18.5-1.5
> Chess M. King 4-0 Sapphire 19.5-0.5
>
>
> The "cooking" has this far given MCPro5 32 ratingpoints, (and not more
than 100!)
> 32 points is not much, but of course it looks much better to be No 2 than
No 9!
>
> Well, I will follow this thread hoping that anybody will have an
acceptable
> solution to take out the plus effect of killer books! Of course I also am
inte-
> rested in "the real" playing strength of the programs. But please
remember, that
> the opening library must be one part of this strength.
>
>
> Goran Grottling (who once started the SSDF rating list...)
>
> PS. BTW, I can confirm your news about Rebel8 on the next rating list.
After 511
> games it has a rating of 2479 and is indeed the new Number One! Here are
our results
> so far:
>
> 1 Rebel 8.0 Pentium 90 MHz, 2479
> Genius 3 P90 11.5-8.5 MCPro 5.0 P90 13.5-6.5 Rebel 6.0 P90
10.5-9.5
> Rebel 7.0 P90 12.5-7.5 Genius 4 P90 9.5-10.5 CM5000 P90
1.5-0.5
> Hiarcs 4 P90 11.5-8.5 Geniu4 486/66 11.5-8.5 Nimzo 3.0 P90
13.5-6.5
> Hiarcs 3 P90 10.5-9.5 MCPro 4.0 P90 15-5 Geniu3 486/66
15-5
> Fritz 3.0 P90 15.5-4.5 Rebel7 486/66 15-5 MCPro5 486/66
19-1
> Geniu2 486/66 11.5-8.5 Kallis198 P90 15-5 WChess P90
11-9
> Hiarc3 486/66 15-5 Rebel6 486/66 14-6 ChGen1 486/66
9.5-1.5
> Decade P90 12-8 MCPr35 486/66 15.5-4.5 Fritz3 486/66
17-3
> Lyon 68030 14-6 Kallis 486/66 20.5-3.5 SPARC 20 MHz
10-3
> Meph. RISC 1-0
>
> Next official list will appear October 23.
>
>
>
>
>
Dirk Frickenschmidt
Hi Bob,
beeing in time trouble, I can only answer some points in short:
To be, the overall "behaviour" of an engine is the decisive thing.
And you are right to assume I will not just be looking at the swedish list.
I wrote some chess articles and a beginners book about computer chess some
years ago and developed my own kind of testing.
For example, I proved the tactical strength of the Superconny of other
programs of its time by a then completely new kind of test:
using a tested human pure tatcics test rating for computers (the so-called
Colditz-test, some of its postions later beeing in the so-called BT
tactical test).
I also introduced the idea of "Thema-Turniere" (not knowing how you call
playing with a certain opening for all) to the "Computer-Schach & Spiele"
journal.
I only mention this to indicate I'm no freak staring at Swedish Elo-Points
all the time.
And as I said, playing style is very important to me soon as playing
strength is amoung let's say top twenty.
But what I hate nevertheless is real advanced book cooking: this kind of
thing tells nothing about the machine (instaed some about the book cook)
and obscures in fact playing strength as well as playing style.
It has *nothing* in common with the kind of engine of the program, and
*this* is what disturbs me a lot!
It's an absolutely worthless feature, and not that nonsense of "fight or
die" some mad stars-and-stripes liked to make of it recently.
It's simply a stupid job good for nothing at all...
And it does make a not acceptable difference in Elo-Points!
(Compare Eds and Goran's mails concerning this!)
So why shouldn't you programmers agree not to use something that much
boring?
P.S. I promise to have a look at Crafty after the "Rigorosum" (oral exam)
of my dissertation is done...
Yours Dirk
--
Yours Dirk
Robert Hyatt <hy...@crafty.cis.uab.edu> schrieb im Beitrag
<541lhp$f...@juniper.cis.uab.edu>...
> Dirk Frickenschmidt (di...@jimknopf.wupper.de) wrote:
> : Hi Kevin,
> :
...
> : For me it's something completely else: My only interest is that in the
> : playing style and playing strength of a program (and as long as
strength is
> : ok - not necesarily best - I'm looking at style in first place).
>
> Good point. One of my old favorites was Dave Kittingers
SuperConstellation
> that would play speculative attacking sacrifices. You had to watch h7
with
> a careful eye. :) Most programs nowadays don't behave like this. I did
> have a pretty marvelous version of Crafty a few months back that was
attacking
> left and right. Lost too many games however, but the "style" was quite
> flamboyant... almost Tal-like.
>
> :
> : Mchess would be an attractive program without killer booking as well,
> : although I doubt it ever was strong enough for the first place in the
> : Swedish list. It was simply perfect for the Swedish way of testing
:-)))
> : But the only reason why I bought it is the remaining playing strength
and
> : style after diregarding the whole cooking. It's okay for me if a
> : programmer's team adopts the book to the playing style: not anything
> : more!!!
> :
>
> It's still hellishly strong. There's little difference between Rebel,
> Mchess Pro, Genius, and others. Not enough that you could really tell
> the difference by looking at the games, until you get to know each
program
> well enough to understand each one's unique differences. I'm beginning
to
> develop a "feel" for Rebel 8, ChessMaster 5000, Genius 4 and Fritz 4,
based
> on hundreds of games played by Lonnie against Crafty. Each one has its
> own set of strengths and weaknesses, and these even change from version
to
> version...
>
> : So I'm definitely not interested in this kind of programming and if it
goes
> : on like this it will sooner or later have influence on my decision:
simply
> : not to buy this kind of programs any more.
> :
> : By the way: the learning function regularly produces more stupid stuff
than
> : interesting games.
> : If you ever saw the variations on one variation produced by two
computers,
> : the results are in my eyes in 80% of the games boring, in 15%
ridiculous
> : and in 5% interesting. I'm simply not interested in this kind of
opening
> : études either.
> :
> : All in all: I'm happy to see the real playing strength and style of
> : programs after
> : a) switching off learning functions immediately
> : b) using variety books instead of tornament book
> : c) playing tournaments with certain openings for black and white and
then
> : switching off beoth books.
> :
> : Yours Dirk!
> :
>
> You'd love Crafty then. Unfortunately, it's book is *so* wide, it gets
> into more trouble than Dennis the Menace. Makes for interesting games,
> and for interesting losses too.. :)
>
>
Albert Silver
> An important goal of the M-Chess Opening Book is to assist
> you, the chess enthusiast, to develop a winning repertoire of your own.
I'm not going to comment on all of the elements of the post but I would
like to relate an experience in which I tried to do just what Mr Hirsch
suggests a few years ago shortly after purchasing Mchess Pro 3.5. Being
short of material with which to study openings I decided to make use of
Mchess Pro's opening book to learn opening theory. I followed the
variations to the end and found that one side was usually quite lost!
After the experience repeated itself a few times I gave up on the idea
as I realized I couldn't quite expect my potential opponents to know all
of this "theory" and thus follow these lines to their bitter end. I'd
also like to point out that this strongly influenced my decision a month
ago when I decided to purchase two programs and chose Genius 4.0 and
Rebel 8.0.
Albert Silver
Garby Leon
Sounds good to me!
And the positions could be chosen at random, making the test 'double
blind.'
Plus, if the openings skewed the results in any way (say one created a
consistent dead win for any computer that played black), it would be
obvious immediately.
regards,
-- garb
Ed Schröder
Dear Sandro,
You must be kidding!
Remember who said:
"Opening preparation against commercial opponents matters somewhat, but
not as much as one might expect, because an SSDF rating is based on
hundreds of games against at least twenty opponents."
:Reply by Sandro Necchi to Shroeder's "The MCHESS5 computer killer
:book...":
:1. Ed did not make a fair article, because if the opening preparation
:could bring the results he wrote M-Chess 5.0 would have been much higher
:in the SSDF list.
Ed did wrote a fair article since the above is not correct!
See my comments below.
:2. It is not true that I told someone, in Aegon, how to out-book other
:programs and that M-Chess 5.0 would be no. 1 on the list due to that. The
:last time I was in Aegon was April-May 1994, some months before the
:release of M-Chess 4.0!!
You told so to Jeroen, it was indeed 1994 and not 1995. I have checked
Jeroen again and your exact words were:
"We will book out all programs and we will be the new no.1 on SSDF"
:I told Jeroen Noomen (the Rebel opening book editor) that I believed
:M-Chess 4.0 would be better than 3.5, also to book improvements, as
:tests showed. I think the difference is quite high.
Nothing wrong with a good opening book!
But a cooked book?
:3. Ed did not say that opening book preparation against other programs is
:made by all opening book editors, not only by me. Did anyone forget how
:well scoring was Hiarcs 3.0 against M-Chess 4.0 due to specific and long
:opening book moves preparation? Not to say about Genius etc...
Of course I can't speak for Mark or Richard but maybe you can place the
cooked games here in RGCC. A few days ago I have placed 30-40 Mchess5
games here in RGCC so people can check already many of your cooks.
:Rebel 8 is no exception, so since he is doing the same why is he
:criticizing M-Chess!!!!! THIS IS FAR MORE DISGUSTING!!!!!
As explained in a previous posting when I saw all the book cooks of
Mchess5 on Rebel6 (and also on Genius3 and Hiarcs3) I ordered my book
editor Jeroen Noomen to:
a) avoid all your cooks (they are quite "genius" I must say!)
b) Add 2-3 cooks EXCLUSIVELY on Mchess5 as a suggestion to you to stop
this development. I assume you have found them already :)
Despite of the 2-3 book lines as mentioned above the Rebel8 opening book
is a normal human alike opening book WITHOUT ANY COOK AT ALL ON COMPUTER
OPPONENTS!
At least I play with open cards.
Hope you do the same, just say you prepare on computer opponents and we
can close this discussion. Remember the games are against you!!
It would have been SO easy for me to take ALL YOUR MCHESS5 COOKED LINES
and put them in MY opening book!!!
Rebel8 scored now on SSDF with Mchess5 cooks
Genius 3 P90 11.5-8.5 13-7
Rebel 6.0 P90 10.5-9.5 16-4
Hiarcs 3 P90 10.5-9.5 16.5-3.5
These old programs can not defend themselves and Rebel8 would have
earned an extra 32 ELO points. No?
:4. We did not make any protest or criticism about other people's opening
:preparation, even when we were faced with specific opening books like in
:Paderborn by the Genius team, because this is allowed by chess!!
That is a different story, the subject now is SSDF!!
My point is that matches like Mchess5-Rebel6, Mchess5-Hiarcs3 the result
is ALREADY KNOWN BEFORE the match is started! These (old) programs can
not defend themselves against your cooks! All the work is done at home.
No escape possible. Do you deny this?
:5. Opening books have disadvantages and advantages, like everything, but
I
:believe they are welcome by the chess program owners because they help
the
:program to play more human -like and from my point of view more fun.
Of course.
:6. Anyone can play chess the way they like and this does not give the
:right to criticize other people, even to a great programmer like Ed.
Sandro, this whole subject started by the initial posting of Marty who
said:
"Opening preparation against commercial opponents matters somewhat, but
not as much as one might expect, because an SSDF rating is based on
hundreds of games against at least twenty opponents."
Great programmer or not, this is *NOT* true and I think I have the right
to say that. I have seen enough, I have enough evidence of that. It does
not "matters somewhat", it matters a LOT!
:7. Going back to opening books, since this preparation against other
:programs is made by everyone it is a lie to make other people believe
that
:this has given to M-Chess 5.0 100 points advantage. Sorry, but nobody can
:do it!!
Maybe you should read Goran's posting, he removed all results between
Mchess5 against Rebel6, Genius2/3 and Hiarcs3. This already would drop
Mchess5 with 32 ELO points! Read again: already 32 ELO points on just 4
programs! I think my estimate of a 100 ELO points gain is pretty good.
:8. Also, to say that I look at other programs play and when they make a
:mistake I add moves to the book is not true at all. I have another system
:and I prepare the book thinking more about human players, than
computers!!
You said something totally different to Jeroen Noomen! At the end of
this posting I have placed the statistics I have made after AUTO232
matches between Mchess5 against Genius3, Rebel6 and Hiarcs3. Just in
case you have missed them.
Mchess5 comes with scores like +8.77 or even with Mate in N moves out
of the book, book lines are based on computer mistakes you simply
add to the Mchess5 book. They are of course no theory at all.
:This poor explanation of how I improve the M-Chess opening book, by Ed,
:simply means he does not know M-Chess 5.0 enough, but I am not willing
:to explain my secrets.
I assume you meant cooking, no I indeed do not want to now that at all.
In fact I hate book cooking since it hides the truth on the real
playing strength of a chess program.
:9. Ed, Marty is spending all year to improve the M-Chess program and you
:will see it with M-Chess 6.0 wich is killing Rebel 8.0, quite often, in
:all phases of the game!
Also if I offer SSDF a new book for Rebel8 after a few months?
Could be very convincing don't you think? :)
A Rebel8.SSDF version, hmm, who knows.
:10. Ed, Opening books are becoming more important due to strength
:increase, like it or not!!
Of course I agree here, but I prefer NO BOOK COOKING on COMPUTER
OPPONEMTS.
: -Sandro Necchi
Ok, I have answered all your 10 questions, I have only two:
1) Will Mchess6 still contain all the Mchess5 book cooks? In case of yes
Mchess6 will again the same huge scores against Rebel6, Hiarcs3
and Genius2/3. As Goran already explained this is 32 ELO points.
Not bad!
2) Will Mchess6 also contain cooks on Rebel7, Hiarcs4 and Genius4 too?
Or maybe, just maybe, you have removed all cooks?
This would make my day and also a lot of concerned people!
Read all the postings here, listen to your customers, the majority
does not like what you are doing!
I also will shut up immediately since the problem is solved.
Sandro here are the satistics again in case you have missed them, the
complete games can be found in RGCC too. Maybe you should take a look
at the games which are very self explaining.
Please note that the games are played in SSDF style so you will see
many duplicates.
--------------------------------------------------------------------------
--
Match Mchess5 - Genius3
Level 40 in 2:00
Machine 2 x P90 (identical)
Move = The move number where Mchess5 left the book.
Score = The score of the first Mchess5 move after leaving the book.
Game Move Score Result
---- ---- ----- ------
1 25 + 3.21 1-0
2 25 + 3.21 1-0
3 27 + 0.55 draw
4 46! + 3.26 draw
5 24 +11.53! 1-0
6 18 +11.04! 1-0
7 18 + 0.14 draw
8 18 +11.04 1-0
9 23 + 0.66 0-1
10 19 + Mat9!!! 1-0
Mchess5 - Genius3 7.5 - 2.5
------------------------------------------------
Match Mchess5 - Hiarcs3
Level 40 in 2:00
Machine 2 x P90 (identical)
Move = The move number where Mchess5 left the book.
Score = The score of the first Mchess5 move after leaving the book.
Game Move Score Result
---- ---- ----- ------
1 28 +14.32 1-0
2 25 + 8.53 1-0
3 24 + 8.80 1-0
4 18 + 1.01 1-0
5 24 + 8.53 1-0
6 25 + 2.20 1-0
7 29 + 2.20 1-0
8 29 + 7.33 1-0
9 35 + 7.33 1-0
10 35 + 7.33 1-0
11 25 + 8.53 1-0
12 29 + 2.20 1-0
13 35 + 7.33 1-0
14 35 + 7.33 1-0
15 35 + 7.33 1-0
16 35 + 7.33 1-0
17 25 + 8.53 1-0
18 18 + 1.01 1-0
19 25 + 8.53 1-0
Mchess5 - Hiarcs3 19 - 0 I find this unacceptable.
-------------------------------------------------------------
Match Mchess5 - Rebel6
Level 40 in 2:00
Machine 2 x P90 (identical)
Move = The move number where Mchess5 left the book.
Score = The score of the first Mchess5 move after leaving the book.
Game Move Score Result
---- ---- ----- ------
1 16 + 0.33 1-0
2 16 + 1.29 1-0
3 19 + Mat8!!! 1-0
4 16 + 7.45!! 1-0
5 28 + 1.49 1-0
6 19 + Mat8!!! 1-0
7 17 + 0.90 1-0
8 19 + Mat8!!! 1-0
9 13 + 0.00 draw
10 19 + Mat8!!! 1-0
11 16 + 1.29 1-0
12 19 + Mat8!!! 1-0
13 17 + 0.88 1-0
14 28 + 1.39 draw
Mchess5 - Rebel6 13 - 1 Also unacceptable
-----------------------------------------------------------
Awaiting your reply.
Best regards,
- Ed Schroder -
posted in RGCC
posted to mche...@aol.com (MCHESS PRO)
Garby Leon
In <546vqv$l...@juniper.cis.uab.edu> hy...@crafty.cis.uab.edu (Robert
Hyatt) writes:
>
>Garby Leon (ga...@ix.netcom.com) wrote:
>:
>: Plus, if the openings skewed the results in any way (say one created
>: a consistent dead win for any computer that played black), it would
>: be obvious immediately.
>:
>: regards,
>:
>: -- garb
>Wouldn't even be a problem, because both programs would get the black
>side of that opening and if the game is really busted, each would get
>one point, which wouldn't bias anything too much...
Right - but it would show the position was faulty in that it didn't
give the programs a chance to 'fight.'
Ideally, of course, you'd want startup positions where the stronger
program would have a chance to win both sides.
Ed Schröder
Dear Marty,
:Reply by Marty Hirsch to Shroeder's "The MCHESS5 computer killer
book...":
:1. M-Chess is not the only program that prepares against other programs.
:2. We do not prepare traps, we only prepare lines we believe to be
:objectively strong. We DO NOT hunt for mistakes by other programs.
I think you do.
Just one example...
Mchess5 - Genius3
1.e4 e5 2.Bc4 Nf6 3.d4 exd4 4.Nf3 Nxe4 5.Qxd4 Nf6 6.Bg5
Be7 7.Nc3 c6 8.O-O-O d5 9.Qh4 Be6 10.Rhe1 h6 11. Bd3 O-O
12.Bxh6 Ne4 13.Qh5 g6 14. Qe5 Bf6 15.Qf4 Nxc3 16.Rxe6 fxe6
17.Qg4 g5 18.Nxg5 Kh8 19.Qh5 Nxa2+ 20.Kb1 Nc3+ 21.bxc3 Qb6+
22.Kc1 Qb2+ 23. Kxb2 Bxc3+ 24.Kb3 Nd7 25.Bxf8+ Kg8 26.Qf7+
Kh8 27.Qh7# 1-0
After the stupid 11.. 0-0?? black is lost after 12.Bxh6!
Is this theory?
I guess not.
All programs like Hiarcs3, Genius3, Rebel6 ETC. ETC. castle here and
lost MANY MANY times on this book line on SSDF.
Any ELO 1500 rated human player never would play 11..0-0??
Still the Mchess5 book comtinues many many moves.
In the above game Mchess comes out of book on move 19 with a Mate
in N moves. More examples available if needed.
It's a cook on computer opponents, no doubt about it.
:3. Contrary to what I've seen in this newsgroup, the Urusoff Gambit is a
:marvelous sacrifice. If you are a serious chess player I strongly
:recommend you to study the M-Chess repertoire: 1) to learn a very
exciting
:variation which you might well enjoy playing and winning with! and 2) to
:consider how you might defend against this should you encounter it in a
:tournament. An important goal of the M-Chess Opening Book is to assist
:you, the chess enthusiast, to develop a winning repertoire of your own.
:4. Having a successful opening book should not be considered a
:disadvantage. How can it be that strong openings by MChess are to be
:termed "book cooks" and removed from consideration, while the opening
:preparations of other programs such as Rebel 8.0 are to be accepted
:without comment?
I already explained this in detail several times here in RGCC.
Here we go again:
As explained in a previous posting when I saw all the book cooks of
Mchess5 on Rebel6 (and also on Genius3 and Hiarcs3) I ordered my book
editor Jeroen Noomen to:
a) avoid all your cooks (they are quite "genius" I must say!)
b) Add 2-3 cooks EXCLUSIVELY on Mchess5 as a suggestion to you to stop
this development. I assume you have found them already :)
Despite of the 2-3 book lines as mentioned above the Rebel8 opening book
is a normal human alike opening book WITHOUT ANY COOK AT ALL ON COMPUTER
OPPONENTS!
At least I play with open cards.
Hope you do the same, just say you prepare on computer opponents and we
can close this discussion. Remember the games are against you.
:5. With regard to duplicate games: There are no chess tournaments where
:you are forbidden to repeat an opening, or to vary your play once your
:opponent is out-of-book. According to Mr. Schroeder, it is somehow
unfair
:for MChess to do either of these things. It seems to me that the
learning
:features make MChess more fun and more interesting.
Mr. Schroeder did not say that!
He said that "not counting double games" on SSDF is a possible solution
to handle cooked book lines and that you can argue about that! Quite a
difference :)
:6. MChess 5.0 won the championship in Paderborn against a field of mostly
:amateurs who had prepared against MChess, not the other way around.
I fully agree, this is a pain too!
:7. MChess 5.0 won the playoff in Paderborn against a "killer book"
:prepared by the Genius team. M-Chess was out of book on move 11, Genius
:on move 22, but M-Chess 5.0 still won the game and the title.
There is a HUGE difference between getting the opponent out of book as
soon as possible and book cooking on COMPUTER opponents.
The "Ruy Lopez / Crafty attack" comes into mind :)
1.e4 e5 2.Nf3 Nc6 3.Bb5 a6 4.Bc4!!
Very original and accepted since after 4.Bc4! Crafty is mostly out of
book after a few moves without any computer cook!
However it would have been a bad thing if Bob had build a huge range
of book lines with all kind of COMPUTER OPPONENT traps on the
4.Bc4! theme, resulting in a score of +7.31 or soo on move 22. This I
call book cooking. Bob did not.
This is exactly what happened on the Mchess5 "1.e4 e5 2.Bc4" theme!
I do not understand why Sandro Necchi (your book editor) who was so open
and honest to Jeroen Noomen (the Rebel book editor) on Aegon 1994 and
now is denying everthing?
In Mchess5 I have seen several opening names with "smashing?" names
like: "Bishop Game, Necchi attack". Other so called "Necchi attacks"
book names examples are available. This at least tells me these lines
are not common theory.
My point is that if you have cooked lines on computer opponents why
don't you say that openly as Sandro did in 1994?
At least I admit OPENLY I have ordered Jeroen to put 2-3 book cooks
EXCLUSIVELY on Mchess5. Simply a piece of your own cake as a serious
suggestion to stop this cooking on computer opponents.
This whole discussion we have now will hurt us ALL.
a) It will hurt me, because I have attacked a highly respected chess
programmer like Marty Hirsch!
b) It will hurt you.
c) And what I find most important "It will hurt the image of computer
chess!!"
Knowing this all on before hand I still find it very very important
to deal with it now, to discuss the matter openly and find solutions
before it will hurt the image of computer chess TOO MUCH!!
I mean you can not expect from other chess programmers after watching
these crazy SSDF results like:
Mchess5 P90 - Rebel 6.0 P90 16-4
Mchess5 P90 - Hiarcs 3 P90 16.5-3.5
Mchess5 P90 - Genius 3 P90 13-7
that we sit down and do nothing??
As said before should I join the book cooking you started?
I think the answer is a clear *NO* from my side.
I like to release my opening books clean from every computer opponenent
cook! But things may change if this new development you started will
become common if other chess programmers start doing the same.
If possible THIS whole discussion we are having now may / can prevent
this ugly thing to happen!
:8. MChess Pro 5.0 won first place (decisively) over Genius, Rebel, and
:Hiarcs in two out of two round-robin matches against the Finnish National
:chess team. Clearly, its strength does not depend on opening preparation
:against known opponents.
I never said anything negative about your chess engine. In fact I think
it's a great chess engine with a very attractive style and that I can
recommand it to every chess lover!
Just remove your cooked book lines.
- Ed Schroder -
:-Marty Hirsch
BTW, I think your new SHUFFLE feature is great!
My compliments.
Robert Hyatt
Robert Hyatt
Dirk Frickenschmidt (di...@jimknopf.wupper.de) wrote:
:
: P.S. I promise to have a look at Crafty after the "Rigorosum" (oral exam)
: of my dissertation is done...
:
: Yours Dirk
If you've followed my posts here over the past two years as Crafty has
evolved, you know that I don't cook books, period. Nor do I "uncook"
them any longer as Bert and I did with Cray Blitz. I'm working on
having Crafty understand which openings suit it's style of play, so
that I can continue to use large PGN files to create the opening
books, rather than hand-coding as we did in years gone by.
For me, then, it's a moot issue. I'm going to use (a) books so very wide
that cooking is nearly impossible, unless the match is going to go on for
thousands of games, because Crafty will have that many different good
alternatives to play; (b) implement book learning (and hopefully, non-
book learning) so that Crafty can learn to avoid those lines that lead
to positions that are not so hot; (c) make this portable so all the
Crafty "clones" playing can share this info among themselves and with me
so we can all learn (and I'll publish specs for the commercial guys to
consider if they want in on this learning action and want to share data.)
Bob
Eric Hallsworth
In article <5450ep$c...@lex.zippo.com>, Graham Laight <?@?> writes
>Eric Hallsworth's list should, in theory, work against programs with cooked
>books, because it also takes games against humans into account. Cooked books
>would be expected to get worse results against humans.
>
is a drop in the ocean compared to the number computer v computer. So
whilst I hope that the inclusion of v human games does a decent job to
determine the overall level of the list, and whilst v human results do
influence both the level of the full list and the individual gradings of
every computer and program, their influence in the latter case is not as
great as probably nearly all of us would like.
>I think I'm right in saying that Mchess never quite got to the top of Eric's
>list. Speaking as a subscriber to "Selective Search", and since I trust Eric's
>honesty, I see Eric's list as a better list than the Swedish one.
>
In fact MCP5 did edge into top place for a couple of months, but is now
back in 4th, though there are only 13 Elo points between the top 5 in my
last Issue..... which was before the Rebel8 results started coming in!
--
Best wishes,
Eric Hallsworth, Computer Chess Magazine, The Red House,
46 High Street, Wilburton, Cambs CB6 3RA
Philippe Beaudoin
Albert Silver wrote:
> I'd also like to point out that this strongly influenced my decision a
> month ago when I decided to purchase two programs and chose Genius 4.0
> and Rebel 8.0.
And be sure I wont buy MChess, even if I was considering it. Just an
advice to MChess people: take 2-3 maketing lessons, your posts in this
group haven't been very attractive to potential customers!
Philippe Beaudoin
Komputer Korner
Ed Schr=F6der wrote:
>snipped =
Ed, even if you get the commercial programmers to all agree, no more
cooked books, there is still the big problem of all the amateur =
kitchens. It will become impossible to police, especially if a
programmer programs into his program a special feature to waste =
time to make it look like the program is thinking when in reality
it is still in it's cooked book.AAAAAAAggggghhhhh!!!!!! Hidden
camouflaged cooked killer books!!!! Oh Oh, I think "I" have just
created a monster. =
-- =
Komputer Korner
The komputer that couldn't kompute the square root of
36^n.
Jay Scott
In article <546vqv$l...@juniper.cis.uab.edu>, Robert Hyatt (hy...@crafty.cis.uab.edu) wrote:
>Wouldn't even be a problem, because both programs would get the black
>side of that opening and if the game is really busted, each would get
>one point, which wouldn't bias anything too much...
It would make weak programs appear stronger and strong programs appear
weaker.
Jay Scott <j...@forum.swarthmore.edu>
Machine Learning in Games:
http://forum.swarthmore.edu/~jay/learn-game/index.html
Robert Hyatt
Jay Scott (j...@forum.swarthmore.edu) wrote:
I was responding to the idea of picking a position, then each program
plays the same opponent twice, once from each side. If the position is
busted, the same side would win both games, resulting in a 1-1 split
which would not affect things very much, unless the two programs are
somewhat separated in their ratings. Then the lower-rated program would
get more by winning than it would for losing, and it's rating would go
up more than justified, just like the stronger program's rating would
go down more than justified.
Mark Uniacke
Hi Sandro,
In article <5464s3$j...@newsbf02.news.aol.com>, MCHESS PRO
<mche...@aol.com> writes
>Reply by Sandro Necchi to Shroeder's "The MCHESS5 computer killer
>book...":
>
>1. Ed did not make a fair article, because if the opening preparation
>could bring the results he wrote M-Chess 5.0 would have been much higher
>in the SSDF list.
We saw the affect of removing the matches against just 4 opponents
MChess5 played in the SSDF in a very recent post from Goran Grottling.
Goran said it made 32 Elo points difference. Taking that into account
would drop MChess5 from 2nd to 9th on the current list.
>
>2. It is not true that I told someone, in Aegon, how to out-book other
>programs and that M-Chess 5.0 would be no. 1 on the list due to that. The
>last time I was in Aegon was April-May 1994, some months before the
>release of M-Chess 4.0!!
>
>I told Jeroen Noomen (the Rebel opening book editor) that I believed
>M-Chess 4.0 would be better than 3.5, also to book improvements, as tests
>showed. I think the difference is quite high.
>
>3. Ed did not say that opening book preparation against other programs is
>made by all opening book editors, not only by me. Did anyone forget how
>well scoring was Hiarcs 3.0 against M-Chess 4.0 due to specific and long
>opening book moves preparation? Not to say about Genius etc...
Any "well scoring" achieved by Hiarcs3 over MChess4 was not due to
cooked books.
Hiarcs3 was NOT specifically booked up for MChess4. If it were so
then why did Hiarcs4 do somewhat worse than Hiarcs3?
Or do you believe the alledged "specific and long opening book moves
preparation" was removed from Hiarcs4?
The SSDF scores were:
Hiarcs3 P90 - MCPro4 P90 13.5 - 6.5
Hiarcs3 P90 - MCPro4 486/66 16 - 4
Hiarcs4 P90 - MCPro4 P90 11.5 - 8.5
Hiarcs4 P90 - MCPro4 486/66 10 - 10
Were'nt both programs released at the same time?
Hiarcs3 appears in the British ratings before MChess4,
although the opposite occured in the SSDF list.
I did not even see MChess4 until sometime into 1995.
>
>Rebel 8 is no exception, so since he is doing the same why is he
>criticizing M-Chess!!!!! THIS IS FAR MORE DISGUSTING!!!!!
I have seen no "killer lines" in Rebel8 created for Hiarcs3 or Hiarcs4.
The SSDF match results bare this out.
In fact, it would have been easy for Ed to book up for Hiarcs3 simply by
following the MChess5 lines. The facts are he obviously did NOT.
>
>4. We did not make any protest or criticism about other people's opening
>preparation, even when we were faced with specific opening books like in
>Paderborn by the Genius team, because this is allowed by chess!!
>
>5. Opening books have disadvantages and advantages, like everything, but I
>believe they are welcome by the chess program owners because they help the
>program to play more human -like and from my point of view more fun.
>
>6. Anyone can play chess the way they like and this does not give the
>right to criticize other people, even to a great programmer like Ed.
Ed is entitled to his opinion as much as anyone and has also been kind
enough to back his views with evidence in the form of games.
>
>7. Going back to opening books, since this preparation against other
>programs is made by everyone it is a lie to make other people believe that
>this has given to M-Chess 5.0 100 points advantage. Sorry, but nobody can
>do it!!
This type of preparation is NOT "made by everyone".
>
>8. Also, to say that I look at other programs play and when they make a
>mistake I add moves to the book is not true at all. I have another system
>and I prepare the book thinking more about human players, than computers!!
>
>
>This poor explanation of how I improve the M-Chess opening book, by Ed,
>simply means he does not know M-Chess 5.0 enough, but I am not willing to
>explain my secrets.
>
>9. Ed, Marty is spending all year to improve the M-Chess program and you
>will see it with M-Chess 6.0 wich is killing Rebel 8.0, quite often, in
>all phases of the game!
>
>10. Ed, Opening books are becoming more important due to strength
>increase, like it or not!!
>
>-Sandro Necchi
>
>
>
>\
Best wishes,
Mark
Author of Hiarcs.
Robert Hyatt
Komputer Korner (kor...@netcom.ca) wrote:
:
: Who would pick those equal positions that have lots of chances for
: both sides? Informator doesn't have any symbol for that. You would
: have to hire a GM specially for this.
: --
: Komputer Korner
: The komputer that couldn't kompute the square root of
: 36^n.
This even I could do. You are primarily looking for positions between
10 and 20 moves into a game, where material is even, the position is
not busted for either side, and the kings are reasonably placed, not
trapped in the center for example.
I'd probably produce them using Crafty, for example, by letting it
follow pgn games to move 10, then doing significant searches starting
at that point to make sure the eval was not way out of line. If so,
reject it and play on, otherwise put this into test database and go to
the next game. Stop when test database reaches some acceptable limit.
chessman
Don Fong <df...@cse.ucsc.edu> wrote in article
<54397q$l...@darkstar.ucsc.edu>...
> In article <01bbbb7c$535c0920$Loca...@ibm.rhrz.uni-bonn.de>,
> Moritz Berger <ber...@athene.informatik.uni-bonn.de> wrote:
> >Maybe you (the customer) would be pissed off if you bought a product in
the
> >good faith that it had a very strong chess engine (rated #1 on the SSDF
> >list), only to find out that against humans its playing strength is more
> >than 100 SSDF/ELO points below its advertised strength? These killer
> >openings are often quite bad against humans and mean an overall worse
book
> >quality! MChess 5 often plays the French Defense, good against other
> >computers (computers don't play the French very well), bad against
humans.
>
also ...remember that SSDF ratings are from a limited pool of computer
opponents ...thus they do not predict, nor does the SSDF state that these
are expectant ratings against human opponents .....but I agree that the
advertising is not exactly honest about what SSDF ELO ratings really mean
> PMFJI. it seems to me that the core problem here is not with
> MCHESS or with SSDF. the problem is that the human consumer is
> making an incorrect -assumption- that the best program in a computer
> competition will also be the best program in a human competition.
> IMHO that assumption is no more reasonable than -assuming- that
> the best racing bike will also be the best mountain bike. or that
> the best commuter car will also be a good snowmobile.
>
[delete
Komputer Korner
Garby Leon wrote:
>
> In <546vqv$l...@juniper.cis.uab.edu> hy...@crafty.cis.uab.edu (Robert
> Hyatt) writes:
> >
> >:
> >: Plus, if the openings skewed the results in any way (say one created
> >: a consistent dead win for any computer that played black), it would
> >: be obvious immediately.
> >:
> >: regards,
> >:
> >: -- garb
>
> >side of that opening and if the game is really busted, each would get
> >one point, which wouldn't bias anything too much...
>
> give the programs a chance to 'fight.'
>
> Ideally, of course, you'd want startup positions where the stronger
> program would have a chance to win both sides.
Who would pick those equal positions that have lots of chances for
dwm
Ed Schröder wrote:
>
> Dear Marty,
>
>
> :8. MChess Pro 5.0 won first place (decisively) over Genius, Rebel, and
> :Hiarcs in two out of two round-robin matches against the Finnish National
> :chess team. Clearly, its strength does not depend on opening preparation
> :against known opponents.
>
> I never said anything negative about your chess engine. In fact I think
> it's a great chess engine with a very attractive style and that I can
> recommand it to every chess lover!
>
> Just remove your cooked book lines.
>
> - Ed Schroder -
>
> :-Marty Hirsch
>
> BTW, I think your new SHUFFLE feature is great!
> My compliments.
Based upon what I'm reading, I'm glad to have Rebel 8 and not
MCHESS 5. I wanted a computer game that plays as human like as possible.
Persons I play seldom are in "book" past move 3 to 5.
Ed, I like your answer!!!
And your convictions!! Keep it up!!
David
Anders Thulin
In article <53ting$n...@news.xs4all.nl>, Ed Schröder <rebc...@xs4all.nl> wrote:
>The MCHESS5 computer killer book...
>Here are the results against:
>Mchess5 - Genius3 (currently no. 1 on SSDF ELO 2420) 7.5 - 2.5
>Mchess5 - Rebel6 (currently no. 3 on SSDF ELO 2415) 13.0 - 1.0
>Mchess5 - Hiarcs3 (currently no. 9 on SSDF ELO 2380) 19.0 - 0.0
>According to the HIGH ratings of Genius3, Rebel6 and Hiarcs3 these
>results are IMPOSSIBLE in normal play (without book traps)
No, that's wrong. They are quite possible, but rather improbable,
assuming that Mchess5 is an average opponent. But if Mchess5 is an
non-average opponent, they're neither impossible nor improbable. Elo
rating does not indicate the strength between two players in
particular (who may not even have met), but the general position of a
player in the entire field.
You can se the same things with human players: the score of one
player against one particular other player is sometimes totally out of
register with respect to ELO ratings of the players.
The last parenthesis seems to suggest there's something
fundamentally bad about traps. I can't make out why from your posting.
>I think you all now can see the impact of killer lines and maybe you
>understand my feelings better and my aversion against cooked books.
Not really. I'm not a chess programmer, though, so that might
account for it.
You present data. Fine. But I find no conclusions, or arguments
based on on that data, except the impossibility argument above.
Are you saying that SSDF rating is wrong? To me it appears you use
them in a wrong way - see above. If you believe that there is a
difference between wins and wins, the Elo model is obviously all wrong
for you, as it assumes wins are equal.
Are you saying that Marty Hirsch is wrong? I leave that for others
-- I can't determine myself.
Are you saying that preparing killer books and trap moves is bad?
Killer moves won't work unless the opponent allows them to. Traps may
open, but unless someone steps in them, they won't close. I'm not
clear if you think that that is a flaw in the losing program, but I
assume you do. Removing the flaw seems to be the best way of coping
with the problem. Using Mchess5 seems to be a good way of finding the
flaws in the three programs you mention.
--
Anders Thulin Anders...@lejonet.se 013 - 23 55 32
Telia Research AB, Teknikringen 2B, S-583 30 Linkoping, Sweden
Alexander Fuchs
Philippe Beaudoin <phil...@bnr.ca> schrieb im Beitrag
<3267B9...@bnr.ca>...
> Albert Silver wrote:
> > I'd also like to point out that this strongly influenced my decision a
> > month ago when I decided to purchase two programs and chose Genius 4.0
> > and Rebel 8.0.
>
> And be sure I wont buy MChess, even if I was considering it.
Yes, that's probably the only way we can get rid of the killer books. Don't
buy the program(s) that have these killer books. Although I like M-chess, I
don't think I am going to buy version 6.
Alexander Fuchs
Anders Thulin
In article <01bbbae1$7214e300$d648e0c2@eil>,
Enrique Irazoqui <en...@bitmailes.net> wrote:
>Whether or not the inclusion of cooked lines in the opening book is a fair
>approach, it may morally be a matter of opinion. In practical terms what I
>find not arguable is that "killer books" give a false idea of the real
>strength of a chess engine.
If strength is measured in some other way than by winning, yes. A
wrestler may be physically stronger than a judo expert, but if the
wrestler is lying down when the match is over, there's no question
about who has won.
Tim Mirabile
Komputer Korner <kor...@netcom.ca> wrote:
Hah! I knew you'd say this. Anyway, unclear or equal positions are good
enough for this purpose. Even if you believe that White scores 56% from
those unclear positions, the fact that each program gets a shot at both
sides will balance out, and the stronger program should score more than 56%
with White and more than 44% with Black. The only real problem I can see
is if you choose an initial position so critical and deep that the outcome
is determined by who happens to pick the correct first move by pure chance.
+----------------------------------------------------------------------+
| Tim Mirabile <t...@mail.htp.com> http://www.webcom.com/timm/ |
| TimM on FICS - telnet://fics.onenet.net:5000/ PGP Key ID: B7CE30D1 |
+----------------------------------------------------------------------+
Urban Koistinen
Ed Schröder (rebc...@xs4all.nl) wrote:
ES: Correct, and if you remove these doubles (won, draw or lost) you will
ES: exclude that part, INCLUDING the cooks! Remember a cooked line can only
ES: gain just *ONE* win and not *FIVE* , *SEVEN* or more.
If the SSDF adopted a policy of removing double games, don't you
think a program using cooks would add some randomness to its play
once the game is easily won to avoid having the game removed?
Alexander Fuchs
Eric Hallsworth <er...@elhchess.demon.co.uk> schrieb im Beitrag
<wEsZGAAR...@elhchess.demon.co.uk>...
> In article <5450ep$c...@lex.zippo.com>, Graham Laight <?@?> writes
> >Eric Hallsworth's list should, in theory, work against programs with
cooked
> >books, because it also takes games against humans into account. Cooked
books
> >would be expected to get worse results against humans.
> >
>
> In fact MCP5 did edge into top place for a couple of months, but is now
> back in 4th, though there are only 13 Elo points between the top 5 in my
> last Issue..... which was before the Rebel8 results started coming in!
>
Is this list available on the net ?
What's the difference in approach between your list and the Swedish one ?
Thanks for your reply!
Alexander Fuchs
Bernd Herbst
=============================================================================
> Weitergeleitet durch Bernd Herbst (1001:1/2)
> Brett : "PRIVAT"
> Von : Harald Faber, 64:1/1000.11@chessnet (Freitag, den 18. Oktober 1996
00:19)
> An : Bernd Herbst
> Betr. : Re: The MCHESS5 computer killer book...
=============================================================================
BH> From: "Ed Schr/der" <rebc...@xs4all.nl>
BH> Subject: The MCHESS5 computer killer book...
BH> Organization: Schroder BV
BH>
BH> The MCHESS5 computer killer book...
BH>
BH> Here are the results against:
BH> Mchess5 - Genius3 (currently no. 1 on SSDF ELO 2420) 7.5 - 2.5
BH> Mchess5 - Rebel6 (currently no. 3 on SSDF ELO 2415) 13.0 - 1.0
BH> Mchess5 - Hiarcs3 (currently no. 9 on SSDF ELO 2380) 19.0 - 0.0
BH>
BH> According to the HIGH ratings of Genius3, Rebel6 and Hiarcs3 these
BH> results are IMPOSSIBLE in normal play (without book traps)
Total agreement!!!
BH> Mchess5 - Rebel6 13 - 1 Also unacceptable
Unbelievable!
But clever..
BH> Coming to the GOAL of this posting:
BH>
BH> - Is this the future of computer chess?
Seems to be.. :-(((
BH> - Spending months of our time on cooked books to get a good rating on
BH> SSDF?
Well you can even sell more programs when you lead the SSDF... ;-(
BH> - Should the programmers of Genius, Hiarcs and Rebel do the same?
I guess you are to, if you do not want to be disadvantaged any longer.
In the german chessnet we are not able to confirm MCP5 any kind of
strength like Rebel or Genius.
BH> I obvious prefer to spend my time on improving the chess engine of Rebel
Of course this is very senseful, but opening also is chess. Remember tests
in the german Computer Schach und Spiele, where testers experiment with a
book converter. One result was, that with the well-done Genius-Tournament-
book Fritz3 defeats Genius (with tournament-book) more often (with the
converted Genius-tournament-book) than without.
Take a look at the final standings of the Welser tournament (see Computer
Schach und SPiele 2/1996). There were all the programs on Pentium100 with
16MB RAM, *using NO opening book*!
Final standings with 2h/60moves:
1.Fritz3!
2.Quest3
3.Rebel7!
4.MChessPro4!!!
5.Kalisto 1.98
6.Genius2!
7.Genius3
8.Hiarcs4
9.WChess 1.03
10.MChessPro5!!!!!
11.Rebel6
12.Hiarcs3
13.Gandalf 2.1
14.Diogenes 2.10
Interesting that MCP4 is higher placed than MCP5, isn't it? :-)
BH> rather than spending months of my time looking for weak points in other
BH> chess programs and add total won lines to the Rebel opening book!
Well neither you nor MArty Hirsch himself will do this, he has got his
"fellows" to do that. ;-)
BH> Personally I find this behavior disgusting since it hides the truth of the
BH> real playing strength of a chess program.
Yes and no. What about a human tournament, where one player beats the
other with good openings? He is clever and in my eyes he earns the wins,
because he is a clever player. I guess in championship the players of
higher leagues even do have deeper understandings and knowledge of
openings which lead them to a win against other players who do not play
that high. So why should he not use his knowledge? Supposing a worse
player finds an opening variation that lets him win against a higher-rated
player, why shouldn't he play that variation? Of course he will only win
once in that way, so I ask myself, is it not possible for computer
programs to learn as a human would? Of course there are functions like
that (I can't remember if Rebel implements this), but do they work
correctly?
BH> But I really wonder if I have any choice left!
Look above, I think you must do the same not to be in disadvantage
although it is not necessairy yet for you because the new SSDF-list
probably will place Rebel8 on top with more than 2480 points
One reason for that can be, and that excites me very often, is that Rebel
again and again finds holes in Genius-Tournament- and Grandmaster-books.
E.g I got one game where Genius3 left the tournament-book in the 7th move,
in move 21 Rebel7 announces a mate in 8! Although he also got early out of
book. Now still going but obviously lost is an actual game where Rebel8
(P90, 28MB HT) kicks Genius4 (486/133 16MB HT, Grandmasterbooks) to an
evaluation of -5.69 in move 16!! Without any opening-book-tricks. If you
want me to I can post you these 2 games..
BH> What to do?
I would say wait, because the new list will appear shortly. If the killer-
books would be that successful, Rebel would never achieve the first rank.
Wait and see, especially what the new MCP6 will do in the opening against
other strong programs and in SSDF. But prepare to do the same with your
Rebel-opening-book, sad it is...
BH> Comments are *VERY* welcome because I want to know what you all think
BH> about this subject.
OK, my opinion is written down...
BH> I mean if nobody really cares why should I care any longer?
What do you care?
BH> Just confused and worried.
Easy to understand your thinking.
Harald...... der Einzigartige, ok ok, mehr einzig als artig.... :-)
=============================================================================
Hallo Ed,
an opinion from a Friend !
Mit grinsendem Freundlich ;-)=)
Bernd InterNet: bhe...@shadow.franken.de
... Faellt der Bauer tot vom Traktor, steht in der Naehe ein Reaktor.
--
Bernd Herbst
Total agreement!!!
Unbelievable!
But clever..
Seems to be.. :-(((
Final standings with 2h/60moves:
1.Fritz3!
2.Quest3
3.Rebel7!
4.MChessPro4!!!
10.MChessPro5!!!!!
BH> What to do?
What do you care?
Hallo Ed,
A Opinion from a Friend !
Answer over me ..
Mit grinsendem Freundlich ;-)=)
Bernd InterNet: bhe...@shadow.franken.de
... Lieber hier ein logIN als anderswo knockOUT!
--
Don Getkey
In article <3267B9...@bnr.ca>, Philippe Beaudoin <phil...@bnr.ca>
writes:
>And be sure I wont buy MChess, even if I was considering it. Just an
>advice to MChess people: take 2-3 maketing lessons, your posts in this
>group haven't been very attractive to potential customers!
>
> Philippe Beaudoin
>
>
Dear Mr.Beaudoin ,
I found everything that Mr.Hirsch or his partner Mr.Necchi said, to
be fairly straight forward and without apology. They seem to have made a
rare attempt to dislodge some of the mis-information about MCP5, and they
way it was designed. I respect this effort, not revile it.
As the owner of 13 different types of chess programs, MCP5 being one
of them, I assure you I have been completely satisfied with it's playing
strength/style. It remains the most entertaining and strongest chess
program I own. My thanks to Mr.Hirsch, for he has created the standard
(in many ways) by which all others are measured.
The only thing that surprises me in all this, is why Mr.Hirsch and
his MCP5 have not been under more severe scrutiny sooner? After all, when
your the "Top Dog" or "King of the Hill", you become an temptingly easy
target.
yours in chess,
Don
Ramsey MN USA
Garby Leon
In <3267D4...@netcom.ca> Komputer Korner <kor...@netcom.ca> writes:
>
>Garby Leon wrote:
>>
>> In <546vqv$l...@juniper.cis.uab.edu> hy...@crafty.cis.uab.edu (Robert
>> Hyatt) writes:
>> >
>> >Garby Leon (ga...@ix.netcom.com) wrote:
>> >:
>> >: Plus, if the openings skewed the results in any way (say one created
>> >: a consistent dead win for any computer that played black), it would
>> >: be obvious immediately.
>> >:
>> >: regards,
>> >:
>> >: -- garb
>>
>> >Wouldn't even be a problem, because both programs would get the black
>> >side of that opening and if the game is really busted, each would get
>> >one point, which wouldn't bias anything too much...
>>
>> Right - but it would show the position was faulty in that it didn't
>> give the programs a chance to 'fight.'
>>
>> Ideally, of course, you'd want startup positions where the stronger
>> program would have a chance to win both sides.
>
>Who would pick those equal positions that have lots of chances for
>both sides? Informator doesn't have any symbol for that. You would
>have to hire a GM specially for this.
>Komputer Korner
>The komputer that couldn't kompute the square root of
>36^n.
I didn't say "lots of chances for both sides" -- I'm assuming the
stronger program can win even if its chances are = . The danger is a
given 'random' position will give a hidden positional (or tactical)
advantage for one side, preventing a real contest.
Chris Whittington
"Ed Schröder" <rebc...@xs4all.nl> wrote:
>
> Dear Marty,
>
Guys, guys, guys.
We know the Christmas release and sales season is upon us.
To paraphrase Clausevitz: 'war on rgcc is an extension of the
SSDF list by other means'.
The blame war you're fighting says (not my words, or thoughts), so
far:
1. Marty is a cheat because he's cooking his books.
2. SSDF is not valid because results are skewed.
I'll add:
3. Hiarcs3, Rebel6, Genius3 are to blame because they
released with opening books with too little variance
a) because they wanted to get into particular lines, because they
thought these lines were good for their program.
b) because they didn't invest sufficient resources into their book
development.
c) because it worked at the time.
Now these programs have been cooked, they complain.
Actually the above 'blame' statement is about as ridiculous as the
first two.
Nobody is doing anything wrong.
Simply what has happened is that the *interaction* between various
people doing nothing wrong has generated a minor 'systems' failure.
Its no good blaming individuals for this.
And its not exactly a catastrophic failure.
Solution:
1. Create more variance in opening books. Fritz for example
is impossible to cook, because the opening variance is so high.
2. Add an on-line learning function to each program, to avoid losses
and promote wins.
3. Play lots more games on the SSDF to average out the results. (If
Mchess keeps beating Hiarcs3, then Hiarcs3 grade will fall also, this
gives less benefit to keep beating it)
4. New proposal to Goran:
Everybody thinks SSDF grades are too *high*. Eg the top results
suggest progams play at or near GM strength, when we know they don't.
We know that a small change to a program can result in a sudden
and dramatic increase in wins against other programs. This is not
because the 'new' program is that much stronger, but for some
other difficult to understand or explain reasons.
I think that SSDF is using some variant of the 400 or 350 +
opponent grade for a win, opposite for a loss, type of grading
system.
I think this generates too *wide* a range of grades. Too low for the
very weak programs, too high for the strong ones.
I think that, for a *computer* grading list there is an argument
for reducing the win/loss adjustment. Maybe even halving it.
Chris Whittington
Ed Schröder
:Ed, even if you get the commercial programmers to all agree, no more
:cooked books, there is still the big problem of all the amateur
:programmer programs into his program a special feature to waste
:it is still in it's cooked book.AAAAAAAggggghhhhh!!!!!! Hidden
:camouflaged cooked killer books!!!! Oh Oh, I think "I" have just
:created a monster.
:The komputer that couldn't kompute the square root of 36^n.
You know my solution to "cooked books" on Computer tournaments?
I simply not go, problem is solved :)
On SSDF only commercial available programs are tested.
Amateur problem also solved.
Your monster idea...
Yes this is very easy to program, however it will become known, people
are too smart for that. Chess programmers are supposed to be smart
too and will understand that, and not do it.
All three problems solved! :)
- Ed -
Ed Schröder
From: hy...@crafty.cis.uab.edu (Robert Hyatt)
:This is a problem anyway. I haven't seen any of the programs that are
:rated at >2400 ("Elo") perform at that level in real games against real
:FIDE IM's and GM's... So from this perspective, the issue is not so
:important because you'll be disappointed killer books or not. A preview
:was posted this morning with Rebel 8 at 2475 or so on the next list.
Each year we have the Aegon tournament.
1994 Rebel got a TPR of 2460 or so
1995 Rebel got a TPR of 2470 or so
1996 Rebel got a TPR of 2530 or so
These TPR's come pretty close to SSDF ratings.
Moreover a few other chess programs did even better than Rebel
both in 1995 and 1996!
Bob, this is 1996, times are changing.
:I have a lot of respect for all the commercial programs, but have not
:seen a one that I think really plays at nearly a FIDE 2500 level. They
:simply don't "know enough." They play tactically wonderful, positionally
:o.k., but long-range planning is the pits, yet that's exactly what a 2500
:player excels at, "where do I want my pieces and pawns 20 moves from
now?"
:Not "where do I want my pieces *now*?" and then two moves later ask the
:same question and get a slightly different answer. :)
I am not underestimating IM's and GM's at all but the fact is that
even GM's have a hard time at Aegon. Speaking for Rebel at Aegon 1996,
Rebel had to face 3 grandmasters and one IM.
Rebel scored 50% against the 3 grandmasters (1 win, 1 draw, 1 lost)
Rebel won from the IM.
And what about the scores of Nimzo and Quest on Aegon 1996?
I think you underestimate today's commercial chess program and maybe
your own Crafty too?
:In any case, take the SSDF as computer vs computer performance and
nothing
:more. The programs at the top will do better against humans than the
ones
:at the bottom because they are way stronger... however, whether #1 is
:better against Kasparov than #2 is anybody's guess...
If I compare the AEGON ELO results with SSDF ELO results things are quite
in balance. I do not see the problem. Also other human <> computer
tournaments or matches justify the SSDF ratings sofar.
> BTW: By avoiding the opening traps of the MChess 5 book and using a few
> cooked lines specifically against MChess itself, Rebel 8 will be very
> likely the top program on the next list by a margin of maybe 50-60 SSDF/ELO
> point (just to prove the point). At least the intermediate results posted
> here seem to suggest this. This shows that Ed could play the game very well
> if he wanted, he could even post monthly killer updates on his web page
> where he currently provides outstanding customer support.
Monthly killer updates on my home page???? **** NEVER ****
Not even if people will pay a $1000 for it!
- Ed Schroder -
Robert Hyatt
Alexander Fuchs (w3fu...@fsrz2.rz.uni-passau.de) wrote:
:
:
: Philippe Beaudoin <phil...@bnr.ca> schrieb im Beitrag
: <3267B9...@bnr.ca>...
: > Albert Silver wrote:
: > > I'd also like to point out that this strongly influenced my decision a
: > > month ago when I decided to purchase two programs and chose Genius 4.0
: > > and Rebel 8.0.
: > And be sure I wont buy MChess, even if I was considering it.
:
: Yes, that's probably the only way we can get rid of the killer books. Don't
: buy the program(s) that have these killer books. Although I like M-chess, I
: don't think I am going to buy version 6.
:
: Alexander Fuchs
I still think you are not buying the right program for the wrong reason.
(get that? not sure I did... :) )
If I were shopping for a program, I'd try to play them all, and pick the
one I liked best. You personally will never know if Mchess Pro has a
"killer book line" or not most likely, since you aren't targeted by such
a line. Just ignore all the hyperbole, and pick what you like best. If
you like MCP's interface, or style of play, or whatever, that's what would
make my mind up, not whether he's cooked somebody else's book or not, because
it wouldn't directly affect me or my enjoyment of that program at all.