You might also want to filter out any positions where either side has the option of a forcing or near forcing 3-fold repetition.
I hope that you (Dariusz) do not have any objections to me duplicating your work but with a different method. This would verify pgn scanner's output. 8moves_GM is not the first large pgn of unique starting positions I have created, but it is the first one that I used engine evaluations to remove positions. I would like to improve the process so that I can create a better set of starting positions.
To be honest, I have no interest in fighting over what positions are used by the Stockfish testing framework. If it was decided that the 8moves_GM pgn was not good enough for the framework, it would not bother me. My true interest is in creating the best set of openings that can be made so that I can use them in the Gaviota framework. And then happily share them with anybody else who thought the positions would be useful to them.
Anyway, I hope that you understand that I am not upset or annoyed or anything else about the pgn being modified. I do not necessarily agree about the bounds being used to filter the positions, but I do hope the end result of your work does improve the efficiency of the testing.
I have difficulty to believe that there can be so much crap in the opening book we've been using so far. Yet it is true: these GM really don't know how to play chess ;-)
I am redoing Darius analysis at the moment (50cp threshold), and so far, my results agree:
- 1040 FEN tested out of 48491
- already 319 bad ones: that's just over 21%
Some of it can be explained by Houdini showing lower scores than Stockfish, but certainly not all of it.
Once I've finished, I'll get a second opinion from Critter. That was, I'll only exclude the positions where both Stockfish and Critter show a score outside [-50,+50], which will hopefully remove many of the inflated SF scores.
I also wonder if any of the positions past 1,000 ever get used at all. Do we know that the interface chooses at random or in sequence? As every computer is given a task of 1,000 games, if they run the positions in sequence, it is the same 1,000 positions and only those 1,000 positions.
If that is the case, more attention to the first 1,000 would make sense as would shrinking it to 1,000 positions to reduce unnecessary data movement.
The openings to keep are therefore:
On 8moves_v3.pgn, here are the current results I got:
Total games : 34700 games
- Pgnscanner 0.88 : 0 game
- Linux script : 0 game
Just to inform you that pgnscanner 0.90 is on line at http://www.zeitnot.fr/index.php/pgnscanner/5-pgnscanner-eng.
I mainly focused on doubles detection improvments (dbl -abs) : speed, reliablility and stability.
see you :)
The functionality you describe is not a piece of cake. However, Pgnscanner already has its own book generator that gives statistical informations in a Fritz's book manner (see "newbk /?", "openbk /?" and "showbk /?" commands). Thereby, it's possible to calculate and give an elo to your engine (givelo command) then create a pgnscanner's book and navigate through it in order to get interesting statistics. But okay, searching for thousands given positions could take a very long time this way.
Perhaps a possible method could be :
1. get only games where Little Blitzer is white
2. calculate and give elo to Little Blitzer (and opponents)
3. build a pgnscanner's book
4. with a pgnscanner command that doesn't exist yet, extract positions from the book that are not good enough according to statistical datas.
5. repeat the process with Little Blitzer as black
If people are interested in this kind of functionnality, I think I could consider it in a few weeks without too much overload.
See you :)
I'm currently implementing a new command (vfilt) for Pgnscanner which automatically filters games according to the evaluation given by an external engine. In my current testings, I'm running stockfish-DD for 1 sec on each last move of the 8moves_v3.pgn file. I set a acceptance interval of [-80 / 80]. At this moment, it ran on 18000 of 34700 games and found 87 games outside of the acceptance interval. Most of them are just near the bounds (eval ~ 80-90), but a few seem to be significantly worse and should be verified for more than 1 sec.
For example :
[Annotator "game 10181, depth=16 val=307 filter=[-80/80] status=rejected"]
1.e4 e5 2.Bc4 Nf6 3.d3 c6 4.Nf3 Be7 5.O-O d6 6.Re1 O-O 7.Nbd2 Re8 8.Bb3 Nbd7 1/2-1/2
[Annotator "game 12225, depth=13 val=147 filter=[-80/80] status=rejected"]
1.d4 Nf6 2.Nf3 d5 3.c4 c6 4.e3 Bf5 5.cxd5 cxd5 6.Nc3 e6 7.Ne5 Nc6 8.g4 Bg6 1/2-1/2
[Annotator "game 11725, depth=17 val=141 filter=[-80/80] status=rejected"]
1.e4 c5 2.Nf3 e6 3.d4 cxd4 4.Nxd4 Nf6 5.Nc3 Bb4 6.e5 Ne4 7.Qg4 Nxc3 8.Qxg7 Rf8 1/2-1/2
Let me know if this function has an interest for the fishcooking project.
Just to inform you that pgnscanner 0.91 is out at http://www.zeitnot.fr/index.php/pgnscanner/5-pgnscanner-eng
A few new commands have been implemented.
The 2 following commands might interest you for SF book's building :
- fenstat command in order to calculate statistical results of games starting from a specified position.
- vfilt command in order to automatically evaluate and filter openings (e.g 8moves_vX.pgn) by an engine such as stockfish.
Thanks to Mindbreaker for his help.
I suppose your request is not related to the Stockfish project.
So with respect to the topic of this thread, I will directly answer you by mail :-)