Low draw rate in training games/data.

[Joseph Ellis]

Currently in training games LCZ has been maintaining a draw rate of ~9%. In recent match games, it is averaging ~33%, and that is with sometime substantial Elo differences between the nets. It would not be unreasonable to estimate a draw rate of ~34% under match conditions for self-play (identical nets).

This means that currently, for our training data, 25% of the total games (1 out of every 4) which would normally have ended in a draw under “best play” according to LCZ, are instead decisive due to a constant temperature of 1.

While I was not sure if producing an incorrect/false result in (minimally) 25% of the training games would be detrimental or not, I became more concerned when considering the numbers exclusively for draws.

Based on the empirical evidence of the match play games, we know that left to its own devices LCZ will draw ~34% of the games against itself. The best case scenario is that no wins are being thrown away in training games (unlikely), which leaves the genuine draw rate at the full 9% for training games. This means that the “normal” result for drawish positions is only achieved 26.4% of the time, and that the expected result is being lost to temperature 73.6% of the time. And those are best case numbers assuming that no part of the 9% of draws is made up of otherwise decisive games decaying into draws due to temperature blunders. 

I understand that the role temperature plays is a vital one. LCZ needs the variety and the ability to explore new positions it provides. However, I am a bit concerned that constant temperature of 1 is a bit too effective. Perhaps there is a way to provide the variety and exploration LCZ needs without significantly altering the normal statistical distribution of the games?

[jkiliani]

The obvious solution to the phenomenon you describe would be to also apply a temperature decay schedule to training games instead of just match games. However I'd be very careful about that change since it just might be that training data needs a very lopsided distribution of decisive to drawn games for the value training targets to properly work.

[pikipi]

Perhaps you want to bring this in the discrod dev channel?

[Joseph Ellis]

Thinking about this some more... if similar changes in the outcome occur for decisive games as well, then over 50% (possibly up to 67%) of the total games generated are coming to an end other than their "natural" conclusion.

Maybe this just doesn't matter that much for learning purposes, but I don't think it can be optimal, or really even a good thing.

[Philippe Wakson]

Well, as we have seen from leela ID125 on TCEC, her low draw rate is mostly due to it's fear of drawing. How many times we've seen it, as the 50 moves rule approach, advancing that pawn and lose the game rather than stalemating and drawing. Without all these very sad losses Leela would gain an auto +200 elo or something.

But the question is why? Why does it fear that draw?

[Thanar]

Simply because it thinks it has an advantage in the position, so it plays on for a win (like other engines would). But Leela's misunderstanding of those endgames is so bad, that it still thinks it has an advantage after sacrificing perhaps a pawn or a piece, so much so, that it gets into a losing position before realizing it, and by then it is too late to go for a draw, so it loses.

[Thanar]

The pawn/piece sac in those positions is done only when the 50 move rule would cause a draw, but it gets reset to 50 by the capture which keeps Leela's eval positive, since the 50 move rule draw (or the fact that the position is now objectively lost) is again beyond Leela's search horizon.

[Philippe Wakson]

The fact that the NN still evaluates this position poorly is the big question. Since it's supposed to draw around 34% of the time but only does 9% of the time, it suggests that more than 25% of her matches (which is over a million games) went there and had the opportunity to reevaluate this pawn advance. Yet Leela keeps on advancing that pawn. Because a NN needs WAY less than a million times the same pattern to weight it accordingly, there must be some sort of bug somewhere, preventing leela from drawing these games

On Saturday, April 14, 2018 at 4:28:23 PM UTC-4, Joseph Ellis wrote:

[Philippe Wakson]

There must be some weight that gives her a better fitness if she keeps on playing the game rather than drawing, regardless of if it's losing position afterwards

On Saturday, April 14, 2018 at 4:28:23 PM UTC-4, Joseph Ellis wrote:

[Philippe Wakson]

Well, since v0.7 who fixed some critical training bug the draw rate is going up at a constant rate. Boom!!

On Saturday, April 14, 2018 at 4:28:23 PM UTC-4, Joseph Ellis wrote:

[Warren D Smith]

I recommend auto-adjusting temperature to try to keep the draw rate at (say) 33%.  

If goes above that, make it hotter, if below make it colder.

If C0 always draws then it will not be able to learn anything from

game-result and the learning curve will hit a ceiling, which indeed happned

for the deepmind paper's curve.  If it never draws it will risk being too deluded

about the nature of chess.