Leela's pv score

[checkersp...@gmail.com]

Hey, I was watching LeelaZero playing against (weak) stockfish at https://www.twitch.tv/y_sensei . I was wondering how the score next to Leela's pv is generated since it uses a neural network and no static eval ?

[lilmafya]

Interesting question... is there any answer to that?

[Karlson Pfannschmidt]

Here you go (from UCTSearch.cpp):

// UCI-like output wants a depth and a cp.

// convert winrate to a cp estimate ... assume winrate = 1 / (1 + exp(-cp / 650))

// (650 can be tuned to have an output more or less matching e.g. SF, once both have similar strength)

int cp = -650 * log(1 / feval - 1);

[luis....@gmail.com]

Why not just output the winrate? It doesn't need to be a percentage. You can output a number between 0 and 1.

I think it's a better metric than the materialistic centipawn evaluation. Why shoehorn Leela in traditional engines parameters?

[Karlson Pfannschmidt]

The winrate is output as well. The cp is displayed by GUIs for example to visualize the eval of an engine over time.

[luis....@gmail.com]

But cp doesn't make sense with Leela. It doesn't "think" that way.

[Karlson Pfannschmidt]

Even modern engines like Stockfish do not only think in terms of material. Usually the complicated evaluation function is calibrated in order to output values compatible with classic centipawns.

Think of it this way: We humans like to think of evaluating positions in terms of centipawns. How can we convert the output of any evaluation function (be it a complicated neural network which outputs probabilities or any other real-valued output of a program) to a value in centipawns which behaves as we would expect it to (comparing a position with a difference in material should also show a proportional decrease in the cp value).

[luis....@gmail.com]

I understand. I'm just saying that I don't see any particular value in keeping centipawn output. Winning probability seems much more natural to me. If anything, I would replace centipawns with winning probability in traditional engines as well.

[Ignacio Urrutia]

I absolutely agree. 0.5 pawns at the beginning do not mean the same as 0.5 pawns when most pieces have disappeared. Sometimes these 0.5 pawns have draw probability of 100% at high level of play, whereas 0.5 pawns at the beginning have a totally different probability distribution of W/D/L.

Current dynamic view of chess positions is less materialistic than back then, when values of pieces were estimated, or when engine design tried to find equivalences between positional advantages and material. 

Estimated result makes a lot more sense. The simplest way would be estimated game result (0= 100% loss, 1 = 100% win), but intermediate results are not accurately described. 0.5 only means equal probability of wins and losses, but says nothing about probability of draw. It's important to differentiate because we want engines to pick the lines with low draw probability, provided that loss probability doesn't grow faster than win probability with that choice. In case of doubt, go for the least drawish line.