Poor endgame

[Joules Kin]

When alphago games were being reviewed by professional Go players, one comment that stood out was that alphago always played a suboptimal endgame. It's like, the AI sees a winning position and starts to slack off. Well, I think something similar is happening to leela too.

I got leela (ID 101) to play giraffe with time odds. Leela had 16 minutes, giraffe had 8. She still doesn't always win games in these matches either, but here's an example of where she did:

https://lichess.org/oUtfKcdL

The last 30 moves or so are the most garbage endgame I've ever seen. Stockfish could find like 20 or so inaccuracies starting from move 74. Leela had a mate in four, and decided to ignore that and queen another pawn.

[asterix aster]

This slacking behaviour is normal for leela zero. It stems from the fact that the network does not take into consideration  the number of moves it takes to win as long as it wins in the end.

[Hurtfew]

I had Leela playing game in 1 minute against an old engine called Der Bringer.  Bringer won 8 games lost 0 drew 2.  One of Leela's draws was a game where Leela had a rook, king and bishop versus king and Leela couldn't find a win...  At some point Leela needs to learn the concept of making progress.  

[evalon32]

How many playouts per move did Leela manage at that time control?

I've been tracking for a while how good Leela is at basic checkmates, and by my reckoning, the recent networks can find a mate with K+R v K against Stockfish

• 100% of the time with 800 playouts
• 90-100% of the time with 100 playouts
• 70-85% of the time with 10 playouts

On Saturday, April 7, 2018 at 10:11:30 PM UTC-4, Hurtfew wrote:

I had Leela playing game in 1 minute against an old engine called Der Bringer.  Bringer won 8 games lost 0 drew 2.  One of Leela's draws was a game where Leela had a rook, king and bishop versus king and Leela couldn't find a win...  At some point Leela needs to learn the concept of making progress.  

On Saturday, April 7, 2018 at 9:06:44 PM UTC-4, asterix aster wrote:

This slacking behaviour is normal for leela zero. It stems from the fact that the network does not take into consideration  the number of moves it takes to win as long as it wins in the end.

And yes, it is similar to the phenomenon in Go. AlphaGo (and MCTS-based programs in general, unless they use dynamic komi) doesn't care about the point margin as long as it wins. LC0 doesn't care about the number of moves.

[Hurtfew]

I did not change the playouts.  I believe the default is 800.  What happened was it repeated the position 3 times and got a draw.

[evalon32]

800 is indeed the default, but that's only used when there's no time limit. You said it was a game in 1 minute. The number of playouts would be whatever your hardware can handle in the time Leela has to move.

[Hurtfew]

Where would that be reported in Arena.  Is that the value in the nps field?

[evalon32]

What you want is simply "nodes" - not sure if Arena displays that. But if you know the time per move, you can divide nps by that and get nodes.

[Hurtfew]

Now I've upped the threads for Leela to 6 and set playouts=3200 and changed to 5min + 5sec and the screen looks like the attached image.  When I was playing 1 min per game Leela only had 2 threads, (Bringer is only ever capable of using 1 thread).

[Hurtfew]

Here is Leela about to lose to mate in 3.  It sees the mate in 3 in its PV but still the eval doesn't indicate the mate.

[Hurtfew]

Okay, I understand what is going on.  The -p "playouts" setting needs to be much higher for faster computers and/or longer time controls.  I did some testing with infinite analysis and it appears that Leela just stops calculating when it the total nodes gets to the playouts setting value.  So if the setting is -p 3200, as soon as you get to 3k on the total nodes it stops calculating.  It literally just stops.  I set it to -p 1000 in infinite mode and it stops as soon as the total nodes searched field gets to 1k.  So on a fast computer it should be set MUCH higher.  My computer gets to 1k in a fraction of a second so I set -p 128000000.  I don't think it's going to hit that even in an endgame at long time control.  It's playing much stronger now.

Thanks evalon32 for the clarification.

[Hurtfew]

I think I see an issue with the playouts counting.  What I'm seeing is that the playouts starts counting at zero each move EXCEPT when Leela's opponent plays the move that Leela is expecting it to play; in that case the playouts count does not start at zero.  Here's an example:

If Leela calculates a total of 50k playouts and determines that it should play Rb6 and expects the opponent to play Kc7 and the opponent does NOT play Kc7 then on Leela's next move it starts counting the playouts at 0, but if the opponent DOES make the move Kc7, when Leela starts calculating on it's next move it doesn't start counting at 0, it starts counting at something like 40k.  If there is a sequence where Leela keeps guessing the opponent's move correctly, the playouts count gets pretty high, (I saw a sequence where it got to 989k).

This seems like a major issue when you combine it with the fact that if it exceeds the -p setting it will stop calculating.  It seems to me that this would happen a lot during self play training since the likelihood for guessing the opponent's move is much higher and the -p setting is pretty low, (-p 1600).  This could also explain poor endgame play.

[kostuek]

You do not need the -p setting if time controls are in use, only use one or the other, since a mix may cause weird behavior. With -p Leela stops the calculating after given playouts, with time controls calculation will go on, as long as there is time. Also, you should use -noponder setting to get a clean behavior. Additionally I set -t 1 to avoid any thread related bugs that may be in place.