Leela: What is it and what are its goals?

[David Larson]

A few days ago I was talking to someone about Leela and was asked, "What's the goal of the project?: I admit I could not answer this question.

From reading these forums it appears that people think the goals are:

• To duplicate the results of Alpha Zero.
• To complete with current and future programs.
  

However when I tried looking for anything that would indicate this I was unable to find anything.

1) http://lczero.org/ 

     Does not state what the project is. The top link is to "Getting Started" but there's nothing on the main page that indicates why a person that stumbles on the site should care.

2) "Getting Started" goes to https://github.com/glinscott/leela-chess/wiki  

     This page has "Moved to https://github.com/LeelaChessZero/lc0/wiki" which is not the best user experience.

3) GitHub page: https://github.com/LeelaChessZero/lc0/wiki

     Also does not state what the project is or why a person new to Leela should care to explore further. The top of the list has the FAQ. 

4) FAQ: https://github.com/LeelaChessZero/lc0/wiki/FAQ

     The FAQ does tell a person what the project is or what it's goals are.

Maybe they are written down somewhere that I couldn't find. If the goals are the two that I listed then that leads to further questions and clarification: How will it be determined if the goals are reached? Once the top goal is reached does that mean the work shifts to trying to achieve the secondary goals?

I think clarifying this would help address much of the arguments and posturing that occurs in this forum. Most of the time the argument comes down to one person or group assuming that the goals of the project are one thing with others assume it is something else. Clarifying and publicly presenting the goals in and easy to find location would also help people understand why they should contribute to the project.

[Graham Jones]

You're right, for a newcomer, things are very unclear. There is something written down here:

https://github.com/LeelaChessZero/lczero

I 've pasted it below. The chance of a newcomer finding this is tiny, and it's not written for newcomers.

You said: "I think clarifying this would help address much of the arguments and posturing that occurs in this forum." 

Optimist!

Introduction

This is an adaptation of GCP's Leela Zero repository to chess, using Stockfish's position representation and move generation. (No heuristics or prior knowledge are carried over from Stockfish.)

The goal is to build a strong UCT chess AI following the same type of techniques as AlphaZero, as described in Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm.

We will need to do this with a distributed project, as it requires a huge amount of computations.

Please visit the LCZero forum to discuss: https://groups.google.com/forum/#!forum/lczero, or the github issues.

[John D]

it's a very good question, my impression has been that it's along the lines of those stated, but lately i believe the goal is to surpass A0. that's just an impression tho, the best place to interact with devs is now actually the discord channel. they literally don't use this forum afaik.

[Ghost]

To replicate success of A0 against SF

[David Larson]

The goal is to build a strong UCT chess AI following the same type of techniques as AlphaZero…

How is this measured?

[Jesse Jordache]

It's a peer review of the Deep Mind proof of concept, as well as a means of satisfying anyone who said "gee, it would be nice if we could have an A0 to play with".

The blog was supposed to clarify issues like this as well as make things from the dev side clear to the contributing audience, but the current blog editor just likes to link to other people's videos on YouTube and call it a day.

[Matt Blakely]

The scientific goal is to build the strongest chess entity in the world (I hope at least)

But there is also a chess goal, to some like me at least, which is to create beautiful chess games of the highest level.  Leela games are often beautiful in ways traditional engines are not.  Human chess players can learn way more from Leela than traditional engines IMO and she has the potential to change the game completely.  

After Leela becomes #1 expect not only copycats but programs like stockfish to try and replicate her ideas and strategies.  

[josé luis]

For me also to have an AI capable of rank chessplayers better than elo..to understand more the greatest chessplayers ever..there have been various research but with AB engines and it was quite good but some players maybe weren t fully understood..with leela its going to be a greater analysis..the same way the best games ever..

[Graham Jones]

Leela' performance against Stockfish 8 compared to AlphaZero's performance against Stockfish 8 under similar conditions to those described in the Deepmind paper.

[Alexander Lyashuk]

In short, the goal of the project is first replicate AlphaZero, and then make it as strong as possible.

The main reason of "replicate AlphaZero" part is actually not to test reproducibility in scientific sense, that's more like a nice side effect.

The main reason is that there were many attempts to create NN engine before, and they failed, while AlphaZero succeeded.

There are many possible improvements which really look beneficial if you think about them, but the problem with implementing them is that if we stuck, we cannot longer just compare what we are doing differently and fix that. We do stuck a lot, and in fact there are lots of surprising subtleties which matter (recent example is sampling rate, we used ~0.95 I guess, and it was too much).

So, while we can follow the guide, we do.

After we reach AlphaZero state (or if we stuck before that and won't be able to find any explanation), the goal will surely be to "create as strong chess engine as possible".

I expect that there will be some zero vs non-zero debate, which may result in two different forks, but I expect non-zero to get much more attention in the end because it will be stronger.

--
You received this message because you are subscribed to the Google Groups "LCZero" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lczero+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lczero/3b486a54-2c96-477a-80bf-fdc96c018d9b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jesse Jordache]

"I expect non-zero to get much more attention in the end because it will be stronger."

On what do you base that?

[Alexander Lyashuk]

On the fact that set {"zero" techniques; "non-zero" techniques} is a superset of set {"zero" techniques}.

Strictly speaking, it may happen that "non-zero" techniques will give no additional benefit (in this case "stronger" should be "not weaker"), but it's very likely that there will be at least "non-zero" technique which will still be beneficial.

I don't expect that "non-zero" to be something like human-written eval function, but rather something along the lines of "running alpha-beta blunder check in parallel", "train on tactical puzzles", "train endgames separately", "run alpha-beta with some manual heuristic ahead of mcts search to know which positions to prefetch into NN cache" (this one is actually pretty close to be implemented. That's still "zero", right? It's only performance optimization, right? As MCTS won't do different things, it will just be faster because of prefetched posisions), etc.

To view this discussion on the web visit https://groups.google.com/d/msgid/lczero/9713fd89-4f2f-4994-9670-f79b1ee3f266%40googlegroups.com.

[Curious]

Hi,

I have long felt the same and even asked the exact same questions on discord. The answer there, if you get one, is that the devs are too busy to update lczero.org.

From a general viewpoint, an organization where no one has time to do anything but adress emergencies is a badly run one. Not saying that is the case here, necessarily. The devs appear competent in the coding aspect, but not in the community building aspect. Tweaking parameters and trying stuff is more fun than updating webpages, afterall.

Everything is supposed to get better once lc0 is promoted to main and lczero-client is scrapped. Then the devs have free time again. We will see.

Blog is not really used. Besides dkappe I cant remember seeing another dev on the forum, and i've never seen this forum even mentioned on discord.

[Alexander Lyashuk]

The problem with driving community is that devs only have free time sporadically (like, one weekend once per two week), and building a community is a continuous process.

And naturally when you don't have so much time, then when you do have, you want to spend it on code changes (also because people are worried that nothing is done for weeks), and not answering forums.

If anyone wants to lead community building effort in any way, that would be really welcome.

--
You received this message because you are subscribed to the Google Groups "LCZero" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lczero+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lczero/7b250cac-041b-42fe-91c0-20f79da19190%40googlegroups.com.

[Edward Panek]

Lead, follow or get out of the way! 

[Hasan ÇALIŞIR]

The time to leave from community but you say "you are welcome to lead community" 

Don't you think it is a bit late?

If you don't know let me inform you "There are 2 more NN in TCEC13 which are stronger than Leela..

This project will be dead in 1 month..

26 Tem 2018 Per 00:20 tarihinde Alexander Lyashuk <moos...@gmail.com> şunu yazdı:

To view this discussion on the web visit https://groups.google.com/d/msgid/lczero/CACGjmsL1rzaPByJ7zBPW-Wwrzvr7fPP_xYdjMQn1M0hMC8c_Fw%40mail.gmail.com.

[Edward Panek]

That's quite the assertion. Any evidence to back that up?

[Curious]

From the activity on discord the devs seem to have lots of time :)

This sort of thing is a problem you encounter as you grow. 6 ML-interested people isn't enough anymore. There must a infrastructure person, one on PR-side, maybe a web-dev and maybe more. And those people should also feel like part of the team; right now it feels like "the devs" don't even acknowledge a need for these roles and such it isn't very tempting to invest one's time.

But that is just my opinion. Luckily others are more optimistic.

[Jesse Jordache]

Uh, yes!  I think.  It fits my definition of zero, anyway, which is "anything that doesn't violate the free learning space that improving from random vs random crates.  Anything that doesn't introduce an unwanted seed that positional artifacts will form around".  That isn't terribly concise but it will do.

Not to get tautological, but {"zero" techniques" ∪ harmful "nonzero techniques"} ⊂ {"zero" techniques}  If I was still running linux I'd have bothered to figure out a faster way of doing that than googling "mathematical symbols".