If restricted to a "human number of nodes" to evaluate, who wins? Leela or GM?

Shah

unread,

Feb 1, 2019, 3:33:17 AM2/1/19

to LCZero

I think that would be like comparing Leela's intuition to human.
An intersting comparison I think.
How many nodes a human GM evaluates?
I don't know... But I am sure people in this forum are more knowledgeable and can say.
Another way to put it:
Did anyone caculate Leela's elo for smallest node values?
Say: 1, 5, 10, 20, 50, 100...??
Thanks!

Ingo Weidner

unread,

Feb 1, 2019, 3:40:06 AM2/1/19

to LCZero

The "official" estimated Elo graph includes 3 graphs with 1 node, 10 nodes and 100 nodes tested against Stockfish 9 ("SF 9"):

https://docs.google.com/spreadsheets/d/1Mi2qwPCK4aVNt9B8aC-HecIyLJ0BEAPvp63L59DTrY4/edit#gid=1729484756

Shah

unread,

Feb 1, 2019, 4:12:30 AM2/1/19

to LCZero

Ingo, many thanks, I wasn't aware of that.

Warren D Smith

unread,

Feb 1, 2019, 9:54:13 AM2/1/19

to LCZero

On Friday, February 1, 2019 at 3:40:06 AM UTC-5, Ingo Weidner wrote:

The "official" estimated Elo graph includes 3 graphs with 1 node, 10 nodes and 100 nodes tested against Stockfish 9 ("SF 9"):
https://docs.google.com/spreadsheets/d/1Mi2qwPCK4aVNt9B8aC-HecIyLJ0BEAPvp63L59DTrY4/edit#gid=1729484756

You seriously expect me to believe leela plays at 2300 strength with no lookahead whatsoever?

I have immense trouble believing anything can do that.

vladimir...@gmail.com

unread,

Feb 1, 2019, 10:34:38 AM2/1/19

to LCZero

пятница, 1 февраля 2019 г., 17:54:13 UTC+3 пользователь Warren D Smith написал:

You seriously expect me to believe leela plays at 2300 strength with no lookahead whatsoever?

I have immense trouble believing anything can do that.

Do you believe in go-playing Leela, which is 1-3 Dan level on KGS without search?

Vassilis

unread,

Feb 1, 2019, 11:32:16 AM2/1/19

to LCZero

Yes Vlad, I believe :)

I also play, and love Go (about 10-11 kyu strength. A "patzer", so to speak)

Warren D Smith

unread,

Feb 1, 2019, 11:57:50 AM2/1/19

to LCZero

Actually, I, and a lot of people, had trouble believing that when the deepmind AlphaGo team claimed this, but I cannot

argue with them. I am better at chess than I am as a go player. I think in a goodly fraction of chess games

a position will arise where I will outcalculate no-lookahead leela and hence win, and I don't believe any

feasible evaluation function can avoid this. And I am nowhere near 2300 strength, I'd guess I am 1200 to 2000

depending on speed of play.

So my mind is boggled by the claim leela without lookahead is 2300.

Also, I realize leela is unconventional, and that the following rule of thumb is being pushed beyond reason, but...

for conventional chessprogs a common rule of thumb is each doubling of #nodes searched, brings 50 to 150 more

elo strength. So, if full strength leela is searching 10^6 = 2^20 nodes, then 1-node leela should be 1000-3000 elo weaker.

So is it possible for me to play no-lookahead leela online someplace?

Warren D Smith

unread,

Feb 1, 2019, 12:21:15 PM2/1/19

to LCZero

as another quantification of this:

what if you train leela purely to solve mate-in-N problems without lookahead?

I suspect for N=0 (i.e. just recognizing whether mated or not) and

N=1, you can probably succeed in training a neural net to do that.

For N=2, already I am having trouble believing you can do it.

For N=3, major trouble believing.

Vassilis

unread,

Feb 1, 2019, 12:30:31 PM2/1/19

to LCZero

Warren yes, but go is a different game.
In chess calculation is much more important! I don't say that in go isn't, but go is more about patterns...
When I play Leela Chess on my laptop, zero-depth, I always win!! Even if my position is not better, Leela blunders. (I'm bellow 2000 at any TC)
I've never manage to win Leela Go (with depth=0) though.

Warren D Smith

unread,

Feb 1, 2019, 12:41:27 PM2/1/19

to LCZero

OK, good, you agree with me, and your experience proves that, leela[no search] is NOT 2300 or anywhere near it.

Fine, but for some mysterious reason the "official" estimated Elo graph says leela[1 node] is 2300 --- WHY??:

https://docs.google.com/spreadsheets/d/1Mi2qwPCK4aVNt9B8aC-HecIyLJ0BEAPvp63L59DTrY4/edit#gid=1729484756

Vassilis

unread,

Feb 1, 2019, 12:45:58 PM2/1/19

to LCZero

Frankly, I don't know.
A developer maybe could answer this?

Message has been deleted

Vassilis

unread,

Feb 1, 2019, 12:58:30 PM2/1/19

to LCZero

I can't read the spreadsheet on my tablet, for some reason, but I believe you.
What does Leela[1 node] mean? Is it the same as setting depth=0, or has a different meaning?

Francesco Tommaso

unread,

Feb 1, 2019, 4:21:53 PM2/1/19

to LCZero

I think she might play at 2300 or very close even with only one node. I just made Leela play on Arena restricting her search to only one node and used net 40459. She played against Hedgehog (2100 Elo Engine on CCRL), which had 10 secs per move on a Ryzen 7 2700. She drew one game and lost the other after long endgame. She might win with a test 30 network.

Stuart Cracraft

unread,

Feb 1, 2019, 4:36:00 PM2/1/19

to Francesco Tommaso, LCZero

I disagree with this.

I had Leela play at 1 node and she hung her queen after about 15-20 moves in the early middlegame.

—Stuart

--
You received this message because you are subscribed to the Google Groups "LCZero" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lczero+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lczero/1b7e94d9-5c03-4eae-bb87-532ef9906391%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

—Stuart

P.S. For scheduling time with me:

https://meetme.so/StuartCracraft

For my resume:

http://www.stuartcracraft.com

For my LinkedIn

http://www.linkedin.com/in/stuartcracraft

My email is

smcra...@me.com

Stefan Albrecht

unread,

Feb 1, 2019, 5:57:30 PM2/1/19

to LCZero

,,,we could ask Kingcrusher about this. I guess he is FIDE "Candidate Master" (CM) and about 2.200 elo...

I remeber last year he was playing in person rapid games online at "easy mode" against leela, but I cant remeber if easy mode was 400 nodes/move or a bit more or less.

After constantly losing, he set Leela to 1 node, and after some training, Leela won also against him. The videos that time were very funny, as he is laughing tears while losing the games.

I am now searching for the vids.....

....found one he played on easy mode, here it is.

https://www.youtube.com/watch?v=jHZ76duAgXA

regards, Stefan

Am Freitag, 1. Februar 2019 09:33:17 UTC+1 schrieb Shah:

Edward Panek

unread,

Feb 1, 2019, 6:51:35 PM2/1/19

to LCZero

I love KC...such a good sport.

Message has been deleted

Daniel Rocha

unread,

Feb 1, 2019, 7:00:27 PM2/1/19

to Vassilis, LCZero

Beware of comparing computer ELOs with human ELOs. What is being compared is software, while hardware configurations are more or less abstracted. A program running on a cellphone will be slower than on a top CPU/GPU.

Em sex, 1 de fev de 2019 às 21:54, Vassilis <vasm...@gmail.com> escreveu:

Stephan, this "easy mode" is definitely not 1 node (from Leela's part)
She can see elementary combinations!

--
You received this message because you are subscribed to the Google Groups "LCZero" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lczero+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lczero/ea3f5909-7d25-47d2-a618-f74f45aa4422%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Daniel Rocha - RJ

danie...@gmail.com

Vassilis

unread,

Feb 1, 2019, 7:10:11 PM2/1/19

to LCZero

Sure, but 1 node is 1 node regardless the configuration.
Leela should be of the same strength anywhere

Vassilis

unread,

Feb 1, 2019, 7:17:32 PM2/1/19

to LCZero

Stefan, this "easy mode" is definitely not 1 node (from Leela's part)
She can see elementary tactics!

If KC really lost to Leela in 1 node, it's not because of her training :) he must did something terribly wrong! (Or impulsive??)

Daniel Rocha

unread,

Feb 1, 2019, 7:34:21 PM2/1/19

to Vassilis, LCZero

There was 2600 GM that played with an early version of Leela. We can guess the strength from there.

--
You received this message because you are subscribed to the Google Groups "LCZero" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lczero+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lczero/5da30098-48d3-4636-af72-54a1a4b7f7c3%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Stuart Cracraft

unread,

Feb 1, 2019, 7:36:34 PM2/1/19

to LCZero

At 3000 nodes per second, Leela on an Ubuntu instance on AWS beat my program named “Talish” which is a weakish 2500 grandmaster in a 10 second vs. 10 second match.

The defeat was sound but obviously too low a sample to know more.

Does anyone have scripts, such as bash or python, which can match or front-end Leela in front of its UCI?

Playing or using Leela with UCI is a real bear and a simple script to move based on something simple with ./lc0 running in the background would be useful.

To view this discussion on the web visit https://groups.google.com/d/msgid/lczero/CAPuzWc1e0%2B8UnKxO81B6OdZQ9Pui3s_OC796yrOuYPmZsOTN%2Bg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

Message has been deleted

Vassilis

unread,

Feb 1, 2019, 11:55:44 PM2/1/19

to LCZero

Hi Stuart.

Instead of a script, you could use Arena. There is a Linux version available:
http://playwitharena.com/?Download:Arena_for_Linux

But if for some reason, Arena doesn't work there is SCID vs PC:
http://scidvspc.sourceforge.net

Both are excellent GUIs but you have to compile the later, and it's not that intuitive as Arena
Unfortunately Arena doesn't work in Manjaro, but works fine in Debian.
So, maybe in Ubuntu too!

Thank you for what you have offered all these years, to computer chess.
I used to have a lot of fun with GnuChess, as a student!

Dietrich Kappe

unread,

Feb 2, 2019, 12:17:05 AM2/2/19

to LCZero

It's fortuitous that there was just a match with IM lovlas against leela with low node counts (I think it peaked at 10.)

See here: https://lichess.org/Ons64qmu

Apparently he was quite fond of her playing style at this level (strong positionaly with a blind spot for some tactics).

If you hunt around, I think a GM had a run at her with similar conditions just before.

Vassilis

unread,

Feb 2, 2019, 1:08:27 AM2/2/19

to LCZero

Hi all

Well, I am forced to reveal you my chess "abilities" by showing you a game I played a few minutes ago against Leela ID 32890 (depth=1)

I am not sure if depth=1 is the same as nodes=1 but anyway...

You are watching the effort of a 1575 elo player (myself), blitzing out moves without any particular plan, and minimal thought.

Please, don't judge my chess understanding from this game (it's early in the morning). I've played way better games, and worse ones. Always against humans in tournaments, where the fun is :)

Now, how on earth my opponent (Leela) is a 2300+ elo player, is beyond my comprehension !!!

Enjoy the game :)

[Event "UCI Game"]
[Site "?"]
[Date "2019.02.02"]
[Round "?"]
[White "Vassilis"]
[Black "LCZero_32890"]
[Result "1-0"]

1.d4 Nf6 2.Nf3 e6 3.e3 d5 4.c4 Be7 5.Be2 O-O 6.O-O dxc4 7.Bxc4 a6 8.Nc3 b5 9.Bd3 Bb7 10.Re1 c5 11.dxc5 Bxc5 12.Qc2 Nbd7 13.Rd1 Rc8 14.Ne5 Qc7 15.Nxd7 Nxd7 16.Bxh7+ Kh8 17.Be4 Be7 18.Bxb7 Qxb7 19.Qe4 Qc7 20.Bd2 Nc5 21.Qf3 Kg8 22.Rac1 Qe5 23.e4 b4 24.Ne2 Nxe4 25.Bf4 Qf5 26.Nd4 Qd5 27.Nc6 Qxc6 28.Rxc6 Rxc6 29.Qxe4 Rfc8 30.Be3 f5 31.Qf3 Bf6 32.Bd4 Bxd4 33.Rxd4 Rc1+ 34.Rd1 Rxd1+ 35.Qxd1 b3 36.axb3 Rd8 37.Qxd8+ Kf7 38.Qd7+ Kf8 39.Qxe6 f4 40.Qxa6 g5 41.Qf6+ Kg8 42.b4 g4 43.b5 Kh7 44.b6 Kg8 45.Qe7 Kh8 46.b7 f3 47.b8=Q# 1-0

Stuart Cracraft

unread,

Feb 2, 2019, 2:12:48 AM2/2/19

to Vassilis, LCZero

Hi.

I don’t know about depth = 1. I have always used go nodes 1 or go movetime 10. Something like that.

Your game is good and I believe you are absolutely accurate regarding your comment.

But my question is why don’t you do “go movetime 10” to give it time?

And what kind of GPU are you using?

I get 3000 nodes per second from Leela on a Deep Learning single GPU instance running on Ubuntu on AWS and could never beat it.

Also my weakish 2500 traditional program was crushed by it.

I think something is cooking with your settings.

Choose go movetime 10 and redo your game and let us know what happens.

Send your entire log so we know you are running it properly.

—Stuart

--
You received this message because you are subscribed to the Google Groups "LCZero" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lczero+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lczero/4d7a000c-bcd8-47df-b69f-9938363e0a57%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Vassilis

unread,

Feb 2, 2019, 3:06:14 AM2/2/19

to LCZero

Hi Stuart

I've played this game under Scid vs PC GUI and not from the command line:

From Menu: Play -> Computer - UCI Engine

and chose Fixed Depth 1 and LCZero_32890 like in the screenshot

I guess the GUI does what it says it does, so I didn't bother more. Leela's UCI parameters were set to default, except threads=1 and no ponder. Also Leela uses syzygy.

if I do “go movetime 10” from the command line, that means Leela thinks for 10 millisecs before making her move right? That could be far more than one node (or depth=1)

My GPU is a GTX960M inside an Asus i7 Laptop

Cheers

gui.png

Stuart Cracraft

unread,

Feb 2, 2019, 3:36:27 AM2/2/19

to Vassilis, LCZero

That’s true and I had forgotten that one should multiply by 1000 due to the ms.

So go movetime 10000 is what I used.

I would avoid fixed depth and fixed nodes.

Time instead.

Give it a try.

Dietrich Kappe

unread,

Feb 2, 2019, 3:46:08 AM2/2/19

to LCZero

Stuart,

Rather than renting an expensive gpu on aws, try one of the distilled networks on cpu. https://github.com/dkappe/leela-chess-weights/wiki/Distilled-Networks

Vassilis

unread,

Feb 2, 2019, 4:35:19 AM2/2/19

to LCZero

Stuart, the only reason I played with fixed depth=1 has to do with the discussion I had with Warren D. Smith
(see above) about Leela's ELO when playing instantly. (If we both interpret correctly the graph, her ELO is 2300 with 1 node play? Seriously? I'm sure it's about something else...)

However, giving Leela 10 sec to think, I have no chance.
And no matter how interesting the game would be I would end up losing without knowing where I went wrong :)
No problem, I give it a try... But I have no illusions.

jimgl...@gmail.com

unread,

Feb 2, 2019, 3:57:20 PM2/2/19

to LCZero

"If you hunt around, I think a GM had a run at her with similar conditions just before."

Is this what you folk are thinking of?

Leela net 125(!) vs GM Andrew Tang:

https://youtu.be/pqYAy-l1c6g

Dietrich Kappe

unread,

Feb 2, 2019, 5:24:43 PM2/2/19

to LCZero

No. I meant a few hours before the IM. The match against GM Tang is pretty ancient.

jimgl...@gmail.com

unread,

Feb 2, 2019, 5:50:38 PM2/2/19

to LCZero

Googling around a little bit the only other GM I find playing Leela is Daniel Naroditsky v 11248 at knight odds.

https://youtu.be/dsz7_Al1dXM

FWIW.

Stuart Cracraft

unread,

Feb 2, 2019, 6:04:23 PM2/2/19

to Dietrich Kappe, LCZero

Thanks - wish I could. I don’t have Leela up yet on my largish Mac OS X MacBook Pro.

If anyone has a good installation script and/or instructions for Mac OS X, I’ll give it a try (and distilled network a try assuming I can get only the cpu version of Leel up.

By the way, this is (so far) just to get a fun human-like player to be able to play against and maybe occasional play for my own 2500 chess program “Talish”.

Thanks everybody.

The Discord is great.

On Feb 2, 2019, at 12:46 AM, Dietrich Kappe <dka...@gmail.com> wrote:

Stuart,

Rather than renting an expensive gpu on aws, try one of the distilled networks on cpu. https://github.com/dkappe/leela-chess-weights/wiki/Distilled-Networks

--
You received this message because you are subscribed to the Google Groups "LCZero" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lczero+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lczero/892931b5-2150-4d6e-b5f5-f16f7af00506%40googlegroups.com.

Dietrich Kappe

unread,

Feb 2, 2019, 6:43:31 PM2/2/19

to LCZero

You have to look at the games of LeelaChess on lichess. Here is one of the games of GM almostadams from yesterday. https://lichess.org/zC8ZXCSg

Don’t know what the various number of node limits were, but probably around 10.

Dietrich Kappe

unread,

Feb 2, 2019, 6:50:01 PM2/2/19

to LCZero

Stuart,

You can run the distilled networks anywhere lc0 runs, including gpu. You can see how it fares on Mac OS X with blas backend on that page via the various cpu tournaments.

On Mac OS X, as long as I have the development tools, I just checkout the latest release from git, then build it with the build.sh script. There’s a little bit of python setup to be able to use meson and ninja in the build, but those are described in the lc0 pages on github.

Let me know if you need some help.

Stuart Cracraft

unread,

Feb 2, 2019, 6:53:25 PM2/2/19

to Dietrich Kappe, LCZero

Hi,

Can you send me the link to the Mac OS X instructions which everyone uses?

I will go through the install and send my log to show what is going on as it is not working to get Leela (cpu version) up on this Mac.

Regards,

Stuart

--
You received this message because you are subscribed to the Google Groups "LCZero" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lczero+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lczero/5f635fb8-8ef3-4d42-a58b-d868f22d251e%40googlegroups.com.

Vassilis

unread,

Feb 3, 2019, 12:21:52 AM2/3/19

to LCZero

Hi Stuart.

The instructions are here:
https://github.com/LeelaChessZero/lc0

Bottom of the page and after the instructions for the Windows build.
Unfortunately I don't have a Mac, so I can't say more...
Let us know if you did it.

Stuart Cracraft

unread,

Feb 3, 2019, 1:14:20 AM2/3/19

to Vassilis, LCZero

Hey,

Here’s what happened.

The main issue is the meson build.

Please see the log below and thankyou very much for your insights!

Stuart

Stuarts-MacBook-Pro:lc0 root# ./build.sh

The Meson build system

Version: 0.48.2

Source dir: /usr/local/src/lc0

Build dir: /usr/local/src/lc0/build/release

Build type: native build

Project name: lc0

Project version: undefined

Native C++ compiler: c++ (clang 10.0.0 "Apple LLVM version 10.0.0 (clang-1000.11.45.5)")

Build machine cpu family: x86_64

Build machine cpu: x86_64

Library libprotobuf found: NO

Found pkg-config: /usr/local/bin/pkg-config (0.29.2)

Dependency protobuf found: NO (tried pkgconfig and framework)

Program protoc found: NO

Using protobuf from cache.

Using protobuf-3.5.1-2w-wrap.zip from cache.

|

|Executing subproject protobuf

|

|Project name: protobuf

|Project version: 3.5.1

|Native C++ compiler: c++ (clang 10.0.0 "Apple LLVM version 10.0.0 (clang-1000.11.45.5)")

|Compiler for C++ supports arguments -DHAVE_PTHREAD: YES

|Compiler for C++ supports arguments -Wno-sign-compare: YES

|Compiler for C++ supports arguments -Wno-unused-parameter: YES

|Compiler for C++ supports arguments -Wno-ignored-qualifiers: YES

|Compiler for C++ supports arguments /wd4146: NO

|Compiler for C++ supports arguments /wd4244: NO

|Compiler for C++ supports arguments /wd4305: NO

|Compiler for C++ supports arguments /wd4506: NO

|Dependency threads found: YES

|Build targets in project: 6

|

|Subproject protobuf finished.

Library pthread found: YES

Library dl found: YES

Library libtensorflow_cc found: NO

Dependency accelerate found: YES (/System/Library/Frameworks/Accelerate.framework)

Library mkl_rt found: NO

Library openblas.dll found: NO

Library openblas found: NO

meson.build:184:6: ERROR: Include dir /System/Library/Frameworks/Accelerate.framework/Frameworks/vecLib.framework/Headers does not exist.

A full log can be found at /usr/local/src/lc0/build/release/meson-logs/meson-log.txt

Stuarts-MacBook-Pro:lc0 root#

--
You received this message because you are subscribed to the Google Groups "LCZero" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lczero+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lczero/5a73e31f-731f-43ef-aa58-4c0fc3c577d5%40googlegroups.com.

Vassilis

unread,

Feb 3, 2019, 1:31:36 AM2/3/19

to LCZero

I'm not sure if this is the reason, but I don't see a backend installed, like libopenblas. This is important
Don't know how to do it in Mac though.
Also, instead of downloading the latest stable release, try the zip file from the master branch. It includes protoc

Message has been deleted

Vassilis

unread,

Feb 3, 2019, 1:40:32 AM2/3/19

to LCZero

I found somewhere this:

brew install openblas

Or if this doesn't work

brew install homebrew/science/openblas

Hope this helps

Stuart Cracraft

unread,

Feb 3, 2019, 2:33:47 AM2/3/19

to Vassilis, LCZero

Tried it. No luck.

Stuarts-MacBook-Pro:lc0 root# brew install openblas

Error: Running Homebrew as root is extremely dangerous and no longer supported.

As Homebrew does not drop privileges on installation you would be giving all

build scripts full access to your system.

Stuarts-MacBook-Pro:lc0 root# exit

logout

Stuarts-MacBook-Pro:~ stuartcracraft$ brew install openblas

Updating Homebrew...

==> Auto-updated Homebrew!

Updated 3 taps (homebrew/cask-versions, homebrew/core and homebrew/cask).

==> New Formulae

ahoy dhall grpcurl libpulsar pass-otp sloc

anycable-go dmg2img hexyl libvirt-glib postgresql@10 sn0int

anyenv dnscontrol interactive-rebase-tool lsd rakudo spice-protocol

atomist-cli entityx istioctl moarvm react-native-cli swagger-codegen@2

aws-iam-authenticator esptool jmxterm needle redis@4.0 switch-lan-play

bluetoothconnector gambit-scheme kubeprod nqp ruby@2.5 tass64

chafa gerbil-scheme kubeseal opencv@3 s3ql termtosvg

cryptominisat ghr libgusb osx-cpu-temp ship tmx

dav1d goreman libnova oxipng simple-scan

==> Updated Formulae

awscli ✔ dosbox-x i2pd mysql-cluster scummvm

boost ✔ dovecot iamy mysql@5.7 sdcv

gdb ✔ doxygen ibex nasm sdf

go ✔ dpkg icemon nativefier serverless

libidn2 ✔ druid imagemagick ncmpcpp sf-pwgen

meson ✔ dscanner immortal neofetch sfml

node ✔ dspdfviewer influxdb neovim shellshare

oniguruma ✔ dub ios-sim netcdf shfmt

python ✔ dvm ios-webkit-debug-proxy netpbm shibboleth-sp

readline ✔ dwarf iso-codes newlisp shogun

sqlite ✔ dwm isync newsboat signify-osx

wget ✔ dynare itstool nghttp2 sile

abcm2ps e2fsprogs jabba nginx simh

abook easyengine jbake ngspice simple-mtpfs

ace eccodes jboss-forge nickle simutrans

afflib ed jenkins nim sip

aircrack-ng efl jenkins-lts nmh siril

akamai eigen jenv nnn skaffold

alexjs ekg2 jfrog-cli-go node-build skinny

algernon elasticsearch jhipster node@10 skopeo

allure elasticsearch@5.6 jid node@6 sl

amazon-ecs-cli elixir joplin node@8 sleuthkit

ammonite-repl emacs-clang-complete-async json11 nodenv smartmontools

amqp-cpp embulk jsoncpp nomad socat

angle-grinder emscripten jsonnet nordugrid-arc soci

angular-cli encfs juju nsd solr

anjuta enet juju-wait nss sonar-scanner

annie enigma kakoune nu source-highlight

ansible envconsul kapacitor numpy source-to-image

ansifilter epubcheck kettle nuxeo sox

antlr erlang kibana nvm spatialite-tools

antlr4-cpp-runtime erlang@20 kibana@5.6 nwchem spdlog

apache-drill eslint kitchen-sync nzbget sphinx

apache-flink etcd knot objc-codegenutils sphinx-doc

apache-geode ethereum knot-resolver octave spidermonkey

ape evince kontena ompl spoof-mac

apibuilder-cli exiv2 kops onscripter spotbugs

appledoc exploitdb kotlin opa sqlcipher

arangodb eye-d3 kpcli opam sqlmap

arcade-learning-environment faas-cli krakend open-babel squashfs

argus-clients fastbit kube-aws open-mpi sshuttle

aria2 fauna-shell kubectx openal-soft statik

ark fb-client kubeless openblas stellar-core

asciinema fbi-servefiles kubernetes-cli opencc stk

asdf fcl kubernetes-helm openclonk stone-soup

asio ffmpeg kyoto-cabinet opencoarrays stubby

ask-cli ffmpeg2theora languagetool openconnect subversion

aspcud ffmpeg@2.8 laszip opencv sundials

astrometry-net ffmpegthumbnailer ldapvi opencv@2 supersonic

atkmm ffms2 ldc opendbx supertux

aubio firebase-cli lean openfortivpn supervisor

audacious fish ledger openldap swagger-codegen

augeas flac leiningen openmsx swi-prolog

autorest fltk leptonica opensaml swiftformat

avian fluid-synth less opentsdb swiftgen

aws-okta fluxctl lftp openvdb swiftlint

aws-sdk-cpp flyway libassuan orc-tools switchaudio-osx

azure-cli fmt libatomic_ops osm-pbf syncthing

babel fn libav osmium-tool synfig

babl folly libbi osrm-backend sysbench

bacula-fd fontforge libbladerf p11-kit sysdig

ballerina fonttools libccd packer takt

basex freeswitch libcds pacvim tarantool

bash freetds libcec pagmo task

bashdb frugal libcerf paket tbb

bat fselect libchaos pandoc tcc

bcal fswatch libcouchbase pandoc-citeproc tcl-tk

bdw-gc fuseki libdazzle pango tcpdump

bear futhark libedit pangomm tcpreplay

bedops fx libetpan parallel telegraf

bento4 gammaray libgda parallelstl telegram-cli

bettercap gauche libgit2 pari teleport

bigloo gauge libglademm passenger terminal-notifier

binaryen gawk libgnomecanvasmm pce terraform

bind gcab libgpg-error pcl terraform-docs

bison gcc@6 libgsf pdal terragrunt

bitcoin gdal libgxps pdf2htmlex tesseract

bitrise gdcm libheif pdfgrep tgui

bitwarden-cli geant4 libhttpserver pdfpc theharvester

blackbox gearman libjwt pdftoedn thors-serializer

blahtexml geeqie liblas pdftoipe tig

blast gegl libmicrohttpd pdnsrec tika

blink1 genstats libnice pdsh tile38

blueutil geoipupdate libosmium percona-server tintin

boost-mpi geos libphonenumber percona-toolkit tippecanoe

boost-python get_iplayer libpng perl tmux-mem-cpu-load

boost-python3 getdns libpointing petsc tmuxinator-completion

botan gexiv2 libpqxx petsc-complex tokei

bower ghc libpsl pgbadger tomcat

brew-php-switcher ghc@8.2 libpst pgcli tomcat@8

buildifier ghostscript libqalculate pgweb tomee-webprofile

buku gibo libreadline-java phoronix-test-suite topgrade

bullet ginac librealsense php tor

bundletool git libressl php-code-sniffer tox

bzt git-flow-avh librsvg php-cs-fixer trace2html

cabextract git-fresh libsamplerate php@7.1 tracebox

cadaver git-open libsass php@7.2 traefik

caddy git-quick-stats libsecret phpmyadmin trafficserver

caf git-secret libsigc++ phpunit translate-shell

caffe git-standup libsodium picard-tools travis

cairomm gitbucket libssh pilosa typescript

cake github-markdown-toc libstfl pioneer ubertooth

calc gitlab-gem libtins pioneers ucloud

camlp5 gitlab-runner libuv pipenv uhd

capnp gjs libvirt pixman unbound

capstone glances libvmaf planck unnethack

cargo-completion glbinding libvoikko plantuml unrar

carla glib libvpx platformio upscaledb

carthage glibmm libvterm plplot urbit

cataclysm global libxc pmd urdfdom

catimg globjects libxml++ pngquant urdfdom_headers

ccache gmic libxml++3 podofo urh

ccrypt gmsh libxml2 ponyc uriparser

ceres-solver gmt libxmlsec1 poppler userspace-rcu

certbot gnatsd libxslt postgis v8

cfitsio gnome-builder linkerd postgres-xc vagrant-completion

cgdb gnome-latex lldpd postgresql vala

cgit gnome-recipes llvm postgresql@9.4 vault

cglm gnu-chess llvm@3.9 postgresql@9.5 vc4asm

chakra gnu-sed llvm@4 postgresql@9.6 vegeta

checkbashisms gnu-smalltalk llvm@5 povray verilator

checkstyle gnu-tar llvm@6 pre-commit vert.x

chrome-cli gnu-units lmdb prefixsuffix vice

chronograf gnupg lmod presto vim

circleci gnupg-pkcs11-scd log4cplus prettier vim@7.4

citus gnuplot logstalgia primesieve vips

clamav gnuplot@4 logstash profanity visp

clang-format gnuradio logtalk prometheus vowpal-wabbit

clblast gnutls lsdvd protobuf vte

cli53 go-bindata lxc protobuf-c vte3

clingo g...@1.10 mackup protobuf-swift vtk

clipsafe go@1.8 macvim protobuf@3.1 w3m

clisp go@1.9 mafft pspg weaver

clojure gobby magnetix pulseaudio weboob

closure-compiler gobject-introspection mailutils pulumi webp

cmake gocryptfs makensis pumba webpack

cockroach godep mame purescript weechat

cocoapods goenv mapcrafter pushpin wellington

cointop golang-migrate mapnik puzzles whois

collectd google-benchmark mapserver pwntools widelands

commandbox gopass mariadb pwsafe wine

composer goreleaser mariadb-connector-c pycodestyle wireguard-go

conan gource mariadb@10.2 pyenv wireguard-tools

confluent-oss gowsdl mas pygitup wireshark

conjure-up gphoto2 maxwell pypy with-readline

console_bridge gpsbabel mdbtools pypy3 woboq_codebrowser

consul gqlplus mdcat python-yq wolfssl

container-diff gr-osmosdr mdds python@2 wp-cli

convox gradle mdk qbs wp-cli-completion

coreutils grafana media-info qemu wpscan

cp2k grakn mednafen qjackctl wtf

cppcheck graph-tool mercurial qmmp wxmaxima

cppcms graphite2 mesa qpdf x264

cquery grep mesos qpid-proton x265

create-dmg groff metashell quex xalan-c

cryfs groovy micronaut quicktype xcodegen

cryptopp grpc midnight-commander r xcproj

crystal grv mikutter rabbitmq xdot

crystal-icr gsmartcontrol mill rdfind xerces-c

csv-fix gssdp minimal-racket re2 xml-security-c

ctop gst-plugins-ugly minio rebar3 xml-tooling-c

curl gst-python minio-mc recutils xonsh

curl-openssl gstreamermm mint redis xqilla

curlpp gtk+3 mitie renameutils xsd

cython gtk-doc mkcert repo xsimd

dar gtkmm mkclean restic xtensor

dartsim gtkmm3 mkl-dnn rethinkdb yafc

dasht gtksourceviewmm mkvtoolnix riak yaml-cpp

dbxml gtksourceviewmm3 mockserver riemann-client yamllint

dcd guile modd rke yara

dependency-check guile@2.0 mogenerator rlwrap yarn

devtodo gupnp-tools monero robot-framework ydcv

dfmt gzip monetdb rocksdb ykman

dhall-json handbrake mongo-cxx-driver rom-tools yle-dl

diamond haproxy mongodb root yosys

diceware harfbuzz mongodb@3.0 roswell you-get

diff-pdf haste-client mongodb@3.2 rp youtube-dl

diff-so-fancy hbase mongodb@3.4 rpm yq

diffutils hcloud mongodb@3.6 rswift z3

digdag healpix mono ruby zabbix

direnv helmfile moreutils ruby-build zbackup

django-completion hfstospell mosh ruby@2.0 zeromq

dlib highlight mosquitto ruby@2.3 zile

dmd howard-hinnant-date mozjpeg ruby@2.4 zim

dnscrypt-wrapper hss mpd rust zmqpp

dnsperf hstr mplayer rustup-init znc

docfx htmldoc mplayershell safe zola

docker http-parser mps-youtube salt zorba

docker-completion hub mpv sbcl zsh

docker-machine hugo mruby sbt zsh-completions

docker-machine-completion hunspell mu scalapack zstd

docker-machine-nfs hwloc mutt sceptre zurl

docker-machine-parallels hydra mycli schismtracker

doctl hyperfine mysql scipy

doitlive hyperscan mysql-client scrcpy

==> Deleted Formulae

apache-arrow cctools ffmbc gv liblastfm pyexiv2

apache-arrow-glib cctools-headers gnome-doc-utils hyper percona-server@5.6 rock

apple-gcc42 cputhrottle gra...@2.14 ld64 php@5.6 ruby@1.8

==> Installing dependencies for openblas: gmp, isl, mpfr, libmpc and gcc

==> Installing openblas dependency: gmp

==> Downloading https://homebrew.bintray.com/bottles/gmp-6.1.2_2.mojave.bottle.tar.gz

######################################################################## 100.0%

==> Pouring gmp-6.1.2_2.mojave.bottle.tar.gz

🍺 /usr/local/Cellar/gmp/6.1.2_2: 18 files, 3.1MB

==> Installing openblas dependency: isl

==> Downloading https://homebrew.bintray.com/bottles/isl-0.20.mojave.bottle.tar.gz

######################################################################## 100.0%

==> Pouring isl-0.20.mojave.bottle.tar.gz

🍺 /usr/local/Cellar/isl/0.20: 71 files, 3.9MB

==> Installing openblas dependency: mpfr

==> Downloading https://homebrew.bintray.com/bottles/mpfr-4.0.1.mojave.bottle.tar.gz

######################################################################## 100.0%

==> Pouring mpfr-4.0.1.mojave.bottle.tar.gz

🍺 /usr/local/Cellar/mpfr/4.0.1: 28 files, 4.6MB

==> Installing openblas dependency: libmpc

==> Downloading https://homebrew.bintray.com/bottles/libmpc-1.1.0.mojave.bottle.tar.gz

######################################################################## 100.0%

==> Pouring libmpc-1.1.0.mojave.bottle.tar.gz

🍺 /usr/local/Cellar/libmpc/1.1.0: 12 files, 354KB

==> Installing openblas dependency: gcc

Warning: Building gcc from source:

The bottle needs the Xcode CLT to be installed.

==> Downloading https://ftp.gnu.org/gnu/gcc/gcc-8.2.0/gcc-8.2.0.tar.xz

######################################################################## 100.0%

==> Patching

patching file gcc/graphite.h

==> ../configure --build=x86_64-apple-darwin18.2.0 --prefix=/usr/local/Cellar/gcc/8.2.0 --libdir=/usr/local/Cellar/gcc/8.2.0/lib/gcc/8 --disable-nls --enable-check

==> make BOOT_LDFLAGS=-Wl,-headerpad_max_install_names

Last 15 lines from /Users/stuartcracraft/Library/Logs/Homebrew/gcc/02.make:

cc1plus: note: use e.g. -Wabi=11 to warn about changes from GCC 7

In file included from /private/tmp/gcc-20190202-55647-asze61/gcc-8.2.0/libstdc++-v3/libsupc++/exception:144,

from /private/tmp/gcc-20190202-55647-asze61/gcc-8.2.0/libstdc++-v3/libsupc++/new:40,

from ../../../../libstdc++-v3/libsupc++/bad_array_length.cc:24:

/private/tmp/gcc-20190202-55647-asze61/gcc-8.2.0/build/x86_64-apple-darwin18.2.0/libstdc++-v3/include/bits/nested_exception.h:40:10: fatal error: bits/move.h: No such file or directory

#include <bits/move.h>

^~~~~~~~~~~~~

compilation terminated.

make[5]: *** [bad_array_length.lo] Error 1

make[5]: *** Waiting for unfinished jobs....

make[4]: *** [all-recursive] Error 1

make[3]: *** [all] Error 2

make[2]: *** [all-stage3-target-libstdc++-v3] Error 2

make[1]: *** [stage3-bubble] Error 2

make: *** [all] Error 2

READ THIS: https://docs.brew.sh/Troubleshooting

Stuarts-MacBook-Pro:~ stuartcracraft$

--
You received this message because you are subscribed to the Google Groups "LCZero" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lczero+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lczero/c1a39ccc-f8fa-4fc2-a91d-e8e921578eb7%40googlegroups.com.

Vassilis

unread,

Feb 3, 2019, 3:02:35 AM2/3/19

to LCZero

Try updating brew first with:

brew update

Have you also tried with:

brew install homebrew/science/openblas

instead of brew install openblas

if this also doesn't work, I run out of ideas

josé luis

unread,

Feb 3, 2019, 5:27:11 AM2/3/19

to LCZero

I think computer are better at chess..no matter TC..at 1 node vs ultrabullet wich is most equivalent..leela would win allmost all games

Vassilis

unread,

Feb 3, 2019, 5:41:14 AM2/3/19

to LCZero

Jose Luis, do you play chess? I mean... tournament chess!

Don't forget that human chess players range from bellow 1000 up to 2800+ ELO.
So this statement is simply not true, even for the best chess engine! TC matters.

josé luis

unread,

Feb 3, 2019, 10:40:33 AM2/3/19

to LCZero

No..just 1100 if I dont remember bad last time I played..but seeing the evidence of the results avaiable.
I saw Leela 1 node drew against komodo full at ccc

josé luis

unread,

Feb 3, 2019, 10:43:10 AM2/3/19

to LCZero

Of course TC matters..but Lc0 or SF are better at every TC..at wich TC is not true?

josé luis

unread,

Feb 3, 2019, 10:46:03 AM2/3/19

to LCZero

Sorry..you are talking at TC odds?

Vassilis

unread,

Feb 3, 2019, 10:50:08 AM2/3/19

to LCZero

OK! Hypothetically speaking...

Just pair Magnus against Leela 1 node. Blitz game for Magnus, for Leela doesn't matter, since she is on 1 node (constant thinking time)

Who would you think will win?

I put all my money on Magnus :)

For long time controls though, there is absolutely no question, Leela is way better... See my point?

Vassilis

unread,

Feb 3, 2019, 10:54:41 AM2/3/19

to LCZero

[...Sorry..you are talking at TC odds?]

Yes, but not very much different. Just give the human player time to move the pieces :)

1 node Leela vs 3 min Magnus

Note, 1 node, not 1 min. It is important

josé luis

unread,

Feb 3, 2019, 11:18:02 AM2/3/19

to LCZero

Ok..with odds is possible..but the fair condition to 1 node is ultrabullet or faster..dont you think so?

Vassilis

unread,

Feb 3, 2019, 11:43:10 AM2/3/19

to LCZero

if with ultra-bullet you mean 1 min + 1 sec increment for the human -vs- 1 node for Leela, I would still put my money on the best human :)

The difference - and the gap - between Leela and best human, is clear with any format in the form of [X min + Y sec] for both engine and human. Then yes, Leela almost always win. And very rarely, a draw may still be possible. Actually we are saying the same thing. I simply don't believe in this 1 node ELO. 2300+

josé luis

unread,

Feb 3, 2019, 11:50:53 AM2/3/19

to LCZero

Maybe you should think how would score Magnus 30 seconds entire game vs Magnus 120 minutes..Aprox. Elo? 2300??

Vassilis

unread,

Feb 3, 2019, 12:17:30 PM2/3/19

to LCZero

So ultra-bullet is 30 seconds the whole game, without any increment? Well I can't tell you for sure then. Don't know if he blunders eventually.

One thing I can tell you for sure, is that instant moves by Magnus are more than 2300 elo each. All these top guys don't even need to think variations, to play intuitively strong moves.

This is how they win in blitz! Yes they calculate from time to time (also very fast) but most of their moves are intuitive. And after they reach a winning position, the rest is a matter of technique!

I remember playing one strong GM once (ELO 2550+) in a clock simultaneous game. Time control had an increment, but I don't remember how much. We were six players ranging from 1500 to 1900 elo, and each time he approached a table he made his move at 5 sec the most! Most of his moves were instant moves. While we were thinking, he was drinking orange juice, and talking to our coach (silently) without looking at our tables. After we all lost 6-0 he had consumed about 5 mins for all of us. I ashamed to say how much time each one of us had spent. He was also remembering critical positions of all games and was kind enough to point out our errors.

If a 2550+ GM can do that, how about a 2800+? The difference is huge

josé luis

unread,

Feb 3, 2019, 1:11:44 PM2/3/19

to LCZero

I understand there are chessplayers playing 15 seconds game..so I believe Magnus could think all instamoves of a game in few seconds..thats most similar to 1 node I guess..
I know most moves Magnus choose were the first moves considers..but..I remember some 5 minutes blitz games by MVL and others and I saw many blunders That Me(not a chessplayer)caught me by surprise..I noticed many times great players that missed the correct move..and at the chess chats many said there maybe could play against top players with huge time odds(not than One can trust chats)..maybe Im wrong..

Vassilis

unread,

Feb 3, 2019, 1:40:37 PM2/3/19

to LCZero

No you are not wrong, people at the chat rooms say a lot of things... I don't blame them, most of them have a false perception about chess in general.

But think for a while what kind of blunders those top GMs usually make...

Are they easy to spot blunders, or sophisticated ones, require a top engine to spot ?

It is easy for someone "from outside" to say a GM move "a blunder" with the help of Stockfish or any other top engine kbitzing. But while kibitzing, (and this is important) Stokfish is NOT in 1 node but in analysis mode. In this mode, it searches each position 20++ plies deep, which makes it stronger than any human! That's why its evaluation spikes (+-) from time to time.

Leela in 1 node also makes blunders, which are easy for a top GM to spot and exploit.

But Leela in normal time controls, (I underline the word time, here) and equal terms, no matter if this is blitz, rapid, or long games, is a different beast :) Clearly superior to any human - like any other top engine.

Dietrich Kappe

unread,

Feb 3, 2019, 3:49:45 PM2/3/19

to LCZero

Looking at the low node games against the IM and GM mentioned above, there was a certain feeling out phase where the human players figured out what low node was and wasn’t capable of.

A few observations:

- 1 node is clearly too low
- with 10 nodes, the humans had the edge after learning, but had to play into very sharp positions to exploit low node leela’s tactical weakness
- above 10 nodes, the humans rapidly lose ground.
- These days, even Magnus would struggle to take points off of 200 node leela.

The good news is that leela (and especially the distilled networks, make for educational and entertaining opponents at lowish nodes.

The same can not be said of ab engines.

Vassilis

unread,

Feb 3, 2019, 5:32:10 PM2/3/19

to LCZero

Thank you Dietrich.

The way you put it, makes a lot of sense and I fully agree to everything you said :)

A final comment about AB engines:
You can lower their strength, by reducing for e.g their search depth.
But that makes them play very unnatural and artificial chess you would never see from an equally weak human player.
Therefore they are not as valuable educational tool as Leela.

rohit raj

unread,

Feb 4, 2019, 12:57:53 AM2/4/19

to LCZero

Dennis Hassabis, founder of deepmind and a very strong chess player himself, had said in his interview with Judith Polgar and Anna Rudolf during WCC, that top players look at 100 positions every move.

Shah

unread,

Feb 4, 2019, 1:32:16 AM2/4/19

to LCZero

Hi Vas,
Strange that no consensus could be reached wrt Leela's stength at 1 node.
Would love to hear what KC has to say about this.

Two puzzles for me:
1. If Leela can (at least most of the time) avoid a mate in 1 when evaluating 1 node only, that means that somehow tactic evaluation capabilities must have been "encoded" into Leela's NN.
Which to me is surprising.
Maybe it implies that the training is not done the optimal way in that part of the network is being "wasted" for tactics?

2. I cannot understand how a 10 node evaluation can significantly increase tactic capabilities compared to 1 node.
Iow, if leela at 1 node has 1-move-tactic-blindspot, leela at 10 nodes should suffer from a similar thing, altough to a lesser degree.
So it is also surprising that some people here are saying that there is, infact, a significant increase in elo when going from 1 to 10 node.

Cheers!

MadChuckle

unread,

Feb 4, 2019, 2:14:10 AM2/4/19

to LCZero

For those who don't know yet, you can actually challenge and play Leela both at full strength and at 10-nodes at Lichess:

Full strength Leela: https://lichess.org/@/LeelaChess
10-node Leela: https://lichess.org/@/LeelaNice

Both are maintained by careless25 (@Discord)

Recently both "IM Lovlas" (Lasse Løvik) and "GM almostadams" (Adam Tukhaev) played with the 1, 10 and full strength Leela (all were on the main LeelaChess lichess channel setup live by Careless25). You can find the games in the history of the bot page there.

Their commentary were along the lines of what is stated here. That 1-node leela is unexpectedly strong positionally but misses some easy sharp tactics. 10-node leela is beatable but you have to play very solid (for an IM).

@100-nodes (I am just guessing now) I think you can win some occasional games but never win a 5-game match, maybe Carlsen can. @200+ nodes I don't think it is possible to wi na 5-game match for any human.

The good thing is weakened leela is playing like a strong human with occasional human like blunders. I believe if you take an earlier net from a run, it would be even more like a moderate level human player.

Leela is quite strong without search or with little search compared to A/B engines, that is one of things that makes NN unique and exciting.

On Friday, February 1, 2019 at 11:33:17 AM UTC+3, Shah wrote:

I think that would be like comparing Leela's intuition to human.
An intersting comparison I think.
How many nodes a human GM evaluates?
I don't know... But I am sure people in this forum are more knowledgeable and can say.
Another way to put it:
Did anyone caculate Leela's elo for smallest node values?
Say: 1, 5, 10, 20, 50, 100...??
Thanks!

Stuart Cracraft

unread,

Feb 4, 2019, 2:46:31 AM2/4/19

to Vassilis, LCZero

no luck

Stuarts-MacBook-Pro:/ stuartcracraft$ brew install homebrew/science/openblas

Error: No available formula with the name "homebrew/science/openblas"

Please tap it and then try again: brew tap homebrew/science

==> Searching for similarly named formulae...

Error: No similarly named formulae found.

Stuarts-MacBook-Pro:/ stuartcracraft$

--
You received this message because you are subscribed to the Google Groups "LCZero" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lczero+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lczero/e25a049d-ecca-444a-ad7b-c03b2cda6e1e%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

—Stuart

P.S. For scheduling time with me:

https://meetme.so/StuartCracraft

For my resume:

http://www.stuartcracraft.com

For my LinkedIn

http://www.linkedin.com/in/stuartcracraft

My email is

smcra...@me.com

Markus Kohler

unread,

Feb 4, 2019, 3:26:05 AM2/4/19

to LCZero

Did you try playing against Leela with one node?

It is really pretty good and not that easy to beat.

BTW I once did an experiment and let leela play against itself (the versions might have been slightly different) with 1 node. The result was a draw and analyzing it with SF10 on lichess did not reveal a single, inaccuracy, mistake or blunder.

I find that is a very remarkable result.

Am Freitag, 1. Februar 2019 15:54:13 UTC+1 schrieb Warren D Smith:

On Friday, February 1, 2019 at 3:40:06 AM UTC-5, Ingo Weidner wrote:
The "official" estimated Elo graph includes 3 graphs with 1 node, 10 nodes and 100 nodes tested against Stockfish 9 ("SF 9"):
https://docs.google.com/spreadsheets/d/1Mi2qwPCK4aVNt9B8aC-HecIyLJ0BEAPvp63L59DTrY4/edit#gid=1729484756

You seriously expect me to believe leela plays at 2300 strength with no lookahead whatsoever?

I have immense trouble believing anything can do that.

Vassilis

unread,

Feb 4, 2019, 5:37:31 AM2/4/19

to LCZero

Hi Markus

I've played Leela recent nets with depth=1 (which is probably not the same as 1 node) and beat her easily. See one example game above!
Now, if someone could tell me how to set up Leela to play in 1 node in SCID vs PC GUI, I would be grateful!
I would like to avoid playing the whole game from command line using uci commands.
Also Arena GUI doesn't work in Manjaro anymore.

Vassilis

unread,

Feb 4, 2019, 5:39:49 AM2/4/19

to LCZero

Hi Stuart!

I wish I could help you more, but I don't own a Mac.
Maybe someone here, who has successfully compiled Leela on his Mac could?

Ilya Kalinin

unread,

Feb 5, 2019, 5:12:52 PM2/5/19

to LCZero

Anybody knows how can I restrict amount of Nodes per move to a fixed number?

Dietrich Kappe

unread,

Feb 5, 2019, 5:49:43 PM2/5/19

to LCZero

In uci it's "go nodes 800" or whatever your preferred node number is. There should be a way of selecting this as a TC in your GUI of choice.

Ilya Kalinin

unread,

Feb 8, 2019, 1:17:55 PM2/8/19

to LCZero

Guys, I just made an account on Chess.com with Leela Chess searching only 1 NODE (in Bullet mode). So It playes super cool, It achieved IM level 2400+, so it's even better than in the graph of elo estimates(1n vs SF9 )~2300 on LCZero.org. I'm just stunned of it's success, only think - Leela plays on International Master level! With no search, by just intuition!!!!

I used 32930 net. And yeah, It sees a LOT of tactics in comparison with humans, sometimes it can miss one but mostly it's good at it. Esspecially it easily finds logical tactics like ones in the opening or in standart patterns. And What it does Perfectly is ATTACKS! It checkmates 2/3 of opponents in the opening or in the middle game!

I'm just stunned!Try yourself and you will see.

P.s Yet it lost a couple of games in less than 10 moves, accidentally(

Warren D Smith

unread,

Feb 8, 2019, 1:31:45 PM2/8/19

to Ilya Kalinin, LCZero

well, (to Ilya Kalinin)
(1) you need to explain the details of *how* we "try it"
(2) from my perspective as a sad human, I pretty much cannot search at all
in "bullet" time control, and basically have to make random moves or
lose on time,
so if it is leela with no search versus me with no search,
then ok, I can believe I'm going to be in trouble.
But I think leela playing no search versus a human with enough
time to think, is going to be a lot worse than 2400.... in fact, I
conjecture I will
usually beat it playing 10 min+7 sec increment, say?

--
Warren D. Smith
http://RangeVoting.org <-- add your endorsement (by clicking
"endorse" as 1st step)

Ilya Kalinin

unread,

Feb 8, 2019, 1:45:24 PM2/8/19

to LCZero

No, 2400+ is a LOT! At this level humans see a lot of tactics. I tried using 3+2 mode as well and it beats everyone. 1 minute + 1 second is quite a lot of time to make a good move in chess. I think at 3+2 mode it is at least 2000.

Ilya Kalinin

unread,

Feb 8, 2019, 1:52:46 PM2/8/19

to LCZero

Reply to (1) "How we try it".

It's quite easy. You download a program called "ChessBot", use lc0 as an engine,and set configuration 1 depth(u will see in console that it's 1 node as well) and use it in chess. com. In order to stabilize the rating faster u should start it with new account(At the beginning elo changes after games are much more significant)

vladimir...@gmail.com

unread,

Feb 8, 2019, 1:54:39 PM2/8/19

to LCZero

Do your opponents know they are playing vs. a bot?

Ilya Kalinin

unread,

Feb 8, 2019, 2:00:20 PM2/8/19

to LCZero

No, they did not. So in fact i was cheating) But u see, it's exceptionally for scientific purposes, Do not blame me, It is test for this discussion

Todd Freitag

unread,

Feb 8, 2019, 3:35:12 PM2/8/19

to LCZero

Although it's not cool that you were cheating your opponents, I'll skip that discussion because there's also another issue with this experiment.

Human players use information about their opponents - sometimes subconsciously and sometimes deliberately. In particular, especially in blitz games, humans will play aggressive moves that they may suspect are not correct, but they don't have time to work it out, and it's difficult for the opponent to deal with the pressure within tight time constraints. Against an engine, this is just suicide. There are many other strategies that are effective against humans at short time controls and ineffective against engines (and vice versa). Some people would play quite differently knowing they are playing a bot.

Warren D Smith

unread,

Feb 8, 2019, 3:58:22 PM2/8/19

to LCZero

On Friday, February 8, 2019 at 1:45:24 PM UTC-5, Ilya Kalinin wrote:

No, 2400+ is a LOT! At this level humans see a lot of tactics. I tried using 3+2 mode as well and it beats everyone. 1 minute + 1 second is quite a lot of time to make a good move in chess. I think at 3+2 mode it is at least 2000.

--well, Nakamura & Carlsen can play at bullet speeds and do a lot of

search & eval,
with in at least one case Carlsen playing a bullet game against
another GM in which (according to stockfish analysis later) he made
zero mistakes!
I don't think I can play well enough to make zero mistakes
no matter how much time you give me to think (well, unless my opponent helps me
by making big blunders early, then I can play with zero mistakes :).

I have no idea how the hell those guys do that. Myself, if I try to
play at bullet speeds, I cannot search, I cannot even evaluate the one node
of my (non)search; all I can do is generate some but not all legal moves, and
semi-evaluate the situation, which is "bad and getting worse" :(.

But I have in mind as "rating", some fairly normal speed of play, not
insane bullet speed.
If I play normal speed & you play with 1 node, I think I'm usually
going to win, no matter who or what you are.

You have to say what your account names are on chess.com, or whatever;
it may be obvious to you, but not to me...

Ilya Kalinin

unread,

Feb 8, 2019, 4:15:49 PM2/8/19

to LCZero

No, You don't understand, it seems. She plays VERY good chess. I mentioned that she is not so bad in tactics(i would rather say good as she delivers checkmates all around,beautiful ones, u have to see it with your own eyes) and as for positional play, she's just perfect. She will outplay you totally positionally and win unless it misses some cool tactics(that happens very rarely), that's for sure. Her main strength is concealed in positional play, I thought it was quite evident.

So try it and you will see..

Vassilis

unread,

Feb 8, 2019, 5:11:30 PM2/8/19

to LCZero

Ilya hi!

The following is a true story of a 1575 elo player, with good results against higher rated players in tournaments. (Good positionally, Blind tactically)

Please read carefully...

Sometime ago I played some test games with Leela, and I found out It was not too hard to beat her at depth=1, with me playing blitz (5'-5") as long as I were concentrated.

Bellow is a sample game of this test...

TECHNICAL DETAILS:

lc0 v0.20.1

net 32890

strength: depth = 1

GUI Scid vs PC

OS Linux Manjaro 64bit

cudann backend, GPU Nvidia GTX960M

Default UCI parameters except threads=1, no ponder

[Event "UCI Game"]

[Site "?"]
[Date "2019.02.02"]
[Round "?"]
[White "Vassilis"]
[Black "LCZero_32890"]
[Result "1-0"]

1.d4 Nf6 2.Nf3 e6 3.e3 d5 4.c4 Be7 5.Be2 O-O 6.O-O dxc4 7.Bxc4 a6 8.Nc3 b5 9.Bd3 Bb7 10.Re1 c5 11.dxc5 Bxc5 12.Qc2 Nbd7 13.Rd1 Rc8 14.Ne5 Qc7 15.Nxd7 Nxd7 16.Bxh7+ Kh8 17.Be4 Be7 18.Bxb7 Qxb7 19.Qe4 Qc7 20.Bd2 Nc5 21.Qf3 Kg8 22.Rac1 Qe5 23.e4 b4 24.Ne2 Nxe4 25.Bf4 Qf5 26.Nd4 Qd5 27.Nc6 Qxc6 28.Rxc6 Rxc6 29.Qxe4 Rfc8 30.Be3 f5 31.Qf3 Bf6 32.Bd4 Bxd4 33.Rxd4 Rc1+ 34.Rd1 Rxd1+ 35.Qxd1 b3 36.axb3 Rd8 37.Qxd8+ Kf7 38.Qd7+ Kf8 39.Qxe6 f4 40.Qxa6 g5 41.Qf6+ Kg8 42.b4 g4 43.b5 Kh7 44.b6 Kg8 45.Qe7 Kh8 46.b7 f3 47.b8=Q# 1-0

So, I decided to repeat the test, after upgrading the Leela engine and the net, and playing her this time at 1 node, under a different GUI (Arena) and also the same time control for me (5'-5").

To my surprise the engine was too strong for me. Like you said, combinations all over the board (with only 1 node), I was totally massacred in every game by Leela. The best result I could get was a draw, a pawn up. Maybe I am still winning, but I was afraid to continue fight with only a few seconds in the clock. This Leela was a beast :) And I cannot explain why this difference compared to the first test. Maybe depth=1 is not the same as 1 node.

See for yourself, and compare the first game with this one... Huge difference

TECHNICAL DETAILS:

lc0 v0.20.2

net 32930

strength: 1 node

Arena for Linux 1.1

OS Linux Debian 64bit

openblas backend, CPU i7 6700HQ

Default UCI parameters except threads=1, no ponder

[Event "Computer chess game"]
[Site "Debian"]
[Date "2019.02.04"]
[Round "?"]
[White "Vassilis"]
[Black "LCZero_32930"]
[Result "1/2-1/2"]
[BlackElo "2000"]
[ECO "E23"]
[Opening "Nimzo-Indian"]
[Time "19:01:48"]
[Variation "Spielmann, 4...c5 5.dxc5 Nc6"]
[WhiteElo "?"]
[Termination "normal"]
[PlyCount "87"]
[WhiteType "human"]
[BlackType "program"]

1. d4 Nf6 2. c4 e6 3. Nc3 Bb4 4. Qb3 c5 5. dxc5 Nc6 6. a3 Bxc5 7. Nf3 d5 8. cxd5 exd5 9. e3 O-O 10. Be2 a6 11. O-O Qd6 12. Na4 Ba7 13. Nb6 Na5 14. Nxc8 Rfxc8 15. Qa4 Nc4 16. Bxc4 dxc4 17. Qc2 b5 18. Bd2 Qe6 19. a4 Bc5 20. axb5 axb5 21. Bc3 Ne4 22. Bd4 Bf8 23. Nd2 Nxd2 24. Qxd2 b4 25. Rac1 Ra2 26. Rc2 c3 27. bxc3 b3 28. Rxa2 bxa2 29. Ra1 Ra8 30. Qc2 Qc4 31. g3 h5 32. Kg2 h4 33. h3 hxg3 34. fxg3 Bd6 35. Qf2 Ra5 36. Qf3 Rb5 37. Qa8+ Rb8 38. Qxa2 Qc6+ 39. Kf2 Qe4 40. c4 Qf5+ 41. Kg2 Qe4+ 42. Kf2 Qf5+ 43. Kg2 Qe4+ 44. Kf2 {3-fold repetition} 1/2-1/2

You are from Russia(?) you guys play a lot of chess.

Please tell me what's going on.

Warren D Smith

unread,

Feb 8, 2019, 6:29:23 PM2/8/19

to Vassilis, LCZero

Presumably depth=1 is a lot different than nodes=1.

Depth=1 with a full width search would be equivalent to about 35 nodes
on average.

Still, beating you nearly always with depth=1 already is pretty impressive.

Warren D Smith

unread,

Feb 8, 2019, 6:34:49 PM2/8/19

to Vassilis, LCZero

Oh sorry, I had it backwards -- your story actually is:
you (Vassilis) were beating leela at depth=1 (nodes=35?)
with older leela, now newer leela is beating you at nodes=1.

OK, then:
1.evidently leela has improved tremendously since version you had played before.
2.this is incredible to me that 1 node (no search at all) can nearly
always beat you.

(It sounds like you are comparable in strength to me.)

Absolutely incredible. How can anything be that good?

Ilya Kalinin

unread,

Feb 8, 2019, 6:35:13 PM2/8/19

to LCZero

Well it depends on gui u use whether depth=1 means 1 node or not, but in my case (program ChessBot), You can choose a depth and it shows all the parameters whilst the game including amount of nodes, so it was 1 node on the depth 1 and it was clear

Ilya Kalinin

unread,

Feb 8, 2019, 6:36:55 PM2/8/19

to LCZero

Warren, Yeah that is just what I meant! It's absolutely incredible!

vladimir...@gmail.com

unread,

Feb 8, 2019, 6:37:12 PM2/8/19

to LCZero

If I understand correctly, at nodes=1 we play only vs. policy network (or policy head), and evaluation part doesn't affect anything at all. The engine just picks the highest-rated move from probability distribution. And at depth=1 we play purely vs. value network/head.

Vassilis

unread,

Feb 8, 2019, 7:02:06 PM2/8/19

to LCZero

But Vladimir it's only one net!

You think it could be used differently in these two cases. Perhaps!
The truth is Leela is unbeatable in 1 node, for my standards. With the 32930 net. The one which loses to Stockfish... at TCEC!
Simply incredible

Vassilis

unread,

Feb 8, 2019, 7:12:03 PM2/8/19

to LCZero

I was wondering why there are 2 UCI commands:
'go depth n' and 'go nodes n'

I'm convinced now it's not the same...

Vassilis

unread,

Feb 8, 2019, 7:20:25 PM2/8/19

to LCZero

Ilya, is chessbot this commercial product?
http://www.chess-bot.com

Is there a Linux version?

Ilya Kalinin

unread,

Feb 8, 2019, 7:45:59 PM2/8/19

to LCZero

No, It is not commercial product, but I can send u the direct link to download it for windows. I do not know if there is any linux version

Vassilis

unread,

Feb 8, 2019, 7:56:44 PM2/8/19

to LCZero

Ok, send me the Windows link, and I'll do some searching to see if there is a Linux version.
Thank you so much, Ilya :)

Ilya Kalinin

unread,

Feb 8, 2019, 7:57:14 PM2/8/19

to LCZero

Aah, sorry, i meant it is commercial project, it's not free but I have cracked version

Vassilis

unread,

Feb 8, 2019, 8:05:56 PM2/8/19

to LCZero

Ok sorry :)
Then it must be the one I thought!

jimgl...@gmail.com

unread,

Feb 8, 2019, 8:11:26 PM2/8/19

to LCZero

Interesting! Good experiment.

But I suggest being honest. Let people know that they will be playing Leela "handicapped to looking at only one node (!)" and I bet you'll get a lot of happy opponent takers.

Your results will be more robust too.

Even tell them "it is in the name of science", and they are helping accelerate the date of the singularity.

Ilya Kalinin

unread,

Feb 8, 2019, 8:20:36 PM2/8/19

to LCZero

ChessBot link https://send.firefox.com/download/d205cfd7ad/#-F_jW42-DHtY26H84T-XWw

Message has been deleted

Vassilis

unread,

Feb 9, 2019, 5:19:24 AM2/9/19

to LCZero

Thank you Ilya :)

Unfortunately ChessBot is a Windows only GUI so not of much use for me!
I know I can run it through wine or inside virtual box, but I prefer Linux native apps.

PS: Why don't you use Arena? www.playwitharena.com
It's free and rich in features.

Again, thank you.

Shah

unread,

Feb 9, 2019, 2:37:24 PM2/9/19

to LCZero

Ilya, thanks for the great information.
So, is it certain and verified that we are indeed talking about 1 single node and not anything else?
If indeed that is the case, it is astonishing.
I cannot even begin to understand it.
But if that is the case, doesn't it imply, that part of leela has somehow specialized in tactical evaluation?
And wouldn't that necessarily be a non optimized network??
As I understand it, the nn "should focus" on strategy, letting the tree search handle tactics.

vladimir...@gmail.com

unread,

Feb 9, 2019, 2:52:41 PM2/9/19

to LCZero

Is it possible to configure engine tournament in Arena when Leela will play at 1 node per move, while all other participants at 1 min + 1 sec. increment?
I would like to find this Leela level in my homebrew rating list.