[Computer-go] CNN with 54% prediction on KGS 6d+ data

[Detlef Schmicker]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

as somebody ask I will offer my actual CNN for testing.

It has 54% prediction on KGS 6d+ data (which I thought would be state
of the art when I started training, but it is not anymore:).

it has:
1
2
3
> 4 libs playing color
1
2
3
> 4 libs opponent color
Empty points
last move
second last move
third last move
forth last move

input layers, and it is fully convolutional, so with just editing the
golast19.prototxt file you can use it for 13x13 as well, as I did on
last sunday. It was used in November tournament as well.

You can find it
http://physik.de/CNNlast.tar.gz

If you try here some points I like to get discussion:

- - it seems to me, that the playouts get much more important with such
a strong move prediction. Often the move prediction seems better the
playouts (I use 8000 at the moment against pachi 32000 with about 70%
winrate on 19x19, but with an extremely focused progressive widening
(a=400, a=20 was usual).

- - live and death becomes worse. My interpretation is, that the strong
CNN does not play moves, which obviously do not help to get a group
life, but would help the playouts to recognize the group is dead.
(http://physik.de/example.sgf top black group was with weaker move
prediction read very dead, with good CNN it was 30% alive or so :(

OK, hope you try it, as you know our engine oakfoam is open source :)
We just merged all the CNN stuff into the main branch!
https://bitbucket.org/francoisvn/oakfoam/wiki/Home
http://oakfoam.com

Do the very best with the CNN

Detlef

code:
if (col==Go::BLACK) {
for (int j=0;j<size;j++)
for (int k=0;k<size;k++)
{
for (int l=0;l<caffe_test_net_input_dim;l++)
data[l*size*size+size*j+k]=0;
//fprintf(stderr,"%d %d %d\n",i,j,k);
int pos=Go::Position::xy2pos(j,k,size);
int libs=0;
if (board->inGroup(pos))
libs=board->getGroup(pos)->numRealLibs()-1;
if (libs>3) libs=3;
if (board->getColor(pos)==Go::BLACK)
{
data[(0+libs)*size*size + size*j + k]=1.0;
//data[size*size+size*j+k]=0.0;
}
else if (board->getColor(pos)==Go::WHITE)
{
//data[j*size+k]=0.0;
data[(4+libs)*size*size + size*j + k]=1.0;
}
else if
(board->getColor(Go::Position::xy2pos(j,k,size))==Go::EMPTY)
{
data[8*size*size + size*j + k]=1.0;
}
}
}
if (col==Go::WHITE) {
for (int j=0;j<size;j++)
for (int k=0;k<size;k++)
{//fprintf(stderr,"%d %d %d\n",i,j,k);
for (int l=0;l<caffe_test_net_input_dim;l++)
data[l*size*size+size*j+k]=0;
//fprintf(stderr,"%d %d %d\n",i,j,k);
int pos=Go::Position::xy2pos(j,k,size);
int libs=0;
if (board->inGroup(pos))
libs=board->getGroup(pos)->numRealLibs()-1;
if (libs>3) libs=3;
if (board->getColor(pos)==Go::BLACK)
{
data[(4+libs)*size*size + size*j + k]=1.0;
//data[size*size+size*j+k]=0.0;
}
else if (board->getColor(pos)==Go::WHITE)
{
//data[j*size+k]=0.0;
data[(0+libs)*size*size + size*j + k]=1.0;
}
else if (board->getColor(pos)==Go::EMPTY)
{
data[8*size*size + size*j + k]=1.0;
}
}
}
if (caffe_test_net_input_dim > 9) {
if (board->getLastMove().isNormal()) {
int j=Go::Position::pos2x(board->getLastMove().getPosition(),size);
int k=Go::Position::pos2y(board->getLastMove().getPosition(),size);
data[9*size*size+size*j+k]=1.0;
}
if (board->getSecondLastMove().isNormal()) {
int
j=Go::Position::pos2x(board->getSecondLastMove().getPosition(),size);
int
k=Go::Position::pos2y(board->getSecondLastMove().getPosition(),size);
data[10*size*size+size*j+k]=1.0;
}
if (board->getThirdLastMove().isNormal()) {
int
j=Go::Position::pos2x(board->getThirdLastMove().getPosition(),size);
int
k=Go::Position::pos2y(board->getThirdLastMove().getPosition(),size);
data[11*size*size+size*j+k]=1.0;
}
if (board->getForthLastMove().isNormal()) {
int
j=Go::Position::pos2x(board->getForthLastMove().getPosition(),size);
int
k=Go::Position::pos2y(board->getForthLastMove().getPosition(),size);
data[12*size*size+size*j+k]=1.0;
}
}

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iQIcBAEBAgAGBQJWZvOlAAoJEInWdHg+Znf4t8cP/2a9fE7rVb3Hz9wvdMkvVkFS
4Y3AomVx8i56jexVyXuzKihfizVRM7x6lBiwjYBhj4Rm9UFWjj2ZvDzBGCm3Sy4I
SpG8D01VnzVR6iC1YTu3ecv9Wo4pTjc7NL5pAxiZDB0V7OTRklfZAYsX4mWyHygn
cr1pIb79/9QfBf/johmuutXJIwYfVG9ShR1+udbxs3aU3QDAbJJ4eTs8oj+NqFpg
JolEEEg3wY693e77SqbUbjxR3kSsysoz9h1nKnR/ZjHByqlwNvSz9ho9eU0rKhaK
GSQ22/c1VPIZhr24FYBbYNYweOzDtonLpuUFCPSnYVels3h/I/LlqV3MeDo6wuZ2
QCPp5+11o4JzvEt7A4zfJCtEOEH0W2/+IjRcIkAVOo65OV/pPsz2EjHehMU6PC6m
vXA/kPx0jqUm1qSb0qCgMq5ZvSqfpcCY7JOlkEwkDBS1fty9sU0hqst3zXR0KGtn
rFuoREmQYi/mkjZfS2Q4AHiZUDbDZUKzRegUA+gR/eKAmJsmWeTDEI9ZAXgxL0cB
p1HGBNDEUKGk+ruq0gIe5vYygyBcJV0BbbBnweDjeZnlG8vLUAVoMF6V/q3gkZb1
P61rfE4d9dohfGBsZ+UWltRyWMj09ieR2G2zCDpIXyxEuoV6CTAlLzDuhmqFa2ma
Fp3lK/uLhOucXwBtStdx
=E47K
-----END PGP SIGNATURE-----
_______________________________________________
Computer-go mailing list
Compu...@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

[Michael Markefka]

Hello Detlef,

I've got a question regarding CNN-based Go engines I couldn't find
anything about on this list. As I've been following your posts here, I
thought you might be the right person to ask.

Have you ever tried using the CNN for complete playouts? I know that
CNNs have been tried for move prediction, immediate scoring and move
generation to be used in an MC evaluator, but couldn't find anything
about CNN-based playouts.

It might only be feasible to play out the CNN's first choice move for
evaluation purposes, but considering how well the performance of batch
sizes scales, especially on GPU-based CNN applications, it might be
possible to setup something like 10 candidate moves, 10 reply
candidate moves and then have the CNN play out the first choice move
for those 100 board positions until the end and then sum up scores
again for move evaluation (and/or possibly apply some other tried and
tested methods like minimax). Given that the number of 10 moves is
supposed to be illustrative rather than representative, other
configurations of depth and width in position generation and
evaluation would be possible.

It feels like CNN can provide a very focused, high-quality width in
move generation, but it might also be possible to apply that quality
to depth of evaluation.

Any thoughts to share?

All the best

Michael

[Petr Baudis]

Hi!

In case someone is looking for a starting point to actually implement
Go rules etc. on GPU, you may find useful:

https://www.mail-archive.com/compu...@computer-go.org/msg12485.html

I wonder if you can easily integrate caffe GPU kernels in another GPU
kernel like this? But without training, reimplementing the NN could be
pretty straightforward.

--
Petr Baudis
If you have good ideas, good data and fast computers,
you can do almost anything. -- Geoffrey Hinton

[Josef Moudrik]

Regarding full CNN playouts, I think that problem is that a playout is a long serial process, given 200-300 moves a game. You need to construct planes and transfer them to GPU for each move and read result back (at least with current CNN implementations afaik), so my guess would be that such playout would take time in order of seconds. So there seems to be a tradeoff, CNN playouts are (probably much) better (at "playing better games") than e.g. distribution playouts, but whether this is worth the implied (probably much) lower height of the MC tree is a question.

Maybe if you had really a lot of GPUs and very high thinking time, this could be the way.

Josef 

[Petr Baudis]

Hi!

Well, for this to be practical the entire playout would have to be
executed on the GPU, with no round-trips to the CPU. That's what my
email was aimed at.

[Josef Moudrik]

Yes, that's why I wrote with current CNN implementations. But I still wonder whether my estimate for the round-trip length is at least of the correct magnitude.

Josef

[Álvaro Begué]

I don't think the CPU-GPU communication is what's going to kill this idea. The latency in actually computing the feed-forward pass of the CNN is going to be in the order of 0.1 seconds (I am guessing here), which means finishing the first playout will take many seconds.

So perhaps it would be interesting to do something like this for correspondence games, but not for regular games.


Álvaro.

[David Ongaro]

Did everyone forget the fact that stronger playouts don't necessarily lead to an better evaluation function? (Yes, that what playouts essential are, a dynamic evaluation function.) This is even under the assumption that we can reach the same number of playouts per move.

[Hideki Kato]

As NNs basically learn the frequency of each move, using the value as
its probability to be chosen in a simulation could be ok.

Hideki

David Ongaro: <6C2FF906-2A00-45C1...@hamburg.de>:

>> > > > > http://physik.de/CNNlast.tar.gz <http://physik.de/CNNlast.tar.gz>

>> > > > >
>> > > > >
>> > > > >
>> > > > > If you try here some points I like to get discussion:
>> > > > >
>> > > > > - - it seems to me, that the playouts get much more important with such
>> > > > > a strong move prediction. Often the move prediction seems better the
>> > > > > playouts (I use 8000 at the moment against pachi 32000 with about 70%
>> > > > > winrate on 19x19, but with an extremely focused progressive widening
>> > > > > (a=400, a=20 was usual).
>> > > > >
>> > > > > - - live and death becomes worse. My interpretation is, that the strong
>> > > > > CNN does not play moves, which obviously do not help to get a group
>> > > > > life, but would help the playouts to recognize the group is dead.

>> > > > > (http://physik.de/example.sgf <http://physik.de/example.sgf> top black group was

>with weaker move
>> > > > > prediction read very dead, with good CNN it was 30% alive or so :(
>> > > > >
>> > > > >
>> > > > > OK, hope you try it, as you know our engine oakfoam is open source :)
>> > > > > We just merged all the CNN stuff into the main branch!
>> > > > > https://bitbucket.org/francoisvn/oakfoam/wiki/Home

><https://bitbucket.org/francoisvn/oakfoam/wiki/Home>
>> > > > > http://oakfoam.com <http://oakfoam.com/>

>> > > > > Compu...@computer-go.org <mailto:Compu...@computer-go.org>
>> > > > > http://computer-go.org/mailman/listinfo/computer-go

><http://computer-go.org/mailman/listinfo/computer-go>
>> > > > _______________________________________________
>> > > > Computer-go mailing list

>> > > > Compu...@computer-go.org <mailto:Compu...@computer-go.org>
>> > > > http://computer-go.org/mailman/listinfo/computer-go

><http://computer-go.org/mailman/listinfo/computer-go>
>> > >
>> > > --
>> > > Petr Baudis
>> > > If you have good ideas, good data and fast computers,
>> > > you can do almost anything. -- Geoffrey Hinton
>> > > _______________________________________________
>> > > Computer-go mailing list

>> > > Compu...@computer-go.org <mailto:Compu...@computer-go.org>
>> > > http://computer-go.org/mailman/listinfo/computer-go

><http://computer-go.org/mailman/listinfo/computer-go>
>>
>> > _______________________________________________
>> > Computer-go mailing list

>> > Compu...@computer-go.org <mailto:Compu...@computer-go.org>
>> > http://computer-go.org/mailman/listinfo/computer-go

><http://computer-go.org/mailman/listinfo/computer-go>
>>
>>
>> --
>> Petr Baudis
>> If you have good ideas, good data and fast computers,
>> you can do almost anything. -- Geoffrey Hinton
>> _______________________________________________
>> Computer-go mailing list

>> Compu...@computer-go.org <mailto:Compu...@computer-go.org>
>> http://computer-go.org/mailman/listinfo/computer-go

><http://computer-go.org/mailman/listinfo/computer-go>
>> _______________________________________________
>> Computer-go mailing list
>> Compu...@computer-go.org
>> http://computer-go.org/mailman/listinfo/computer-go

>---- inline file
>_______________________________________________

>Computer-go mailing list

>Compu...@computer-go.org

>http://computer-go.org/mailman/listinfo/computer-go
--
Hideki Kato <mailto:hideki...@ybb.ne.jp>

[Álvaro Begué]

Of course whether these "neuro-playouts" are any better than the heavy playouts currently being used by strong programs is an empirical question. But I would love to see it answered...

[Michael Markefka]

Thank you for the feedback, everyone.

Regarding the CPU-GPU roundtrips, I'm wondering whether it'd be
possible to recursively apply the output matrix to the prior input
matrix to update board positions within the GPU and without any
actual (possibly CPU-based) evaluation until all branches come up with
game ending states. I assume illegal moves would mostly fall away when
sticking to the top ten or top five move considerations provided by
the CNN.

As for performance, I could imagine initialization being relatively
slow, but wouldn't be surprised if the GPU-based CNN performance could
offer a branch size, running through many parallel boards with
comparatively minor performance impact, where this outweighed the
initial overhead again.

Whether this would provide a better evaluation function than MCTS I
don't know, but just like Alvaro I would love to see this tried, even
if just to rule it out for the moment.

I've got a GTX 980 Ti on a 4790k with 16 GB at home. For a low key
test I could run Windows (CUDA installed and running, tested with
pylearn2) or Ubuntu from a live setup on USB and would be willing to
run test code, if somebody provided a package I could simply download
and execute.

All the best

Michael

[Igor Polyakov]

I doubt that the illegal moves would fall away since every professional
would retake the ko... if it was legal

[Michael Markefka]

I think ko moves are taken into account on one of in the input planes
for most configurations. At least I hope remember that correctly.
Could it be achieved to create such a plane from the prior input
matrix and following output matrix by difference?

[Aja Huang]

On Tue, Dec 8, 2015 at 4:37 PM, Josef Moudrik <j.mo...@gmail.com> wrote:

Regarding full CNN playouts, I think that problem is that a playout is a long serial process, given 200-300 moves a game. You need to construct planes and transfer them to GPU for each move and read result back (at least with current CNN implementations afaik), so my guess would be that such playout would take time in order of seconds. So there seems to be a tradeoff, CNN playouts are (probably much) better (at "playing better games") than e.g. distribution playouts, but whether this is worth the implied (probably much) lower height of the MC tree is a question.

You may want to take a look at this paper:

Convolutional Monte Carlo Rollouts in Go

http://arxiv.org/pdf/1512.03375v1.pdf

Aja

[Hiroshi Yamashita]

Hi Detlef,

Thank you for publishing your data and latest oakform code!
It was very helpful for me.

I tried your 54% data with Aya.

Aya with Detlef54% vs Aya with Detlef44%, 10000 playout/move
Aya with Detlef54%'s winrate is 0.569 (124wins / 218games).

CGOS BayseElo rating
Aya with Detlef44% (aya786n_Detlef_10k) 3040
Aya with Detlef54% (Aya786m_Det54_10k ) 3036
http://www.yss-aya.com/cgos/19x19/bayes.html

Detlef54% is a bit stronger in selfplay, but they are similar on CGOS.
Maybe Detlef54%'s prediction is strong, and Aya's playout strength
is not enough.

Speed for a position on GTS 450.
Detlef54% 21ms
Detlef44% 17ms

Cumulative accuracy from 1000 pro games.

move rank Aya Detlef54% Mixture
1 40.8 47.6 48.0
2 53.5 62.4 62.7
3 60.2 70.7 71.0
4 64.8 75.8 76.1
5 68.1 79.5 79.9
6 71.0 82.3 82.6
7 73.2 84.5 84.8
8 75.2 86.3 86.6
9 76.9 87.8 88.1
10 78.3 89.0 89.3
11 79.6 90.2 90.6
12 80.8 91.2 91.4
13 81.9 92.0 92.2
14 82.9 92.7 92.9
15 83.8 93.3 93.5
16 84.6 93.9 94.1
17 85.4 94.3 94.5
18 86.1 94.8 95.0
19 86.8 95.2 95.4
20 87.4 95.5 95.7

Mixture is pretty same as Detlef54%.
I changed learning method from MM to LFR.
Aya's own accuracy is from LFR rank, not MM gamma.
So comparison is difficult.

Cumulative accuracy Detlef44%
http://computer-go.org/pipermail/computer-go/2015-October/008031.html

Regards,
Hiroshi Yamashita

_______________________________________________

[Detlef Schmicker]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

I am fighting with the problem most seem to have with the strong move
predictions at the moment, MCTS is not increasing the players a lot :)

I wonder, if somebody measured the performance of the pure CNN54
against pachi 10k (or 100k), to get a comparison with the darkforest CNN.

It is not too much work, but you probably did it already.

Thanks,

Detlef

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iQIcBAEBAgAGBQJWglFcAAoJEInWdHg+Znf42q4P/AnMdgqhps4RSJG3NoLiwEUq
QmT4mQd58WbuxnXRO4xiyIKGTQq13+FOpqVu7RgFPXxQaKS+8Hi1qpGVjg8aE8Zh
bnHb3D+p30hv9lCT8e4xNQ2B1JZsgOlM3MsbeFdQB+vxca3kUcnCf9oMvHo0W8TL
Tl8q7sDbI1bW0Z16lCKfDHdwyiBhDjETPP9j1wlfZgXyqD5JMCqwxcUkOrxlsh96
ZhX5bCnbN5CAPKedTxQVz8GcPwo74TIXCb+UmzklVOBC3pGJ3WrtWmNyHiPwiJ75
qYEzolICvW+wE+RbCfeiGaaL1CY9B5N2GKSCPQdzd0UYUwBrXsUMG3mTJ5Kwg26G
+nIg/KBnWCbgjN9WpHVkAsRewkAGezom7OSp2y1KyrIORcQc3FW8LLWxhzXjBNuj
3VFx9iT6zSiO+5kjUINdejVh4cT19Oao+ZVWZuPyBf9y/dcUn01NE2tCr+xIcqFq
7p+R0y9VA15f/KDufgJHUeeaPCdox6YU4VlxlbQoKdQt/X6iQftxPEDcBe39kxRy
R7SGJ6sMYxJBbsnNFfb547jBpeJRunHaX2dswjZtKleEUSTGXKgs77/ju3kbgC8n
WZuvvs6QcPqPsAyFFBsYbpOelP2NT7jpMX7IdkiGLb5wpblUhtnkV+nTy2ccTG1e
veWukuo97oFFhUBSHQtV
=+mE4
-----END PGP SIGNATURE-----

[Josef Moudrik]

Hi,

Winrate of your pure CNN againts pachi retsugen is:

GAMES WINRATE S.D.    PAIRING

224   0.558   0.033   19-7.5-1-pachi-=10000-detlef_54

221   0.407   0.033   19-7.5-1-pachi-=20000-detlef_54

I used the

https://github.com/jmoudrik/deep-go-wrap

for the player.

Regards,

Josef