Oakfoam a very weak go program?

[Hong Lu]

Hi all, 

I am new to Oakfoam go program and I have been evaluating the software in the past comparing it to fuego, pachi  and gnugo on strength.  In my own testing, I found that Oakfoam 

lost to gnugo, fuego, in 5 games match.  It even lost to fuego in 6 handicap advantage.  Am i missing anything here?  I expected Oakfoam go program to be a lot stronger 

given it uses  DCNN  and feels similarity to fuego. on its engine features. 

I have successfully compiled oakfoam version 0.2.1-dev(1.0).  I have caffe enabled in compiling.  However, my caffe version is CPU only so that I commented out some of 

GPU device related codes as suggest in this group.  Below is configuration file I used for the testing: 

param thread_count 2

param cnn_num_of_gpus 1

param playouts_per_move 10000

param playouts_per_move_max 10000

param playouts_per_move_min 500

param uct_expand_after 20

The above is saved as file  oakfoam.config,  then when I run it, I run it like oakfoam -c oakfoam.config in command line. 

The above configuration file idea came from some posts here mentioning the  NG04  bot configuration.  My hardware is duel core linux Ubuntu machine for the above testing environment.

Please correct me if I did wrong on above configurations.

Thanks, 

Hong Lu

Shoreline at DGS/KGS.

[Detlef Schmicker]

Hi Hong Lu,

the standard configuration is very weak. To get it strong you need to load a neural network and use a nvidia grafic card!

with this it is a quite strong program (NG-06 on cgos)

http://www.yss-aya.com/cgos/19x19/bayes.html

If you have this available I would be happy to share the necessary files with you!

Detlef

--
You received this message because you are subscribed to the Google Groups "Oakfoam" group.
To unsubscribe from this group and stop receiving emails from it, send an email to oakfoam+u...@googlegroups.com.
To post to this group, send email to oak...@googlegroups.com.
Visit this group at https://groups.google.com/group/oakfoam.
For more options, visit https://groups.google.com/d/optout.

[Hong Lu]

Hi Detlef, 

Thanks for the clarification on the strength and hardware requirement/   This GPU requirement makes oakfoam similar as Darkforest Go requires. 

Currently I don't have nor plan to have GPU purposed in near term.   Any way, Thanks for the info

Hong

[Detlef Schmicker]

I added the actual configuration in my branch

https://bitbucket.org/dsmic/oakfoam/src

the subdirectory contains the files necessary to run the cgos version.

Feel free to try it without gpu!

if you are a programmer:

I did not test, if the cpu only code of caffe is thread save (the gpu code is not), therefore I dont know if the lock

CNNmutex in engine.cc can be disabled. It might be possible to set 

param cnn_num_of_gpus 1

to let caffe use more threads?!

[Hong Lu]

Hi Detlef,

Thanks for releasing the  kgs bot  parameter and training data file.  I will test out the CPU_ONLY  mode in my computer to see what oakfoam's strength is under low end hardware setup.


By the way,  the code you released will compile, but will do core dump in my CPU only set up.  I need to do below patch to make the software runs:

diff  your version versus my version src/engine/Engine.cc  files  in order to make the oakfoam binary works under CPU_ONLY caffe setup.

40c40
<   Caffe::SetDevice(0);
---
>   //Caffe::SetDevice(0);
42c42,43
<   Caffe::set_mode(Caffe::GPU);
---
>   //Caffe::set_mode(Caffe::GPU);
>   Caffe::set_mode(Caffe::CPU);
45c46
<   Caffe::DeviceQuery();
---
>   //Caffe::DeviceQuery();
892,893c893,894
<   //Caffe::set_mode(Caffe::CPU);
<   Caffe::set_mode(Caffe::GPU);
---
>   Caffe::set_mode(Caffe::CPU);
>   //Caffe::set_mode(Caffe::GPU);

After I compile and run  oakfoam -c  nicego-cnn-06.gtp

The program runs, but will an error below (I assume it is not major issue):
I0201 15:00:21.937793 20795 net.cpp:746] Ignoring source layer tanh3
I0201 15:00:21.937811 20795 net.cpp:746] Ignoring source layer power_soll3
I0201 15:00:21.937832 20795 net.cpp:746] Ignoring source layer loss6
Attention, fixed coded as caffe support broken?!   --->>> !!!!!!!!!!!!!!!!!!!!!!!! 0 shape 20
seed: 1485205653

Hong Lu

Shoreline at DGS/KGS.

[Detlef Schmicker]

Thanks a lot, I will put your patch into my version tomorrow ...

No, the messages can be ignored, they come from my training net, which include more layers....

Detlef

--

[Hong Lu]

In deed,  my original thinking to use this

param cnn_num_of_gpus 1  is to enable multi-threaded function of CNN. 

By the way, it did not work, and will generate core dump under this mode in new code. 

I also commented out  2 lines of Engine.cc CNNmutex  code,  also got core dump in CPU_ONLY mode testing 

Just like you said,   caffee  CPU_ONLY mode is not thread safe.

I am now running the CPU_ONLY code with additional parameter to your config gtp file in my dual core Ubuntu machine:
param thread_count 2
param time_move_max 60

so far so good,  no core dump,  oakfoam can take 120% of CPU maximum,  not fully  using the 2 CPUs,  but still better than single CPU mode.
I will report back the strength test in my low end machine,  oakfoam verses fuego  matches.

Hong Lu

[Detlef Schmicker]

I did some ugly hacks with the num gpus.

I dont remember exactly, but if you set the thred_count and num_gpus before loading the neural net in the gtp file I think it makes several copies of the net and should be thread safe ...

I did this as a quick and dirty hack, as I was planing to rent an Amazon EC with 4 gpus for a tournament, vut did not do it :)

Detlef

> Am 01.02.2017 um 22:35 schrieb Hong Lu <hong...@gmail.com>:
>
> Lu

[Hong Lu]

I actually tested with thred_count num gpus 1  at beginning 2 lines of the file  nicego-cnn-06.gtp, still got core dumped in CPU_ONLY machine. 

I think in CPU_ONLY  machine,  it is not thread safe. 

Hong

[Hong Lu]

follow up on the early pass bug,  here is another example of Oakfoam issued PASS way early and lost game and resigned.  

Here is log file of the location where Oakfoam issued PASS:

Attention, fixed coded as caffe support broken?!   --->>> !!!!!!!!!!!!!!!!!!!!!!!! 0 shape 20

seed: 1485951277

loading opening book from 'book.dat'... done

[genmove]: r:0.180 v:0.000 plts:10007 ppms:1.04 cnncs:4 cnnps:0.42 rd:-0.180 r2:1.00 fs:-13.98 eq:0 eq2:0 fsd:0.53 un:1/1(-0) bs:1  un:(PASS)st:(0,0,0,0,0,0,0,0,0,0,0,0,0,98,0,93,6,16,0,0,0,51,394,337,0) ravepreset: -nan expand_num: 8.24979

play Black pass

[Hong Lu]

Latest Update:

There still seems to be have early Oakfoam pass issues in the software, not totally fixed. 

However,  under my 1 minute NG06 setting in low end hardware,   Oakfoam  DCNN version  software strength looks very impressive under my new testing. 

Oakfoam  won 9 to 1 against fuego  in even games,   It also won 3 to 2  in 5 games against DCNN version of Pachi as black. 

This would put Oakfoam as strongest go program among three open source go program.