The new no-gimmicks NUMA branch

633 views
Skip to first unread message

mcos...@gmail.com

unread,
Sep 23, 2016, 9:27:58 AM9/23/16
to FishCooking
I have rewritten NUMA branch starting this time from Texel code and removing all the countermove table stuff, so that between current master and this branch:


The ONLY difference is the numa-aware binding of the running threads.

I have started a test here:


Once we have a test result as our baseline, next step is to add numa-aware memory allocation of the per-thread tables, this simply means that the memory allocation will be done after the binding of the thread so that possibly the OS will allocate memory in the same node of the running thread.

Joachim Müller (JojoM)

unread,
Sep 25, 2016, 8:46:49 AM9/25/16
to FishCooking
I tried to run this test and received the following error:

numa.cpp:31:18: fatal error: numa.h: No such file or directory

 #include <numa.h>

                  ^

compilation terminated.

<builtin>: recipe for target 'numa.o' failed

make[2]: *** [numa.o] Error 1

make[2]: *** Waiting for unfinished jobs....

syzygy/tbprobe.cpp: In function 'probe_ab(Position&, int, int, int*)':

syzygy/tbprobe.cpp:205:14: warning: array subscript is above array bounds [-Warray-bounds]

         p[i++] = pop_lsb(&bb) ^ mirror;

              ^

syzygy/tbprobe.cpp: In function 'Tablebases::probe_dtz(Position&, int*)':

syzygy/tbprobe.cpp:315:14: warning: array subscript is above array bounds [-Warray-bounds]

         p[i++] = pop_lsb(&bb) ^ mirror;

              ^

make[2]: Leaving directory '/tmp/tmpoJslV6/mcostalba-Stockfish-1e7e83b/src'

Makefile:484: recipe for target 'gcc-profile-make' failed

make[1]: *** [gcc-profile-make] Error 2

make[1]: Leaving directory '/tmp/tmpoJslV6/mcostalba-Stockfish-1e7e83b/src'

Makefile:408: recipe for target 'profile-build' failed

make: *** [profile-build] Error 2


Exception running games:

Traceback (most recent call last):

  File "worker.py", line 88, in worker

    run_games(worker_info, password, remote, run, task_id)

  File "/u/muellerj/X0/fishtest-master/worker/games.py", line 383, in run_games

    setup_engine(new_engine, binaries_url, worker_dir, run['args']['resolved_new'], repo_url, worker_info['concurrency'])

  File "/u/muellerj/X0/fishtest-master/worker/games.py", line 152, in setup_engine

    build(worker_dir, sha, repo_url, destination, concurrency)

  File "/u/muellerj/X0/fishtest-master/worker/games.py", line 131, in build

    subprocess.check_call(MAKE_CMD + ' -j %s' % (concurrency), shell=True)

  File "/usr/lib64/python2.7/subprocess.py", line 541, in check_call

    raise CalledProcessError(retcode, cmd)

CalledProcessError: Command 'make profile-build COMP=gcc ARCH=x86-64-modern -j 32' returned non-zero exit status 2

Task exited

Ronald de Man

unread,
Sep 25, 2016, 8:53:06 AM9/25/16
to FishCooking
On Linux, install libnuma including header files.

Which package you need probably depends on your distribution. On Fedora you need numactl and numactl-devel (installing the latter probably installs the former as well).

rod...@tjader.com.br

unread,
Sep 25, 2016, 9:13:25 AM9/25/16
to FishCooking

This does not compile on Linux, as it is missing the definitions of NodeInfo, str2Num and num2Str.

Joachim Müller (JojoM)

unread,
Sep 25, 2016, 9:17:22 AM9/25/16
to FishCooking
So it is a pity for Fishtest I do not have the rights to install the required components on this machine.

Joachim Müller (JojoM)

unread,
Sep 25, 2016, 9:18:22 AM9/25/16
to FishCooking, rod...@tjader.com.br
Not that would be in a position to install Windows either ... ;)

Ronald de Man

unread,
Sep 25, 2016, 10:55:27 AM9/25/16
to FishCooking
On Sunday, September 25, 2016 at 3:17:22 PM UTC+2, Joachim Müller (JojoM) wrote:
So it is a pity for Fishtest I do not have the rights to install the required components on this machine.


Whether you want to go this far is entirely up to you, but in theory you could download the libnuma source from here:

ftp://oss.sgi.com/www/projects/libnuma/download/

and compile it yourself. But you will have to fiddle with the Stockfish Makefile and will have to know how to add an include path and get libnuma either dynamically or statically linked. If you are not familiar with that, then it is better not to bother and avoid unnecessary frustration ;-)

Joachim Müller (JojoM)

unread,
Sep 25, 2016, 11:58:21 AM9/25/16
to FishCooking
On Sunday, September 25, 2016 at 4:55:27 PM UTC+2, Ronald de Man wrote:
On Sunday, September 25, 2016 at 3:17:22 PM UTC+2, Joachim Müller (JojoM) wrote:
So it is a pity for Fishtest I do not have the rights to install the required components on this machine.

Whether you want to go this far is entirely up to you,

As I said, I do not have the permission to change the setup of this machine.
 
but in theory you could download the libnuma source from here:

ftp://oss.sgi.com/www/projects/libnuma/download/

and compile it yourself. But you will have to fiddle with the Stockfish Makefile and will have to know how to add an include path and get libnuma either dynamically or statically linked. If you are not familiar with that, then it is better not to bother and avoid unnecessary frustration ;-)

I would need a step-by-step fool-proof instruction on how to exactly  achieve this.

joost.van...@gmail.com

unread,
Sep 25, 2016, 12:43:00 PM9/25/16
to FishCooking
> I would need a step-by-step fool-proof instruction on how to exactly  achieve this.

Something like:

wget ftp://oss.sgi.com/www/projects/libnuma/download/numactl-2.0.11.tar.gz
tar -xzvf numactl-2.0.11.tar.gz
cd numactl-2.0.11
./configure --prefix=/data/vjoost/chess/numactl-2.0.11/install
make -j
make -j install
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/data/vjoost/chess/numactl-2.0.11/install/lib/
export CPLUS_INCLUDE_PATH=$CPLUS_INCLUDE_PATH:/data/vjoost/chess/numactl-2.0.11/install/include/

will install the library and the exports (which needs to be executed every time you create a new shell) will make it available to gcc's default search. You should replace '/data/vjoost/chess/numactl-2.0.11/' with whatever place you downloaded to (can be your home).

For me the latest numa branch doesn't compile (the older did).

peter.zs...@gmx.at

unread,
Sep 25, 2016, 12:47:04 PM9/25/16
to FishCooking
Hello just my 2 cents:
No problems to compile numa-branch on Linux for me (Ubuntu 16.04).
I only had to install libnuma-dev.

About the test: I ran it on my machine on Windows, but only 200 games. It took about 16hours! Draw rate seems to be in the 900-range. I do not support this test anymore.

br CoffeeOne

Marco Costalba

unread,
Sep 25, 2016, 4:32:08 PM9/25/16
to Peter Zsifkovits, FishCooking
Thanks Peter for your support!

I will stop teh test, indeed it seems teh 2 branches are equivalent also in speed.

Eventually, if you have time and interest, you may do some speed test on those 2 branches, I expect speed to be largely equivalent too.

Peter Zsifkovits

unread,
Sep 26, 2016, 5:02:25 PM9/26/16
to Marco Costalba, FishCooking
Hello,

Welcome.
I made the speedtest now of the 2 versions, which were compared on fishtest (the 200 games):

I used: 
Engine 1: Master: https://github.com/mcostalba/Stockfish/tree/28240d375c837f2342163e84675bc9230124cd30
Engine 2: Numa-Texel: https://github.com/mcostalba/Stockfish/tree/1e7e83b2b1ee492767695a55414a12bab3620e95

both versions compiled with non-profile(!) build with GCC 6.2, 20 runs each, 64 Thread Test, 8 Numa Nodes.
Variation is very high, but that is to be expected in a 64 thread test. Total time was still below 2 hours, so not problematic.
Do not compare the absolute values with any tests before from me (different CPUs, different compiler).

Build Tester: 1.4.6.0
Windows Server 2012 R2 (Version 6.3, Build 0, 64-bit Edition)
AMD Opteron(tm) Processor 6386 SE             
SafeMode: No
Running In VM: No
HyperThreading Enabled: Yes
CPU Warmup: Yes
Command Line: bench 1024 64 24  default depth
Tests per Build: 20
ANOVA: n/a

                Engine# (NPS)                     Speedup     Sp     Conf. 95%    S.S.
2  (30 639 302,0  ) ---> 1  (26 971 119,8  ) ---> 13,600%  1 113 793,3    Yes       No

See more details in the attached screenshot.

About Linux: Sorry, I mixed up things, I did not test this actual branch (which i benched on Windows here) yet.
So if you decide to continue with the "numa-texel" variant, of course linux compilation should be fixed, if there is an issue. I can play with it under Linux on weekend.

br
numa-texel-64.JPG

lp

unread,
Sep 26, 2016, 5:22:19 PM9/26/16
to FishCooking
I had numa with texel files from the time of common table table, but I could not test it because of hardware on linux, and was not sure if it is working on windows.
But it does compile on linux. So if you want to check if it is working correctly,  here is Marco branch with few changes.
Reply all
Reply to author
Forward
0 new messages