NUMA test compilation

[Joachim Müller (JojoM)]

During compilation of the current NUMA patch as taken from Thomas Zipproth, I get the following error:

g++ -Wall -Wcast-qual -fno-exceptions -fno-rtti -std=c++11 -fprofile-generate -pedantic -Wextra -Wshadow -m64 -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o thread.o thread.cpp

In file included from numasf.cpp:1:0:

numasf.h:16:18: fatal error: numa.h: No such file or directory

 #include <numa.h>

                  ^

compilation terminated.

<builtin>: recipe for target 'numasf.o' failed

make[2]: *** [numasf.o] Error 1

make[2]: *** Waiting for unfinished jobs....

In file included from thread.cpp:29:0:

numasf.h:16:18: fatal error: numa.h: No such file or directory

 #include <numa.h>

                  ^

compilation terminated.

Can I do something about that?

[mibere]

Same on OS X 10.11.6


g++ -Wall -Wcast-qual -fno-exceptions -fno-rtti -std=c++11  -pedantic -Wextra -Wshadow -m64 -arch x86_64 -mmacosx-version-min=10.9 -DNDEBUG -O3 -mdynamic-no-pic -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o movepick.o movepick.cpp
g++ -Wall -Wcast-qual -fno-exceptions -fno-rtti -std=c++11  -pedantic -Wextra -Wshadow -m64 -arch x86_64 -mmacosx-version-min=10.9 -DNDEBUG -O3 -mdynamic-no-pic -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o numasf.o numasf.cpp

In file included from numasf.cpp:1:

./numasf.h:16:10: fatal error: 'numa.h' file not found
#include <numa.h>
         ^
1 error generated.
make[1]: *** [numasf.o] Error 1
make: *** [build] Error 2

Exception running games:
Traceback (most recent call last):
  File "worker.py", line 88, in worker
    run_games(worker_info, password, remote, run, task_id)
  File "/Users/xxx/fishtest/worker/games.py", line 383, in run_games
    setup_engine(new_engine, binaries_url, worker_dir, run['args']['resolved_new'], repo_url, worker_info['concurrency'])
  File "/Users/xxx/fishtest/worker/games.py", line 152, in setup_engine
    build(worker_dir, sha, repo_url, destination, concurrency)
  File "/Users/xxx/fishtest/worker/games.py", line 129, in build
    subprocess.check_call(make_cmd, shell=True)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 540, in check_call
    raise CalledProcessError(retcode, cmd)
CalledProcessError: Command 'make build ARCH=osx-x86-64-modern || make build ARCH=x86-64-modern' returned non-zero exit status 2
Task exited

[Ronald de Man]

On Saturday, November 5, 2016 at 6:59:41 PM UTC+1, Joachim Müller (JojoM) wrote:

During compilation of the current NUMA patch as taken from Thomas Zipproth, I get the following error:

Note that this is Mohammed's patch that we have seen earlier and not something of Zipproth's own making.

In file included from thread.cpp:29:0:

numasf.h:16:18: fatal error: numa.h: No such file or directory

 #include <numa.h>

                  ^

compilation terminated.

Can I do something about that?

Same problem as you had before:
 https://groups.google.com/d/msg/fishcooking/2FG7cChRxu0/jD4eRkFiBAAJ

So the same solution would apply. But as I said then, it is going to be a very frustrating process if you cannot do this without "step-by-step fool-proof" instructions.

[Thomas Zipproth]

Most likely libnuma is missing.
sudo apt-get install libnuma-dev
https://www.howtoinstall.co/en/ubuntu/precise/libnuma-dev

One Remark:
That code is original Stockfish testing code with only small output modifications from me.
It seemed to have advantages both on Numa and non-Numa machines (shared History Table), so I used it in Brainfish where avoiding additional complexity is not so important as in Stockfish.

On Saturday, November 5, 2016 at 6:59:41 PM UTC+1, Joachim Müller (JojoM) wrote:

[Joachim Müller (JojoM)]

Thank you for trying to help and excuse me for falsely attributing the patch to Thomas instead if Mohammed. I do not have super user / admin rights on the machine in question so I am sorry I cannot participate in this test.

[Ronald de Man]

On Saturday, November 5, 2016 at 8:02:46 PM UTC+1, Joachim Müller (JojoM) wrote:

Thank you for trying to help and excuse me for falsely attributing the patch to Thomas instead if Mohammed. I do not have super user / admin rights on the machine in question so I am sorry I cannot participate in this test.

If at least the libnuma library is installed on the machine (and if it is a NUMA machine running Linux, how could it not have libnuma installed -- see if you have libnuma.so in /usr/lib64 or in a directory with a similar name), then that is sufficient to run the program. You could compile it on another machine that has the libnuma header files or get an executable from someone else.

[Joachim Müller (JojoM)]

I'll see what I can do ...

[Joachim Müller (JojoM)]

@Ronald: In /usr/lib64 I can indeed, giving the command ls -lh | grep numa, locate the following file:

-rwxr-xr-x  1 root root   48K Dec 29  2015 libnuma.so.1

Will that be of any help? 

[Joachim Müller (JojoM)]

By the way, the current numasf test has been carried out exclusively by Windows machines ;)

[Ronald de Man]

Yes, it means you can run executables that use libnuma.

Also this simplifies things a lot if you want to compile the numasf patch yourself: there is no need to install or even compile libnuma!

Just take the attached numa.h file and place it in your stockfish/src directory. (I think there is no other numa.h file there, only a numasf.h, otherwise you will have to rename this numa.h to numa2.h or so.)

Then replace the line
#include <numa.h>
with
#include "numa.h"
(or numa2.h)

Now compile as usual. I think this should work.

[Joachim Müller (JojoM)]

Within the fishtest-master directory, what would I have to modify in order to get worker.py to "do the right thing" (and to not produce the error named above)? I cannot see a directory named stockfish/src in there. So where should I put the numa.h file you attached? And in which file (and where exactly) should the #include "numa.h" replacement be made?

[Ronald de Man]

On Sunday, November 6, 2016 at 4:21:13 PM UTC+1, Joachim Müller (JojoM) wrote:

Within the fishtest-master directory, what would I have to modify in order to get worker.py to "do the right thing" (and to not produce the error named above)? I cannot see a directory named stockfish/src in there. So where should I put the numa.h file you attached? And in which file (and where exactly) should the #include "numa.h" replacement be made?

You should put numa.h wherever you have numasf.cpp and numasf.h that is causing the error. And you have to change the #include in numasf.h, because that is where the line is located. See the error message you posted. And if you get more complaints about missing numa.h, then you have to change the #include in the files that cause those complaints.

[Joachim Müller (JojoM)]

So then it all comes down to the question: Where are this files stored after their download from Fishtest? I'll have to find out ...

[pasquale....@gmail.com]

fishtest download the source code in a temporary folder.
Look in games.py, from line 109

def build(worker_dir, sha, repo_url, destination, concurrency):
"""Download and build sources in a temporary directory then move exe to destination"""
tmp_dir = tempfile.mkdtemp()
os.chdir(tmp_dir)

with open('sf.gz', 'wb+') as f:
f.write(requests.get(github_api(repo_url) + '/zipball/' + sha, timeout=HTTP_TIMEOUT).content)
zip_file = ZipFile('sf.gz')
zip_file.extractall()
zip_file.close()

[pasquale....@gmail.com]

I'm a newbie in python but this should be useful (code not tested)

def build(worker_dir, sha, repo_url, destination, concurrency):
"""Download and build sources in a temporary directory then move exe to destination"""
tmp_dir = tempfile.mkdtemp()
os.chdir(tmp_dir)

with open('sf.gz', 'wb+') as f:
f.write(requests.get(github_api(repo_url) + '/zipball/' + sha, timeout=HTTP_TIMEOUT).content)
zip_file = ZipFile('sf.gz')
zip_file.extractall()
zip_file.close()

for name in zip_file.namelist():
if name.endswith('/src/'):
src_dir = name
os.chdir(src_dir)

custom_make = os.path.join(worker_dir, 'custom_make.txt')

# hack numasf.h to include a local numa.h

with open('tmp', 'w') as out:
with open('numasf.h') as f:
new_numasf = f.read()
new_numasf = new_numasf.replace('#include <numa.h>', '#include "/home/JojoM/SF-hack/numa.h")
out.write(new_numasf)
shutil.copyfile('tmp', 'numasf.h')

[Lyudmil Antonov]

In the unzipped directory, stockfish.exe, base.exe, cutechess-cli.exe and other files (2-moves and 8-moves opening books etc.) are stored in /fishtest-master/worker/testing/ directory

[Joachim Müller (JojoM)]

So what you mean is I should just paste those lines into games.py and put numa.h alongside it?

[Joachim Müller (JojoM)]

Or (see above) should I put numa.h into the testing directory?

[pasquale....@gmail.com]

The snippet of code is only a hint on how write a patch for games.py

It replaces in numasf.h the line '#include <numa.h>' with a local numa.h in your path (e.g. /home/JojoM/SF-hack/numa.h)


=====

# hack numasf.h to include a local numa.h

with open('tmp', 'w') as out:
with open('numasf.h') as f:
new_numasf = f.read()
new_numasf = new_numasf.replace('#include <numa.h>', '#include "/home/JojoM/SF-hack/numa.h")
out.write(new_numasf)
shutil.copyfile('tmp', 'numasf.h')

======



-pp

[Joachim Müller (JojoM)]

I'll give it a try when the next numa test comes up.

[pasquale....@gmail.com]

You can install locally a fishtest development server (it's very easy) and load a numa test already passed on fishtest

https://github.com/glinscott/fishtest/wiki/Installing-a-development-fishtest-server-on-Ubuntu

http://tests.stockfishchess.org/tests/view/581e027a0ebc5910626b9b87

[Joachim Müller (JojoM)]

Thanks very much, I'll give it a go when I have time ;)

[Ronald de Man]

On Sunday, November 6, 2016 at 8:14:54 PM UTC+1, Joachim Müller (JojoM) wrote:

So then it all comes down to the question: Where are this files stored after their download from Fishtest? I'll have to find out ...

It is also possible to store the numa.h I attached in, say, /home/jojom/include and then add /home/jojom/include to the CPATH environment variable. In that case you do not need to modify anything in the SF source at all.

But then you need to know how to set environment variables and launch fishtest with the variable correctly set.

https://gcc.gnu.org/onlinedocs/gcc/Environment-Variables.html

[Joachim Müller (JojoM)]

Please do me a favour and instruct me on (1) how to edit the CPath variable, and (2) how to launch the worker accordingly.

I hear, Marco is willing to start NUMA testing with multi-core hardware provided by Ipman. That being said, I might still be able to help when the results are examined in the framework later on. As I did last year, when Lazy SMP was introduced.

[Ipman Chess]

I'm still waiting for a reply from Marco..e-mail sent on 06-11-2016

Maybe he is working on numa-code..

Ipman.

Op maandag 7 november 2016 10:55:23 UTC+1 schreef Joachim Müller (JojoM):

[Marco Costalba]

Sorry I have quite limited time in these weeks but I don't forget what I have said. 

[Ipman Chess]

No problem Marco..you know my e-mail and you can always sent me something when it's ready for testing!

Best regards,
Ipman.

Op vrijdag 11 november 2016 16:45:26 UTC+1 schreef Marco Costalba:

[mcos...@gmail.com]

Now it's ready :-)

Please pull from

https://github.com/mcostalba/Stockfish/tree/numa

I have just fixed a possible race, so for you and people interesting to test it, please pull it again. Thanks.

[Joachim Müller (JojoM)]

Will NUMA be tested in the framework and if successful be merged into SF master?

[peter.zs...@gmx.at]

Hello,

As far as I have seen, this patch should be tested only under Windows, right?
Is it only relevant for more than 64 Threads, or does it makes sense to test with 64 Threads (and less)?

br

[peter.zs...@gmx.at]

I posted some benchmark results in the pull request:
https://github.com/official-stockfish/Stockfish/pull/883

[peter.zs...@gmx.at]

Let me post something off-topic here, but it is somehow funny, at least for me :D
I bought Houdini 5. H5 has an option to enable/disable NUMA per uci option, too.

The funny thing is H5 has a bench command very similar to stockfish, too.
All I needed was to create a file with the name "benchpos.txt".
I created the bench positions from Stockfish benchmark.cpp.
Then I could run "bench <hash-size> <nr-of-threads> <depth>"

H5 fills the hash quite quickly, but does not reach high depth, so I used:
bench 2048 64 18 also 3 runs each. The differences in the individual runs were much higher than with stockfish or cfish, but for a rough comparison it should be OK:

Houdini 5 default avg: 84,9 MN/s
Houdini 5 with "setoption name NUMA Enabled value false" avg: 54,8 MN/s
so +54,9% speed gain for Houdini 5 numa awareness.