compiling on arch linux

[Mathieu Moalic]

Hello,

I have an error compiling the current master branch on github:

$ make compile -j 20 install

g++ -c -I /opt/cuda/include -I /usr/include/python3.10 -Ofast -std=c++17 -IBorisLib -IBorisCUDALib -fopenmp Boris/AnisotropyBiaxial.cpp -o Boris/Boris_o/AnisotropyBiaxial.o
g++ -c -I /opt/cuda/include -I /usr/include/python3.10 -Ofast -std=c++17 -IBorisLib -IBorisCUDALib -fopenmp Boris/AnisotropyBiaxialCUDA.cpp -o Boris/Boris_o/AnisotropyBiaxialCUDA.o
g++ -c -I /opt/cuda/include -I /usr/include/python3.10 -Ofast -std=c++17 -IBorisLib -IBorisCUDALib -fopenmp Boris/Anisotropy.cpp -o Boris/Boris_o/Anisotropy.o

....

....

....

nvcc -rdc=true -c -std=c++14 -IBorisLib -IBorisCUDALib -w -arch=sm_80 Boris/TransportCUDA.cu -o Boris/Boris_cuo/TransportCUDA.o
nvcc -rdc=true -c -std=c++14 -IBorisLib -IBorisCUDALib -w -arch=sm_80 Boris/TransportCUDA_Spin.cu -o Boris/Boris_cuo/TransportCUDA_Spin.o
nvcc -rdc=true -c -std=c++14 -IBorisLib -IBorisCUDALib -w -arch=sm_80 Boris/TransportCUDA_Spin_Display.cu -o Boris/Boris_cuo/TransportCUDA_Spin_Display.o
nvcc -rdc=true -c -std=c++14 -IBorisLib -IBorisCUDALib -w -arch=sm_80 Boris/viDMExchangeCUDA.cu -o Boris/Boris_cuo/viDMExchangeCUDA.o
nvcc -rdc=true -c -std=c++14 -IBorisLib -IBorisCUDALib -w -arch=sm_80 Boris/ZeemanCUDA.cu -o Boris/Boris_cuo/ZeemanCUDA.o
nvcc -arch=sm_80 -dlink -w Boris/Boris_cuo/*.o -o Boris/Boris_cuo/rdc_link.o
nvlink error   : Undefined reference to '_ZN20ManagedAtom_MeshCUDA29Get_Atomistic_EnergyChange_SCEi6cuVAL3IfERPiiRS1_' in 'Boris/Boris_cuo/Atom_Mesh_CubicCUDA_Histo.o'
nvlink error   : Undefined reference to '_ZN15ManagedMeshCUDA20Get_EnergyChange_AFMEi6cuVAL3IfES1_RPiiRS1_' in 'Boris/Boris_cuo/Mesh_AntiFerromagneticCUDA_MonteCarlo.o'
nvlink error   : Undefined reference to '_ZN15ManagedMeshCUDA19Get_EnergyChange_FMEi6cuVAL3IfERPiiRS1_' in 'Boris/Boris_cuo/Mesh_FerromagneticCUDA_Histo.o'
make: *** [makefile:64: install] Error 255
make: *** Waiting for unfinished jobs....

I trimmed the output as it is like 300 lines.

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Thu_Feb_10_18:23:41_PST_2022
Cuda compilation tools, release 11.6, V11.6.112
Build cuda_11.6.r11.6/compiler.30978841_0

Nvidia RTX3080ti

Regards,

Mathieu 

[Serban Lepadatu]

Hi Mathieu,

The errors shown at the end are linker errors, but did the compilation step complete fully without errors?

I would split the compilation and linking steps. So, run the following separately, and make sure each completes without errors:

First clean to start again fresh:

$ make clean

Compile:

$ make compile -j 20

Link:

$ make install

Regards,

Serban

--
You received this message because you are subscribed to the Google Groups "Boris Computational Spintronics" group.
To unsubscribe from this group and stop receiving emails from it, send an email to boris-computational-s...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/boris-computational-spintronics/827337dd-3e82-4fb2-845f-881eb53d01den%40googlegroups.com.

[Mathieu Moalic]

$ make compile -j 20

 gives no errors

$ make install 

nvcc -arch=sm_$(arch) -dlink -w $(CUOBJ_DIR)/*.o -o $(CUOBJ_DIR)/rdc_link.o  

-> no errors

g++ -I /opt/cuda/include -I /usr/include/python3.10 $(OBJ_DIR)/*.o $(CUOBJ_DIR)/*.o -L/opt/cuda/lib64 -fopenmp -ltbb -lsfml-graphics -lsfml-window -lsfml-system -lfftw3 -lX11 -lcudart -lcufft -lcudadevrt -o BorisLin

/usr/bin/ld: Boris/Boris_o/BorisConsole.o: warning: relocation against `_Py_NoneStruct' in read-only section `.text'
/usr/bin/ld: Boris/Boris_o/BorisConsole.o: in function `BPython_Message_RunStage(_object*, _object*)':
BorisConsole.cpp:(.text+0x294): undefined reference to `_PyArg_ParseTuple_SizeT'
/usr/bin/ld: BorisConsole.cpp:(.text+0x2bc): undefined reference to `PyUnicode_FromString'
/usr/bin/ld: Boris/Boris_o/BorisConsole.o: in function `BPython_Message(_object*, _object*)':
BorisConsole.cpp:(.text+0x5bf): undefined reference to `_PyArg_ParseTuple_SizeT'
/usr/bin/ld: BorisConsole.cpp:(.text+0x6b6): undefined reference to `PyUnicode_FromString'
/usr/bin/ld: Boris/Boris_o/BorisConsole.o: in function `sys_stdout_redirect(_object*, _object*)':
BorisConsole.cpp:(.text+0x84a): undefined reference to `_PyArg_ParseTuple_SizeT'
/usr/bin/ld: BorisConsole.cpp:(.text+0x8d3): undefined reference to `_Py_NoneStruct'
/usr/bin/ld: BorisConsole.cpp:(.text+0x8ee): undefined reference to `_Py_NoneStruct'
/usr/bin/ld: Boris/Boris_o/BorisConsole.o: in function `make_Simulation(CLArgs)':
BorisConsole.cpp:(.text+0xf37): undefined reference to `PyImport_AppendInittab'
/usr/bin/ld: BorisConsole.cpp:(.text+0xf3c): undefined reference to `Py_Initialize'
/usr/bin/ld: BorisConsole.cpp:(.text+0xf48): undefined reference to `PySys_GetObject'
/usr/bin/ld: BorisConsole.cpp:(.text+0x11ae): undefined reference to `PyUnicode_FromString'
/usr/bin/ld: BorisConsole.cpp:(.text+0x11bd): undefined reference to `PyList_Append'
/usr/bin/ld: BorisConsole.cpp:(.text+0x125e): undefined reference to `PyUnicode_FromString'
/usr/bin/ld: BorisConsole.cpp:(.text+0x126d): undefined reference to `PyList_Append'
/usr/bin/ld: BorisConsole.cpp:(.text+0x153e): undefined reference to `PyUnicode_FromString'
/usr/bin/ld: BorisConsole.cpp:(.text+0x154d): undefined reference to `PyList_Append'
/usr/bin/ld: Boris/Boris_o/BorisConsole.o: in function `PyInit_PythonEmbeddedModule()':
BorisConsole.cpp:(.text+0x1cd): undefined reference to `PyModule_Create2'
/usr/bin/ld: Boris/Boris_o/BorisConsole.o: in function `BPython_StopPythonScriptFlag(_object*, _object*)':
BorisConsole.cpp:(.text+0x1fb): undefined reference to `PyLong_FromLong'
/usr/bin/ld: Boris/Boris_o/BorisConsole.o: in function `BPython_Message_PrepareRunWithCode(_object*, _object*)':
BorisConsole.cpp:(.text+0x231): undefined reference to `PyUnicode_FromString'
/usr/bin/ld: Boris/Boris_o/BorisConsole.o: in function `BPython_Message_RunWithCode(_object*, _object*)':
BorisConsole.cpp:(.text+0x25d): undefined reference to `PyLong_FromLong'
/usr/bin/ld: Boris/Boris_o/BorisConsole.o: in function `BPython_Message_Run(_object*, _object*)':
BorisConsole.cpp:(.text+0x311): undefined reference to `PyUnicode_FromString'
/usr/bin/ld: Boris/Boris_o/Commands_PythonScripting.o: in function `Simulation::RunPythonScript(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)':
Commands_PythonScripting.cpp:(.text+0x2345): undefined reference to `PySys_GetObject'
/usr/bin/ld: Commands_PythonScripting.cpp:(.text+0x2358): undefined reference to `PyUnicode_FromString'
/usr/bin/ld: Commands_PythonScripting.cpp:(.text+0x2363): undefined reference to `PyList_Append'
/usr/bin/ld: Commands_PythonScripting.cpp:(.text+0x2c11): undefined reference to `PyRun_SimpleStringFlags'
/usr/bin/ld: warning: creating DT_TEXTREL in a PIE
collect2: error: ld returned 1 exit status
make: *** [makefile:65: install] Error 1

My guess is that the linker does not find the proper python headers maybe ?

I fixed a similar issue during compile by changing in makefile:

Boris/Boris_o/%.o: Boris/%.cpp

       g++ -c -Ofast -std=c++17 -IBorisLib -IBorisCUDALib -fopenmp $< -o $@

into 

Boris/Boris_o/%.o: Boris/%.cpp

       g++ -c -I /opt/cuda/include -I /usr/include/python3.10 -Ofast -std=c++17 -IBorisLib -IBorisCUDALib -fopenmp $< -o $@

I added the same Includes tags in install but it does not find them.

[Mathieu Moalic]

ls /usr/include/python3.10
abstract.h         ceval.h          datetime.h             fileobject.h          Imaging.h              iterobject.h    methodobject.h     osmodule.h    pyframe.h      pystate.h    QuantHeap.h      sliceobject.h   tupleobject.h
Bcn.h              classobject.h    descrobject.h          fileutils.h           ImagingUtils.h         Jpeg2K.h        modsupport.h       patchlevel.h  pyhash.h       pystrcmp.h   QuantOctree.h    structmember.h  typeslots.h
Bit.h              codecs.h         dictobject.h           floatobject.h         ImDib.h                Jpeg.h          moduleobject.h     pycapsule.h   pylifecycle.h  pystrhex.h   QuantPngQuant.h  structseq.h     unicodeobject.h
bltinmodule.h      code.h           dynamic_annotations.h  frameobject.h         ImPlatform.h           listobject.h    namespaceobject.h  pyconfig.h    pymacconfig.h  pystrtod.h   QuantTypes.h     sysmodule.h     warnings.h
boolobject.h       compile.h        enumobject.h           funcobject.h          import.h               longintrepr.h   object.h           py_curses.h   pymacro.h      Python.h     rangeobject.h    TiffDecode.h    weakrefobject.h
bytearrayobject.h  complexobject.h  errcode.h              genericaliasobject.h  internal               longobject.h    objimpl.h          pydtrace.h    pymath.h       pythonrun.h  Raw.h            token.h         ZipCodecs.h
bytesobject.h      context.h        eval.h                 genobject.h           interpreteridobject.h  marshal.h       opcode.h           pyerrors.h    pymem.h        pythread.h   setobject.h      traceback.h
cellobject.h       cpython          exports.h              Gif.h                 intrcheck.h            memoryobject.h  osdefs.h           pyexpat.h     pyport.h       QuantHash.h  Sgi.h            tracemalloc.h

[Serban Lepadatu]

Hi Mathieu,

Looks like it cannot find the python shared library required for the last linking step.

On your line: g++ -I /opt/cuda/include -I /usr/include/python3.10 $(OBJ_DIR)/*.o $(CUOBJ_DIR)/*.o -L/opt/cuda/lib64 -fopenmp -ltbb -lsfml-graphics -lsfml-window -lsfml-system -lfftw3 -lX11 -lcudart -lcufft -lcudadevrt -o BorisLin

you shouldn't need the first "-I/..." fields as these are only used to include header files, which are not needed in this linking step. Instead you should specify the path to the Python3.10 shared object library using the -L flag.

I don't know where on your system the required library file is, but it's quite possibly in "-L /usr/lib/python3.10". You can check using $locate *python*, and see where the python .so file is (sorry can't remember the full name, but it's an .so file). Then, try this:

g++ "-L /usr/lib/python3.10  $(OBJ_DIR)/*.o $(CUOBJ_DIR)/*.o -L/opt/cuda/lib64 -fopenmp -ltbb -lsfml-graphics -lsfml-window -lsfml-system -lfftw3 -lX11 -lcudart -lcufft -lcudadevrt -o BorisLin

Regards,

Serban

https://groups.google.com/d/msgid/boris-computational-spintronics/ae4dbc9b-01f3-4021-b7ce-11a5baf54b13n%40googlegroups.com

.

[Mathieu Moalic]

You are correct I got confused between the header files and share objects. Here is with your suggestion:

I find "libpython3.so" in /usr/lib:

$ \ls -l  /usr/lib/libpython3.*
>>> lrwxrwxrwx 1 root root      20 Mar 24 00:05 /usr/lib/libpython3.10.so -> libpython3.10.so.1.0
>>> -rwxr-xr-x 1 root root 3883816 Mar 24 00:05 /usr/lib/libpython3.10.so.1.0
>>> -rwxr-xr-x 1 root root   13864 Mar 24 00:05 /usr/lib/libpython3.so

I add it to with the -L flag:

  g++ $(OBJ_DIR)/*.o $(CUOBJ_DIR)/*.o -L/usr/lib/ -L/opt/cuda/lib64 -fopenmp -ltbb -lsfml-graphics -lsfml-window -lsfml-system -lfftw3 -lX11 -lcudart -lcufft -lcudadevrt -o BorisLin

Sadly, the output remains the same.

[Serban Lepadatu]

Hi Mathieu,

Forgot to say, in addition to specifying the path, you also need to specify which library to link in, as -lpython3.10. Thus you should use:

g++ $(OBJ_DIR)/*.o $(CUOBJ_DIR)/*.o -L/usr/lib/ -L/opt/cuda/lib64 -fopenmp -lpython3.10 -ltbb -lsfml-graphics -lsfml-window -lsfml-system -lfftw3 -lX11 -lcudart -lcufft -lcudadevrt -o BorisLin

It seems the makefile on github is still only configured for v3.4, which didn't use embedded python, will need to update it.

Let me know if this works.

Regards,

Serban

https://groups.google.com/d/msgid/boris-computational-spintronics/5ac0277e-a3a0-49cc-847a-0927bea1db0bn%40googlegroups.com

.

[Mathieu Moalic]

Thank you, the compiling works now !