OpenBLAS not found on ARM

[Alfredo Canziani]

I’m trying to install Torch on ARM (an Nvidia Jetson TK1), but I’m not able to “find” Open BLAS.
I’ve installed it myself from xianyi repository and it looks like I succeed doing so

Generating openblas_config.h in /opt/OpenBLAS/include
Generating f77blas.h in /opt/OpenBLAS/include
Generating cblas.h in /opt/OpenBLAS/include
Copying LAPACKE header files to /opt/OpenBLAS/lib
Copying the static library to /opt/OpenBLAS/lib
Copying the shared library to /opt/OpenBLAS/lib
Install OK!

Despite the fact I wouldn’t know how to test it.
Then, running luarocks install torch I get

-- The C compiler identification is GNU 4.8.2
-- The CXX compiler identification is GNU 4.8.2
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Try OpenMP C flag = [-fopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Success
-- Try OpenMP CXX flag = [-fopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Success
-- Found OpenMP: -fopenmp  
-- Compiling with OpenMP support
-- Compiling with OpenMP support
-- No OMAP3 processor on this on this machine.
-- No OMAP4 processor on this on this machine.
-- Neon found with compiler flag : -mfpu=neon -D__NEON__
-- Looking for mmap
-- Looking for mmap - found
-- Performing Test C_HAS_SSE1_1
-- Performing Test C_HAS_SSE1_1 - Failed
-- Performing Test C_HAS_SSE1_2
-- Performing Test C_HAS_SSE1_2 - Failed
-- Performing Test C_HAS_SSE1_3
-- Performing Test C_HAS_SSE1_3 - Failed
-- Performing Test C_HAS_SSE2_1
-- Performing Test C_HAS_SSE2_1 - Failed
-- Performing Test C_HAS_SSE2_2
-- Performing Test C_HAS_SSE2_2 - Failed
-- Performing Test C_HAS_SSE2_3
-- Performing Test C_HAS_SSE2_3 - Failed
-- Performing Test C_HAS_SSE3_1
-- Performing Test C_HAS_SSE3_1 - Failed
-- Performing Test C_HAS_SSE3_2
-- Performing Test C_HAS_SSE3_2 - Failed
-- Performing Test C_HAS_SSE3_3
-- Performing Test C_HAS_SSE3_3 - Failed
-- Performing Test C_HAS_SSE4_1_1
-- Performing Test C_HAS_SSE4_1_1 - Failed
-- Performing Test C_HAS_SSE4_1_2
-- Performing Test C_HAS_SSE4_1_2 - Failed
-- Performing Test C_HAS_SSE4_1_3
-- Performing Test C_HAS_SSE4_1_3 - Failed
-- Performing Test C_HAS_SSE4_1_4
-- Performing Test C_HAS_SSE4_1_4 - Failed
-- Performing Test C_HAS_SSE4_2_1
-- Performing Test C_HAS_SSE4_2_1 - Failed
-- Performing Test C_HAS_SSE4_2_2
-- Performing Test C_HAS_SSE4_2_2 - Failed
-- Performing Test C_HAS_SSE4_2_3
-- Performing Test C_HAS_SSE4_2_3 - Failed
-- Performing Test C_HAS_SSE4_2_4
-- Performing Test C_HAS_SSE4_2_4 - Failed
-- Performing Test CXX_HAS_SSE1_1
-- Performing Test CXX_HAS_SSE1_1 - Failed
-- Performing Test CXX_HAS_SSE1_2
-- Performing Test CXX_HAS_SSE1_2 - Failed
-- Performing Test CXX_HAS_SSE1_3
-- Performing Test CXX_HAS_SSE1_3 - Failed
-- Performing Test CXX_HAS_SSE2_1
-- Performing Test CXX_HAS_SSE2_1 - Failed
-- Performing Test CXX_HAS_SSE2_2
-- Performing Test CXX_HAS_SSE2_2 - Failed
-- Performing Test CXX_HAS_SSE2_3
-- Performing Test CXX_HAS_SSE2_3 - Failed
-- Performing Test CXX_HAS_SSE3_1
-- Performing Test CXX_HAS_SSE3_1 - Failed
-- Performing Test CXX_HAS_SSE3_2
-- Performing Test CXX_HAS_SSE3_2 - Failed
-- Performing Test CXX_HAS_SSE3_3
-- Performing Test CXX_HAS_SSE3_3 - Failed
-- Performing Test CXX_HAS_SSE4_1_1
-- Performing Test CXX_HAS_SSE4_1_1 - Failed
-- Performing Test CXX_HAS_SSE4_1_2
-- Performing Test CXX_HAS_SSE4_1_2 - Failed
-- Performing Test CXX_HAS_SSE4_1_3
-- Performing Test CXX_HAS_SSE4_1_3 - Failed
-- Performing Test CXX_HAS_SSE4_1_4
-- Performing Test CXX_HAS_SSE4_1_4 - Failed
-- Performing Test CXX_HAS_SSE4_2_1
-- Performing Test CXX_HAS_SSE4_2_1 - Failed
-- Performing Test CXX_HAS_SSE4_2_2
-- Performing Test CXX_HAS_SSE4_2_2 - Failed
-- Performing Test CXX_HAS_SSE4_2_3
-- Performing Test CXX_HAS_SSE4_2_3 - Failed
-- Performing Test CXX_HAS_SSE4_2_4
-- Performing Test CXX_HAS_SSE4_2_4 - Failed
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of void*
-- Check size of void* - done
-- Checking for [mkl_gf - mkl_gnu_thread - mkl_core - pthread - m]
--   Library mkl_gf: not found
-- Checking for [mkl_gf - mkl_intel_thread - mkl_core - pthread - m]
--   Library mkl_gf: not found
-- Checking for [mkl_intel - mkl_gnu_thread - mkl_core - pthread - m]
--   Library mkl_intel: not found
-- Checking for [mkl_intel - mkl_intel_thread - mkl_core - pthread - m]
--   Library mkl_intel: not found
-- Checking for [mkl_gf - mkl_sequential - mkl_core - m]
--   Library mkl_gf: not found
-- Checking for [mkl_intel - mkl_sequential - mkl_core - m]
--   Library mkl_intel: not found
-- Checking for [mkl_gf - mkl_gnu_thread - mkl_core - pthread - m]
--   Library mkl_gf: not found
-- Checking for [mkl_gf - mkl_intel_thread - mkl_core - pthread - m]
--   Library mkl_gf: not found
-- Checking for [mkl_intel - mkl_gnu_thread - mkl_core - pthread - m]
--   Library mkl_intel: not found
-- Checking for [mkl_intel - mkl_intel_thread - mkl_core - pthread - m]
--   Library mkl_intel: not found
-- Checking for [mkl - guide - pthread - m]
--   Library mkl: not found
-- MKL library not found
-- Checking for [openblas - gfortran]
--   Library openblas: BLAS_openblas_LIBRARY-NOTFOUND
-- Checking for [openblas - gfortran - pthread]
--   Library openblas: BLAS_openblas_LIBRARY-NOTFOUND
-- Checking for [goto2 - gfortran]
--   Library goto2: BLAS_goto2_LIBRARY-NOTFOUND
-- Checking for [goto2 - gfortran - pthread]
--   Library goto2: BLAS_goto2_LIBRARY-NOTFOUND
-- Checking for [acml - gfortran]
--   Library acml: BLAS_acml_LIBRARY-NOTFOUND
-- Checking for [Accelerate]
--   Library Accelerate: BLAS_Accelerate_LIBRARY-NOTFOUND
-- Checking for [vecLib]
--   Library vecLib: BLAS_vecLib_LIBRARY-NOTFOUND
-- Checking for [ptf77blas - atlas - gfortran]
--   Library ptf77blas: BLAS_ptf77blas_LIBRARY-NOTFOUND
-- Checking for [blas]
--   Library blas: BLAS_blas_LIBRARY-NOTFOUND
-- Cannot find a library with BLAS API. Not using BLAS.

Any idea of what I may test?

[ConvNetLearner]

You installed OpenBLAS in /opt/OpenBLAS/, but lib/TH/cmake/FindBLAS.cmake does not seem to search in that directory. You'll have to either change where OpenBLAS is installed (usr/local/lib, /usr/lib, /usr/local/lib64, /usr/lib64) or change FindBLAS.cmake to include /opt/OpenBLAS in its library search.

[Alfredo Canziani]

OK, I can try to include stuff to the cmake file, but why does this work fine on x86 architectures?
The only thing I’ve changed in Torch’s installation script has been

export CMAKE_LIBRARY_PATH=/usr/lib/gcc/x86_64-linux-gnu:/usr/lib/gcc/x86_64-linux-gnu/4.6:$CMAKE_LIBRARY_PATH

with

export CMAKE_LIBRARY_PATH=/usr/lib/gcc/arm-linux-gnueabihf:/usr/lib/gcc/arm-linux-gnueabihf/4.8:$CMAKE_LIBRARY_PATH

Without the fist, even on x86 Ubuntu boxes the Blas is not found (I guess gfortran is not found).
That line fixes everything on x86 but not on ARM (of course, I made it point to the correspondent location on ARM architecture).
Why is this happening?

OK, I can try to include stuff to the `cmake` file, but why does this work fine on x86 architectures?
The only thing I've changed in Torch's installation script has been
```
export CMAKE_LIBRARY_PATH=/usr/lib/gcc/x86_64-linux-gnu:/usr/lib/gcc/x86_64-linux-gnu/4.6:$CMAKE_LIBRARY_PATH

```
with
```
export CMAKE_LIBRARY_PATH=/usr/lib/gcc/arm-linux-gnueabihf:/usr/lib/gcc/arm-linux-gnueabihf/4.8:$CMAKE_LIBRARY_PATH
```
Without the fist, even on x86 Ubuntu boxes the Blas is not found (I guess `gfortran` is not found).
That line fixes everything on x86 but not on ARM (of course, I made it point to the correspondent location on ARM architecture).
Why is this happening?

[ConvNetLearner]

I don't think CMAKE_LIBRARY_PATH is involved here at all. After all you installed OpenBLAS in /opt/OpenBLAS.

Did you verify that LD_LIBRARY_PATH contains the same things (or equivalent things) on the both x86 and ARM?  I noticed the use of "PATH xxx yyy ENV LD_LIBRARY_PATH" in lib/TH/cmake/FindBLAS.cmake.  If everything fails, you just need to add debugging print-outs in FindBLAS.cmake to find out why.

[soumith]

find gcc version

gcc -v

Example, if gcc version is 4.8.2

$GCCVER=4.8.2

$ARCH=arm-linux-gnueabihf

then, find the correct folder that contains the gcc libs:

ls /usr/lib/gcc/$ARCH/$GCCVER

Does that work? If not, adjust the directory

then run the following command:

export LD_LIBRARY_PATH=/usr/lib/gcc/$ARCH/$GCCVER:/opt/OpenBLAS/lib:$LD_LIBRARY_PATH

then run ezinstall.

It will work.

--
You received this message because you are subscribed to the Google Groups "torch7" group.
To unsubscribe from this group and stop receiving emails from it, send an email to torch7+un...@googlegroups.com.
To post to this group, send email to tor...@googlegroups.com.
Visit this group at http://groups.google.com/group/torch7.
For more options, visit https://groups.google.com/d/optout.

[Alfredo Canziani]

Summarising: I have been testing this stuff on 2 different ARM platforms.
Platform 1: Nvidia Jetson TK1

ubuntu@tegra-ubuntu:~$ echo $LD_LIBRARY_PATH
/opt/OpenBLAS/lib:/usr/lib/gcc/arm-linux-gnueabihf/4.8.2:
ubuntu@tegra-ubuntu:~$ ls /opt/OpenBLAS/lib
libopenblas.a  libopenblas_armv7-r0.2.9.rc2.a  libopenblas_armv7-r0.2.9.rc2.so  libopenblas.so  libopenblas.so.0
ubuntu@tegra-ubuntu:~$ ls /usr/lib/gcc/arm-linux-gnueabihf/4.8.2
cc1          crtend.o       libasan.a          libcaf_single.a  libgcov.a           libgomp.so              libstdc++.a
cc1plus      crtendS.o      libasan_preinit.o  libgcc.a         libgfortran.a       libgomp.spec            libstdc++.so
collect2     f951           libasan.so         libgcc_eh.a      libgfortranbegin.a  liblto_plugin.so        libsupc++.a
crtbegin.o   finclude       libatomic.a        libgcc_s_sf.so   libgfortran.so      liblto_plugin.so.0      lto1
crtbeginS.o  include        libatomic.so       libgcc_s.so      libgfortran.spec    liblto_plugin.so.0.0.0  lto-wrapper
crtbeginT.o  include-fixed  libbacktrace.a     libgcc_s.so.1    libgomp.a           libssp_nonshared.a      sf

This is the log file

Platform 2: ZedBoard

linaro@linaro-ubuntu-desktop:~$ echo $LD_LIBRARY_PATH
/opt/OpenBLAS/lib/:/usr/lib/gcc/arm-linux-gnueabihf/4.6:
linaro@linaro-ubuntu-desktop:~$ ls /opt/OpenBLAS/lib/
libopenblas.a libopenblas.so libopenblas.so.0 libopenblas_armv7p-r0.2.9.rc2.a libopenblas_armv7p-r0.2.9.rc2.so
linaro@linaro-ubuntu-desktop:~$ ls /usr/lib/gcc/arm-linux-gnueabihf/4.6
cc1 crtbegin.o crtend.o finclude libgcc.a libgcc_s.so.1 libgfortran.so libgomp.a liblto_plugin.so libssp_nonshared.a libsupc++.a
cc1plus crtbeginS.o crtendS.o include libgcc_eh.a libgcov.a libgfortran.spec libgomp.so liblto_plugin.so.0 libstdc++.a lto-wrapper
collect2 crtbeginT.o f951 include-fixed libgcc_s.so libgfortran.a libgfortranbegin.a libgomp.spec liblto_plugin.so.0.0.0 libstdc++.so lto1

This is the log file

[Alfredo Canziani]

Could you give me some more tips? What (and how) show I print out? I’m totally new to Cmake and I wouldn’t know where to start…

[Alfredo Canziani]

OK, I fixed the FindBLAS.cmake myself. Now I am able to build torch with BLAS support also on ARM architectures.

Eventually, I’ve found BLAS and gfortran. I believe cmake‘s ENV behaves differently on ARM.

-- Checking for [openblas - gfortran]
--   Library openblas: /opt/OpenBLAS/lib/libopenblas.so
--   Library gfortran: /usr/lib/gcc/arm-linux-gnueabihf/4.8.2/libgfortran.so

Here’s the patch.

ubuntu@tegra-ubuntu:~/torch7$ git diff --cached
diff --git a/lib/TH/cmake/FindBLAS.cmake b/lib/TH/cmake/FindBLAS.cmake
index 282ad8f..9a9c681 100644
--- a/lib/TH/cmake/FindBLAS.cmake
+++ b/lib/TH/cmake/FindBLAS.cmake
@@ -67,7 +67,7 @@ MACRO(Check_Fortran_Libraries LIBRARIES _prefix _name _flags _list)
       else ( APPLE )
         find_library(${_prefix}_${_library}_LIBRARY
           NAMES ${_library}
-          PATHS /usr/local/lib /usr/lib /usr/local/lib64 /usr/lib64
+          PATHS /usr/local/lib /usr/lib /usr/local/lib64 /usr/lib64 /opt/OpenBLAS/lib /usr/lib/gcc/arm-linux-gnueabihf/4.8.2
           ENV LD_LIBRARY_PATH )
       endif( APPLE )
       mark_as_advanced(${_prefix}_${_library}_LIBRARY)

The first one is quite general, since OpenBLAS is installed in /opt/OpenBLAS/lib by default, whereas the second one (i.e. the location of gfortran‘s libraries) is platform dependent.
I really tried to do all sort of combinations with LD_LIBRARY_PATH and CMAKE_LIBRARY_PATH with no success.
Now I’d like to integrate the fix in Torch, but I wouldn’t know where to start and how to make it platform independent.