Dynamic binding with multithreaded ATLAS

[Botong]

Hi,

I am using ATLAS in a cluster setting. Currently by following the
instructions in BUILDING_ATLAS, I have successfully compiled a static
binding jblas-1.2.0 jar file which allow me to run jblas on different
machines using the same copy of multithreaded ATLAS.

But as the machines in the cluster have different hardware setting, I
need to compile a single jblas jar file, but when called in different
machines, it will utilize the ATLAS tuned and installed in each
machine. In this case, is it Dynamic binding I should resort to?

In the static jblas package, there are two libraries called
"libjblas_arch_flavor.so" and "libjblas.so". What is the functionality
of them? Is it one for JNI library and one for ATLAS/BLAS library?


For dynamic binding, I get stuck in almost the last step, could you
help me with this. I am compiling these in a machine wiith Fedora OS,
amd64 archirecture. After I get the jar file and do a test run. But it
wouldn't pass the sanity check:

[root@ip-10-83-77-171 jblas]# java -server -jar jblas-dynamic-Linux-
amd64-1.2.1.jar --debug
Simple benchmark for jblas

Running sanity benchmarks.

checking vector addition... ok
-- org.jblas CONFIG BLAS native library not found in path. Copying
native library from the archive. Consider installing the library
somewhere in the path (for Windows: PATH, for Linux: LD_LIBRARY_PATH).
-- org.jblas DEBUG Preloading ArchFlavor library.
-- org.jblas CONFIG ArchFlavor native library not found in path.
Copying native library libjblas_arch_flavor from the archive. Consider
installing the library somewhere in the path (for Windows: PATH, for
Linux: LD_LIBRARY_PATH).
-- org.jblas DEBUG Attempting to load "libjblas_arch_flavor.so".
-- org.jblas DEBUG Trying path "/libjblas_arch_flavor.so".
-- org.jblas DEBUG Trying path "/bin/libjblas_arch_flavor.so".
-- org.jblas DEBUG Trying path "/lib/static/Linux/amd64/
libjblas_arch_flavor.so".
-- org.jblas DEBUG Trying path "/lib/static/Linux/amd64/
libjblas_arch_flavor.so".
-- org.jblas DEBUG Trying path "/lib/dynamic/Linux/amd64/
libjblas_arch_flavor.so".
-- org.jblas CONFIG Loading libjblas_arch_flavor.so from /lib/dynamic/
Linux/amd64/.
-- org.jblas DEBUG tempfile.getPath() = /tmp/
jblas5233815495294025615libjblas_arch_flavor.so
-- org.jblas DEBUG Copying took 0.0 seconds.
-- org.jblas DEBUG Attempting to load "libjblas.so".
-- org.jblas DEBUG Trying path "/libjblas.so".
-- org.jblas DEBUG Trying path "/bin/libjblas.so".
-- org.jblas DEBUG Trying path "/lib/static/Linux/amd64/sse3/
libjblas.so".
-- org.jblas DEBUG Trying path "/lib/static/Linux/amd64/sse3/
libjblas.so".
-- org.jblas DEBUG Trying path "/lib/dynamic/Linux/amd64/sse3/
libjblas.so".
-- org.jblas CONFIG Loading libjblas.so from /lib/dynamic/Linux/amd64/
sse3/.
-- org.jblas DEBUG tempfile.getPath() = /tmp/
jblas7487923993934332451libjblas.so
-- org.jblas DEBUG Copying took 0.0020 seconds.
checking matrix multiplication... ok
checking existence of dsyev...... ok
[-0.21065603010602588, -0.6404447607135406, -0.5071295294530739;
-0.5090854060895623, -0.11644450194791618, 0.7626214072101205;
-0.8075147820730988, 0.40755575681770734, -0.3813107036050603;
0.21065603010602574, 0.64044476071354, -0.12581882584801418]
[17.23368793961409; 1.4142135623730951; 1.2819751242557092E-16]
[-0.47060485181697004, 0.7822176211129874, 0.40824829046386313;
-0.571448748634892, 0.08233869695926208, -0.8164965809277259;
-0.6722926454528144, -0.6175402271944637, 0.408248290463863]
[17.23368793961409; 1.414213562373095; 1.2819751242557092E-16]
checking existence of dgesvd...... ok
Checking complex return values... (z = -21.0 + 88.0i)
Check whether we're catching XERBLA errors. If you see something like
"** On entry to DGEMM parameter number 4 had an illegal value", it
didn't work!
** On entry to DGEMM parameter number 4 had an illegal value
[root@ip-10-83-77-171 jblas]#




All the steps I type in the console are listed here in case you need
to double check:

wget http://www.netlib.org/lapack/lapack-lite-3.1.1.tgz

wget http://sourceforge.net/projects/math-atlas/files/Stable/3.8.3/atlas3.8.3.tar.bz2/download

git clone git://github.com/mikiobraun/jblas.git

yum -y install gcc-gfortran

-------------------Lapack-----------------
untar lapack

cp INSTALL/make.inc.LINUX make.inc
vi make.inc

FORTRAN = gfortran
OPTS = -O2 -fPIC
DRVOPTS = $(OPTS)
NOOPT = -fPIC
LOADER = gfortran
LOADOPTS = -fPIC

make -k blaslib lapacklib

-------------------Atlas-------------------

tar xvf atlas3.8.3.tar.bz2

mkdir build
cd build

../configure --with-netlib-lapack=/root/lapack-lite-3.1.1/
lapack_LINUX.a -b 64 -Si cputhrchk 0 -Fa alg -fPIC
(removed "-A 21" before "-b 64")

make
make time

cd lib
make ptshared (to get the .so lib files. Without this jblas configure
would fail without --static-libs)

--------------------JBLAS------------------

./configure --libpath=/root/ATLAS/build/lib/ --ptatlas

yum -y install ant-nodeps

yum -y install lapack-devel

make

ant dynamic-lean-jar


---------------- Run ----------------------------

export LD_LIBRARY_PATH=/root/ATLAS/build/lib

java -server -jar jblas-dynamic-Linux-amd64-1.2.0.jar




Thanks for your help, and let me know if you need other detail
information.

Sincerely,
Botong

[Mikio Braun]

Hi Botong,

sorry for the late reply.

So in principle, you're doing everything correctly, I think.
libjblas_arch_flavor.so contains code to read out the processor type
(to see whether it supports sse2 and so on), while libjblas.so
contains the actual bindings to the processor. The arch flavor part is
only necessary for dynamic loading, so you should be fine if you just
add libjblas.so to the path.

It seem like overriding the XERBLA with a local copy didn't work.
Actually, this is not a hard problem, it only means that jblas cannot
wrap Fortran error messages into Java exceptions. XERBLA is called
when there are some errors in the parameters (like dimensions don't
match). What this also means is that every time, there is an error,
your JVM will just stop, unfortunately.

It probably has something to do with your version of GCC. Which
version do you have (try with "gcc -v?")

Can you also paste the outputs of "make clean all"?

-M

> --
> You received this message because you are subscribed to the Google Groups "jblas-users" group.
> To post to this group, send email to jblas...@googlegroups.com.
> To unsubscribe from this group, send email to jblas-users...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/jblas-users?hl=en.
>
>

--
Dr. Mikio Braun, Beckerstr. 11, 12157 Berlin
Privat: 030 / 42 10 56 42, Büro: 030 / 314 78627, Handy: 0172 / 97 45 676

[Botong]

Hi Mikio,

Sorry for the delay. Here is my gcc info:

[root@ip-10-98-181-187 ~]# gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/4.5.1/lto-
wrapper
Target: x86_64-redhat-linux
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --
infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/
bugzilla --enable-bootstrap --enable-shared --enable-threads=posix --
enable-checking=release --with-system-zlib --enable-__cxa_atexit --
disable-libunwind-exceptions --enable-gnu-unique-object --enable-
linker-build-id --enable-languages=c,c++,objc,obj-c+
+,java,fortran,ada,lto --enable-plugin --enable-java-awt=gtk --disable-
dssi --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre --enable-
libgcj-multifile --enable-java-maintainer-mode --with-ecj-jar=/usr/
share/java/eclipse-ecj.jar --disable-libjava-multilib --with-ppl --
with-cloog --with-tune=generic --with-arch_32=i686 --build=x86_64-
redhat-linux
Thread model: posix
gcc version 4.5.1 20100924 (Red Hat 4.5.1-4) (GCC)



and here is the result of "make clean all" after the exact same step
in my original post.

[root@ip-10-98-181-187 jblas]# make clean all
rm -f native/*.o native/*.so src/main/resources/lib/dynamic/Linux/
amd64/*.so src/main/resources/lib/dynamic/Linux/amd64/sse3/*.so src/
main/java/org/jblas/NativeBlas.java src/main/c/NativeBlas.c generated-
sources
ruby scripts/fortranwrapper.rb --complexcc c99 org.jblas NativeBlas \
./lapack-lite-3.1.1/BLAS/SRC/[sdcz]copy.f \
./lapack-lite-3.1.1/BLAS/SRC/[sdcz]swap.f \
./lapack-lite-3.1.1/BLAS/SRC/[sdcz]axpy.f \
./lapack-lite-3.1.1/BLAS/SRC/[sdcz]scal.f \
./lapack-lite-3.1.1/BLAS/SRC/[cz][sd]scal.f \
./lapack-lite-3.1.1/BLAS/SRC/[sdcz]dot*.f \
./lapack-lite-3.1.1/BLAS/SRC/[sd]*nrm2.f \
./lapack-lite-3.1.1/BLAS/SRC/[sd]*asum.f \
./lapack-lite-3.1.1/BLAS/SRC/i[sdcz]amax.f \
./lapack-lite-3.1.1/BLAS/SRC/[sdcz]gemv.f \
./lapack-lite-3.1.1/BLAS/SRC/[sdcz]ger*.f \
./lapack-lite-3.1.1/BLAS/SRC/[sdcz]gemm.f \
./lapack-lite-3.1.1/SRC/[sd]gesv.f \
./lapack-lite-3.1.1/SRC/[sd]sysv.f \
./lapack-lite-3.1.1/SRC/[sd]syev.f \
./lapack-lite-3.1.1/SRC/[sd]syev[rdx].f \
./lapack-lite-3.1.1/SRC/[sd]posv.f \
./lapack-lite-3.1.1/SRC/[sdcz]geev.f \
./lapack-lite-3.1.1/SRC/[sd]getrf.f \
./lapack-lite-3.1.1/SRC/[sd]potrf.f \
./lapack-lite-3.1.1/SRC/[sdcz]gesvd.f \
./lapack-lite-3.1.1/SRC/[sd]sygvd.f
complex calling convention = c99
Using dumped routines...
ant javah
Buildfile: build.xml

prepare:

javah:
[javac] Compiling 1 source file to /root/jblas/target/classes

BUILD SUCCESSFUL
Total time: 2 seconds
touch generated-sources
gcc -fPIC -DHAS_CPUID -Iinclude -I/usr/lib/jvm/java-6-sun/include -I/
usr/lib/jvm/java-6-sun/include/linux -c src/main/c/NativeBlas.c -o
target/c/NativeBlas.o
gcc -shared -L/root/ATLAS/build/lib/ -o target/c/NativeBlas.so target/
c/NativeBlas.o -latlas -lptf77blas -llapack -lptcblas
mkdir -p src/main/resources/lib/dynamic/Linux/amd64/sse3
mv "target/c/NativeBlas.so" "src/main/resources/lib/dynamic/Linux/
amd64/sse3/libjblas.so"
gcc -fPIC -DHAS_CPUID -Iinclude -I/usr/lib/jvm/java-6-sun/include -I/
usr/lib/jvm/java-6-sun/include/linux -c src/main/c/jblas_arch_flavor.c
-o src/main/c/jblas_arch_flavor.o
gcc -shared -L/root/ATLAS/build/lib/ -o src/main/c/
jblas_arch_flavor.so src/main/c/jblas_arch_flavor.o -latlas -
lptf77blas -llapack -lptcblas
mkdir -p src/main/resources/lib/dynamic/Linux/amd64
mv "src/main/c/jblas_arch_flavor.so" "src/main/resources/lib/dynamic/
Linux/amd64/libjblas_arch_flavor.so"
rm src/main/c/jblas_arch_flavor.o
[root@ip-10-98-181-187 jblas]#


BTW, does JBLAS or ATLAS have any sparse matrix libraries?


Thanks,
Botong

> > wgethttp://sourceforge.net/projects/math-atlas/files/Stable/3.8.3/atlas3....

> > For more options, visit this group athttp://groups.google.com/group/jblas-users?hl=en.

>
> --
> Dr. Mikio Braun, Beckerstr. 11, 12157 Berlin

> Privat: 030 / 42 10 56 42, Büro: 030 / 314 78627, Handy: 0172 / 97 45 676- 隐藏被引用文字 -
>
> - 显示引用的文字 -

[Mikio Braun]

Hi Botong,

sorry for the delay. So, any news? You're GCC looks pretty recent, so
I'm not quite sure while you have the link problems. As I've said, you
can use jblas with ATLAS as it is, it will only crash on errors in
LAPACK instead of throwing an exception...

If this is a big problem, I'll dig into it further. Just let me know.

-M

> For more options, visit this group at http://groups.google.com/group/jblas-users?hl=en.

[Botong]

Ok, thanks.

Botong

> ...
>
> read more »