Unsatisfied link error

[Olli-Pekka]

Hi,
First of all I want to say that this is a very nice software package.

I tried the 1.0.1 version and it worked fine. java -jar jblas.. test
went fine. The performance with matrix-matrix multiplications is great
compared to Colt, for example.

However, I needed the singluar value decomposition so I tried the
version 1.0.2. I was able to build it but when I try to run I get


java -jar jblas-1.0.2.jar
Simple benchmark for jblas

Running sanity benchmarks.

checking vector addition... ok
-- org.jblas CONFIG BLAS native library not found in path. Copying
native library from the archive. Consider installing the library
somewhere in the path (for Windows: PATH, for Linux: LD_LIBRARY_PATH).
-- org.jblas CONFIG ArchFlavor native library not found in path.
Copying native library libjblas_arch_flavor from the archive. Consider
installing the library somewhere in the path (for Windows: PATH, for
Linux: LD_LIBRARY_PATH).
-- org.jblas CONFIG Loading libjblas_arch_flavor.so from /lib/static/
Linux/amd64/.
-- org.jblas CONFIG Loading libjblas.so from /lib/static/Linux/amd64/
sse3/.
checking matrix multiplication... ok
checking existence of dsyev...... ok
Exception in thread "main" java.lang.UnsatisfiedLinkError:
org.jblas.NativeBlas.dgesvd(CCII[DII[DI[DII[DII[DII)I
at org.jblas.NativeBlas.dgesvd(Native Method)
at org.jblas.NativeBlas.dgesvd(NativeBlas.java:362)
at org.jblas.Singular.sparseSVD(Singular.java:48)
at org.jblas.util.SanityChecks.checkSVD(SanityChecks.java:136)
at org.jblas.util.SanityChecks.main(SanityChecks.java:160)
at org.jblas.benchmark.Main.main(Main.java:110)

Do you have any solution for this?

Cheers,
Olli-Pekka, UC Berkeley

[mikiobraun]

Hi Olli-Pekka,

I just added singular value decomposition recently. It's probably best
if you clone the repository from master and try to compile that. The
DoubleMatrix and FloatMatrix variants should compile just fine, but
the Complex*Matrix variants still have a bug.

Your bug seems as if the wrapper functions for some functions are
missing, which is a bit odd. I'll try that out tomorrow when I'm back
in the office and will get back to you.

-M

[Olli-Pekka]

Hi,
Thanks! I am currently using the latest version from the repo.
DoubleMatrix variant is fine for me. Looking forward hearing from you!
I can build the jar from Netbeans and also from command line using
ant.

Cheers,
Olli-Pekka

[mikiobraun]

Hello Olli,

I just pushed some fixes.

Please try it with

./configure --static-libs --download-lapack
make realclean all
ant static-lean-jar
java -jar jblas-static-Linux-i386-1.0.3.jar --debug

and send me the outputs of it all in case it doesn't compile.

I don't have an Apple notebook right now, so I cannot test it there,
unfortunately... .

-M

[Olli-Pekka]

Hi,
Thanks for your help!
I am still having bad luck with the configuration part

./configure --static-libs --download-lapack
checking for java, javac... ok
determining operating system... ok (Linux)
determining architecture... ok (amd64)
determining architecture flavor... ok (sse3)
looking for nm... ok
locating the Java Development Kit... ok (/usr/lib/jvm/java-6-
sun-1.6.0.20)
Setting up gcc and flags... ok (gcc)
deciding whether to use g77 or gfortran... ok (gfortran)
looking for version of make... ok (make)
search for lapack sources (configure by --lapack=dir)... ok (./lapack-
lite-3.1.1)
determining build type... ok (static)
getting library path...... ok (["/usr/lib", "/lib", "/usr/lib/sse2"])
determining whether to build for lapack or atlas... ok (atlas)
looking for libraries...... *failed*

Configuration failed!

Reason: Could not locate libraries for the following symbols: dsyev_.
I am using atlas installed from Ubuntu package manager.

I hope this can give you a hint what is going wrong on my computer. If
you need more specifics, just let me know!

Cheers,
Olli-Pekka

[Mikio Braun]

Hi,

there should be one more file called configure.log or so. Can you send it as well?

-M

> I just pushed so...

[Olli-Pekka]

Hi,
Here it is

Searching for command java
Searching for command javac
Setting FOUND_JAVA to true
Setting OS_NAME to Linux
Setting OS_ARCH to amd64
Searching for command gcc
Setting OS_ARCH_WITH_FLAVOR to amd64/sse3
Searching for command nm
Setting FOUND_NM to true
Searching for file /usr/lib/jvm/java-6-sun-1.6.0.20/include/jni.h
Setting JAVA_HOME to /usr/lib/jvm/java-6-sun-1.6.0.20
Searching for command gcc
Searching for command make
Searching for command ld
Setting CC to gcc
Setting CFLAGS to -fPIC
Setting INCDIRS to -Iinclude -I/usr/lib/jvm/java-6-sun-1.6.0.20/
include -I/usr/lib/jvm/java-6-sun-1.6.0.20/include/linux
Setting SO to so
Setting LIB to lib
Setting RUBY to ruby
Setting LDFLAGS to -shared
Setting F77 to gfortran
Setting LD to gcc
Setting CCC to c99
Setting MAKE to make
Searching for file ./lapack-lite-3.1.1/BLAS/SRC/dgemm.f
Searching for file ./lapack-lite-3.1.1/SRC/dsyev.f
Setting LAPACK_HOME to ./lapack-lite-3.1.1
Setting LINKAGE_TYPE to static
Setting libpath to /usr/lib/lib/usr/lib/sse2
Setting BUILD_TYPE to atlas


I also tried the 2nd approach that you describe in the BUILDING_ATLAS.
I was able to compile lapack,atlas and configure jblas successully (it
found all the libraries). However when I tried to do 'make realclean
all' I got the following error:

rm -f native/*.o native/*.so native-libs/static/Linux/amd64/*.so
native-libs/static/Linux/amd64/sse3/*.so src/org/jblas/NativeBlas.java
generated-sources
rm -f fortranwrapper.dump
ruby scripts/fortranwrapper.rb --complexcc c99 org.jblas NativeBlas \
./lapack-lite-3.1.1/BLAS/SRC/[sdcz]copy.f \
./lapack-lite-3.1.1/BLAS/SRC/[sdcz]swap.f \
./lapack-lite-3.1.1/BLAS/SRC/[sdcz]axpy.f \
./lapack-lite-3.1.1/BLAS/SRC/[sdcz]scal.f \
./lapack-lite-3.1.1/BLAS/SRC/[cz][sd]scal.f \
./lapack-lite-3.1.1/BLAS/SRC/[sdcz]dot*.f \
./lapack-lite-3.1.1/BLAS/SRC/[sd]*nrm2.f \
./lapack-lite-3.1.1/BLAS/SRC/[sd]*asum.f \
./lapack-lite-3.1.1/BLAS/SRC/i[sdcz]amax.f \
./lapack-lite-3.1.1/BLAS/SRC/[sdcz]gemv.f \
./lapack-lite-3.1.1/BLAS/SRC/[sdcz]ger*.f \
./lapack-lite-3.1.1/BLAS/SRC/[sdcz]gemm.f \
./lapack-lite-3.1.1/SRC/[sd]gesv.f \
./lapack-lite-3.1.1/SRC/[sd]sysv.f \
./lapack-lite-3.1.1/SRC/[sd]syev.f \
./lapack-lite-3.1.1/SRC/[sd]syev[rdx].f \
./lapack-lite-3.1.1/SRC/[sd]posv.f \
./lapack-lite-3.1.1/SRC/[sdcz]geev.f \
./lapack-lite-3.1.1/SRC/[sd]getrf.f \
./lapack-lite-3.1.1/SRC/[sd]potrf.f \
./lapack-lite-3.1.1/SRC/[sdcz]gesvd.f
complex calling convention = c99
ant javah
Buildfile: build.xml

prepare:

javah:
[javac] Compiling 1 source file to /home/optossav/NetBeansProjects/
mikiobraun-jblas-2e65911/bin

BUILD SUCCESSFUL
Total time: 1 second
touch generated-sources
gcc -fPIC -DHAS_CPUID -Iinclude -I/usr/lib/jvm/java-6-sun-1.6.0.20/
include -I/usr/lib/jvm/java-6-sun-1.6.0.20/include/linux -c native/
NativeBlas.c -o native/NativeBlas.o
gcc -shared -o native/NativeBlas.so native/NativeBlas.o -Wl,-
z,muldefs ./ATLAS/build/lib/liblapack.a ./ATLAS/build/lib/
libf77blas.a ./ATLAS/build/lib/libcblas.a ./ATLAS/build/lib/libatlas.a
-l:libgfortran.a
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/4.4.1/
libgfortran.a(transfer.o): relocation R_X86_64_32 against `.bss' can
not be used when making a shared object; recompile with -fPIC
/usr/lib/gcc/x86_64-linux-gnu/4.4.1/libgfortran.a: could not read
symbols: Bad value
collect2: ld returned 1 exit status
make: *** [native/NativeBlas.so] Error 1

Do you prefer one method over the other?

Cheers,
OP

On Jun 2, 10:21 am, Mikio Braun <mikiobr...@googlemail.com> wrote:
> Hi,
>
> there should be one more file called configure.log or so. Can you send it as
> well?
>
> -M
>

[Mikio Braun]

Hi,

hm... it seems I removed the relevant parts in the log some time
ago... . One idea was that you probably don't have the full lapack
libraries installed. Atlas provides only a subset of all lapack
routines, and dsyev is meant to check that. Try also installing the
package "liblapack-dev".

> gcc -shared -o native/NativeBlas.so native/NativeBlas.o -Wl,-
> z,muldefs ./ATLAS/build/lib/liblapack.a ./ATLAS/build/lib/
> libf77blas.a ./ATLAS/build/lib/libcblas.a ./ATLAS/build/lib/libatlas.a
> -l:libgfortran.a
> /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/4.4.1/
> libgfortran.a(transfer.o): relocation R_X86_64_32 against `.bss' can
> not be used when making a shared object; recompile with -fPIC
> /usr/lib/gcc/x86_64-linux-gnu/4.4.1/libgfortran.a: could not read
> symbols: Bad value
> collect2: ld returned 1 exit status
> make: *** [native/NativeBlas.so] Error 1

Ah, I think this is an old bug I also thought I had fixed... . On
64bit, the linker is more picky with respect to -fPIC and shared
libraries (I think for 32bit, all code is basically position
independent). Can you try editing configure.out by hand and replace
"-l:libgfortran.a" with "-lgfortran" and then start again at "make
realclean all"?

-M

--
Dr. Mikio Braun, Beckerstr. 11, 12157 Berlin
Privat: 030 / 42 10 56 42, Büro: 030 / 314 78627, Handy: 0172 / 97 45 676

[Olli-Pekka]

Ok,
getting closer now I installed liblapack-dev and:

./configure --static-libs --download-lapack
checking for java, javac... ok
determining operating system... ok (Linux)
determining architecture... ok (amd64)
determining architecture flavor... ok (sse3)
looking for nm... ok
locating the Java Development Kit... ok (/usr/lib/jvm/java-6-
sun-1.6.0.20)
Setting up gcc and flags... ok (gcc)
deciding whether to use g77 or gfortran... ok (gfortran)
looking for version of make... ok (make)
search for lapack sources (configure by --lapack=dir)... ok (./lapack-
lite-3.1.1)
determining build type... ok (static)
getting library path...... ok (["/usr/lib", "/lib", "/usr/lib/sse2"])
determining whether to build for lapack or atlas... ok (atlas)

looking for libraries...... {"atlas"=>"/usr/lib", "cblas"=>"/usr/lib",
"f77blas"=>"/usr/lib", "lapack"=>"/usr/lib"}
ok

Configuration succesfull, writing out results to configure.out

Now the configure.out looks like this

BUILD_TYPE=atlas
CC=gcc
CCC=c99
CFLAGS=-fPIC -DHAS_CPUID
F77=gfortran
FOUND_JAVA=true
FOUND_NM=true
INCDIRS=-Iinclude -I/usr/lib/jvm/java-6-sun-1.6.0.20/include -I/usr/
lib/jvm/java-6-sun-1.6.0.20/include/linux
JAVA_HOME=/usr/lib/jvm/java-6-sun-1.6.0.20
LAPACK_HOME=./lapack-lite-3.1.1
LD=gcc
LDFLAGS=-shared
LIB=lib
LINKAGE_TYPE=static
LOADLIBES=-Wl,-z,muldefs /usr/lib/liblapack.a /usr/lib/libf77blas.a /
usr/lib/libcblas.a /usr/lib/libatlas.a -l:libgfortran.a
MAKE=make
OS_ARCH=amd64
OS_ARCH_WITH_FLAVOR=amd64/sse3
OS_NAME=Linux
RUBY=ruby
SO=so

Then 'make realclean all'

make realclean all

gcc -shared -o native/NativeBlas.so native/NativeBlas.o -Wl,-

z,muldefs /usr/lib/liblapack.a /usr/lib/libf77blas.a /usr/lib/
libcblas.a /usr/lib/libatlas.a -l:libgfortran.a
/usr/bin/ld: /usr/lib/liblapack.a(sgeev.o): relocation R_X86_64_32
against `.rodata' can not be used when making a shared object;
recompile with -fPIC
/usr/lib/liblapack.a: could not read symbols: Bad value

collect2: ld returned 1 exit status
make: *** [native/NativeBlas.so] Error 1

There's still something weird. I tried the change you told me to do
regarding the building against self compiled atlas to this case also.
It didn't help. Error is pretty much same.

Cheers,
OP

[Mikio Braun]

Hi OP,

thanks for your kind patience! And sorry for all the hassle. Probably
the easiest thing is if I could send you a fat-jar for linux/64bit.
I'll try to compile one tomorrow for you to try. If you still want to,
there are two things, you can try: Building against your own ATLAS and
building a jar file which does not include the ATLAS routines but
links them dynamically.

Now I remembered that building against the ubuntu static libs (ending
in *.a) won't work as those are not compiled with the -fPIC flag
(position independent code). In short, you cannot build a dynamic lib
including static libs unless you compiled them with -fPIC in the first
place. Therefore, you have to use the self-compiled ones (adding -fPIC
everywhere while compiling ATLAS as well as LAPACK/BLAS, I think that
is covered in the BUILDING_ATLAS file.)

Then, you can try to do the "-l:libgfortran.a" -> "-lfortran" change
and it hopefully compiles successfully.

You can also try to run configure without options, and compile to a
jar file which still depends on atlas etc. (I called this "dynamic"
for some reason):

./configure
make realclean all
ant dynamic-lean-jar
java -jar jblas-dynamic-Linux-amd64-1.0.3.jar --debug

[Olli-Pekka]

Hi,
Great feel free to send it to me, I can try it.

I did the dynamic thing. The jar itself works. One of the sanity
checks didn't go through.

Simple benchmark for jblas

Running sanity benchmarks.

checking vector addition... ok
-- org.jblas CONFIG BLAS native library not found in path. Copying
native library from the archive. Consider installing the library
somewhere in the path (for Windows: PATH, for Linux: LD_LIBRARY_PATH).

-- org.jblas DEBUG Preloading ArchFlavor library.

-- org.jblas CONFIG ArchFlavor native library not found in path.
Copying native library libjblas_arch_flavor from the archive. Consider
installing the library somewhere in the path (for Windows: PATH, for
Linux: LD_LIBRARY_PATH).

-- org.jblas DEBUG Attempting to load "libjblas_arch_flavor.so".
-- org.jblas DEBUG Trying path "/libjblas_arch_flavor.so".
-- org.jblas DEBUG Trying path "/bin/libjblas_arch_flavor.so".
-- org.jblas DEBUG Trying path "/lib/static/Linux/amd64/
libjblas_arch_flavor.so".
-- org.jblas DEBUG Trying path "/lib/static/Linux/amd64/
libjblas_arch_flavor.so".
-- org.jblas DEBUG Trying path "/lib/dynamic/Linux/amd64/
libjblas_arch_flavor.so".
-- org.jblas CONFIG Loading libjblas_arch_flavor.so from /lib/dynamic/
Linux/amd64/.
-- org.jblas DEBUG tempfile.getPath() = /tmp/
jblas5885041327465854918libjblas_arch_flavor.so
-- org.jblas DEBUG Copying took 0.0 seconds.
-- org.jblas DEBUG Attempting to load "libjblas.so".
-- org.jblas DEBUG Trying path "/libjblas.so".
-- org.jblas DEBUG Trying path "/bin/libjblas.so".
-- org.jblas DEBUG Trying path "/lib/static/Linux/amd64/sse3/
libjblas.so".
-- org.jblas DEBUG Trying path "/lib/static/Linux/amd64/sse3/
libjblas.so".
-- org.jblas DEBUG Trying path "/lib/dynamic/Linux/amd64/sse3/
libjblas.so".
-- org.jblas CONFIG Loading libjblas.so from /lib/dynamic/Linux/amd64/
sse3/.
-- org.jblas DEBUG tempfile.getPath() = /tmp/
jblas1305148370259384930libjblas.so
-- org.jblas DEBUG Copying took 0.0010 seconds.

checking matrix multiplication... ok
checking existence of dsyev...... ok

[-0.21065603010602588, -0.6404447607135408, 0.6567982667010701;
-0.5090854060895623, -0.11644450194791611, 0.1544565862797897;
-0.8075147820730989, 0.4075557568177074, -0.0772282931398948;
0.21065603010602577, 0.6404447607135403, 0.7340265598409653]
[17.23368793961409; 1.4142135623730954; 8.797820196354852E-17]
[-0.47060485181697004, 0.7822176211129873, -0.40824829046386324;
-0.571448748634892, 0.08233869695926219, 0.8164965809277258;
-0.6722926454528144, -0.6175402271944638, -0.4082482904638629]
[17.23368793961409; 1.414213562373095; 8.797820196354852E-17]
checking existence of dgesvd...... ok
Check whether we're catching XERBLA errors. If you see something like
"** On entry to DGEMM parameter number 4 had an illegal value", it
didn't work!
** On entry to DGEMM parameter number 4 had an illegal value

However, the SVDs etc. work. I guess it would be best if all the
checks went ok.

Thanks for your patience! This is a nice package!

Cheers,
OP

[Mikio Braun]

Hello OP,

I put a jar with ATLAS statically compiled in here:
http://mikiobraun.de/jblas/jblas-static-Linux-amd64-1.0.3.jar

Let me know whether it works for you!

[Olli-Pekka]

Works! Awesome. All tests pass too.

Thank you so much! Btw. in case you're interested we are planning to
use jblas as a part of or research:
http://traffic.berkeley.edu/index.html
http://lagrange.ce.berkeley.edu/drupal/index.php

Currently our code is implemented with JAMA and we're planning to use
jblas as a replacement.

Cheers,
Olli-Pekka

[Mikio Braun]

Hello Olli-Pekka,

thanks for the links. By the way, if you find some specific routines
missing from jblas, just drop me a line. Usually, it's quite simple to
add it.

Best,

Mikio

> --
> You received this message because you are subscribed to the Google Groups "jblas-users" group.
> To post to this group, send email to jblas...@googlegroups.com.
> To unsubscribe from this group, send email to jblas-users...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/jblas-users?hl=en.

[朱孟麒]

Hi mikiobraun,

I encountered same issue, but I can not download "http://mikiobraun.de/jblas/jblas-static-Linux-amd64-1.0.3.jar"

could your share it to me? like send my gmail ,thank you very much

Best Regards

Mikoo

在 2010年6月3日星期四 UTC+8下午8:30:12，mikiobraun写道：

[Mikoo]

Hi Mikio

Just forgot wrote down my gmail: benha...@gmail.com

在 2010年6月3日星期四 UTC+8下午8:30:12，mikiobraun写道：

Hello OP,