cgo lapack linker errors

[Brendan Tracey]

I just noticed that I can no longer compile lapack/cgo. I'm wondering if I'm doing something wrong.

I usually have done:
cd ~/software
git clone https://github.com/Xianyi/OpenBLAS
cd OpenBLAS
make
cd $GOPATH/src/github.com/gonum/lapack/cgo
CGO_LDFLAGS="-L/Users/brendan/software/OpenBLAS -lopenblas" go test

This whole procedure works fine with blas/cgo (including failing if I don't include the CGO_LDFLAGS), and it used to run fine with lapack. Now I get linker errors that none of the symbols are available. Did the -lopenblas change which symbols are included?

If it's useful:
gcc --version
Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 7.3.0 (clang-703.0.29)
Target: x86_64-apple-darwin15.4.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin


Thanks

[Vladimír Chalupecký]

It might be somehow related to the recent switch to C.LAPACKE_*_work calls in the clapack package (commits 0a88dcb and 383e92f). Recently I was getting linker errors in lapack/cgo with MKL (l_mkl_11.3.0.109.tgz) because some of the more obscure *_work routines were not in the library. Updating to the latests version of MKL (l_mkl_11.3.3.210.tgz) solved the issue.

I'm not having any problems with OpenBLAS (on Linux) so I guess the cause will be somewhere else.

Could you confirm that it is really the case the none of the symbols are available?

You could try removing the binary package from pkg directory: rm -r $GOPATH/pkg/darwin_amd64/github.com/gonum/lapack*

You could try cleaning OpenBLAS before compiling:

cd ~/software/OpenBLAS

git checkout develop

git pull

make clean

make

Not related to your issue but since you are on Apple, I'd be curious if linking lapack/cgo against Accelerate works with the new *_work calls.

[Dan Kortschak]

It would be unfortunate if we had to revert the work change; it omits
code paths that have been a source of external bugs for us, so I'd
really like to keep it the way that it is. Do we have a survey of
LAPACKE implementations and what API they expose?

On Thu, 2016-06-23 at 18:06 -0700, Vladimír Chalupecký wrote:
> It might be somehow related to the recent switch to C.LAPACKE_*_work calls
> in the clapack package (commits 0a88dcb and 383e92f). Recently I was
> getting linker errors in lapack/cgo with MKL (l_mkl_11.3.0.109.tgz) because
> some of the more obscure *_work routines were not in the library. Updating
> to the latests version of MKL (l_mkl_11.3.3.210.tgz) solved the issue.
> I'm not having any problems with OpenBLAS (on Linux) so I guess the cause
> will be somewhere else.
>

> Could you confirm that it is really the case the *none* of the symbols are

[Vladimír Chalupecký]

It would be unfortunate if we had to revert the work change; it omits
code paths that have been a source of external bugs for us, so I'd
really like to keep it the way that it is.

Me too.

 

Not related to your issue but since you are on Apple, I'd be curious if 
linking lapack/cgo against Accelerate works with the new *_work calls. 

Now I realize that Accelerate might not have LAPACKE, so most likely it was not working before either.

Do we have a survey of LAPACKE implementations and what API they expose?

I might be wrong but it seems to me that there is only one LAPACKE implementation that follows and changes with the versions of LAPACK.

lapacke.h (and therefore the generated clapack.go) that we currently have in cgo/clapack is from LAPACK 3.6.0, so with older versions of LAPACK the linker will fail (as it does for me on Fedora 23 that has version 3.5.0).

So in my previous post I should have written "*_work routines added in LAPACK 3.6.0" instead of "some of the more obscure *_work routines".

What is our policy on the version(s) of LAPACK(E) that we support (i.e., have in clapack)? Only the latest?

If that's the case, we should update it to 3.6.1, there are some fixes in routine signatures.

Also, it seems to me that the 'clapack' package could be more accurately called 'lapacke'.

[Brendan Tracey]

I pulled from master on OpenBLAS (no changes because I had downloaded yesterday) and did a clean install. I pulled from Lapack master. Deleted $GOPATH/pkg.

Still same problems.

It does appear that it's just the *_work routines. Small excerpt:

  "_LAPACKE_dormhr_work", referenced from:
      __cgo_836158d91a5b_Cfunc_LAPACKE_dormhr_work in clapack.cgo2.o
     (maybe you meant: __cgo_836158d91a5b_Cfunc_LAPACKE_dormhr_work)
  "_LAPACKE_dormlq_work", referenced from:
      __cgo_836158d91a5b_Cfunc_LAPACKE_dormlq_work in clapack.cgo2.o
     (maybe you meant: __cgo_836158d91a5b_Cfunc_LAPACKE_dormlq_work)
  "_LAPACKE_dormql_work", referenced from:
      __cgo_836158d91a5b_Cfunc_LAPACKE_dormql_work in clapack.cgo2.o
     (maybe you meant: __cgo_836158d91a5b_Cfunc_LAPACKE_dormql_work)


The error is somewhat strange. It's suggesting "maybe you meant" that's the same as the one it can't find. (side note: this is why I hate linkers, this kind of cryptic error message that's hard to search for). It must be something with the way cgo works on OSX, right? The routines must be exported, otherwise you all wouldn't be able to see them -- we're using the same repository. I've also been experiencing https://github.com/golang/go/issues/16129 which is possibly related.

[Vladimír Chalupecký]

How did you install your Fortran compiler? Isn't this some conflict or misunderstanding between the Fortran (gfortran?) and C (clang?) compilers?