can't build/install cvxopt from OS X Lion

[jiaer]

Helps/hints would be appreciated.

Here is the erro message from a failed test.

Traceback (most recent call last):

  File "lp.py", line 7, in <module>

    sol = solvers.lp(c, G, h)  

  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cvxopt/coneprog.py", line 2794, in lp

    from cvxopt import base, blas, misc

  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cvxopt/misc.py", line 20, in <module>

    from cvxopt import base, blas, lapack, cholmod, misc_solvers

ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cvxopt/lapack.so, 2): Symbol not found: _ilaenv_

  Referenced from: /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cvxopt/lapack.so

  Expected in: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib

 in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cvxopt/lapack.so

[Martin]

Hi,

I have compiled and installed CVXOPT 1.1.3 in Lion, and it appears to work without any problems (the lp.py example runs fine). The ilaenv symbol is present in my BLAS library. You can check your BLAS library with the follow command:

nm /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib | grep ilaenv

000000000000d530 t _ilaenv_

000000000000d5a1 t _ilaenv_safe

I'm using Python 2.7 (default in Lion) and GCC 4.2.1.

Martin

[jiaer]

the _ilaenv_ seems to be there 

nm /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib | grep ilaenv

000000000000d530 t _ilaenv_

000000000000d5a1 t _ilaenv_safe

my python version is

Python 2.7.2 (v2.7.2:8527427914a2, Jun 11 2011, 15:22:34) 

[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin

The only thing I did was to installed python from the official website. Could this be a problem? How can I restore the version shipped by Lion?

Thanks a lot!

Jiaer

[Martin]

I don't think the official Python is the problem, but it is easy to try the built-in version. The official Python adds the followling lines to the ".profile" file in your home directory:

# Setting PATH for Python 2.7                                                   

# The orginal version is saved in .profile.pysave                               

PATH="/Library/Frameworks/Python.framework/Versions/2.7/bin:${PATH}"           

export PATH

Simply comment out the last two lines and reopen Terminal to revert to the built-in version of Python. Be aware that another user has reported some strange behavior with the lp/conelp solver in Lion. I believe that this issue is still unresolved.

Martin

[Shamus Patry]

I was having similar issues as jiaer, I had not updated xcode with lion and I had the path from the official version.  But I've resolved both those issues, and it seems to be working! (the LP and QP examples on the website work anyway).  Without the path from the official version it uses llvm-gcc-4.2, with the path it uses gcc-4.2 and seems to target 10.6 rather than 10.7.  And it seems that the built in BLAS and LAPACK were recompiled for 64-bit.  The nm command from the older version of xcode couldn't recognize libBLAS.dylib (or anything in the Accelerate framework) as valid.

However during the build there were a number of warnings. I'm just wondering how many of them are normal (for any target, osx, or lion specifically), or if I still have something messed up in my environment.

1-warning: implicit conversion shortens 64-bit value into a 32-bit value - happens throughout the build

2-warning: ignoring #pragma ivdep - throughout the build, but much less often

3-warning: initialization from incompatible pointer type - only seems to be lapack.o

I suspect 1 is due to lion, and I've read that apple a nonstandard BLAS/LAPACK build, and would guess 3 relates to that.

Thanks

Shamus

[Martin]

The official precompiled Python version did not work for me either, so it appears that the system supplied version of python is the way to go for now. 

The implicit conversion warnings should be harmless. Python integers are stored as long int while BLAS and LAPACK use int for integer parameters. This conversion is currently performed implicitly by the compiler, and with proper typecasts in a future update, these warnings should disappear. 

The ivdep pragmas appear in the SuiteSparse source code, and as far as I know, they're only used by Intel's C compiler, and hence it is OK to ignore the gcc warnings.

I think that the incompatible pointer type in lapack.c is due to the fact that the macros SIZEOF_INT and SIZEOF_LONG are not defined in Lion (use "cpp -dM /dev/null" to list defined macros), and therefore this warning is potentially dangerous and may require a fix. I'll try to post a fix as soon as possible. Perhaps it is sufficient to replace SIZEOF_INT with INT_MAX and SIZEOF_LONG with LONG_MAX since INT_MAX and LONG_MAX are defined under both Ubuntu and Mac OS X.

Martin

[Martin]

The macros SIZEOF_INT and SIZEOF_LONG are not available in Lion, but they are defined in a Python header file (pyconfig.h), so this is not the issue.

The incompatible pointer warnings only occur when compiling a 32-bit binary in Lion (the default is a universal binary with both 32-bit and 64-bit). The warnings are due to some missing typecasts. It should be safe to ignore these until they are fixed in a future update.

Python will build 64-bit extensions if you add the following line to your .profile in your home directory:

export ARCHFLAGS='-arch x86_64'

[Shamus Patry]

I'm guessing one of the warnings isn't good.  I built CVXMOD on top of CVXOPT, the simple examples worked, but I tried the l1 regularization example here http://cvxmod.net/examples/spike.html

I got the following error out (I have run this example just fine with pythonxy install on Windows7)

using conic solver.

     pcost       dcost       gap    pres   dres   k/t

 0:  1.0742e-16  4.3973e-17  3e+00  1e+00  1e+00  1e+00

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

  File "<stdin>", line 5, in l1reg

  File "/Library/Python/2.7/site-packages/cvxmod/base.py", line 2496, in solve

    res = cvxopt.solvers.conelp(c, G, h, dims, A, b)

  File "/Library/Python/2.7/site-packages/cvxopt/coneprog.py", line 1037, in conelp

    W = misc.compute_scaling(s, z, lmbda, dims, mnl = 0)

  File "/Library/Python/2.7/site-packages/cvxopt/misc.py", line 284, in compute_scaling

    W['d'] = base.sqrt( base.div( s[mnl:mnl+m], z[mnl:mnl+m] ))

  File "/Library/Python/2.7/site-packages/cvxopt/__init__.py", line 232, in div

    return +reduce(base.ediv, args)

ArithmeticError: division by zero

Also, I tried to add ARCHFLAGS, and still got the 32/64 bit cast warnings :(

[Dima Pasechnik]

while trying to port Sage to MacOSX 10.7, we get things like:

Testing in src/examples/doc/chap4 
Testing  acent ...
 0.  Newton decr. = 1.193e+01
Traceback (most recent call last):
  File "acent", line 66, in <module>
    x, ntdecrs = acent(A, b)  
  File "acent", line 50, in acent
    if -sum(log(1-step*y)) < ALPHA*step*lam: break
ValueError: domain error
Error: /Applications/sage_builds/Lion/new/sage-4.8.alpha0/local/bin/python acent failed
Testing  acent.py ...
 0.  Newton decr. = 1.193e+01
Traceback (most recent call last):
  File "acent.py", line 64, in <module>
    x, ntdecrs = acent(A, b)  
  File "acent.py", line 48, in acent
    if -sum(log(1-step*y)) < ALPHA*step*lam: break
ValueError: domain error
Error: /Applications/sage_builds/Lion/new/sage-4.8.alpha0/local/bin/python acent.py failed

The full log is here:

http://sage.math.washington.edu/home/palmieri/misc/cvxopt-1.1.3.log

and reported on

http://trac.sagemath.org/sage_trac/ticket/12011

Any clue as to how to fix these?

Thanks,

Dima

[Martin]

I get the same error when I compile CVXOPT with the built-in blas/lapack library (Accelerate), but the error disappears when I link against another blas library (ref blas or atlas). I suspect that there is a bug in the Apple supplied blas library in Lion.