segment fault

76 views
Skip to first unread message

王栋

unread,
Jun 27, 2017, 5:04:18 AM6/27/17
to CVXOPT
Hi all:

i found a error segment fault when i use cvxopt as follows:

from cvxopt import solvers

print 'pMatrix size is' + str(pMatrix.size)
print 'qMatrix size is' + str(qMatrix.size)
print 'gMatrix size is' + str(gMatrix.size)
print 'hMatrix size is' + str(hMatrix.size)

sol=solvers.qp(pMatrix, qMatrix, gMatrix, hMatrix)
resMatrix = sol['x']
print 'resMatrix size  is ' + str(resMatrix.size)
print 'the weights is:'
print resMatrix[0:20]

the print output is:
pMatrix size is(355063, 355063)
qMatrix size is(355063, 1)
gMatrix size is(710116, 355063)
hMatrix size is(710116, 1)

the code give error:
sol=solvers.qp(pMatrix, qMatrix, gMatrix, hMatrix)

the cvxopt version is 1.1.9

anybody can help me? 
thanks ahead

Martin

unread,
Jun 27, 2017, 5:07:42 AM6/27/17
to CVXOPT
Thanks for reporting this. Can you provide a minimal example? What platform are you using?

王栋

unread,
Jun 27, 2017, 9:45:10 PM6/27/17
to CVXOPT
the platform is:

Distributor ID: Debian
Description: Debian GNU/Linux 8.7 (jessie)
Release: 8.7
Codename: jessie


在 2017年6月27日星期二 UTC+8下午5:07:42,Martin写道:

王栋

unread,
Jun 27, 2017, 10:10:58 PM6/27/17
to CVXOPT
the matrix is generated dynamically, and some matrix is sparce matrixs,how can i get and save to file?


在 2017年6月27日星期二 UTC+8下午5:07:42,Martin写道:
Thanks for reporting this. Can you provide a minimal example? What platform are you using?

王栋

unread,
Jun 27, 2017, 10:24:05 PM6/27/17
to CVXOPT
i read the document carefully, and get the matrix. please check the attachment, it is named "example.tar.gz", contain 4 matrix i used


在 2017年6月27日星期二 UTC+8下午5:07:42,Martin写道:
Thanks for reporting this. Can you provide a minimal example? What platform are you using?
example.tar.gz

Martin

unread,
Jun 28, 2017, 2:40:26 AM6/28/17
to CVXOPT
Unfortunately I don't know how to read the files that you provided. It would be very helpful if you could provide a minimal, self-contained script (along with your data) that reproduces the error.

王栋

unread,
Jun 28, 2017, 4:20:21 AM6/28/17
to CVXOPT
ok,please check the attachment again.
it can be executed directely

在 2017年6月28日星期三 UTC+8下午2:40:26,Martin写道:
Desktop.rar

Martin

unread,
Jun 28, 2017, 5:51:26 AM6/28/17
to CVXOPT
Your example runs without any issues on my machine (macOS 10.12.5, Python 2.7.12, cvxopt 1.1.9). 

A couple of follow-up questions:
  • Is your OS 32-bit or 64-bit? (arch or uname -m)
  • Which version of Python are you using? (python -V)
  • How did you install CVXOPT?

王栋

unread,
Jun 28, 2017, 8:15:30 AM6/28/17
to CVXOPT
thank you.

my os is:
Distributor ID: Debian
Description: Debian GNU/Linux 8.7 (jessie)
Release: 8.7
Codename: jessie

when i run my program on ubuntu ,it is also OK:
Distributor ID: Ubuntu
Description: Ubuntu 16.04.2 LTS
Release: 16.04
Codename: xenial

i guess it is something wrong on Debian.
i use too much time to confirm the program

在 2017年6月28日星期三 UTC+8下午5:51:26,Martin写道:

Martin

unread,
Jun 28, 2017, 9:50:38 AM6/28/17
to CVXOPT
OK. Ubuntu is also based on Debian. My current guess is that either your Debian system is 32-bit and there is a bug in CVXOPT or there is a bug in one of the libraries that CVXOPT links against.

王栋

unread,
Jun 28, 2017, 9:55:09 PM6/28/17
to CVXOPT
my Debian system is 64-bit.

i guess there is a bug in one of the libraries that CVXOPT links against.

thanks for your help again.

在 2017年6月28日星期三 UTC+8下午9:50:38,Martin写道:

王栋

unread,
Jun 28, 2017, 10:59:34 PM6/28/17
to CVXOPT
at another machine, use same ubuntu system, it also get segment fault. the error info is:

Thread 11 "python" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffdbbd3700 (LWP 7932)]
0x00007ffff1e64ce0 in dscal_kernel_8_zero ()
   from /home/wangdong/anaconda2/lib/python2.7/site-packages/cvxopt/.libs/libopenblasp-r0-39a31c03.o
(gdb) bt
#0  0x00007ffff1e64ce0 in dscal_kernel_8_zero ()
   from /home/wangdong/anaconda2/lib/python2.7/site-packages/cvxopt/.libs/libopenblasp-r0-39a31c03.o
#1  0x00007ffff1e64f2c in dscal_k_HASWELL ()
   from /home/wangdong/anaconda2/lib/python2.7/site-packages/cvxopt/.libs/libopenblasp-r0-39a31c03.o
#2  0x00007ffff0848d13 in trmv_kernel ()
   from /home/wangdong/anaconda2/lib/python2.7/site-packages/cvxopt/.libs/libopenblasp-r0-39a31c03.o
#3  0x00007ffff0a0e9a5 in blas_thread_server ()
   from /home/wangdong/anaconda2/lib/python2.7/site-packages/cvxopt/.libs/libopenblasp-r0-39a31c03.o
#4  0x00007ffff77c56ba in start_thread (arg=0x7fffdbbd3700) at pthread_create.c:333
#5  0x00007ffff6deb82d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
(gdb) 


在 2017年6月28日星期三 UTC+8下午9:50:38,Martin写道:
OK. Ubuntu is also based on Debian. My current guess is that either your Debian system is 32-bit and there is a bug in CVXOPT or there is a bug in one of the libraries that CVXOPT links against.

Martin

unread,
Jun 29, 2017, 2:48:49 AM6/29/17
to CVXOPT
Thanks, that's helpful. It looks like an issue with OpenBLAS. 

I can see that you are using anaconda. How did you install CVXOPT? From the conda-forge channel or some other channel? (If you are not sure, you can find out with conda list cvxopt).

王栋

unread,
Jun 29, 2017, 3:25:06 AM6/29/17
to CVXOPT
i use pip, as follows:
cvxopt                    1.1.9                     <pip>

pip show cvxopt:

Name: cvxopt
Version: 1.1.9
Summary: Convex optimization package
Author: M. Andersen, J. Dahl, and L. Vandenberghe
License: GNU GPL version 3
Location: /home/wangdong/anaconda2/lib/python2.7/site-packages
Requires:

python --version:
Python 2.7.13 :: Anaconda 4.4.0 (64-bit)

yes, i also guess an issue with OpenBLAS.
in .bashrc i set:
export OPENBLAS_NUM_THREADS=1
after that, it can run correctyly with single thread

but when i set OPENBLAS_NUM_THREADS is 6 (my machine is 12 core), it still run with single thread
but when i set OPENBLAS_NUM_THREADS bigger than 10 (my machine is 12 core), it still run with segment fault

在 2017年6月29日星期四 UTC+8下午2:48:49,Martin写道:

Martin

unread,
Jun 29, 2017, 4:08:08 AM6/29/17
to CVXOPT
You could try to build CVXOPT with ATLAS instead of OpenBLAS:

1. Install libatlas-dev and make sure that it is the current default: 

sudo apt-get install libatlas-dev
update
-alternatives --config libblas.so.3
update
-alternatives --config liblapack.so.3

2. Uninstall CVXOPT and build/reinstall CVXOPT from source:

pip uninstall cvxopt
pip install cvxopt
--no-binary cvxopt

王栋

unread,
Jun 29, 2017, 4:56:30 AM6/29/17
to CVXOPT
i followed your method.but i find my program runs very slowly with single thread.

how can i run with multithread? my program has any problem?

在 2017年6月29日星期四 UTC+8下午4:08:08,Martin写道:

Martin

unread,
Jun 29, 2017, 5:11:56 AM6/29/17
to CVXOPT
I think the maximum number of threads that ATLAS will use is set at compile time. Perhaps Ubuntu provides a single-threaded ATLAS library or a version with a small number of maximum threads. You may have to compile ATLAS yourself to change that, but compiling ATLAS takes a while.

Alternatively, you could try linking CVXOPT against MKL.

Sergio

unread,
Jun 29, 2017, 6:00:08 AM6/29/17
to CVXOPT
ATLAS binaries in Debian and Debian derivatives are not optimized (because tons of variants would be needed).

To get good performance with ATLAS on debian derivatives, you need to recompile it on your own hardware. The build process assures that the build adapts to your own system.  This may require many tests to be run on your system to determine what routines and parameters are optimal for it.  Depending on your specific architecture the build may take a very very long time because of these tests.  On some systems pre-determined profiles can be used that speed up things, but on others all the tests are run, taking ages (even a whole day).

Instructions to build atlas are in /usr/share/doc/libatlas3-base/README.Debian

Martin

unread,
Jun 29, 2017, 6:36:36 AM6/29/17
to CVXOPT
Sergio, thanks for clarifying that. 

An alternative idea would be to try building OpenBLAS on your system, following a tip included in the OpenBLAS FAQ:

"Please build OpenBLAS with larger NUM_THREADS. For example, make NUM_THREADS=32 or make NUM_THREADS=64. In Makefile.system, we will set MAX_CPU_NUMBER=NUM_THREADS."
Reply all
Reply to author
Forward
0 new messages