Crash with large matrices

[Paolo Toccaceli]

Hello,

I am running cvxopt 1.1.8 on a host running CentOS with kernel 2.6.32 and I am experiencing a crash when manipulating matrices beyond a certain size.
I installed cvxopt using Anaconda 4.2.0 (64-bit) with Python 3.5.2.

Steps to reproduce (from an ipython session)
In [1]: import cvxopt
In [2]: p = cvxopt.matrix(1,(16384,16384))
In [3]: p+1
Segmentation fault (core dumped)

A smaller matrix works

In [1]: import cvxopt
In [2]: p = cvxopt.matrix(1,(16383,16383))
In [3]: p+1
Out[3]: <16383x16383 matrix, tc='i'>

It seems it's the overall size

In [1]: import cvxopt
In [2]: p = cvxopt.matrix(1,(32768,8192))   # one dim doubled, the other halved
In [3]: p+1
Segmentation fault (core dumped)

I tried to find in the docs if there is a size limit, but I did not see anything.
I don't think it is an out-of-memory issue as the node has 132GB of RAM and top indicates there's plenty of memory (the ipython session appears to allocate 2GB when I create the matrix).

Any ideas anyone?

Regards

    Paolo

[Joachim Dahl]

It's probably a 32 bit overflow bug, where a index-counter is 32 bit instead of 64 bit.  It should be possible to fix given your examples, but I wonder how you reach that limit in practice?  If it's an issue you have with your optimization models, then likely you are using dense matrices in places where you should be using sparse matrices.

--
You received this message because you are subscribed to the Google Groups "CVXOPT" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cvxopt+unsubscribe@googlegroups.com.
To post to this group, send email to cvx...@googlegroups.com.
Visit this group at https://groups.google.com/group/cvxopt.
For more options, visit https://groups.google.com/d/optout.

[Martin]

This appears to be an issue with 1.1.8 that has been fixed in 1.1.9. An updated conda package is in the pipeline (https://github.com/conda-forge/cvxopt-feedstock/pull/12).

[Paolo Toccaceli]

My problem involves a block diagonal matrix (a few big blocks) and I did go down the route of using sparse matrices at first, but the solver became very slow already on a smaller size.

Anyway, I just checked and the crash appears to occur with sparse matrices too.
In [1]: import cvxopt
In [2]: s = cvxopt.spmatrix(1,(0,16384),(0,16384))
In [3]: s+1
Segmentation fault (core dumped)

Paolo

To unsubscribe from this group and stop receiving emails from it, send an email to cvxopt+un...@googlegroups.com.

[Paolo Toccaceli]

Martin,

thanks for the quick feedback. Glad to know it's being fixed :-)

[Joachim Dahl]

In this case, I think it's safe to say you should specify your matrices as sparse ones.

To unsubscribe from this group and stop receiving emails from it, send an email to cvxopt+unsubscribe@googlegroups.com.

[Martin]

CVXOPT 1.1.9 is now available from the conda-forge channel.