How many cores can MOSEK effectively utilize?

[maur...@gmail.com]

When solving a convex problem (more specifically convex objective and linear constraints) using the interior-point optimizer, how many cores can MOSEK effectively utilize? What sorts of computations can be performed in parallel? Thank you

[Erling D. Andersen]

What kind of convex objective do you have? Quadratic? 

First of all we recommend reformulating the problems as conic optimization problem if possible. Also the conic optimizer is better parallelized than the convex optimizer.

MOSEK will try to exploit all the cores available. However, for smallish problems you will not see much benefit. 

[maur...@gmail.com]

I have a mix of quadratic, exponential, and logarithmic functions in the objective. So I don't believe I can reformulate into a conic problem. I am using mskscopt in the MOSEK MATLAB interface. My test environment only has 4 cores, I'm just curious up to what number of cores good scaling can be expected, and at what number of cores will there be greatly diminishing returns. Thank you

[Erling D. Andersen]

We are extending MOSEK to handle the exponential cones which allows you to do the exponential and logarithmic stuff you currently only can do with Scopt. 

Anything about speed up is guess work at best. You have to try it out.

Btw you can dump scopt models to disk (I can tell you gow). And if you are willing to give them to me I can convert them conic form and tell you speedup for both the separable and conic form.

I have a computer with a huge number of cores.

[maur...@gmail.com]

Please let me know about dumping models to the disk. That would be very helpful.

[Erling D. Andersen]

You can upload the files to our dropbox mentioned at

   https://www.mosek.com/support/

This is the function you need (param and cmd is optional):

function  [res] = mskscwri(filename,opr,opri,oprj,oprf,oprg,c,a,blc,buc,blx,bux,param,cmd)

%

% Syntax : mskscwri(filename,opr,opri,oprj,oprf,oprg,c,a,blc,buc,blx,bux,param,cmd)

%

% Purpose: Write an scopt problem to the files filename.sco and filename.mps

%

[maur...@gmail.com]

Hi Dr. Andersen. I uploaded two sample problems to the dropbox, one has a power objective and the other is exponential. On my system using 4 cores (i5 4670k) each problem is solved in roughly 2 and a half minutes. Would greatly appreciate if you can provide a sense of how the solution times scale as cores increase, I imagine that at some point there are diminishing returns. 

On another note I tried to set MSK_IPAR_NUM_THREADS to 1, 2 and 3 but my CPU usage still showed 100% when running the problem. Not sure why.

[Erling D. Andersen]

Thanks. Will get back shortly.

[Erling D. Andersen]

Are you minimizing or maximizing?

[Erling D. Andersen]

Assuming you are maximizing and using the current and very preliminary version 9. 

Below I show the time for 1,2,4,8,12 threads. There is some benefits up to 8 threads, Well, for the bigger one 12 threads is still useful

Optimizer time (medium)

Name	time runtest-conic-1t-d-default-diaz	time runtest-conic-2t-d-default-diaz	time runtest-conic-4t-d-default-diaz	time runtest-conic-8t-d-default-diaz	time runtest-conic-12t-d-default-diaz	iter runtest-conic-1t-d-default-diaz	iter runtest-conic-2t-d-default-diaz	iter runtest-conic-4t-d-default-diaz	iter runtest-conic-8t-d-default-diaz	iter runtest-conic-12t-d-default-diaz
Sum	127.322	85.502	63.593	59.984	58.370	68	68	67	69	69
N. wins	0	0	0	0	1
Name	time runtest-conic-1t-d-default-diaz	time runtest-conic-2t-d-default-diaz	time runtest-conic-4t-d-default-diaz	time runtest-conic-8t-d-default-diaz	time runtest-conic-12t-d-default-diaz	iter runtest-conic-1t-d-default-diaz	iter runtest-conic-2t-d-default-diaz	iter runtest-conic-4t-d-default-diaz	iter runtest-conic-8t-d-default-diaz	iter runtest-conic-12t-d-default-diaz
c-diaz_test_a106	127.322	85.502	63.593	59.984	58.370	68	68	67	69	69

Optimizer time (large)

Name	time runtest-conic-1t-d-default-diaz	time runtest-conic-2t-d-default-diaz	time runtest-conic-4t-d-default-diaz	time runtest-conic-8t-d-default-diaz	time runtest-conic-12t-d-default-diaz	iter runtest-conic-1t-d-default-diaz	iter runtest-conic-2t-d-default-diaz	iter runtest-conic-4t-d-default-diaz	iter runtest-conic-8t-d-default-diaz	iter runtest-conic-12t-d-default-diaz
Sum	188.154	117.202	92.523	77.288	73.761	99	96	100	99	99
N. wins	0	0	0	0	1
Name	time runtest-conic-1t-d-default-diaz	time runtest-conic-2t-d-default-diaz	time runtest-conic-4t-d-default-diaz	time runtest-conic-8t-d-default-diaz	time runtest-conic-12t-d-default-diaz	iter runtest-conic-1t-d-default-diaz	iter runtest-conic-2t-d-default-diaz	iter runtest-conic-4t-d-default-diaz	iter runtest-conic-8t-d-default-diaz	iter runtest-conic-12t-d-default-diaz
c-diaz_test_c47	188.154	117.202	92.523	77.288	73.761	99	96	100	99	99

[Erling D. Andersen]

Time is seconds btw.

[maur...@gmail.com]

Thank you, this is very helpful. Yes, it is a utility maximization problem. Quite impressive that gains can still be made from 8 to 12 threads, kudos to you and your team. As larger numbers of cores become common, do you think it's worthwhile to try increasing parallelization in MOSEK's algorithms (non-integer problems)? How much room for improvement is still left?

On Tuesday, February 6, 2018 at 6:06:39 AM UTC-5, Erling D. Andersen wrote:

Time is seconds btw.

[Erling D. Andersen]

If you increase the size of the problem you are solving, then more 12 cores be could useful. However, frequently the cores cannot get data fast enough to keep all the cores busy.

So the memory is bottleneck. 

 

For that reason and Amdahls law it is hard to improve the speed ups but some improvement is always possible.