Paralleling gurobi in matlab

[Sebastián Eduardo Arpón Barba]

Hey guys.

Im trying to use gurobi in the matlab interface. 

for i=1:ns

        c2(:,i)=(T(:,:,i)*xk-h(:,i)+nuk(:,i))'*W(:,:,i)-(chik(i)+tk-sk(i)+eta(i))*q(:,i);

        model.Q=sparse(Q2(:,:,i));%matrix of quadratic terms of the objective function 

        model.modelsense(i)='min';% minimizing or maximizing

        model.obj=c2(:,i);%linear vector of the objective function

        model.A=sparse(zeros(1,D2));%constrain matrix A  A PROBAR CON MATRIZ VACIA []

        model.rhs=0;%right hand vector of contraints  A PROBAR CON MATRIZ VACIA []

        model.sense='=';%kind of constraints          

        model.lb=0 * ones(D2,1); %lower bound of every variable assume non negativy if this line is absent

        params.outputflag = 0;

        result=gurobi(model,params);%solving the model

        zk(:,:,i)=result.x;

        clear params;

        clear model;

end

the problem is when I try to parallel this with a parfor

parfor i=1:ns

        c2(:,i)=(T(:,:,i)*xk-h(:,i)+nuk(:,i))'*W(:,:,i)-(chik(i)+tk-sk(i)+eta(i))*q(:,i);

        model.Q=sparse(Q2(:,:,i));%matrix of quadratic terms of the objective function 

        model.modelsense='min';% minimizing or maximizing

        model.obj=c2(:,i);%linear vector of the objective function

        model.A=sparse(zeros(1,D2));%constrain matrix A  A PROBAR CON MATRIZ VACIA []

        model.rhs=0;%right hand vector of contraints  A PROBAR CON MATRIZ VACIA []

        model.sense='=';%kind of constraints          

        model.lb=0 * ones(D2,1); %lower bound of every variable assume non negativy if this line is absent

        params.outputflag = 0;

        result=gurobi(model,params);%solving the model

        zk(:,:,i)=result.x;

        clear params;

        clear model;

end 

I don't know how to use it.

Plz some help me

[Jakob Sch.]

Hi Sebastián,

As far as I can see, this will not make sense. Gurobi already uses all available cpus so there is no benefit in parallelizing your code. Instead you could triy to reuse the model and just change the coefficients if the number of variables and constraints is constant and there is not too much that is changed between the different runs.

Best regards,
Jakob

[Sebastián Eduardo Arpón Barba]

Jakob. The problem is that ns (the limit of the index) in a very little problem is 100000. For a real life problem could reach 1000000 or even more. So I want to use all my cpu power every core one solving a little problem (let say 100 variables). So i can solve 64 problems at time instead of one by one. 

[Heejin Choi]

I have a question regarding reusing the model with matlab. 

It seems that we pass all the model as a struct to gurobi() method, for instance gurobi(model), and model is a struct defining the model, and it seems to build a new model from scratch with the model parameter.

Whereas other interface such as C++, has update method, and can change a variable in the model using set method. 

Do you mean that in gurobi() matlab method, it doesn't rebuild model every time?

Thank you.

[Heejin Choi]

Also my CPU doesn't go up to 100%, and what can be the ways to use full CPU? It seems that gurobi is always stuck on one core.

On Tuesday, November 25, 2014 10:04:59 AM UTC-6, Jakob Sch. wrote:

[Jakob Sch.]

Hi Heejin,

In certain states of the branch-and-bound procedure it is possible that only one core is used. Especially in the beginning there are several tasks (preprocessing, heuristics, solving the root node) that usually utilize only one core. If you start the branch-and-bounding procedure gurobi uses all cores unless it is told otherwise.

Best regards,
Jakob

[Heejin Choi]

Actually, I think branch and bound is for MIP, however, I am only doing it linear programming.  In my case, I should get 100% CPU?

I add a screenshot. It is showing that I am calling gurobi infinitely, but CPU is only around 20%. I observe the same thing if I call gurobi c++ library inside of mex infinitely.

Is there a way to tell gurobi how many cores or threads to use? If I can get 100% CPU, this will boost up the performance greatly. I attached my model and parameter files.

Thank you.

- Heejin 

[Sonja Mars]

Hi,

> Actually, I think branch and bound is for MIP, however, I am only doing it linear programming. In my case, I should get 100% CPU?

This depends on the algorithm that is used to solve your problem. For example the Simplex algorithm does not run in parallel, it will only use one core.

Can you post the Gurobi log file/output you get, when solving your LP? That would make things easier to understand.


Thanks and best regards,
Sonja

-----------------------------------------------------------------
Dr. Sonja Mars
Gurobi Optimization

[Heejin Choi]

Thank you for your response, Dr. Sonja Mars.

These are the outputs.

Optimize a model with 2005 rows, 1512 columns and 5427 nonzeros

Coefficient statistics:

  Matrix range    [1e+00, 1e+00]

  Objective range [6e-02, 7e+00]

  Bounds range    [1e+00, 1e+00]

  RHS range       [0e+00, 0e+00]

Iteration    Objective       Primal Inf.    Dual Inf.      Time

       0   -1.8288178e+00   6.000000e+00   0.000000e+00      0s

       3   -1.1476996e+00   0.000000e+00   0.000000e+00      0s

Solved in 3 iterations and 0.00 seconds

Optimal objective -1.147699608e+00

The optimization parameters are from grbtune. I tried with method=1 or method=2 and both seems to running on single core. 

I uploaded my model file (mps file) and parameter file in previous reply in case if you want to see it.

Also do you know happened to know how to solve the problem calling gurobi c++ library inside mex in parallel using matlab parfor loop? This gives segmentation fault.

Thank you for your help.

[01543627906436788]

barrier (method = 2) uses all cores.

Try setting to method = 3 that most of the time the fastest setting. (But uses the most memory)

You can't call the same gurobi model in a parallel loop.

Only on thread can edit the gurobi model and can optimize. Gurobi uses internally more threads if possible

[Sonja Mars]

Hi,

It looks like this model is just too small and be solved too quickly to benefit from running on more than one core.

Best regards,
Sonja