Run CPLEX in YALMIP in MATLAB on Server

[Christian D]

Dear All,

I have written a Matlab code which itself calls YALMIP to run an optimization problem through running CPLEX. There a several problems I do not get further. Please note that I can run CPLEX only on a server.

1.) When I type "cplex" in my command line in linux it appears. But when I run my Matlab code which calls Yalmip to call CPLEX, CPLEX it does not know.
2.) I do not understand how this all should be put together when compiling. I mean it know that one can compile matlab code in advance and than just hand in the compiled file, right? Now how does this change with Yalmip and CPLEX, is this automatically included in the compilation if the thing with CPLEX would work? 
3.) The algorithm is based on parallel computing, however I call at the moment the Yalmip in a for-loop which could be done in one. So how do I get this parallelised. Do I use the Matlab Parallel Computing Toolbox or modify somehow my code in Yalmip (as there is an example for affine changes which can be expressed in one) or should I parallelise over CPLEX which also has a parallel setting? 



In general I am confused how to put all this together besides getting frustrated with the Linux commands. I am a beginner in Linux and since days I am trying to get it work.I would really appreciate any help of you guys!

Best regards,
Chris

[Johan Löfberg]

All YALMIP cares about is that you have the MATLAB interface in your path. You never work in the OS with YALMIP

>> which cplex
C:\Program Files\IBM\ILOG\CPLEX_Enterprise_Server126\CPLEX_Studio\cplex\matlab\x64_win64\@Cplex\Cplex.p  % Cplex constructor
>>

YALMIP cannot be parallelized with parfor etc. cplex parallelization etc is outside YALMIP and YALMIP does not care/know if the solver happens to be parallelized.

[Christian D]

Thanks to you Mr. Löfberg. However, I just not get it run. It is version CPLEX 1.4 and Matlab R2014a. When I type CPLEX in my command windows there it appears "Welcome to IBM(R) ILOG(R) CPLEX(R) Interactive Optimizer 12.4.0.0 ...". When I try to find the path it gives /usr/sepp/bin/cplex. 

Then I do addpath('/usr/sepp/bin/cplex'); which get me Warning: Directory access failure: /usr/sepp/bin/cplex 

What is concerning me the most is that when I do "yalmiptest" it shows CPLEX not found BUT when I call yalmip with 

            ops = sdpsettings;

            ops.cplex;

            solvesdp(F,obj,ops);

It solves my problem ?! I read that yalmip has an internal solver which is not that good as CPLEX but I do not understand what it really does. Its confusing me totally. I call CPLEX and it does not use CPLEX?


Okay thanks for clearly telling me that parallelization is not possible in Yalmip. So the thing is I am calling Yalmip in a for-loop which is really bad performance. And I think calling all in one will increase my computational time towards infinity since I need integer programming (NP hard...). Any idea maybe? But main priority is to get CPLEX work (the real CPLEX...).

Thanks.

Chris

[Johan Löfberg]

I am pretty sure that the MATLAB interface is not located in bin/cplex. Surely the directory should be something like blabla/cplex/blabla/matlab, like the directtory I showed you from my machine

If you solve a MILP in YALMIP without cplex installed (or any other MILP solver), it will use the naive internal implementation bnb (using linprog as local solver if you have the optimization toolbox).

As I said, the solver most likely parallelizes on your setup, so unless the YALMIP overhead is massive, the total time should be the same regardless if you try to solve multiple problems with a parfor-loop, if you solve them in a for-loop. The total cputime spent would be the same,

However, as bnb manages to solve your problem, it is obviously a very very simple problem.

[Johan Löfberg]

In R2014a, the optimization toolbox ships with a MILP solver (intlinprog) which might have been used by YALMIP here (just look at the output from solvesdp, it says which solver was invoked)

[Christian D]

It is not that simple one decoupled subproblem takes about 60 seconds (with CPLEX and much longer without) and I have 1000 subproblems - and this all is for one iteration. I need about 10 000 iterations. -> 60*1000*10000. This is why parallelization & CPLEX would be nice.

[Mark L. Stone]

That's about 19 years.  Is parallelization going to get you to a reasonable run time?

[Christian D]

Haha yes 19 Years.

I tried first parallelization with YALMIP in a loop, however MATLAB gave me some internal errors considering parfor (it worked particularly but sporadically gave an error and never came to an end). Moreover calling YALMIP takes additional time. It runs best to use parfor and then call CPLEX directly without YALMIP (which took me some time to comprehend that YALMIP is not really a big help). Moreover setting the MIP tolerance with the parameter numbers finally ends up to fasten up the process by magnitudes. 

Thanks for your help.

[Johan Löfberg]

YALMIP cannot be used with parfor (unless you use optimizer objects, then it works. optimizer objects reduces overhead also, so perhaps you should take a look at it. when it has completed after 19 years I might have made yalmip complaint with parfor, but then it is not required any longer since the problem will be solved in 1 second then)