Out of memory in a for loop

[T J]

I'm running into an out-of-memory problem when I run my code in a for-loop. I'm using YALMIP with GUROBI as the underlying solver. If run my code for a single iteration, it correctly solves the problem with no issues, but once I run many iterations, I get the out-of-memory error. I tried declaring my variables outside the loop but that didn't help either. Any suggestions. Thank you.

[Johan Löfberg]

Reproducible code required for any detailed analysis

A typical mistake is that you define variables inside the loop when they could be defined outside the loop. This will cause a build-up of internal variables inside YALMIP

Perhaps you are doing some minor change to the model which just as well (and much more efficiently) could be dealt with using the optimizer framework.

[T J]

Thank you. I will try to post a reproducible code of the relevant optimization part.

[Johan Löfberg]

To summarize off-line discussion

This caused memory issues

for k=1:1e4  
    x = sdpvar(n,1,'full','complex');
    t = sdpvar(1,1,'full');
    [y,A,a] = data;
    Objective = t;
    c1 = norm((y.' + A*x),'inf') <= t
    c2 = norm(B*x+a.') <= b* norm(a.');
    c3 = norm(y.'+A*x) <= c;
    solvesdp([c1,c2,c3],Objective,options)
end

Pulling out the definition of x and t will not help, as the norm operators will create new internal variables every iteration. However, a simple fix is to simply run yalmip('clear') every iteration.

for k=1:1e4
    yalmip('clear')
    x = sdpvar(n,1,'full','complex');
    t = sdpvar(1,1,'full');
    [y,c,A,a] = data...
    Objective = t;
    c1 = norm((y.' + A*x),'inf') <= t
    c2 = norm(B*x+a.') <= b*norm(a.');
    c3 = norm(y.'+A*x) <= c;
    solvesdp([c1,c2,c3],Objective,options)
end

The code is still rather inefficient though, as it completely redefines the model, which leads to large overhead (75% spent in YALMIP, 25% in solver Mosek)

This is a perfect candidate for a optimizer object. Hence a model parameterized in A, y,c and a is created instead, and the code will be along the lines

x = sdpvar(n,1,'','complex');
t = sdpvar(1,1);
yp = sdpvar(1,m,'','complex');
cp = sdpvar(1);    
Ap = sdpvar(m,n,'full','complex');
ap = sdpvar(1,r,'','complex');
bnorma = sdpvar(1);
c1 = norm((yp.' + Ap*x),'inf') <= t;
c2 = norm(B*x+ap.') <= bnorma;
c3 = norm(yp.'+Ap*x) <= cp;
P = optimizer([c1,c2,c3],t,sdpsettings('solver','mosek'),{yp,cp,Ap,ap,bnorma},x);
   
for k=1:1e4
    [y,c,A,a] = data...
    [xsolution,diagnostics,~,~,P] = P{{y,c,A,a,b*norm(a.')}};
    if diagnostics
        error('Fail!')
    end
end