Exponential Cone Stalling

[AS95]

I am solving an exponential cone optimization problem via MOSEK. 

I tried the very same problem via JuMP in Julia, and YALMIP in MATLAB. 

YALMIP always terminates without any "stalling" issues. However, JuMP gives me stalling issues: 

`SLOW_PROGRESS::TerminationStatusCode = 19`

I am wondering: does YALMIP do something extra to prevent stalling issues in general? Specifically, I tried 

`s = sdpsettings;

s.mosek`

to see all the settings of MOSEK used. In general, I see that all of the parameter settings YALMIP uses match with the default parameters stated on MOSEK's website. 

In general, can it be that YALMIP does not have stalling issues because there is some tolerance that is different than the standard tolerance, or, does YALMIP apply some other non-trivial steps to prevent stalling?

Thank you for your time (whoever is reading)!

[Erling D. Andersen]

This is most likely a random effect that has to do with that Julia and Yalmip presents slightly different problems to Mosek.

I would suggest you post the Mosek log output from Julia and Yalmip for the same problem. Given those it is way more likely that something useful can be said.

[Johan Löfberg]

Going from an abstract user model to the low-level numerical format involves several steps with various degrees of freedom, and all of them an have an influence. Pretty much impossible to answer generically, so you will have to look at the low-level model sent to mosek and see if you can find anything (are extra variables introduced in some normalization, do they represent the primal or the dual, are non-standard operators modelled in the same way, etc etc). 

[AS95]

Dear Erling,

Thank you very much for your reply! I am really confused I hope the log files can help me understand better what is happening. Thank you for your suggestions. I am attaching the solver logs from YALMIP and JuMP. Note that the number of constraints and variables are different because YALMIP directly uses logsumexp constraints without auxiliary variables but with JuMP I also introduce all the cone constraints. 

Objective values (This is a minimization problem.)

YALMIP's objective: 0.664015155969750

JuMP's objective: 0.6640152505926787

YALMIP

MOSEK Version 9.2.47 (Build date: 2021-6-15 12:33:47)
Copyright (c) MOSEK ApS, Denmark. WWW: mosek.com
Platform: MACOSX/64-X86

Problem
  Name                   :                
  Objective sense        : min            
  Type                   : CONIC (conic optimization problem)
  Constraints            : 2620            
  Cones                  : 1024            
  Scalar variables       : 30259          
  Matrix variables       : 0              
  Integer variables      : 0              

Optimizer started.
Presolve started.
Linear dependency checker started.
Linear dependency checker terminated.
Eliminator started.
Freed constraints in eliminator : 512
Eliminator terminated.
Eliminator started.
Freed constraints in eliminator : 0
Eliminator terminated.
Eliminator - tries                  : 2                 time                   : 0.00            
Lin. dep.  - tries                  : 1                 time                   : 0.00            
Lin. dep.  - number                 : 0              
Presolve terminated. Time: 0.03    
Problem
  Name                   :                
  Objective sense        : min            
  Type                   : CONIC (conic optimization problem)
  Constraints            : 2620            
  Cones                  : 1024            
  Scalar variables       : 30259          
  Matrix variables       : 0              
  Integer variables      : 0              

Optimizer  - threads                : 16              
Optimizer  - solved problem         : the primal      
Optimizer  - Constraints            : 1084
Optimizer  - Cones                  : 1024
Optimizer  - Scalar variables       : 28723             conic                  : 3072            
Optimizer  - Semi-definite variables: 0                 scalarized             : 0              
Factor     - setup time             : 0.02              dense det. time        : 0.00            
Factor     - ML order time          : 0.00              GP order time          : 0.00            
Factor     - nonzeros before factor : 3.70e+04          after factor           : 3.87e+04        
Factor     - dense dim.             : 0                 flops                  : 2.56e+06        
ITE PFEAS    DFEAS    GFEAS    PRSTATUS   POBJ              DOBJ              MU       TIME  
0   1.6e+00  1.3e+00  1.1e+03  0.00e+00   1.090858272e+03   0.000000000e+00   1.0e+00  0.08  
1   1.3e-01  1.0e-01  6.2e+01  6.93e-02   1.522290028e+02   -4.169904127e+00  7.8e-02  0.16  
2   3.6e-02  2.9e-02  4.3e+00  2.05e+00   2.211175348e+01   -3.218893603e+00  2.2e-02  0.17  
3   5.3e-03  4.2e-03  1.9e-01  1.31e+00   5.058085913e-01   -2.618746649e+00  3.3e-03  0.18  
4   1.1e-03  9.0e-04  1.2e-02  1.51e+00   -7.909188463e-01  -1.283457090e+00  7.0e-04  0.19  
5   5.5e-04  4.4e-04  6.3e-03  7.57e-02   -6.659258501e-01  -1.041155146e+00  3.4e-04  0.20  
6   3.4e-04  2.7e-04  2.2e-03  1.06e+00   -7.130025375e-01  -9.002719075e-01  2.1e-04  0.20  
7   1.6e-04  1.3e-04  7.0e-04  1.25e+00   -6.942261805e-01  -7.751067849e-01  1.0e-04  0.21  
8   1.1e-04  8.8e-05  3.7e-04  1.00e+00   -6.801523861e-01  -7.324371095e-01  6.8e-05  0.22  
9   8.6e-05  6.9e-05  2.4e-04  1.26e+00   -6.732753785e-01  -7.104385655e-01  5.3e-05  0.23  
10  7.9e-05  6.3e-05  2.1e-04  1.38e+00   -6.728871085e-01  -7.057468225e-01  4.9e-05  0.24  
11  5.3e-05  4.2e-05  1.0e-04  1.40e+00   -6.686687693e-01  -6.877949888e-01  3.3e-05  0.25  
12  4.6e-05  3.7e-05  8.1e-05  1.41e+00   -6.685223513e-01  -6.844690056e-01  2.9e-05  0.26  
13  4.4e-05  3.5e-05  7.4e-05  1.36e+00   -6.684605247e-01  -6.833186572e-01  2.7e-05  0.27  
14  2.9e-05  2.4e-05  3.9e-05  1.37e+00   -6.671489541e-01  -6.762408844e-01  1.8e-05  0.28  
15  2.3e-05  1.9e-05  2.6e-05  1.34e+00   -6.664626803e-01  -6.732483918e-01  1.4e-05  0.28  
16  1.8e-05  1.4e-05  1.7e-05  1.31e+00   -6.659112982e-01  -6.708195054e-01  1.1e-05  0.29  
17  1.5e-05  1.2e-05  1.3e-05  1.27e+00   -6.656626895e-01  -6.697489632e-01  9.4e-06  0.30  
18  8.4e-06  6.7e-06  5.2e-06  1.26e+00   -6.649404012e-01  -6.670404203e-01  5.2e-06  0.31  
19  7.5e-06  6.0e-06  4.4e-06  1.20e+00   -6.648880060e-01  -6.667459626e-01  4.7e-06  0.32  
20  2.8e-06  2.3e-06  9.5e-07  1.20e+00   -6.643368478e-01  -6.649826894e-01  1.7e-06  0.33  
21  2.3e-06  1.8e-06  6.9e-07  1.12e+00   -6.642863056e-01  -6.648072369e-01  1.4e-06  0.34  
22  1.5e-06  1.2e-06  3.4e-07  1.12e+00   -6.641756856e-01  -6.644970264e-01  9.0e-07  0.34  
23  8.1e-07  6.5e-07  1.4e-07  1.09e+00   -6.640958235e-01  -6.642697568e-01  5.0e-07  0.35  
24  7.0e-07  5.6e-07  1.1e-07  1.06e+00   -6.640850403e-01  -6.642358445e-01  4.4e-07  0.36  
25  2.8e-07  2.3e-07  2.8e-08  1.05e+00   -6.640402098e-01  -6.641002091e-01  1.8e-07  0.37  
26  1.8e-07  1.4e-07  1.3e-08  1.03e+00   -6.640297641e-01  -6.640669720e-01  1.1e-07  0.38  
27  9.1e-08  7.3e-08  4.9e-09  1.02e+00   -6.640220026e-01  -6.640409128e-01  5.6e-08  0.39  
28  2.8e-08  2.2e-08  8.0e-10  1.01e+00   -6.640168302e-01  -6.640225384e-01  1.7e-08  0.40  
29  1.4e-08  1.1e-08  2.9e-10  1.01e+00   -6.640159164e-01  -6.640188293e-01  8.7e-09  0.41  
30  2.3e-09  1.7e-09  1.7e-11  1.00e+00   -6.640152449e-01  -6.640156856e-01  1.3e-09  0.42  
31  7.1e-09  2.9e-10  1.1e-12  1.00e+00   -6.640151604e-01  -6.640152340e-01  2.2e-10  0.42  
32  1.9e-08  3.7e-11  5.4e-14  9.97e-01   -6.640151464e-01  -6.640151560e-01  2.9e-11  0.45  
Optimizer terminated. Time: 0.49    


Interior-point solution summary
  Problem status  : PRIMAL_AND_DUAL_FEASIBLE
  Solution status : OPTIMAL
  Primal.  obj: -6.6401514642e-01   nrm: 1e-01    Viol.  con: 3e-09    var: 6e-12    cones: 0e+00  
  Dual.    obj: -6.6401515597e-01   nrm: 1e+00    Viol.  con: 0e+00    var: 7e-12    cones: 0e+00  
Optimizer summary
  Optimizer                 -                        time: 0.49    
    Interior-point          - iterations : 32        time: 0.45    
      Basis identification  -                        time: 0.00    
        Primal              - iterations : 0         time: 0.00    
        Dual                - iterations : 0         time: 0.00    
        Clean primal        - iterations : 0         time: 0.00    
        Clean dual          - iterations : 0         time: 0.00    
    Simplex                 -                        time: 0.00    
      Primal simplex        - iterations : 0         time: 0.00    
      Dual simplex          - iterations : 0         time: 0.00    
    Mixed integer           - relaxations: 0         time: 0.00    


sol =

  struct with fields:

    yalmipversion: '20210331'
    matlabversion: '9.10.0.1684407 (R2021a) Update 3'
       yalmiptime: 32.498298604999995
       solvertime: 0.602747395000000
             info: 'Successfully solved (MOSEK)'
          problem: 0

JuMP

Problem
  Name                   :                
  Objective sense        : min            
  Type                   : CONIC (conic optimization problem)
  Constraints            : 179200          
  Cones                  : 51200          
  Scalar variables       : 204860          
  Matrix variables       : 0              
  Integer variables      : 0              

Optimizer started.
Presolve started.
Linear dependency checker started.
Linear dependency checker terminated.
Eliminator started.
Freed constraints in eliminator : 0
Eliminator terminated.
Eliminator - tries                  : 1                 time                   : 0.00            
Lin. dep.  - tries                  : 1                 time                   : 0.04            
Lin. dep.  - number                 : 0              
Presolve terminated. Time: 0.17    
Problem
  Name                   :                
  Objective sense        : min            
  Type                   : CONIC (conic optimization problem)
  Constraints            : 179200          
  Cones                  : 51200          
  Scalar variables       : 204860          
  Matrix variables       : 0              
  Integer variables      : 0              

Optimizer  - threads                : 16              
Optimizer  - solved problem         : the primal      
Optimizer  - Constraints            : 76750
Optimizer  - Cones                  : 51201
Optimizer  - Scalar variables       : 179211            conic                  : 153610          
Optimizer  - Semi-definite variables: 0                 scalarized             : 0              
Factor     - setup time             : 0.16              dense det. time        : 0.03            
Factor     - ML order time          : 0.01              GP order time          : 0.00            
Factor     - nonzeros before factor : 4.61e+05          after factor           : 5.12e+05        
Factor     - dense dim.             : 112               flops                  : 5.68e+06        
ITE PFEAS    DFEAS    GFEAS    PRSTATUS   POBJ              DOBJ              MU       TIME  
0   1.6e+00  1.3e+00  4.1e+04  0.00e+00   8.278383991e-01   -4.122122248e+04  1.0e+00  0.50  
1   2.2e-01  1.8e-01  2.5e+03  4.90e-01   1.577918154e+00   -7.033012527e+03  1.4e-01  0.67  
2   4.0e-02  3.3e-02  1.7e+02  1.61e+00   1.657910814e+00   -9.565504726e+02  2.5e-02  0.83  
3   6.2e-03  5.0e-03  9.9e+00  1.08e+00   1.365089453e+00   -1.412080101e+02  3.9e-03  0.99  
4   3.0e-04  2.5e-04  9.5e-02  1.01e+00   8.778742259e-01   -6.007657031e+00  1.9e-04  1.15  
5   7.8e-05  6.3e-05  1.2e-02  1.01e+00   8.081816140e-01   -9.587993152e-01  4.9e-05  1.31  
6   6.6e-05  5.4e-05  9.7e-03  1.02e+00   7.920327073e-01   -7.058558899e-01  4.2e-05  1.46  
7   4.5e-05  3.7e-05  5.5e-03  1.02e+00   7.604597672e-01   -2.595754485e-01  2.9e-05  1.62  
8   4.3e-05  3.5e-05  5.1e-03  1.03e+00   7.569287581e-01   -2.143769108e-01  2.7e-05  1.78  
9   2.8e-05  2.3e-05  2.6e-03  1.03e+00   7.281090949e-01   1.125093927e-01   1.7e-05  1.94  
10  1.8e-05  1.5e-05  1.4e-03  1.03e+00   7.083841597e-01   3.058073023e-01   1.2e-05  2.09  
11  1.5e-05  1.2e-05  9.7e-04  1.05e+00   6.998118505e-01   3.832636405e-01   9.2e-06  2.25  
12  1.2e-05  1.0e-05  7.5e-04  1.06e+00   6.972264838e-01   4.312127244e-01   7.8e-06  2.40  
13  1.2e-05  9.7e-06  7.1e-04  1.06e+00   6.965317106e-01   4.398582867e-01   7.5e-06  2.55  
14  9.6e-06  7.8e-06  5.1e-04  1.07e+00   6.920786350e-01   4.881984812e-01   6.1e-06  2.70  
15  6.4e-06  5.2e-06  2.7e-04  1.07e+00   6.840965414e-01   5.517123265e-01   4.0e-06  2.85  
16  6.2e-06  5.1e-06  2.6e-04  1.09e+00   6.836441283e-01   5.555741644e-01   3.9e-06  3.01  
17  4.2e-06  3.5e-06  1.4e-04  1.09e+00   6.784547161e-01   5.930927581e-01   2.7e-06  3.16  
18  3.8e-06  3.1e-06  1.2e-04  1.09e+00   6.773102709e-01   6.007554980e-01   2.4e-06  3.31  
19  2.3e-06  1.9e-06  5.6e-05  1.09e+00   6.723060671e-01   6.274805189e-01   1.5e-06  3.46  
20  2.0e-06  1.7e-06  4.6e-05  1.09e+00   6.714146647e-01   6.320500305e-01   1.3e-06  3.60  
21  1.3e-06  1.0e-06  2.2e-05  1.09e+00   6.686029237e-01   6.447699717e-01   8.0e-07  3.74  
22  1.1e-06  9.3e-07  1.9e-05  1.07e+00   6.681841288e-01   6.467147512e-01   7.2e-07  3.89  
23  6.8e-07  5.5e-07  8.5e-06  1.07e+00   6.664926126e-01   6.540226462e-01   4.3e-07  4.03  
24  3.8e-07  3.1e-07  3.5e-06  1.06e+00   6.654202789e-01   6.585576697e-01   2.4e-07  4.16  
25  2.7e-07  2.2e-07  2.1e-06  1.04e+00   6.650284034e-01   6.601810119e-01   1.7e-07  4.30  
26  1.1e-07  9.1e-08  5.5e-07  1.04e+00   6.644180789e-01   6.624166935e-01   7.1e-08  4.44  
27  9.1e-08  7.4e-08  4.1e-07  1.02e+00   6.643480364e-01   6.627206878e-01   5.8e-08  4.58  
28  3.9e-08  3.1e-08  1.1e-07  1.02e+00   6.641480282e-01   6.634746782e-01   2.4e-08  4.72  
29  2.4e-08  1.7e-08  4.4e-08  1.01e+00   6.640880451e-01   6.637176037e-01   1.3e-08  4.85  
30  7.5e-09  5.4e-09  7.7e-09  1.01e+00   6.640356097e-01   6.639189772e-01   4.2e-09  4.99  
31  6.2e-09  4.5e-09  5.8e-09  1.00e+00   6.640321406e-01   6.639354027e-01   3.5e-09  5.13  
32  7.3e-09  1.9e-09  1.7e-09  1.00e+00   6.640224330e-01   6.639804694e-01   1.5e-09  5.26  
33  3.9e-09  4.1e-10  1.6e-10  1.00e+00   6.640164988e-01   6.640076574e-01   3.2e-10  5.40  
34  5.0e-09  1.0e-10  2.0e-11  1.00e+00   6.640154546e-01   6.640132590e-01   7.9e-11  5.54  
35  4.9e-09  1.0e-10  1.9e-11  1.00e+00   6.640154497e-01   6.640132883e-01   7.8e-11  5.68  
36  1.9e-08  5.4e-11  7.6e-12  1.00e+00   6.640153065e-01   6.640141330e-01   4.2e-11  5.82  
37  2.0e-08  5.4e-11  7.5e-12  1.00e+00   6.640153041e-01   6.640141474e-01   4.1e-11  5.96  
38  2.0e-08  5.4e-11  7.4e-12  1.00e+00   6.640153040e-01   6.640141483e-01   4.1e-11  6.09  
39  1.9e-08  5.2e-11  7.1e-12  1.00e+00   6.640152996e-01   6.640141749e-01   4.0e-11  6.37  
40  1.9e-08  5.2e-11  7.1e-12  1.00e+00   6.640152995e-01   6.640141754e-01   4.0e-11  6.51  
41  1.9e-08  5.2e-11  7.1e-12  1.00e+00   6.640152994e-01   6.640141759e-01   4.0e-11  6.64  
42  1.9e-08  5.2e-11  7.1e-12  1.00e+00   6.640152994e-01   6.640141761e-01   4.0e-11  6.78  
43  1.9e-08  5.2e-11  7.1e-12  1.00e+00   6.640152989e-01   6.640141788e-01   4.0e-11  6.92  
44  1.8e-08  5.1e-11  7.0e-12  1.00e+00   6.640152972e-01   6.640141890e-01   4.0e-11  7.06  
45  1.8e-08  5.1e-11  7.0e-12  1.00e+00   6.640152969e-01   6.640141912e-01   4.0e-11  7.20  
46  1.7e-08  5.0e-11  6.7e-12  1.00e+00   6.640152935e-01   6.640142121e-01   3.9e-11  7.33  
47  1.4e-08  4.8e-11  6.4e-12  1.00e+00   6.640152884e-01   6.640142436e-01   3.7e-11  7.47  
48  1.4e-08  4.8e-11  6.4e-12  1.00e+00   6.640152877e-01   6.640142479e-01   3.7e-11  7.61  
49  1.4e-08  4.8e-11  6.3e-12  1.00e+00   6.640152866e-01   6.640142543e-01   3.7e-11  7.75  
50  1.4e-08  4.8e-11  6.3e-12  1.00e+00   6.640152866e-01   6.640142546e-01   3.7e-11  7.88  
51  1.5e-08  4.7e-11  6.1e-12  1.00e+00   6.640152843e-01   6.640142687e-01   3.6e-11  8.02  
52  1.5e-08  4.4e-11  5.5e-12  1.00e+00   6.640152745e-01   6.640143277e-01   3.4e-11  8.16  
53  1.7e-08  4.1e-11  5.0e-12  1.00e+00   6.640152659e-01   6.640143793e-01   3.2e-11  8.30  
54  1.8e-08  4.0e-11  4.9e-12  1.00e+00   6.640152638e-01   6.640143920e-01   3.1e-11  8.44  
55  1.8e-08  4.0e-11  4.9e-12  1.00e+00   6.640152637e-01   6.640143924e-01   3.1e-11  8.58  
56  2.4e-08  3.6e-11  4.2e-12  1.00e+00   6.640152518e-01   6.640144664e-01   2.8e-11  8.72  
57  2.4e-08  3.6e-11  4.2e-12  1.00e+00   6.640152515e-01   6.640144683e-01   2.8e-11  8.85  
58  2.4e-08  3.6e-11  4.2e-12  1.00e+00   6.640152515e-01   6.640144685e-01   2.8e-11  8.99  
59  2.4e-08  3.6e-11  4.2e-12  1.00e+00   6.640152515e-01   6.640144685e-01   2.8e-11  9.22  
60  2.4e-08  3.6e-11  4.2e-12  1.00e+00   6.640152515e-01   6.640144686e-01   2.8e-11  9.36  
61  2.4e-08  3.6e-11  4.1e-12  1.00e+00   6.640152506e-01   6.640144738e-01   2.8e-11  9.64  
62  2.4e-08  3.6e-11  4.1e-12  1.00e+00   6.640152506e-01   6.640144739e-01   2.8e-11  9.91  
63  2.4e-08  3.6e-11  4.1e-12  1.00e+00   6.640152506e-01   6.640144739e-01   2.8e-11  10.17
64  2.4e-08  3.6e-11  4.1e-12  1.00e+00   6.640152506e-01   6.640144739e-01   2.8e-11  10.42
Optimizer terminated. Time: 10.69

[Johan Löfberg]

Not only can the way operators are modelled cause different size (although logsumexp is not done in any particularly clever way in YLMIP, it is standard expcone representation), the size of the model will also depends on primal-dual interpretations of the symbolic model. Here, it appears the default choice in YALMIP is better suited than the default choice in jump. Hence, you either have to find out if jump has support for automatically deriving and switching to a dualized model, or you will have to manually derive the dual model, solve that, and then recover your variables from the duals of the dual. This is not trivial, to say the least...

[AS95]

Thank you very much for your answer, Johan! 

In addition to stalling, I also see that MOSEK works slower in JuMP's model. I will try to understand if dualizing the problem helps.

[AS95]

Update:

The stalling issue is resolved by using dual optimizer in JuMP. Thank you for the excellent suggestions, Johan and Erling.

I now define the model as: model = Model(dual_optimizer(MosekTools.Optimizer))

Just out of curiosity: How does YALMIP decide when to solve the dual? Is it maybe when the number of constraints is more than the number of variables? 

[Erling D. Andersen]

In principle the number of cones should be same whether you solve the primal or dual.  But the Jump model has many more cones.

To me it seems that Jump model is much worse than the Yalmip model. And unfortunately the Mosek presolve cannot fix that

Btw Mosek has a heuristic for deciding whether it solve the primal or dual. It is NP hard in general to figure out which form is the best IMO.

[Erling D. Andersen]

I think the initial to go for is to make sure that Jump formulation and Yalmip formulation has the same number  of cones.

I would assume Johan employs

  https://docs.mosek.com/modeling-cookbook/expo.html?highlight=log%20sum#log-sum-exp

but maybe he tune the special case when the sum has one element only.

[Johan Löfberg]

YALMIP always makes a dual interpretation. To do anything else, you have to invoke the dualization (but I am not sure it supports expcone, don't remember)

[Johan Löfberg]

although it probably is possible to trick YALMIP by introducing a silly binary variable to the expcone model, and then YALMIP will be forced to use a primal setup in the low-level call to mosek. I guess.