Greenhorn ask for support: ERROR: Solver (cbc) returned non-zero return code (-11)

[in...@frame-solution.de]

Hi everybody,

I am transferring a less complex energy system (optimal dispatch) model from GAMS to Pyomo. 

Unfortunately I have no experience with Pyomo and so I am quite lost with an error massage that I received.

As long as the number of time steps (hours) is below 5100 everything runs well. Above this number I receive:

ERROR: Solver (cbc) returned non-zero return code (-11)

I was wondering whether either the amount of memory on my machine (16GB) is too low or the primal value runs out of a tolerable range.

I checked the later assumption by down-scaling the problem by factor 1000 (transferring MW in GW) but the error remains the same.

My attempt to solve the problem by using the NEOS server failed too because of a time-out error above a time steps of more than ~150 hours.

 

After spending more than two days of searching for an answer at the web I now hope for some support from one of you.

Thank you in advance,

Frank

-- Python 3.7, the latest Pyomo version on a Mac and as solver CBC. --

Here is some of the error output:

ERROR: Solver (cbc) returned non-zero return code (-11)

ERROR: Solver log: Welcome to the CBC MILP Solver Version: 2.10.1 Build Date:

    Mar 20 2019

    command line - /Applications/Own_applications/miniconda3/bin/cbc

    -printingOptions all -import

    /var/folders/yf/fyp7bglx1s198yn7wbxvkzyc0000gn/T/tmpjsrc7afw.pyomo.lp

    -stat=1 -solve -solu

    /var/folders/yf/fyp7bglx1s198yn7wbxvkzyc0000gn/T/tmpjsrc7afw.pyomo.soln

    (default strategy 1) Option for printingOptions changed from normal to all

    Presolve 533936 (-3486049) rows, 419701 (-396316) columns and 1991218

    (-43740671) elements Statistics for presolved model

    Problem has 533936 rows, 419701 columns (168000 with objective) and

    1991218 elements Column breakdown: 143976 of type 0.0->inf, 113709 of type

    0.0->up, 0 of type lo->inf, 66000 of type lo->up, 96000 of type free, 0 of

    type fixed, 6 of type -inf->0.0, 10 of type -inf->up, 0 of type 0.0->1.0

    Row breakdown: 47976 of type E 0.0, 0 of type E 1.0, 0 of type E -1.0,

    54000 of type E other, 0 of type G 0.0, 0 of type G 1.0, 0 of type G

    other, 431952 of type L 0.0, 0 of type L 1.0, 0 of type L other, 0 of type

    Range 0.0->1.0, 8 of type Range other, 0 of type Free Presolve 533936

    (-3486049) rows, 419701 (-396316) columns and 1991218 (-43740671) elements

    Perturbing problem by 0.001% of 0.64511948 - largest nonzero change

    5.3360971e-05 ( 0.82849427%) - largest zero change 5.336076e-05 0  Obj

    9531.3691 Primal inf 1390266.9 (150000) Dual inf 748.15507 (96000) 0  Obj

    9531.3691 Primal inf 1390266.9 (150000) Dual inf 3.0075277e+15 (210000)

    1282  Obj 9211.5566 Primal inf 1368434.3 (149851) Dual inf 3.0193564e+15

    (208815) 2566  Obj 8955.7058 Primal inf 1347616.4 (149697) Dual inf

    3.006139e+15 (207587) 3831  Obj 8651.3757 Primal inf 1325789

....

    Obj 2209.904 Dual inf 269803.11 (70536) 989487  Obj 2205.481 Dual inf

    469108.17 (67829) 990487  Obj 2197.9506 Primal inf 4.4680375e-12 (1) Dual

    inf 7.5598167e+11 (71065) 991487  Obj 2192.7803 Dual inf 207435.85 (71121)

    992443  Obj 2187.2496 Dual inf 460709.35 (71218) 993202  Obj 2181.7041

    Dual inf 467853.61 (68373) 994049  Obj 2178.3434 Dual inf 309324.58

    (68869) 994981  Obj 2173.1192 Dual inf 854822.94 (71253) 995626  Obj

    2168.6121 Dual inf 2

Traceback (most recent call last):

  File "ODM_V3401_190326_1000_perf.py", line 257, in <module>

    results = opt.solve(instance) # solves and updates instance

  File "/Applications/Own_applications/miniconda3/lib/python3.7/site-packages/pyomo/opt/base/solvers.py", line 596, in solve

    "Solver (%s) did not exit normally" % self.name)

pyutilib.common._exceptions.ApplicationError: Solver (cbc) did not exit normally 

[Watson, Jean-Paul]

I would also suspect a memory issue, but it would be useful to see the contents of the soln file - if it exists. CBC does indeed to seem to find a feasible solution, but I don’t see the normal CBC termination output, e.g., final objective value and such. 

Can you add “tee=True” and “keepFiles=True” keyword arguments to the solve() call, and post the results? The tee=True option will perhaps help catch a segfault or abrupt termination. The keepFiles=True option should ensure that the .soln file exists at termination - I’m curious to see if that is empty / exists.

Jpw

--
You received this message because you are subscribed to the Google Groups "Pyomo Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyomo-forum...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[in...@frame-solution.de]

Thank you! I will add these two options and will run the model again. Should take some time until it breaks.

Best F

[in...@frame-solution.de]

I run the model twice. In both cases terminal informs that three files are saved: .cbc.log, pyomo.soln and pyomo.lp. However only the .lp is in this directory. I checked twice and removed all older temps to be sure. The .lp has 600 MB, a little bit to big for sharing - probably. I attach the terminal output. Again, Thank you for helping me!

In parallel I try to reduce the complexity and number of constraints. 

Best. Frank

On Wednesday, March 27, 2019 at 10:30:02 AM UTC+1, JPW wrote:

[Siirola, John D]

The lack of a .soln file is not all that surprising.  In Python’s subprocess, return code -11 is for a segfault, which means the CBC process exited abruptly without the chance to write the soln file.

 

SEGFAULT can be raised by an out-of-memory problem (malloc / new return NULL when the memory allocation fails, but most programs are lazy and don’t check that new or malloc returned a non-NULL pointer.  The resulting symptom is a segfault when the program dereferences the NULL pointer) or a general programming issue in CBC.  Given the size of your problem and how long CBC is running for, I would probably bet on it being a memory limit.

 

Completely separately, would you be willing to share your model (off-list)?  I would be interested in using it as part of our internal performance profiling suite.  (Plus, there are two constraints that appear to take a disproportionate amount of time to construct and I would like to see what you are doing there).

 

john

[in...@frame-solution.de]

Good Morning John,

Thanks for these explanations. Yes, I can share the model, pls. contact me therefore here: meissner(at)frame-solution.de. I will add more comments in the model so that it is more clear what I (try to) do there. 

I would be very interested in results of performance test. 

Best, 

Frank

To unsubscribe from this group and stop receiving emails from it, send an email to pyomo...@googlegroups.com.