Gurobi Error Code

[mapocho]

hi, i'm a new user of gurobi for ampl and i try to understand a look
up for this error code

Solve interrupted (error code 10001)
Best objective 3.6201496000e+07, best bound 2.9507284000e+07, gap
18.4915%
Optimize a model with 8123 Rows, 7491 Columns and 2194804 NonZeros

,but i can't find anything.

i really appreciate if you can tell me what dos it mean?

[Greg Glockner]

This is an out-of-memory error. If this is a MIP that requires a large number of nodes to solve, you may want to try setting the nodefilestart parameter to reduce memory use; for details, see http://www.gurobi.com/html/doc/ampl/parameters.html

[mapocho]

Thank Greg for your answer and yes it is, it's a mip problem with a
lot of nodes, but I tried with your information and nothing happened.
Let me show you how I did it:

ampl: model aaa.mod;
ampl: data aaa.dat;
ampl: option solver gurobi;
ampl: options gurobi_options 'nodefiledir c:\ nodefilestart 1';
ampl: solve;

and when memory was exceeded non file was created and it finished with
the same problem (error 10001).

Please tell me if i am doing something wrong.

Thank you

[Zonghao Gu]

Could you change

    ampl: options gurobi_options 'nodefiledir c:\ nodefilestart 1';

to
   ampl: options gurobi_options 'nodefilestart=0.1 outlev=1';

to see whether it will write node files?

--
To unsubscribe, reply using "remove me" as the subject.

[mapocho]

Thanks Zonghao, but i tried it and nothing happened again, same
problem.Could you tell me what do they parameters mean?, i mean, if i
have a huge mip problem, won't it exceed memory limit? , what kind of
files does it write on mi disk (extension format)?, Where do i spected
to find these files?

I really appreciate your help.

Thanks.

On 3 abr, 08:41, Zonghao Gu <g...@gurobi.com> wrote:
> Could you change
>
>     ampl: options gurobi_options 'nodefiledir c:\ nodefilestart 1';
>
> to
>    ampl: options gurobi_options 'nodefilestart=0.1 outlev=1';
>
> to see whether it will write node files?
>

[Greg Glockner]

Could you clarify a few things:

When you set nodefilestart=0.1 as Gu explained, do you still have the out-of-memory error? Can you give us some further details about the model size and your computer? Finally, it would be helpful if you could post the full output produced when you use the outlev=1 option as Gu explained.

Thanks.

[mapocho]

Yes I can:

1) When I set nodefilestart=0.1 as Gu explained, I still have the out-
of-memory error.

2) About my computer compentents, here are some details:

-Proccesor: Intel® Core™ i7-720QM Quad Core 1.6GHz (2.8GHz Turbo Mode,
6MB Cache) (it has 8 processors)

-Ram Memory: 6GB Dual Channel DDR3 (1333 Mhz)

-Hard Drive: 500 GB SATA (7200RPM)

-OS : Windows 7 Home Premium (64 bit)

3) About model size, it's a mip problem with 10409 Rows, 9443 Columns
and 1866451 NonZeros.

4) Here is the full output produced when I use the outlev=1 option as
Gu explained:

Logging started 04/03/10 17:37:47

Optimize a model with 10409 Rows, 9443 Columns and 1866451 NonZeros
Presolve removed 430 rows and 455 columns
Presolve time: 3.43s
Presolved: 9979 Rows, 8988 Columns, 1855025 Nonzeros

Root relaxation log...

Iteration Objective Primal Inf. Dual Inf. Time
7169 1.5876219e+08 2.363783e+03 0.000000e+00 5s
Warning: 1 variables are dropped from basis.
Warning: switch to quad precision
8394 1.6270485e+08 0.000000e+00 0.000000e+00 9s

Root relaxation: objective 1.627048e+08, 8394 iterations, 8.69 seconds
Total elapsed time = 38.88s
Total elapsed time = 42.56s

Nodes | Current Node | Objective Bounds |
Work
Expl Unexpl | Obj Depth IntInf | Incumbent BestBd Gap | It/
Node Time

0 0 1.6270e+08 0 2711 - 1.6270e+08 - -
43s
0 0 7.1055e+08 0 1635 - 7.1055e+08 - -
139s
0 0 7.1055e+08 0 609 - 7.1055e+08 - -
290s
H 0 0 1.0581e+09 7.1055e+08 32.8% -
304s
0 0 9.0761e+08 0 802 1.0581e+09 9.0761e+08 14.2% -
332s
H 0 0 1.0528e+09 9.0761e+08 13.8% -
342s
0 0 9.7469e+08 0 847 1.0528e+09 9.7469e+08 7.42% -
358s
0 0 1.0048e+09 0 906 1.0528e+09 1.0048e+09 4.56% -
372s
0 0 1.0138e+09 0 912 1.0528e+09 1.0138e+09 3.71% -
386s
0 0 1.0187e+09 0 863 1.0528e+09 1.0187e+09 3.24% -
396s
H 0 0 1.0444e+09 1.0187e+09 2.46% -
402s
0 0 1.0207e+09 0 854 1.0444e+09 1.0207e+09 2.27% -
407s
0 0 1.0223e+09 0 870 1.0444e+09 1.0223e+09 2.11% -
415s
0 0 1.0232e+09 0 881 1.0444e+09 1.0232e+09 2.02% -
425s
0 0 1.0232e+09 0 881 1.0444e+09 1.0232e+09 2.02% -
429s
0 2 1.0232e+09 0 881 1.0444e+09 1.0232e+09 2.02% -
440s
4 6 1.0246e+09 2 328 1.0444e+09 1.0238e+09 1.97% 323
445s
11 11 1.0249e+09 5 337 1.0444e+09 1.0239e+09 1.96% 215
452s
14 16 1.0250e+09 5 324 1.0444e+09 1.0239e+09 1.96% 262
456s
23 26 1.0293e+09 6 431 1.0444e+09 1.0244e+09 1.91% 309
465s
29 30 1.0265e+09 7 340 1.0444e+09 1.0250e+09 1.86% 339
470s

Cutting planes:
Gomory: 52
Implied bound: 3552
Clique: 1
Flow cover: 2066

Explored 35 nodes (65397 simplex iterations) in 473.26 seconds
Thread count was 8 (of 8 available processors)

Solve interrupted (error code 10001)

Best objective 1.0443880210e+09, best bound 1.0249753141e+09, gap
1.8588%
Optimize a model with 10409 Rows, 9443 Columns and 1866451 NonZeros

Iteration Objective Primal Inf. Dual Inf. Time
0 1.0360000e+09 1.547369e+03 0.000000e+00 0s
3263 1.0443880e+09 0.000000e+00 0.000000e+00 3s

Solved in 3263 iterations and 2.71 seconds
Optimal objective 1.044388021e+09

Thanks in advice

[Zonghao Gu]

It looks like the computer doesn't have enough memory to run simplex for 8 nodes in parallel. The model is quite dense, simplex factorization could have 1e+7 nonzeros, storing two copies of nonzeros in quad precision, see below "Warning: switch to quad precision", needs about 100 bytes for each nonzeros. Plus two copies for LP matrix and others. Each thread may need about 1GB memory for simplex. I'd suggest to try  1 thread and nodefilestart = 0.1 GB to see whether it will start node file. If nodefile isn't an issue, you could try 4 threads or 2 threads.

[mapocho]

Thanks for your help. I solved my model with only 4 threads.

One last question, what if i want to solve a model even larger, could
be possible with nodefilestart option? or do i need more ram?
(I'm asking this, because i don't really understand this parameter
(nodefilestart))

Thanks again.

On 3 abr, 19:01, Zonghao Gu <g...@gurobi.com> wrote:
> It looks like the computer doesn't have enough memory to run simplex for 8
> nodes in parallel. The model is quite dense, simplex factorization could
> have 1e+7 nonzeros, storing two copies of nonzeros in quad precision, see
> below "Warning: switch to quad precision", needs about 100 bytes for each
> nonzeros. Plus two copies for LP matrix and others. Each thread may need
> about 1GB memory for simplex. I'd suggest to try  1 thread and nodefilestart
> = 0.1 GB to see whether it will start node file. If nodefile isn't an issue,
> you could try 4 threads or 2 threads.
>

[Greg Glockner]

nodefilestart is used to save memory by writing node files to the disk. But it appears that the memory issues for your model aren't caused by the size of the nodefile. Rather, it appears that the memory issues are caused by requiring quad precision, plus the number of threads. Increasing your system memory might help. Considering the large objective, you might want to look into the numerics of the model formulation to see the reason why Gurobi may need to switch to quad precision.

[Florian S.]

I'm having memory problems as well, but it could be a little bit different. I optimized my model very often and never had memory problems. This time I optimized on a slightly different instance where it took more than 40000 sec for Gurobi to reach the MIPGap (1 %). It looks like Gurobi started using more and more memory with time elapsing - is that a normal behavior? I thought, that memory usage should be constant with the number of threads (12 threads). Or does Gurobi save any information in memory of explored nodes which could explain this.

Here is my Gurobi log:

Changed value of parameter MIPGap to 0.01
   Prev: 0.0001   Min: 0.0   Max: 1e+100   Default: 0.0001
Changed value of parameter TimeLimit to 48000.0
   Prev: 1e+100   Min: 0.0   Max: 1e+100   Default: 1e+100

Optimize a model with 514635 rows, 474216 columns and 3584886 nonzeros
Model has 5361 quadratic constraints
Presolve removed 506622 rows and 460574 columns (presolve time = 5s) ...
Presolve removed 506622 rows and 460574 columns
Presolve time: 5.65s
Presolved: 19074 rows, 19292 columns, 117469 nonzeros
Variable types: 8130 continuous, 11162 integer (11162 binary)
Found heuristic solution: objective -0.0000000

Root relaxation: objective 4.055084e+04, 14971 iterations, 1.00 seconds

    Nodes    |    Current Node    |     Objective Bounds      |     Work

 Expl Unexpl |  Obj  Depth IntInf | Incumbent    BestBd   Gap | It/Node Time

     0     0 40550.8385    0  577   -0.00000 40550.8385     -      -    7s
H    0     0                    1246.9978395 40550.8385  3152%     -    8s
H    0     0                    23405.488085 40550.8385  73.3%     -    8s
     0     0 40055.0765    0 1238 23405.4881 40055.0765  71.1%     -    9s
     0     0 39426.6247    0 1707 23405.4881 39426.6247  68.5%     -   13s
H    0     0                    24850.551669 39426.6247  58.7%     -   16s
H    0     0                    26411.992404 39426.6247  49.3%     -   16s
     0     0 38931.7602    0 1739 26411.9924 38931.7602  47.4%     -   17s
H    0     0                    26489.000395 38931.7602  47.0%     -   19s
...
     0     2 33248.1018    0 1255 30967.7863 33248.1018  7.36%     -   69s
     2     4 33245.5336    2 1360 30967.7863 33247.7920  7.36%  2176   70s
    24    29 32125.5003    9 1398 30967.7863 33132.2046  6.99%  2382   75s
H   30    32                    31021.162017 32947.4276  6.21%  2272   78s
H   34    39                    31052.665953 32947.4276  6.10%  2092   78s
    51    51 31851.0910   12  689 31052.6660 32824.4905  5.71%  2120   80s
    75    70 31823.1548   14  677 31052.6660 32824.4905  5.71%  1964   86s
H   76    71                    31084.799013 32824.4905  5.60%  1938   86s
   104    60 31666.8209   16  666 31084.7990 32720.8373  5.26%  1629   96s
   664   311 31872.0855    9 1585 31084.7990 32522.9974  4.63%   596  102s
   810   362 31464.4939   14 1027 31084.7990 32522.9974  4.63%   575  105s
   997   450 infeasible   19      31084.7990 32478.2224  4.48%   533  111s
  1196   532 infeasible   14      31084.7990 32427.6652  4.32%   526  117s
  1349   596 31289.5187   21 1079 31084.7990 32418.4456  4.29%   511  122s
  1521   666 32251.0377   10 1749 31084.7990 32338.9848  4.03%   495  125s
  1858   828 31629.2724   19 1663 31084.7990 32338.9848  4.03%   477  172s
  1860   829 31175.3988   22  929 31084.7990 32338.9848  4.03%   477  178s
  1861   830 31318.6904   13 1978 31084.7990 32338.9848  4.03%   477  190s
  1862   831 31512.5358   14  874 31084.7990 32338.9848  4.03%   476  197s
  1863   831 31318.6904   13  755 31084.7990 32338.9848  4.03%   476  201s
  1864   832 31727.9429   15  831 31084.7990 32338.9848  4.03%   476  206s
...
 5750844 2867200 30856.1866   88  462 30560.5333 30889.3487  1.08%  21.2 43655s
*5751005 2867314             147    30560.533287 30889.3487  1.08%  21.2 43655s
*5751006 2867314             147    30560.533287 30889.3487  1.08%  21.2 43655s
*5751008 2867314             147    30560.533287 30889.3487  1.08%  21.2 43655s
*5751009 2867314             147    30560.533287 30889.3487  1.08%  21.2 43655s
*5751013 2867314             147    30560.533287 30889.3487  1.08%  21.2 43655s
*5751014 2867314             147    30560.533287 30889.3487  1.08%  21.2 43655s
*5751016 2867314             147    30560.533287 30889.3487  1.08%  21.2 43655s
*5751017 2867314             147    30560.533287 30889.3487  1.08%  21.2 43655s
*5751020 2867314             147    30560.533287 30889.3487  1.08%  21.2 43655s
*5751021 2867314             147    30560.533287 30889.3487  1.08%  21.2 43655s
*5751023 2867314             147    30560.533287 30889.3487  1.08%  21.2 43655s
*5751024 2867314             147    30560.533287 30889.3487  1.08%  21.2 43655s
*5751186 2867439             147    30560.533287 30889.3487  1.08%  21.2 43655s
*5751187 2867439             147    30560.533287 30889.3487  1.08%  21.2 43655s
*5751189 2867439             147    30560.533287 30889.3487  1.08%  21.2 43655s
*5751190 2867439             147    30560.533287 30889.3487  1.08%  21.2 43655s
*5751193 2867439             147    30560.533287 30889.3487  1.08%  21.2 43655s
*5751194 2867439             147    30560.533287 30889.3487  1.08%  21.2 43655s
*5751196 2867439             147    30560.533287 30889.3487  1.08%  21.2 43655s
*5751197 2867439             147    30560.533287 30889.3487  1.08%  21.2 43655s
*5751201 2867439             147    30560.533287 30889.3487  1.08%  21.2 43655s
*5751202 2867439             147    30560.533287 30889.3487  1.08%  21.2 43655s
 5751204 2867430 30564.7707  118  420 30560.5333 30889.3487  1.08%  21.2 43677s
*5751224 2867440             147    30560.533287 30889.3487  1.08%  21.2 43677s
*5751225 2867440             147    30560.533287 30889.3487  1.08%  21.2 43677s
*5751246 2867437             147    30560.533287 30889.3487  1.08%  21.2 43677s
*5751247 2867437             147    30560.533287 30889.3487  1.08%  21.2 43677s
*5751249 2867437             147    30560.533287 30889.3487  1.08%  21.2 43677s
*5751250 2867437             147    30560.533287 30889.3487  1.08%  21.2 43677s
stochastic.sh: Row 1:  2969 killed               python2.7 scheduling_stochasticV2.py $i

I'm using Gurobi in Python on a CentOS machine with 12 cores and have appr. 12 GB of free RAM for that process.

[Jakob Sch.]

Hi Florian,

As you can see from the log (second column) you have at the end almost 3 million unexplored nodes. These information has to be stored in memory by gurobi which is likely the reason why you run into problems. You can also see a clear tailing-off effect in your model (high decrease in the gap that flattens during the solving process). If you want to obtain provably optimal solutions, you'll very likely have to upgrade your machine in order to solve instances of that size.

Best regards,
Jakob

[Florian S.]

In this case I don't need a provable optimal solution. I just need a workaround so that gurobi stops when a certain amount of memory is being used. TimeLimit needs to be kept at 48000 sec. Is there any way to reduce the memory usage in these cases?

If understood correctly Nodefilestart and limiting the number of threads wouldn't solve the memory issue here, right?

...

[Greg Glockner]

> If understood correctly Nodefilestart and limiting the number of threads wouldn't solve the memory issue here, right?

Incorrect. NodefileStart will make a big improvement because of the large number of nodes.