Get best bound from solver after a given time

[Vincent Brunelle]

Hi,

For some reasons, I am interested in finding the best bound of a Pyomo model after the time limit is reached. I generally succeed at getting harder to find infos from the solver log using simple Excel macros, but now, it seems that the logs that I got only have the best bound value in the nodes and cuts table, which makes it harder to make a macro for this. Even if my VBA code returns a value that seems right, I can't really trust this script because of all the variations between my CPLEX logs.

I am considering running all my problems with Pyomo again, and I believe that it is a good time to log these infos directly from the model instance. Is it something that can be done using only the Pyomo librairy? I know that the solverFactory.solve() method returns a results object and that some infos can be taken from it, but I can't succeed to find documentation on this object, let alone knowing if it can do what I need.

Thanks for your kind support,

Vincent

[Vincent Reinbold]

Hi Vincent,

Based on some tests, I think SolverFactory().solve() accept an argument named timelimit.

results = SolverFactory("glpk").solve(m,  tee=True, timelimit=20)
But it depends on the solver your are using, (here it is glpk). The argument tee allows you to display the solver outputs. 

For general purpose, one can also pass the options using options dictionary as follow :

solver = SolverFactory('glpk')
solver.options['tmlim'] = TIME_LIMIT
solver.solve(MODEL)
You also have the possibility to do this using command line. See pyomo solve --help  for more information. 

I could not find a proper documentation for each SolverFactory, sorry. Maybe looking to documentation of the solver itself.
Here is a link to the pyomo documentation "Sending options to the solver" : 
https://pyomo.readthedocs.io/en/latest/working_models.html#sending-options-to-the-solver

Greetings,
 
Vincent 

[Vincent Brunelle]

Hi,

First of all, thanks for your very kind help. I must however say that sending options to the solver is not the problem. Perhaps the title of my post was misleading, and altough you gave a correct answer, it is not an answer to my problem. I am very sorry and I appreciate your help very much.

I do know how to put a time limit. In fact, I am solving a great quantity of problems, and many of them were stopped with the time limit. They generally have an incumbent solution (best integer solution) that I can retrieve from the solver after solving, and this become my "best integer solution".Now, for the problems that have been stopped with the timelimit, they are typically hard to solve. The time limit was set to something like 12 hours, and I even made one with a time limit of 72 hours and the optimality gap was still far from 0%, so it is the kind of problem that are very, very hard to solve to optimality. It can take a few hours more, or a few years more to solve, and obviously, I can't afford to wait that long to find out. Anyway, I still need to know how far I am from the best possible solution (the best bound), after the time limit was reached. I then want to use the value of the best bound (not the best integer) at the 12 hours time limit in order to compare the solution of an algorithm designed to increase the quality of the solution. This way, it helps to provide at least one measure of the gap between the algorithm's solution and the optimality (we know that the optimal integer solution is included between the solution at the 12 hours time limit and the best bound at the 12 hours time limit).

So, the problem is not to give parameters to a solver, but to retrieve a property of the solution, if it is possible.

If I could, I would give bonus points to someone who can help me navigate through the properties of the results object that is returned by the opt.solve() method, since I believe that the answer may be in there.

For those of you who asks themselves why I don't look into the log, here is the context. Strangely enough, no informations on the solution are printed by CPLEX at the end of the log produced with tee=True, even if it prints the node/cuts table. It only tells me the number of cuts and the time that was used by CPLEX, without telling me the objective value of the best integer solution and the value of the best bound. I suspect an update of CPLEX, or the fact that I use a persistent solver, since I was able to find these informations at the end of the log before I used the persistent solver and before I updated CPLEX version.

Here is an example of a complete log that I get from CPLEX on one of the instances that were solved to optimality in a ratter short time. Pasting a 12 hours long log would not have been very different except for its incredible length. :

CPXPARAM_TimeLimit                               43200
CPXPARAM_Read_DataCheck                          1
CPXPARAM_Threads                                 6
CPXPARAM_MIP_Display                             3
Tried aggregator 1 time.
MIP Presolve eliminated 1937 rows and 11174 columns.
Reduced MIP has 393 rows, 3692 columns, and 7448 nonzeros.
Reduced MIP has 60 binaries, 0 generals, 0 SOSs, and 0 indicators.
Presolve time = 0.09 sec. (10.90 ticks)
Tried aggregator 1 time.
Reduced MIP has 393 rows, 3692 columns, and 7448 nonzeros.
Reduced MIP has 60 binaries, 0 generals, 0 SOSs, and 0 indicators.
Presolve time = 0.02 sec. (3.04 ticks)
Probing time = 0.00 sec. (0.29 ticks)
Clique table members: 20.
MIP emphasis: balance optimality and feasibility.
MIP search method: dynamic search.
Parallel mode: deterministic, using up to 6 threads.
Root relaxation solution time = 0.13 sec. (5.15 ticks)

        Nodes                                         Cuts/
   Node  Left     Objective  IInf  Best Integer    Best Bound    ItCnt     Gap

      0     0  -1.99663e+07    10                -1.99663e+07      305        
*     0+    0                      -9.88977e+07  -1.99663e+07            79.81%
Found incumbent of value -9.8897734e+07 after 0.39 sec. (28.21 ticks)
      0     0  -3.59828e+07    14  -9.88977e+07      Cuts: 37      366   63.62%
*     0+    0                      -5.34952e+07  -3.59828e+07            32.74%
Found incumbent of value -5.3495226e+07 after 0.44 sec. (40.28 ticks)
      0     0  -3.66155e+07    12  -5.34952e+07      Cuts: 26      396   31.55%
*     0+    0                      -3.73967e+07  -3.66155e+07             2.09%
Found incumbent of value -3.7396716e+07 after 0.45 sec. (48.71 ticks)
      0     0  -3.73759e+07    10  -3.73967e+07      Cuts: 25      440    0.06%
      0     0  -3.73962e+07     8  -3.73967e+07      Cuts: 12      446    0.00%
      0     0        cutoff        -3.73967e+07                    446     ---
Elapsed time = 0.48 sec. (60.44 ticks, tree = 0.01 MB, solutions = 3)

Cover cuts applied:  3
Flow cuts applied:  23
Mixed integer rounding cuts applied:  10
Lift and project cuts applied:  4
Gomory fractional cuts applied:  1

Root node processing (before b&c):
  Real time             =    0.48 sec. (61.17 ticks)
Parallel b&c, 6 threads:
  Real time             =    0.00 sec. (0.00 ticks)
  Sync time (average)   =    0.00 sec.
  Wait time (average)   =    0.00 sec.
                          ------------
Total (root+branch&cut) =    0.48 sec. (61.17 ticks)

In green, you have the value of the best bound and in yellow, you have the objective value of the best integer solution.

I am sure that this will help you understand why I said in my first post that using VBA could help find the value of the best integer solution (I simply search for the value on the line where it is written "Found incumbent of value "), but finding the value of the best bound is harder and can be complex using a script without producing mistakes. Instead of diving into another Excel VBA nightmare, I believe there is a way to fetch this information cleanly, using tools that Pyomo already provides.

As always, thanks for your very kind support,

Vincent