GUROBI on PYOMO is hanging

[Marc Meketon]

To call GUROBI (instant cloud license) from PYOMO I used the below code.  It solves after a few minutes, but it takes minutes for small problems to return the solution, and for larger ones (but not that large) after a few hours PYOMO does not report that the solution has been returned.

Am I doing something wrong???  This is a show stopper for me.

start_time = time.time()

opt = po.SolverFactory("gurobi")

opt.options["TimeLimit"] = 1000

opt.options["MIPGap"] = 0.01

opt.options["NormAdjust"]= 0

opt.options["DegenMoves"] = 1

opt.options["Heuristics"] = 0.5

opt.options["PrePasses"] = 1

print("Send to Gurobi Cloud service")

results = opt.solve(model, tee=True)   # I SEE THE OUTPUT FROM GUROBI, INCLUDING THAT IT SOLVED

model.solutions.load_from(results)

print("Solving: --- %s minutes ---" % ((time.time() - start_time)/60.0))  # NEVER SEE THIS LINE

[Gabriel Hackebeil]

I want to guess that this has to do with how we access attributes on the Gurobi model when we extract the solution. We are likely doing something like looping over the variable objects on the Gurobi model one at a time, asking each for its “.X”. I imagine that each request for “.X” involves a message between you and the cloud service.

We have recently rewritten our Cplex and Gurobi Python interfaces to support a new persistent style of using them, and I believe we started doing bulk extraction of the solution there (i.e., grb_model.getAttr(varlist, ‘X’)). Maybe the author of that new code can chime in here to verify that.

You could also try checking out the latest master version of Pyomo from Github and letting us know if the timing changes.

Gabe

--
You received this message because you are subscribed to the Google Groups "Pyomo Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyomo-forum...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Michael Bynum]

I have not yet updated the direct/persistent interfaces to gurobi to do bulk extraction of variable values. It is on my ToDo list. However, I’m not convinced that is the problem here. It looks to me like you are using the lp file interface (it is default). 

[Michael Bynum]

Can you add “report_timing=True” to your call to solve? This may give us a better idea of where things are stalling.

Michael

[Michael Bynum]

After looking more closely, it looks like the lp file interface also grabs variable values from gurobi one at a time. Thus, Gabe is probably correct about the problem. However, it seems that this may be an issue for both the lp file interface and the direct/persistent interfaces. 

[Gabe Hackebeil]

Good double catch. I hadn’t noticed that the LP file interface was being used, and I also forgot that it used a gurobipy script to extract the solution.

Gabe

[Marc Meketon]

Just to clarify - pulling down the latest master will not help because the bulk update for Gurobi has not be implemented, is that right?  The "report_timing=True" did not work since PYOMO never returned after a long-enough period of time (I cancelled it after awhile so as not to burn up my budget).

I tried 

opt = po.SolverFactory("gurobi", solver_io="python")

and

opt = po.SolverFactory("gurobi", solver_io="nl")

based on the thread https://groups.google.com/forum/#!topic/pyomo-forum/JqkhmFOduiQ 

Both of these seem not to be options allowed with Gurobi Instant Cloud:

The first one failed with:

Traceback (most recent call last):

 ...

  File "C:\Anaconda3_64\lib\site-packages\pyomo\opt\base\solvers.py", line 539, in solve

    self.available(exception_flag=True)

  File "C:\Anaconda3_64\lib\site-packages\pyomo\solvers\plugins\solvers\gurobi_direct.py", line 203, in available

    raise pyutilib.common.ApplicationError("No Gurobi <-> Python bindings available - Gurobi direct solver functionality is not available")

pyutilib.common._exceptions.ApplicationError: No Gurobi <-> Python bindings available - Gurobi direct solver functionality is not available

The latter failed with:

WARNING: "[base]\site-packages\pyomo\solvers\plugins\solvers\ASL.py", 83, _default_executable

        Could not locate the 'gurobi_ampl' executable, which is required for solver asl

Traceback (most recent call last):

  ...

  File "C:\Anaconda3_64\lib\site-packages\pyomo\opt\base\solvers.py", line 539, in solve

    self.available(exception_flag=True)

  File "C:\Anaconda3_64\lib\site-packages\pyomo\opt\solver\shellcmd.py", line 122, in available

    raise ApplicationError(msg % self.name)

pyutilib.common._exceptions.ApplicationError: No executable found for solver 'asl'

[Marc Meketon]

Sonja Mars of Gurobi sent a fix for the file gurobi_run.py that changes a couple of lines to implement a bulk reading of the variables.

However, there is another problem with gurobi_run.py.  For long optimization runs, it is common to reach limits (e.g., time limit or mip-gap limit) before reaching optimality.  A solution is developed, but PYOMO crashes with an error that the enum SolverStatus is getting an "aborted" value but that value is not part of the enum.

The issue is that the code in gurobi_run.py sets two statuses.  One called 'status', and one called 'solution_status'.  When there the optimizer stops due to a limit, the variable 'status' is set to 'aborted'.  The 'solution_status' seems to be set correctly to 'stoppedByLimit'.

However, other parts of PYOMO then issue a message that says:

WARNING - Loading a SolverResults object with an 'aborted' status, but containing a solution

Presolve time=11.31 seconds

Solve time=142.48 seconds

Postsolve time=2.15 seconds

Traceback (most recent call last):

…

  File "C:\Anaconda3_64\lib\site-packages\pyomo\core\base\PyomoModel.py", line 241, in load_from

    "with bad status: %s" % str(results.solver.status))

ValueError: Cannot load a SolverResults object with bad status: aborted

 

My temporary fix is to force 'status' to say 'ok' when these limits are hit by making the appropriate changes in gurobi_run.py.  However, this code needs to be improved by a PYOMO expert - I'm sure whatever I did is a hack.