GUROBI on PYOMO is hanging

329 views
Skip to first unread message

Marc Meketon

unread,
Jan 25, 2018, 4:54:05 PM1/25/18
to Pyomo Forum
To call GUROBI (instant cloud license) from PYOMO I used the below code.  It solves after a few minutes, but it takes minutes for small problems to return the solution, and for larger ones (but not that large) after a few hours PYOMO does not report that the solution has been returned.

Am I doing something wrong???  This is a show stopper for me.

start_time = time.time()

opt = po.SolverFactory("gurobi")
opt.options["TimeLimit"] = 1000
opt.options["MIPGap"] = 0.01
opt.options["NormAdjust"]= 0
opt.options["DegenMoves"] = 1
opt.options["Heuristics"] = 0.5
opt.options["PrePasses"] = 1
print("Send to Gurobi Cloud service")

results = opt.solve(model, tee=True)   # I SEE THE OUTPUT FROM GUROBI, INCLUDING THAT IT SOLVED
model.solutions.load_from(results)

print("Solving: --- %s minutes ---" % ((time.time() - start_time)/60.0))  # NEVER SEE THIS LINE


Gabriel Hackebeil

unread,
Jan 25, 2018, 5:56:33 PM1/25/18
to Pyomo Forum
I want to guess that this has to do with how we access attributes on the Gurobi model when we extract the solution. We are likely doing something like looping over the variable objects on the Gurobi model one at a time, asking each for its “.X”. I imagine that each request for “.X” involves a message between you and the cloud service.

We have recently rewritten our Cplex and Gurobi Python interfaces to support a new persistent style of using them, and I believe we started doing bulk extraction of the solution there (i.e., grb_model.getAttr(varlist, ‘X’)). Maybe the author of that new code can chime in here to verify that.

You could also try checking out the latest master version of Pyomo from Github and letting us know if the timing changes.

Gabe

--
You received this message because you are subscribed to the Google Groups "Pyomo Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyomo-forum...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Michael Bynum

unread,
Jan 25, 2018, 6:10:01 PM1/25/18
to pyomo...@googlegroups.com
I have not yet updated the direct/persistent interfaces to gurobi to do bulk extraction of variable values. It is on my ToDo list. However, I’m not convinced that is the problem here. It looks to me like you are using the lp file interface (it is default). 

Michael Bynum

unread,
Jan 25, 2018, 6:15:09 PM1/25/18
to pyomo...@googlegroups.com
Can you add “report_timing=True” to your call to solve? This may give us a better idea of where things are stalling.

Michael

Michael Bynum

unread,
Jan 25, 2018, 6:33:26 PM1/25/18
to pyomo...@googlegroups.com
After looking more closely, it looks like the lp file interface also grabs variable values from gurobi one at a time. Thus, Gabe is probably correct about the problem. However, it seems that this may be an issue for both the lp file interface and the direct/persistent interfaces. 

Gabe Hackebeil

unread,
Jan 25, 2018, 6:46:45 PM1/25/18
to pyomo...@googlegroups.com
Good double catch. I hadn’t noticed that the LP file interface was being used, and I also forgot that it used a gurobipy script to extract the solution.

Gabe

Marc Meketon

unread,
Jan 26, 2018, 4:42:38 AM1/26/18
to Pyomo Forum
Just to clarify - pulling down the latest master will not help because the bulk update for Gurobi has not be implemented, is that right?  The "report_timing=True" did not work since PYOMO never returned after a long-enough period of time (I cancelled it after awhile so as not to burn up my budget).

I tried 
opt = po.SolverFactory("gurobi", solver_io="python")
and
opt = po.SolverFactory("gurobi", solver_io="nl")

Both of these seem not to be options allowed with Gurobi Instant Cloud:

The first one failed with:
Traceback (most recent call last):
 ...
  File "C:\Anaconda3_64\lib\site-packages\pyomo\opt\base\solvers.py", line 539, in solve
    self.available(exception_flag=True)
  File "C:\Anaconda3_64\lib\site-packages\pyomo\solvers\plugins\solvers\gurobi_direct.py", line 203, in available
    raise pyutilib.common.ApplicationError("No Gurobi <-> Python bindings available - Gurobi direct solver functionality is not available")
pyutilib.common._exceptions.ApplicationError: No Gurobi <-> Python bindings available - Gurobi direct solver functionality is not available

The latter failed with:
WARNING: "[base]\site-packages\pyomo\solvers\plugins\solvers\ASL.py", 83, _default_executable
        Could not locate the 'gurobi_ampl' executable, which is required for solver asl
Traceback (most recent call last):
  ...
  File "C:\Anaconda3_64\lib\site-packages\pyomo\opt\base\solvers.py", line 539, in solve
    self.available(exception_flag=True)
  File "C:\Anaconda3_64\lib\site-packages\pyomo\opt\solver\shellcmd.py", line 122, in available
    raise ApplicationError(msg % self.name)
pyutilib.common._exceptions.ApplicationError: No executable found for solver 'asl'



Marc Meketon

unread,
Jan 26, 2018, 10:14:59 AM1/26/18
to Pyomo Forum
Sonja Mars of Gurobi sent a fix for the file gurobi_run.py that changes a couple of lines to implement a bulk reading of the variables.

However, there is another problem with gurobi_run.py.  For long optimization runs, it is common to reach limits (e.g., time limit or mip-gap limit) before reaching optimality.  A solution is developed, but PYOMO crashes with an error that the enum SolverStatus is getting an "aborted" value but that value is not part of the enum.

The issue is that the code in gurobi_run.py sets two statuses.  One called 'status', and one called 'solution_status'.  When there the optimizer stops due to a limit, the variable 'status' is set to 'aborted'.  The 'solution_status' seems to be set correctly to 'stoppedByLimit'.

However, other parts of PYOMO then issue a message that says:

WARNING - Loading a SolverResults object with an 'aborted' status, but containing a solution

Presolve time=11.31 seconds

Solve time=142.48 seconds

Postsolve time=2.15 seconds

Traceback (most recent call last):

  File "C:\Anaconda3_64\lib\site-packages\pyomo\core\base\PyomoModel.py", line 241, in load_from

    "with bad status: %s" % str(results.solver.status))

ValueError: Cannot load a SolverResults object with bad status: aborted

 

My temporary fix is to force 'status' to say 'ok' when these limits are hit by making the appropriate changes in gurobi_run.py.  However, this code needs to be improved by a PYOMO expert - I'm sure whatever I did is a hack.

Reply all
Reply to author
Forward
0 new messages