Pyomo and Cplex callbacks

[riccardo cantoni]

Hi everybody,

I'm relatively new to Pyomo and I'm working on a project that requires the generation of cuts at every node of the branch & bound tree generated by cplex. Let me clarify: I have a pyomo abstract model that I use to create instances, I'm using the Cplex Direct interface to get access the cplex solver, but before solving the problem I would like to pass a callback function, more specifically the UserCutCallback, to cplex in such a way it generates new cuts at every node of the branch and bound tree. I already Know that it's possible to do that using the cplex API but I would like to use pyomo instead. Looking at the documentation I found a method, the set_callback method, of the CplexDirect interface that seems suitable for my problem, but It raises an exception.

It follows the code I've written along with the exception I got:

# model definition
model = AbstractModel()
...

# create an instance
instance = model.create_instance(data = 'data.dat')

# the callback that cplex should call during the branch-and-cut search
def cut_callback(solver, model):
    print("CB-Cut")

opt = SolverFactory('cplex_direct')
opt.set_callback('cut-callback', cut_callback)

# never reached
opt.solve(instance)

and here the exception raised:

~/miniconda3/envs/opt/lib/python3.6/site-packages/pyomo/opt/bas/solvers.py in set_callback(self, name, callback_fn)
    815         if not self._allow_callbacks:
    816             raise pyutilib.common.ApplicationError(
--> 817                 "Callbacks disabled for solver %s" % self.name)
    818         if callback_fn is None:
    819             if name in self._callback:

ApplicationError: Callbacks disabled for solver cplexdirect 

Do you know if it's possible to get access the cplex callbacks from Pyomo? It would be a very useful feature

Tanks,

Riccardo

[Giorgio Balestrieri]

Ciao Riccardo,

it seems you are using "cplex_direct". I'm not sure what that is, but I believe callbacks do work on "normal" cplex, so they are probably simply not enabled for "cplex direct".

As you can see in the source code, _allow_callbacks is False by default in the base OptSolver class, meaning specific solvers need to enable callbacks explicitly.

Not sure about the rationale here, but you might have better luck using cplex instead of cplex_direct.

Best,

Giorgio

[riccardo cantoni]

Ciao Giorgio,

"cplex-direct" is a solver interface that directly interfaces with cplex through the cplex API, and  does not use any file io, as it happens for the "cplex" interface. Thus, I thought it was the most suitable interface for setting a callback, since I think the only way to inform cplex about the presence of a callback is through the API.

In any case I tried all the cplex interfaces, that are: cplex-direct, cplex-persistent and cplex, and all of them have the  _allow_callbacks attribute set to False, and so I always got the same exception. Looking inside the source code, I didn't find any method that sets _allow_callbacks to True, it's always False.  Maybe this feature is not yet implemented, or maybe I'm missing something.

Please, if you know how to use callbacks from pyomo or if notice that I forgot something important to enable the callbacks let me Know!!

Thanks,

Riccardo

[Giorgio Balestrieri]

Hi Riccardo,

thanks for the information about "cplex-direct", I wasn't aware of that. I did a bit of research and this example might be helpful.

For some reason they use "_cplex_direct" (notice the leading underscore), not sure what the difference is.

By the way, I think it might be worth opening an issue on github to give developers more visibility in this, it should at least be clear whether it's possible or not to use callbacks with a given solver.

Let us know if by any chance it works!

Best,

Giorgio

[Michael Bynum]

The short answer is that callbacks are not currently supported. We have prototyped callbacks for gurobi in another branch, but it is not ready to be used yet, unfortunately. 

Also, the direct interface to cplex was re-written about a year ago, which is when "_cplex_direct" was changed to "cplex_direct". I believe this change was released in Pyomo5.4. 

Michael