Gurobi Optimization with Python's Multiprocessing

[JMFilipe]

I'm using a metaheuristic (genetic algorithm) to fixate some binary variables and them using gurobi to perform a linear optimization using these values. 

The problem is well formulated and works when using a single thread operation, however when I try to use the multiprocessing library (different models running in parallel) sometimes 

the results are not as expected. For instance, for 100 processes running in parallel 3 or 4 have wrong values. 

This is an approximation of my code, it is too large to copy and past here without losing interpretability.

I know that each gurobi environment is not thread safe, so inside the function I create a new one for each model. 

Is there anything wrong with my code or could be other issue? 

import gurobipy as gurobi
import multiprocessing
import numpy as np


def create_and_run_model(position):

    time_series = range(48)

    # ****************************************************
    #      * Values from MetaHeuristic *
    # ****************************************************

    turbine_on = 1 * (position.round() == 1)
    pump_on = 1 * (position.round() == 2)

    # ****************************************************
    #      * Optimization Problem - Solver *
    # ****************************************************

    prob = gurobi.Model(env=gurobi.Env(""))

    var1 = [prob.addVar(name="var1{}".format("_%03d" % (incre,)), lb=0, ub=9999) for incre in time_series]
    var2 = [prob.addVar(name="var2{}".format("_%03d" % (incre,)), lb=0, ub=9999) for incre in time_series]
    # other variables....

    prob.update()

    f_obj = gurobi.quicksum([(turbine_on[i]*var1[i] - pump_on[i]*var2[i]) for i in time_series])

    prob.setObjective(f_obj, gurobi.GRB.MAXIMIZE)

    for i in time_series:
        increment = "%03d" % (i,)
        prob.addConstr(var1[i] + var2[i] <= 1, "constr1" + increment)
        # other constraints....

    prob.optimize()

    if prob.status == gurobi.GRB.Status.INFEASIBLE or prob.status == gurobi.GRB.Status.UNBOUNDED:
        result = 9999999
    else:
        result = round(-prob.objVal, 2)

    prob.reset()

    return result


def call_model(positions):
    pool = multiprocessing.Pool(processes=7)
    fitness = np.array(pool.map(create_and_run_model, positions))
    return fitness

[Renan Garcia]

Is it possible there is some non-determinism in the data structures you use to define the models? Or perhaps the data structures themselves aren't thread safe? I would try writing each model to a unique MPS file after each solve. If 2 different runs give different results, you can compare the model files to make sure they are indeed identical.

One other thing to keep in mind is that the results may be non-deterministic if you are using the TimeLimit parameter.

--

---
You received this message because you are subscribed to the Google Groups "Gurobi Optimization" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gurobi+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[JMFilipe]

I was able to solve the problem using you suggestion of comparing both models. They were different because I had a conditional clause in the model creation, and map() and pool.map() were behaving differently, thus affecting the model.

Thanks