Discrepancy between Gurobi time and time.clock() (python)

[Fabien Tricoire]

Hi,

we repeatedly solve a MIP with changing time limits. Sometimes Gurobi returns before the time limit is reached, with status TIME_LIMIT. We would like to understand why. We measure time with time.clock(). For instance in one case time.clock() returns 5653.23, then we call the following:

model.setParam('TimeLimit', 1547.02 )

model.optimize()

Then the call to optimize() returns and status is TIME_LIMIT, but time.clock() returns 7161.13, although we would expect it to return 7200.25 (which is 5653.23 + 1547.02 ). So optimize() returns 40 seconds earlier than we expect. What are we missing?

kind regards,

Fabien Tricoire

[Fabien Tricoire]

Is there any chance that we could get an answer from Gurobi on this issue?

[Kostja Siefen]

Hello Fabien,

if you think this is a Gurobi issue, please send a log that shows the value of your TimeLimit parameter and the log messages when hitting the time limit. On Windows for example, time.clock() returns wall-clock seconds elapsed since the first call to the function. So make sure you call this functions at least once before measuring time.

Best wishes,

Kostja

[Fabien Tricoire]

Hi Kostja and thanks for the reply,

the information you ask for is in my original post:

1) time.clock() returns 5653.23

2) we call model.setParam('TimeLimit', 1547.02)

3) we call model.optimize()

4) model.optimize returns, status is TIME_LIMIT

5) time.clock() returns 7161.13

So there you have it, model.optimize() ran for 1507.90 seconds then returned because of the time limit, even though the time limit was 1547.02

Do you need any other information?

cheers,

Fabien

[Kostja Siefen]

Hi Fabien,

is this reproducible behavior? Can you please show the corresponding lines in your log? For example, if you set TimeLimit to 7200 you should see something like

Stopped in 88799 iterations and 7200.06 seconds

Time limit reached

Kostja

[Fabien Tricoire]

Yes, I attach a sample execution output to that message. Notable lines in that file:

1) lines 1-4:

time.clock() = 6608.99

Changed value of parameter TimeLimit to 591.22

   Prev: 858.46  Min: 0.0  Max: 1e+100  Default: 1e+100

time.clock() = 6608.99

2) line 155-159:

Explored 6707 nodes (2078785 simplex iterations) in 591.42 seconds

Thread count was 1 (of 20 available processors)

Time limit reached

3) line 161:

time.clock() = 7164.57

However 6608.99 + 591.42 = 7200.41, so there is a 36-second discrepancy in that case. It seems that the method used by Gurobi to measure time is not the same as time.clock(). Can you tell us how Gurobi measures time?

Fabien

[Kostja Siefen]

Hi Fabien,

The behavior of time.clock() depends on your platform and there can be various issues (see http://www.python.org/dev/peps/pep-0418/). 

Gurobi always uses wall-clock seconds, so you should re-run your time measurements with a function like time.time() that always measures wall-clock seconds cross-platform.

Kostja

[Fabien Tricoire]

Thanks Kostja! This answers my original question.

I understand that time.clock() is not consistent across platforms, but according to Python documentation os.times() works on both UNIX and Windows and counts accumulated CPU time. So Gurobi could rely on that instead.

Measuring wall clock time is not a good way to measure the CPU effort of a program, especially for benchmarking purpose. I understand that the current functionality is useful if you want to run Gurobi until a certain date, but what if you want to run it until a certain amount of time resource has been used, independently of other programs running on the same computer?

CPLEX allows to switch between both systems of measuring time: http://www-01.ibm.com/support/knowledgecenter/SSSA5P_12.2.0/ilog.odms.cplex.help/Content/Optimization/Documentation/CPLEX/_pubskel/CPLEX874.html

Is there a similar switch with Gurobi?

Fabien

[Kostja Siefen]

Hi Fabien,

Gurobi has no termination criteria based on CPU time. CPU time is a questionable concept when it comes to multi-core / multi-threaded processing especially in a virtualized environment.

Best wishes,

Kostja