How to get intermediate optimization results in execution time

[Mayron Rodrigues]

Hi Optano

I am utilizing Optano to execute an optimization problem by calling the GUROBI solver. However, I often encounter lengthy execution times exceeding 3600 seconds. Consequently, I am seeking ways to obtain intermediate objective value, GAP and lower boundin execution time

Thanks

Mayron

[Jannick Lange]

Hi Mayron,

The runtime of an optimization problem depends on a multitude of different options. From afar (and sometimes also from nearby), it is impossible to quantify the specific reasons.

* You can, for example, try to export your model as MPS file and run the Gurobi Tuner:

** https://support.gurobi.com/hc/en-us/articles/14126076996881-Parameter-Tuning-Tool

** https://www.gurobi.com/documentation/10.0/refman/command_line_tuning.html

* It might be worthwile to review your model formulation, and try to improve some formulations

** E.g., computing tighter "Big M" values, if you use any in your constraints

** Disaggregate sums, whereever possible

*** e.g. instead of using a constraint with 4 binaries "3 * x <= a + b + c" convert it to 3 constraints:

*** x <= a

*** x <= b

*** x <= c

*** This will improve the quality of the LP relaxation during the solving process and thus might reduce the overall runtime.

You can get some intermediate information regarding Gurobi's internal status by using the "GurobiSolver.Status" callback.

For example, this callback method will compute + print the MIP Gap whenever Gurobi invokes the appropriate callback type:

using (var solver = new GurobiSolver(grbConfig))
{
     solver.Status += statusInfo =>
     {
          if (statusInfo.LowerBound.HasValue && statusInfo.Incumbent.HasValue)
          {
               double gap;
               if (statusInfo.LowerBound.Value.IsAlmost(statusInfo.Incumbent.Value))
               {
                    gap = 0d;
               }
               else if (statusInfo.LowerBound.Value.IsAlmost(0))
               {
                    gap = double.PositiveInfinity;
               }
               else
               {
                    gap = Math.Abs(statusInfo.LowerBound.Value - statusInfo.Incumbent.Value) / Math.Abs(statusInfo.LowerBound.Value);
               }

               Console.Write($"Current gap: {gap * 100:0.####}%");
               Console.WriteLine(statusInfo.SolverTime.HasValue ? $" - Runtime: {statusInfo.SolverTime.Value:0.##} seconds" : "");
          }
     };
     // ...
}

Please let us know, if you have any further questions.

Best Regards

Jannick

[Jannick Lange]

PS:

I just noticed that I used a wrong formula for the computation of the mip gap. The denominator should be the best objective value, not the best dual bound:

// ...
gap = Math.Abs(statusInfo.LowerBound.Value - statusInfo.Incumbent.Value) / Math.Abs(statusInfo.Incumbent.Value);
// ...

Best Regards

Jannick

[Mayron Rodrigues]

Hello  Jannick,

Thank you very much. Your code worked perfectly!

I have another question:

I am testing Gurobi Instant Cloud as well and would like to set up the license without use gurobi.lic file.

I used the code below but didn´t work

solverConfig.LimitedLicenseUsage = LimitedLicenseUsage.Always;
solverConfig.CloudAccessID = "def091---------------------------e2f8ca79f7";
solverConfig.CloudSecretKey = "OG----------------------hMC00Zm";
solverConfig.CSAppName = "MODEL";
solverConfig.LicenseID = 9999999;

Can you help me?

Thanks

Mayron

[OPTANO Team]

Hi

What error do you get?

Best

jp

[OPTANO Team]

For your ease, you may use a gurobi.lic file either in the gurobi path or the execution directory of your application and remove all license parameter from the code. 

Does it use the instant cloud then? 

[Mayron Rodrigues]

Hi jp,

I don´t get any error but the optimization don´t run on cloud, it run on my local machine.

I exclude the license file and when I run again, throw the follow exception: 

Unable to modify parameter LicenseID after environment started

   at HELIC.OPT.Otimizador.Solve() in C:\Users\mp16\source\repos\@POCD\DFHL\otimizador\HELIC.OPT\Otimizador.cs:line 306
   at HELIC.OPT.AmbienteOtimizacao.CallSolver() in C:\Users\mp16\source\repos\@POCD\DFHL\otimizador\HELIC.OPT\AmbienteOtimizacao.cs:line 337
   at HELIC.OPT.AmbienteOtimizacao.<Otimizar>b__145_1() in C:\Users\mp16\source\repos\@POCD\DFHL\otimizador\HELIC.OPT\AmbienteOtimizacao.cs:line 304
   at System.Threading.Tasks.Task`1.InnerInvoke()
   at System.Threading.Tasks.Task.<>c.<.cctor>b__272_0(Object obj)
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
--- End of stack trace from previous location ---
   at HELIC.OPT.AmbienteOtimizacao.Otimizar(String jsonDados) in C:\Users\mp16\source\repos\@POCD\DFHL\otimizador\HELIC.OPT\AmbienteOtimizacao.cs:line 302
   at HELIC.ViewModel.VMMain.Otimizar(Object parameter) in C:\Users\mp16\source\repos\@POCD\DFHL\helic\HELIC.ViewModel\VMMain.cs:line 559

Está mensagem foi copiada para a área de trabalho

[OPTANO Team]

Hi, 

Please clean the build. I'd expected the file is still present in the build directory and the gurobi lib reads it. 

best

jp

[Jannick Lange]

Hi Mayron,

The first thing that I notice is the setting ' solverConfig.LimitedLicenseUsage = LimitedLicenseUsage.Always '.

This will prevent Gurobi from using any of your licenses. It forces Gurobi to use a built-in, size-limited license that only works for small models (2k vars/cols).

From your sample code, it seems as though not all parameters that are required for starting a Cloud Environment are set.

OPTANO.Modeling expects the following values to be set:

public bool IsCloudLicenseSpecified()
{
     return !(string.IsNullOrEmpty(this.CloudAccessID)
          || string.IsNullOrEmpty(this.CloudSecretKey)
          || string.IsNullOrEmpty(this.CloudPool)
          || string.IsNullOrEmpty(this.CloudHost));
}

I never have had access to a Gurobi Cloud license, so it might be possible, that we are overly restrictive/enforce optional parameters to be set.

Do you have sensible credentials to set those 4 properties?

At what point in time do you try to set your cloud parameters in the configuration? Before or after you initialize the GurobiSolver?

The exception seems as though you try to modify a running GRBEnv, which is not allowed.

Best Regards

Jannick

[Mayron Rodrigues]

Jannick,

I removed the  solverConfig.LimitedLicenseUsage = LimitedLicenseUsage.Always line and tried run again. Did not work. Code throw the same exception.

Do you have sensible credentials to set those 4 properties?

No, I don´t. Gurobi lic file for Cloud has only this 4 parameters: CLOUDACCESSID, CLOUDKEY, LICENSEID and CSAPPNAME. See lic file below:

# This is a license file created by the Gurobi Instant Cloud
# Created on Mon May 08 2023 18:18:52 GMT-0300 (Horário Padrão de Brasília)
# License Id: 123456789
# Place this file in your home directory that takes precedence
# or in one of the following shared locations:
#    * C:\gurobi\ on Windows
#    * /opt/gurobi/ on Linux
#    * /Library/gurobi/ on Mac OS X
# Or set the environment variable GRB_LICENSE_FILE to point to this file,
# it will override the default locations
# Do not share or send by email this license file because it contains your secret key
CLOUDACCESSID=xxxxxxx
CLOUDKEY=xxxxxxx
LICENSEID=123456789
CSAPPNAME=HELIC

At what point in time do you try to set your cloud parameters in the configuration? Before or after you initialize the GurobiSolver?

I tried initialize before and after the code: using (var solver = new OPTANO.Modeling.Optimization.Solver.Gurobi1000.GurobiSolver(solverConfig))

Did not work in both cases.

See the exception stack:

em Gurobi.GRBEnv.Set(IntParam param, Int32 newvalue)
   em OPTANO.Modeling.Optimization.Solver.Gurobi1000.GurobiSolver.SetRefManSolverConfiguration()
   em OPTANO.Modeling.Optimization.Solver.Gurobi1000.GurobiSolver.SetConfigurationAdapterSpecific()
   em OPTANO.Modeling.Optimization.SolverBase.BuildConfigureAndSolveOnAdapter(Int32 prioLevel, Dictionary`2 variableValues, Boolean isResolve, TimeSpan gloablElapsedTime)
   em OPTANO.Modeling.Optimization.SolverBase.BuildConfigureAndSolveOnAdapter(Int32 prioLevel, Dictionary`2 variableValues, Boolean isResolve)
   em OPTANO.Modeling.Optimization.SolverBase.Solve(Model model, Dictionary`2 variableValues)
   em HELIC.OPT.Otimizador.Solve() em C:\Users\mp16\source\repos\@POCD\DFHL\otimizador\HELIC.OPT\Otimizador.cs:linha 153

Thanks

[Jannick Lange]

Ah, I see.

The SetRefManSolverConfiguration method is actually invoked again before a model is solved in order to set the latest parameter values in Gurobi.

I need to make sure that the configuration-specific parameters will not be re-set, once the environment has been initialized. This will require a new release.

In the meantime, I think that you do not need to set the parameters in the solverConfig, if you have a properly built licesne file.

Can you try to remove the cloud parameters from your Gurobi solverConfig file?

Afterwards, follow these instructions to help Gurobi find the file: https://support.gurobi.com/hc/en-us/articles/360013417211-Where-do-I-place-the-Gurobi-license-file-gurobi-lic-

Best Regards

Jannick