OPTANO throwing unexpected exception

[Nima Esmaili Mokaram]

Hey all 👋,

We're experiencing a very strange behavior from OPTANO where the library is throwing the below exception. What's very strange is that the exception message reads "You should not see this Exception. (I.e. please report it, if you do so.)" and that's why I'm here 😄.

Additional Info:

- Cplex

- OPTANO.Modeling version 3.3.0.7466

Type: System.AggregateException

Message:

   One or more errors occurred. (Exceeded time limit for solving process. Effective TimeLimit: -0:00:01

   You should not see this Exception. (I.e. please report it, if you do so.))

Properties:

   HResult: 0x80131500

InnerExceptions[0]:

   Type: System.TimeoutException

   Message:

      Exceeded time limit for solving process. Effective TimeLimit: -0:00:01

      You should not see this Exception. (I.e. please report it, if you do so.)

   Properties:

      HResult: 0x80131505

   Source: optano.modeling

   Stack Trace:

      at OPTANO.Modeling.Optimization.SolverBase.BuildConfigureAndSolveOnAdapter(Int32 prioLevel, Dictionary`2 variableValues, Boolean isResolve, TimeSpan gloablElapsedTime)

      at OPTANO.Modeling.Optimization.SolverBase.AdjustQualityConservationBoundAndTryResolve(List`1 toleranceConstraints, Int32 iteration, Double[] absouluteFallbackTolerance, IGrouping`2 objectiveStage, Dictionary`2 lastSolutionVariables, TimeSpan globalElapsedAfterFailedSolve)

      at OPTANO.Modeling.Optimization.SolverBase.SolveNonNative(Dictionary`2 variableValues, Boolean isResolve)

      at OPTANO.Modeling.Optimization.SolverBase.Solve(Model model, Dictionary`2 variableValues)

      at Flips.Optano.cplex128Solve(SolverSettings settings, Model optanoModel)

      at Flips.Optano.solve(OptanoSolverType solverType, SolverSettings settings, Model model)

      at Flips.Solver.solve(SolverSettings settings, Model model)

Any one has any ideas why this is happening and how we can prevent it from happening?

Please let me know if any additional information would be useful

Best,

-Nima

[OPTANO Team]

Hi,

could you provide a (minimal) code example that raises this issue?

What is the timelimit you have used in the SolverConfiguration?

Best

jp

[Nima Esmaili Mokaram]

We are not able to consistently reproduce this is issue 😟.

This issue has happened on different occasions, the latest instance of it was with TimeLimit = 5 minutes.

-Nima

[Jannick Lange]

Hi Nima,

I took a look at the internal code, and I think I might have found a path that might lead to this exception:

Can you please check the log, if Modeling printed the following warning:

•  "Model became infeasible by adding quality-conservation constraint. Infeasibility is due to floating point arithmetic. Adding specified tolerance to quality-conservation constraint's bound."
• (Additionally, can you please check if CPLEX prints any warnings/hints regarding the numeric stability of the model?)
  

It seems as though we use an outdated version of the elapsed solving time, when we check if there's still time left for (re-)solving another hierarchy level.

Later on, the actual solve method uses the "correct" elapsed solving time and realized that it should not have been called. It then throws the observed exception, giving this information in the message text.

We will fix this issue with the next release of OPTANO Modeling.

If you do not observe the afore mentioned information ("Model became infeasible by adding ..."), please let us know.

Regarding the 2nd thing that I asked you to check:

After the solver finds an optimal solution for a given objective z(x) = min c * x (let's call the optimal objective value 'z_opt'), Modeling will add a constraint "c * x <= z_opt", so that the objective value of z will not deteriorate when solving subsequent objective stages.

Usually, adding this constraint should not render the model infeasible. If it becomes does, this might indicate numerical instabilites in the model formulation, that might be resolvable by reformulating the model, or by adjusting the numeric emphasis parameters.

The following link might be helpful investigation/mitigation:

• https://www.ibm.com/docs/en/icos/12.10.0?topic=problems-numeric-difficulties
• https://www.ibm.com/docs/en/icos/12.10.0?topic=optimizer-overcoming-numeric-difficulties
• https://www.ibm.com/docs/en/icos/12.10.0?topic=solutions-limitations-due-numeric-difficulties

Best Regards

Jannick

[Nima Esmaili Mokaram]

Hi Jannick,

Thanks for the response.

I do see the warning you mentioned. It happens right before the exception is thrown (created a gist for the logs here: https://gist.github.com/niesmo/488f156c204c392fe55514a02b9f32d5).

I do not see any other warnings or anything regarding "numeric stability of the model". You can double check the logs incase I missed something. I also don't know if there's some configuration that needs to be set for us to see more logs from CPLEX. If there is, please let me know and we can enable that for future times.

> We will fix this issue with the next release of OPTANO Modeling.

That's great news. How frequent are the releases or can you give me a high-level timeline?

Thanks,

Nima

[Jannick Lange]

Hi Nima,

Sorry for the delayed response. I did not receive (or missed) a notification email for your reply.

I will publish a patch for this today.

Currently, I'm also working on a new feature set that will, among others, add support for native multi-objective handling in CPLEX (see e.g. https://www.ibm.com/docs/en/icos/20.1.0?topic=optimization-solving-multiple-objective-problems).

The update is scheduled for release in late November.

In your log it seems as though you're using very large coefficients in your objective function. The objective value is in the range of 10e8 to 10e9.

If you're mixing this with some constraints where you're using small coefficients (e.g. 10e0 to 10e3), you'll very likely run into numerical issues.

The total range difference between some pairs of coefficients exceeds the default integrality- and feasibility tolerances of CPLEX, which are in the 10e6 (or 10e5) range.

See also this blog post: https://www.ibm.com/support/pages/cplex-tolerance-usage

This is also reflected by the long time CPLEX requires from reaching a MIP Gap of 0.00% until it finally finishes the optimization (~3 seconds vs. 180 seconds).

Can you run the CPLEX model analyzer? I'd expect it to report a quite large "Attention level"

• https://www.ibm.com/docs/en/cofz/12.10.0?topic=tmpp-mip-kappa-detecting-coping-ill-conditioned-mip-models
• Note: Optano Modeling does not expose this parameter (I added a feature request for this. Maybe it will already be included in today's patch).
• As a workaround: You can export your model as mps or lp, and then call CPLEX manually via its command line interface. There, you can add the parameter:

------------------------------------------------------------------------------------------------------------------------------------------------
Welcome to IBM(R) ILOG(R) CPLEX(R) Interactive Optimizer 12.10.0.0
  with Simplex, Mixed Integer & Barrier Optimizers
Copyright IBM Corp. 1988, 2019.  All Rights Reserved.

Type 'help' for a list of available commands.
Type 'help' followed by a command name for more
information on commands.

CPLEX> help

add             add constraints to the problem
baropt          solve using barrier algorithm
change          change the problem
display         display problem, solution, or parameter settings
enter           enter a new problem
feasopt         find relaxation to an infeasible problem
help            provide information on CPLEX commands
mipopt          solve a mixed integer program
netopt          solve the problem using network method
optimize        solve the problem
populate        get additional solutions for a mixed integer program
primopt         solve using the primal method
quit            leave CPLEX
read            read problem or advanced start information from a file
set             set parameters
tools           tools for analysis and debugging of models
tranopt         solve using the dual method
write           write problem or solution information to a file
xecute          execute a command from the operating system

Enter enough characters to uniquely identify commands & options. Commands can be
entered partially (CPLEX will prompt you for further information) or as a whole.
CPLEX> set mip strategy kappastats 2
New value for strategy to gather statistics on the kappa of subproblems: 2
CPLEX>

------------------------------------------------------------------------------------------------------------------------------------------------

Then read your model and optimize it.

Best Regards

Jannick

[Nima Esmaili Mokaram]

Hi Jannick,

I will keep an eye out for a release on the OPTANO.modeling package (https://www.fuget.org/packages/OPTANO.Modeling).

Multi-objective support would be amazing as we actually are setting multiple objectives right now 😊. 

What's interesting is that I went to generate the LP or the MPS output files and realized that they do not work with multiple objectives as of right now.

I removed the secondary objective to generate the MPS file which worked ✔. I then set the strategy kappastats = 2 and optimized (in addition to the other parameters that we already set).

The Attention level was super low (unlike what you mentioned) = 0.000206

Here's the output of the run in a gist: https://gist.github.com/niesmo/2fafc5aeee1dd9039ed26ceb6973fe86

As you can see, we got the warning about the numerical difficulties ("CPLEX encountered numerical difficulties while solving this model.") at the end. 

Regarding the large objective, when we setup the model, we noticed that as well (specially because the optimization would converge super quickly) and did few things. The things I remember are

1. Variable/Objective Scaling (1e-3)

2. Set the gap and abs gap for MIP to 0 (as you can see in the settings).

Let me know what you think about the results.

Best,

-Nima

[Jannick Lange]

Hi Nima,

The low attention level comes as a suprise to me. Especially considering the huge coefficient range (1 to 6e8).

Are the large RHS values actually required? Are those some BigM-values, or do those coefficients actually represent some "real" values?

I'd continue with the following:

• Try to scale down the RHS values (e.g. by converting units from "pieces" to "millions of pieces", or by sharpening your Big-M computation)
• Play around with smaller values for the three tolerances in CPLEX (e.g. setting them to 1e-9)

I don't think that setting the Abssolute/Relative MipGap to 0 is beneficial. Most likely, CPLEX will continue to solve "for perpetuity".

Usually, such binary models are quite hard to solve "exactly", since there are many tiny improvements to be made by swapping around some active/inactive variables.

I'd go for a reasonable relative gap of about 1e-4 to 1e-6, because further "improvements" are below the default tolerance for integrality/optimality anyway, and only give a false sense of "accuracy".

Regarding the exception that you ran into:

I just uploaded a new version of OPTANO Modeling to nuget. It should be listed in a couple of minutes.

• https://www.nuget.org/packages/OPTANO.Modeling/3.8.1.532

Best,

Jannick

[Jannick Lange]

PS:

Besides the check for numerical difficulties, CPLEX also has another assistant for model analysis:

• https://www.ibm.com/docs/en/icos/20.1.0?topic=parameters-data-consistency-checking-modeling-assistance
• You can try to "set read datacheck 2" and re-optimize your model via the CLI tool.
  

Best,

Jannick

[Nima Esmaili Mokaram]

Hi Jannick,

Thanks for 

1. Releasing an update for the OPTANO.Modeling package

2. Recommendation on MIP Gap

3. Recommending data consistency checking and modeling assistance

Few things:

(1) Regarding the new OPTANO.Modeling package (3.8.1.532), there seems to be 2 additional dlls that are included 

- MipCL213WrapperCpp.dll

- Scip70WrapperCpp.dll

Was this intentional? It might have been an error in some of the packaging steps.

(2) Setting the MIP Gap = 0 was intentional at first, but you having recommended that, I'm going to try different MIP Gap settings and look at the difference in solutions

(3) I set the read datacheck = 2 and optimized. There are a lot of warnings (which is likely what you were expecting). I have put the warnings in this gist here (https://gist.github.com/niesmo/2fd3c07c2488e50300cd59eeac539f68).

I'm not familiar with the warnings yet (I'm reading about them right now). Which of the warnings do you think I should focus on the most?

Best,

-Nima

[Nima Esmaili Mokaram]

Also, one more thing:

We were doing some testing with the new OPTANO.Modeling version (3.8.1.532) when we encountered a NullReferenceException exception from OPTANO. Here's the stack trace:

Type: System.NullReferenceException

Message: Object reference not set to an instance of an object.

Properties:

   HResult: 0x80004003

Source: optano.modeling

Stack Trace:

   at OPTANO.Modeling.Optimization.SolverBase.ComputeAndStoreAllObjectiveValues(Solution intermediateSolution)

   at OPTANO.Modeling.Optimization.SolverBase.FinishNonNativeSolve(Solution intermediateSolution, List`1 toleranceConstraints, Int64 totalNodeCount, Dictionary`2 prioToMergedName, Dictionary`2 priorityLevelToFinalGap, Boolean wasTimeout, Boolean userInterruptedSolve)

   at OPTANO.Modeling.Optimization.SolverBase.SolveNonNative(Dictionary`2 variableValues, Boolean isResolve)

   at OPTANO.Modeling.Optimization.SolverBase.Solve(Model model, Dictionary`2 variableValues)

I created a gist for the logs from this run here: https://gist.github.com/niesmo/807e49a25601ed73babaab719d3076b2

Any idea what could be causing this?

Best,

Nima

[Jannick Lange]

Hi Nima,

I just returned from vacation. Thank you for your patience.

Regarding your questions:

(1) The DLLs are intentionally delivered. They are required for using the MipCL-/Scip Solver. However, if you do not intend to use any of those solvers, you can savely remove them.

(3) I'd start looking at the large coefficients, as they will likely be the cause for the infeasibility in the later objective stages.

Regarding the exception:

In the run that is given in the gist, it seems as though

a) CPLEX "thinks" that the quality-conservation constraint to maintain the objective value from the first objective stage leads to an infeasibility in the second stage

** l 105: Warning: No solution found from 44 MIP starts.

b) CPLEX did not find any feasible solution (nor did it manage to "prove" infeasibility due to the quality-conservation constraint) for the second objective stage

The possibility of this happening was not foreseen when we wrote the emulation for multi-stage objectives, which caused a null-pointer due to missing solution/objective values in the 2nd stage.

I will start working on a patch to prevvent modeling from crashing in such a case.

Note, however, that you'd be left without any feasible solution, if CPLEX will show the same behavior w.r.t. the claimed/actual infeasibility/invalidity of the second stage, when fixing the objective value that was found in the first stage.

That issue is best approached by trying to reduce the coeeficient/RHS range of your model.

Best,
Jannick

[Nima Esmaili Mokaram]

Hey Jannick,

Any update on this? and by this, I'm more specifically thinking about the exception that's thrown when using the (3.8.1.532) version. Is there a plan to release a patch to fix that part?

Best,

Nima

[OPTANO Team]

I'll check with the dev-team.

We're about to relase a new modeling version in the next days.

Best

jp

[Jannick Lange]

Hi Nima!

We just released a new version of Optano.Modeling, that should fix the NullPointer issue.

• https://www.nuget.org/packages/OPTANO.Modeling/3.9.0.535

Best,

Jannick

esmaili...@gmail.com schrieb am Freitag, 12. November 2021 um 15:43:05 UTC+1:

[Nima Esmaili Mokaram]

Hi Jannick,

Thank you. We'll update our services and see how it works out. I'll report back here on how it goes and if we find other things.

Best,

Nima

[Jannick Lange]

Hi Nima,

We just published a new version of OPTANO.Modeling, that brings native support for multi-objective optimization to CPLEX 12.10.

It also includes an adapter for CPLEX 20.1.0.

* https://www.nuget.org/packages/OPTANO.Modeling/3.10.0.538

Best,

Jannick

[Nima Esmaili Mokaram]

Hi Jannick,

Thanks for the updated. We updated to the latest version of the library (3.10.0.538) when you posted here. Things are working well and as expected so far.

I will let you know if there's anything that comes up related to things we have been talking about here.

Best,

Nima