Objective function normalization issues with quadratic OF

[Salvador Doménech]

Hi all,

I've been using Optano for power systems modeling, and, while it performs great on most use cases, it seems to take too long to normalize a quadratic objective function.

In power systems modeling we use quadratic terms in the objective function to represent market power in power markets. In my specific use case, I'm trying to solve a model with 1.5M variables and around 1M constraints, with an objective expression with around 600k terms, 100k of which are quadratic. Normalizing this expression takes around 1s using OPTANO.Modeling.Optimization.Exporter.ExpressionNormalizer.Visit(exp), but around 24h using exp.Normalize(). I've also tried to implement my own normalization algorithm (that would admittedly only work with polynomial expressions), and it normalizes it in 4s.

However, when trying to solve the problem with Gurobi or CPLEX, in both cases it seems that they normalize the Objective expression in all cases using exp.Normalize(), even when Configuration.NormalizingBehavior = Configuration.NormalizingBehavior.Manual, and having normalized the expression beforehand using the other methods above. I know this because it takes about the same time as exp.Normalize() takes between the solver.Solve call and when it starts printing the solver log to the console or to the log file.

Therefore, my question is: is there anyway to skip OF normalization when calling the solver? Or, is there anyway to inject an ExpressionNormalizer class to use it if normalization is required?

Thank you in advance!

[Jannick Lange]

Hi Salvador,

please excuse the delayed response. I have been on holidays.

Since we currently use a 3rd party library for non-linear normalization and the whole objective needs to be converted twice, the whole process is (apparently) very inefficient.

In the current version of OPTANO Modeling, the objective is unfortunately always normalized when using CPLEX, since there are some code paths where no explicit check for the "normalizing behavior" setting in place. 

I will release a fix for this shortly as v3.26.1 on NuGet.

Please keep in mind, that this will disable _all_ expression normalization and puts the "burden" on the user. As far as the internal documentation goes, the CPLEX expression converter cannot handle non-normalized quadratic expressions.

Thus, you should make sure that all of your quadratic expressions are normalized, when using "NormalizationBehavior.Manual". E.g. X * (Y + Z) -> AY + AZ.

Best regards,
Jannick