IDACalcICB returned "IDA_LINESEARCH

Martin

unread,

Mar 22, 2021, 6:32:07 AM3/22/21

to CasADi

Hello,

I am trying to solve a NMHE problem using IPOPT from CasADi with a DAE system formulated in IDAS.

On some occasions, I get the messages below. The DAE constraints are met for the starting values for the NLP, this is occuring during optimization, like in the example below at iteration 41.. There, I would expect IPOPT to provide consistent initial conditions to IDAS?

I tried to modify the 'abstol' option as mentioned in some other posts, but this only changes the number of the iteration when this happens.

(BTW: how can I find the default options using the Matlab interface? - they are not in the documentation nor does the command integratorobj.print_options reveal more than a description with only very few default values included...)

Glad for any hints!

Thanks,

KR

M.

**************************************************

*** Solving the Primal Dual System for Iteration 41:

**************************************************

Number of doubles for MUMPS to hold factorization (INFO(9)) = 7545

Number of integers for MUMPS to hold factorization (INFO(10)) = 4838

Factorization successful.

Number of trial factorizations performed: 1

Perturbation parameters: delta_x=0.000000e+000 delta_s=0.000000e+000

delta_c=0.000000e+000 delta_d=0.000000e+000

residual_ratio = 2.200487e-015

Factorization successful.

residual_ratio = 1.395430e-016

**************************************************

*** Finding Acceptable Trial Point for Iteration 41:

**************************************************

--> Starting line search in iteration 41 <--

The current filter has 13 entries.

minimal step size ALPHA_MIN = 5.000000E-007

Starting checks for alpha (primal) = 9.90e-001

Checking acceptability for trial step size alpha_primal_test=9.899651e-001:

New values of barrier function = 5.6332457451514649e+008 (reference 9.9809704976545751e+007):

New values of constraint violation = 8.0199822100085257e+003 (reference 1.2788838596531691e+003):

Checking sufficient reduction...

Failed...

Trying second order correction number 1

Factorization successful.

Checking acceptability for trial step size alpha_primal_test=9.899651e-001:

New values of barrier function = 3.1323885040622228e+008 (reference 9.9809704976545751e+007):

New values of constraint violation = 1.1722834531783469e+003 (reference 1.2788838596531691e+003):

Checking sufficient reduction...

Succeeded...

Checking filter acceptability...

Succeeded...

Second order correction step accepted with 1 corrections.

**************************************************

*** Update HessianMatrix for Iteration 42:

**************************************************

The linesearch algorithm failed with too small a step.

CasADi - WARNING("solver:nlp_hess_l failed:Error in Function::operator() for 'nlp_hess_l' [MXFunction] at .../casadi/core/function.cpp:1368:

Error in Function::operator() for 'adj1_F' [MXFunction] at .../casadi/core/function.cpp:1368:

Error in Function::operator() for 'asens1_F' [IdasInterface] at .../casadi/core/function.cpp:1368:

.../casadi/interfaces/sundials/idas_interface.cpp:569: IDACalcICB returned "IDA_LINESEARCH_FAIL". Consult IDAS documentation.") [.../casadi/core/oracle_function.cpp:223]

Number of Iterations....: 42

Number of objective function evaluations = 58

Number of objective gradient evaluations = 22

Number of equality constraint evaluations = 58

Number of inequality constraint evaluations = 0

Number of equality constraint Jacobian evaluations = 44

Number of inequality constraint Jacobian evaluations = 0

Number of Lagrangian Hessian evaluations = 43

Total CPU secs in IPOPT (w/o function evaluations) = 0.085

Total CPU secs in NLP function evaluations = 1.926

EXIT: Invalid number in NLP function or derivative detected.

Joel Andersson

unread,

Mar 22, 2021, 2:32:32 PM3/22/21

to CasADi

Have you tried just performing the sensitivity analysis, without embedding it in an NLP solver problem? Here is the MATLAB example: https://github.com/casadi/casadi/blob/master/docs/examples/matlab/sensitivity_analysis.m.

If you need to know what numerical values that triggers the failure, you should be able to set the "print_in" or "dump_in" options to the integrator instance. A possible reason could be that the Jacobian of the DAE right-hand-side becomes singular at the end of the time horizon.

Joel

Martin

unread,

Mar 22, 2021, 5:53:36 PM3/22/21

to CasADi

Thanks for your quick answer!!

No, since it does not always fail, I thought this might not provide any additional information.

Are you referring to the Jacobian of the algebraic constraint with respect to the algebraic variable(s)?

I already checked that my DAE is of type semi-explicit index 1.

But actually this might be violated, if one of the parameters - which I am trying to estimate in an augmented state MHE setting - becomes zero.

This is in general prohibited due to the state constraints - but do you think it can happen during the iterations that IPOPT still wants to evaluate this point? The allowed region is far from the singularity...

Really glad for any hints how to avoid this behaviour.

Thanks again,

Kind regards,

Martin

unread,

Mar 23, 2021, 9:09:33 AM3/23/21

to CasADi

In the meantime I modified the sensitivity analysis example to match my system dynamics.

I made two versions:

(1) not adding a quadrature function. this is basically what I was using before in my NMHE definition.

Result: both IDAS and collocation work without any errors.

(2) adding the Lagrange term of my cost function as quadrature to the integrator.

Result: IDAS works fine, collocation brings the following error: (see below)

Now I would not want to use the collocation integrator in this form, without reformulating the whole problem by lifting it into a higher dimension. So for the moment I am fine with IDAS, and this result is rathe for completeness.

But another, more general question arises:

Should I reformulate my MHE problem such that the Langrange costs are in the quadrature equation, as in the sensitivity example?

So far I was only thinking of formulating the costs as discrete function at the measurement samples, rather than integrating over the shooting intervals. This would introduce some interpolation error between measurement samples....

But does it speed up the optimization process?

Thanks,

Kind regards,

Martin

Error using casadi.Function/call (line 935)

Error in Function::operator() for 'adj1_I' [MXFunction] at .../casadi/core/function.cpp:1368:

Error in Function::operator() for 'asens1_I' [Collocation] at .../casadi/core/function.cpp:1368:

Error in Function::operator() for 'asens1_I_backward_rootfinder' [KinsolInterface] at .../casadi/core/function.cpp:1368:

.../casadi/interfaces/sundials/kinsol_interface.cpp:678: nlpsol process failed. Set 'error_on_fail' option to false to ignore this error. Module "KINSol" returned flag "KIN_MXNEWT_5X_EXCEEDED".

The description of this flag is:

"Five consecutive steps have been taken that satisfy the inequality || D_u p ||_L2 > 0.99 mxnewtstep, where p denotes the current step and mxnewtstep is a scalar upper bound on the scaled step length. Such a failure may mean that || D_F F(u)||_L2 asymptotes from above to a positive value, or the real scalar

mxnewtstep is too small."

Consult KINSOL documentation for more information.

Error in casadi.Function/paren (line 1759)

res = self.call(arg);

Error in casadi.Function/subsref (line 1744)

[varargout{1:nargout}]= paren(self, s.subs{:});

Error in casadi_sensitivity_analysis (line 143)

res = I_adj('x0', x0, 'p', p0, 'adj_qf', 1);

Martin

unread,

Mar 23, 2021, 8:17:33 PM3/23/21

to CasADi

For anyone also wanting to debug the inputs to the integrator instance, I want to post my experience:

setting the "print_in" or "dump_in" options to the integrator instance as proposed by Joel above works fine
be aware that the default setting for the "dump_in" starts writing many many files to your current directory
on my Windows machine, this led to Windows Explorer break down because of too many files, I had to use Powershell to clean / delete them
for me, a better option as Matlab user was to just let the "print_in" write all the output into the Matlab console, with Matlab's "diary Myfile.out" command set before.
this let to one single file with all the other console outputs together, which was easy to search.

@Joel,

I could not find any strange inputs to the integrator, and the estimated parameter which would make the algebraic equation singular is never even close to zero.

Repeately, I got this output:

The linesearch algorithm failed with too small a step.

CasADi - WARNING("solver:nlp_g failed:Error in Function::operator() for 'nlp_g' [MXFunction] at .../casadi/core/function.cpp:1368:

Error in Function::operator() for 'F' [IdasInterface] at .../casadi/core/function.cpp:1368:

.../casadi/interfaces/sundials/idas_interface.cpp:569: IDACalcIC returned "IDA_LINESEARCH_FAIL". Consult IDAS documentation.") [.../casadi/core/oracle_function.cpp:223]

This is different from the one posted first, in that the error occurs in the integration function itself, and not in the adj/sens functions (which was probalby additionally happening due to some bad tuning of my estimator parameters).