Parametric sensitivities of NLP solutions

[AY Wer]

Hi,

Thanks for providing this software.

Does the computation of parametric sensitivities of NLP solutions work with IPOPT?

I have been following the sample code provided in the "nlp_sensitivities.py" file from the example pack.

Ideally I would like to use IPOPT for both the calls to

sol = solver(x0=w0, lbx=lbw, ubx=ubw, lbg=lbg, ubg=ubg, p=0)

and to

hsolver = solver.factory('h', solver.name_in(), ['sym:hess:f:p:p'])
hsol = hsolver(x0=sol['x'], lam_x0=sol['lam_x'], lam_g0=sol['lam_g'],

               lbx=lbw, ubx=ubw, lbg=lbg, ubg=ubg, p=0)

However, it would be already good to be able to solve the original problem (the first call) using IPOPT.

Replacing 'sqpmethod' by IPOPT for both calls converge, but it seems an error occurs when the results are returned:

    428             sens_solver = solver.factory('h', solver.name_in(), ['jac:x:p'])
--> 429             dxsol_dx0 = sens_solver(res["x"],arg["p"],arg["lbx"],arg["ubx"],arg["lbg"],arg["ubg"],res["lam_x"],res["lam_g"])
 
~\AppData\Local\Programs\Python\Python37\lib\site-packages\casadi\casadi.py in __call__(self, *args, **kwargs)
  13304       if len(args)>0:
  13305     # Ordered inputs -> return tuple
> 13306         ret = self.call(args)
  13307         if len(ret)==0:
  13308           return None

~\AppData\Local\Programs\Python\Python37\lib\site-packages\casadi\casadi.py in call(self, *args)
  12184 
  12185         """
> 12186         return _casadi.Function_call(self, *args)
  12187 
  12188 

RuntimeError: .../casadi/core/function_internal.hpp:1219: Evaluation failed

When following the recipe exactly, it sometimes works, but on larger problems the same behavior occurs, depending seemingly on the initial values given.

Any pointers how to improve the chances of successfully generating the sensitivities?

Regards

[AY Wer]

It seems to be important to use the sqpmethod solver. Other solvers seem not to work. Also I tried to start from the solution provided from IPOPT, but that only works if the multipliers returned from IPOPT are not given to sqpmethod. For larger problems, the performance of sqpmethod is still not satisfactory compared to IPOPT, but other solvers did not work for me.

[AY Wer]

I think one reason is a bug in either sqpmethod or qpqr. If the tolerances are fulfilled at the first iterate the following line is  printed indefinitely:

Iter  Sing        fk      |pr|   con      |du|   var     min_R   con  last_tau  Note
    0   421         0  1.5e-010   283         0    -1         0   627         0  

[AY Wer]

It seems that using the multipliers from IPOPT always leads to an "Evaluation failed".
I had the most success in enabling blockSQP and initializing it with the results from IPOPT.
Unfortunately, the evaluation also fails if blockSQP terminates before finishing one iteration. Is there a way to set minIter for blockSQP?

Regards

[AY Wer]

For anybody facing similar problems, here is what seems to work well.

1. Solve the problem with small tolerances with Ipopt:                                 
   opts["ipopt.tol"] =1e-9 

   opts["ipopt.dual_inf_tol"]=1e-9

   opts["ipopt.recalc_y"]="yes"

   opts["calc_multipliers"] = True
   solver = cas.nlpsol("solver", "ipopt", self.nlp, opts)
   #fill arg
   res = solver(**arg)
   
   
   
2. Use blockSQP with medium tolerances. Limit the SQP iterations to 1 and most importantly, do NOT pass the multipliers from Ipopt, this will force the algorithm to make at least a single iteration 
   opts2 = dict(expand=True,error_on_fail=False,opttol= 1e-4,max_iter=1)#

   solver2=cas.nlpsol('solver','blocksqp',nlp,opts2)

   res = solver2(x0=res["x"], p=arg["p"], lbx=arg["lbx"], ubx=arg["ubx"], lbg=arg["lbg"], ubg=arg["ubg"])#,lam_g0=res["lam_g"])
    
   
   
   
3. Generate the sensitivties using the blockSQP solver 
   sens_solver = solver2.factory('h', solver2.name_in(), ['jac:x:p'])
   

   dxsol_dp = sens_solver(res["x"], arg["p"], arg["lbx"], arg["ubx"], arg["lbg"], arg["ubg"],

                                          res["lam_x"], res["lam_g"])