nlpsol slow with gpflow

[Rugile Pevceviciute]

Hello!

I am getting familiar with CasaDi (awesome work by the way, great tool!) and I am trying to implement MPC controller where the model is represented as a GP (using gpflow framework). I have followed the blog post about combining the 2 frameworks and used the given implementation of a custom callback for gpflow models. However, when I try to run MPC, the nlpsol is very slow. The bottleneck seems to be hessian and jacobian computations. Any ideas / pointers on how I could improve the runtime?

******************************************************************************

This program contains Ipopt, a library for large-scale nonlinear optimization.

 Ipopt is released as open source code under the Eclipse Public License (EPL).

         For more information visit http://projects.coin-or.org/Ipopt

******************************************************************************

This is Ipopt version 3.12.3, running with linear solver mumps.

NOTE: Other linear solvers might be more efficient (see Ipopt documentation).

Number of nonzeros in equality constraint Jacobian...:     1464

Number of nonzeros in inequality constraint Jacobian.:        0

Number of nonzeros in Lagrangian Hessian.............:     1092

Total number of variables............................:      225

                     variables with only lower bounds:        0

                variables with lower and upper bounds:       75

                     variables with only upper bounds:        0

Total number of equality constraints.................:      150

Total number of inequality constraints...............:        0

        inequality constraints with only lower bounds:        0

   inequality constraints with lower and upper bounds:        0

        inequality constraints with only upper bounds:        0

iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls

   0  1.7766173e+02 9.43e-01 0.00e+00  -1.0 0.00e+00    -  0.00e+00 0.00e+00   0

   1  1.8393889e+02 9.39e-01 4.31e+00  -1.7 3.22e+01    -  2.99e-02 1.54e-02h  2

   2  2.8589751e+02 9.35e-01 1.61e+01  -1.7 1.17e+01    -  6.45e-02 4.29e-02h  1

   3  3.2962474e+02 9.24e-01 2.69e+01  -1.7 1.70e+01    -  4.79e-02 1.12e-02h  1

   4  6.8426284e+02 8.71e-01 2.36e+01  -1.7 1.61e+01    -  6.75e-02 5.82e-02h  1

   5  1.0779780e+03 8.30e-01 1.12e+02  -1.7 3.03e+00   2.0 2.70e-01 4.71e-02h  1

   6  1.9119399e+03 7.63e-01 1.21e+02  -1.7 3.71e+00   1.5 2.24e-01 8.05e-02h  1

   7  3.1712533e+03 6.81e-01 8.26e+01  -1.7 2.30e+00   1.9 5.49e-01 1.08e-01h  1

   8  6.8929681e+03 4.89e-01 1.82e+02  -1.7 1.77e+00   2.4 8.50e-01 2.82e-01h  1

   9  8.9100141e+03 3.90e-01 2.77e+02  -1.7 9.20e-01   2.8 7.18e-03 2.23e-01h  1

iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls

  10  9.1197067e+03 3.89e-01 2.72e+02  -1.7 6.44e-01   2.3 6.18e-01 3.32e-02h  1

  11  1.2067525e+04 3.72e-01 3.07e+02  -1.7 6.05e-01   2.7 7.95e-03 4.26e-01h  1

  12  1.2375313e+04 3.56e-01 2.82e+02  -1.7 4.04e-01   2.3 4.01e-01 9.61e-02h  1

  13  1.3779395e+04 3.14e-01 2.10e+02  -1.7 3.59e-01   2.7 3.10e-01 4.39e-01h  1

  14  1.4107513e+04 3.07e-01 1.69e+02  -1.7 3.81e-01   2.2 5.11e-01 2.16e-01h  1

  15  1.4430298e+04 2.91e-01 1.37e+02  -1.7 3.09e-01   2.6 1.00e+00 2.43e-01h  1

  16  1.4752131e+04 2.60e-01 9.16e+01  -1.7 4.00e-01   2.2 5.32e-01 3.50e-01h  1

  17  1.4771450e+04 2.59e-01 1.07e+02  -1.7 1.06e+00   1.7 9.51e-01 4.16e-02h  1

  18  1.4993729e+04 2.58e-01 6.53e+01  -1.7 3.85e-01   2.1 4.86e-01 3.61e-01h  1

  19  1.5014295e+04 2.56e-01 7.24e+01  -1.7 1.03e+00   1.6 1.00e+00 6.45e-02h  1

iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls

  20  1.5179676e+04 2.39e-01 4.91e+01  -1.7 3.68e-01   2.1 7.02e-01 3.88e-01h  1

  21  1.5201507e+04 2.32e-01 4.54e+01  -1.7 9.74e-01   1.6 1.00e+00 9.42e-02h  1

  22  1.5336021e+04 2.29e-01 3.92e+01  -1.7 3.45e-01   2.0 1.00e+00 4.34e-01h  1

Number of Iterations....: 22

                                   (scaled)                 (unscaled)

Objective...............:   1.5336021099463585e+04    1.5336021099463585e+04

Dual infeasibility......:   3.9210747636443557e+01    3.9210747636443557e+01

Constraint violation....:   2.2895277143882908e-01    2.2895277143882908e-01

Complementarity.........:   2.2512478063791905e-01    2.2512478063791905e-01

Overall NLP error.......:   5.4465745412877844e+00    3.9210747636443557e+01

Number of objective function evaluations             = 25

Number of objective gradient evaluations             = 23

Number of equality constraint evaluations            = 25

Number of inequality constraint evaluations          = 0

Number of equality constraint Jacobian evaluations   = 23

Number of inequality constraint Jacobian evaluations = 0

Number of Lagrangian Hessian evaluations             = 22

Total CPU secs in IPOPT (w/o function evaluations)   =      0.984

Total CPU secs in NLP function evaluations           =     60.008

EXIT: Maximum CPU time exceeded.

                t_proc [s]   t_wall [s]    n_eval

ControlSolver         64.2         36.7         1

        nlp_f     0.000416     0.000415        25

        nlp_g         2.65         1.61        25

   nlp_grad_f     0.000716     0.000715        24

   nlp_hess_l           42         23.9        22

    nlp_jac_g         19.5         11.1        24

Solver Status: Maximum_CpuTime_Exceeded

I have also tried passing the approximate hessian option ({"ipopt.hessian_approximation": "limited-memory"}) to the solver and the results are slightly better,but similar:

       

Objective...............:   9.1762660840148983e+03    9.1762660840148983e+03

Dual infeasibility......:   3.5456945390891627e-08    3.5456945390891627e-08

Constraint violation....:   1.0913720621674727e-10    1.0913720621674727e-10

Complementarity.........:   1.0008720887590746e-11    1.0008720887590746e-11

Overall NLP error.......:   5.5773373554833134e-09    3.5456945390891627e-08

Number of objective function evaluations             = 111

Number of objective gradient evaluations             = 109

Number of equality constraint evaluations            = 111

Number of inequality constraint evaluations          = 0

Number of equality constraint Jacobian evaluations   = 109

Number of inequality constraint Jacobian evaluations = 0

Number of Lagrangian Hessian evaluations             = 0

Total CPU secs in IPOPT (w/o function evaluations)   =      1.180

Total CPU secs in NLP function evaluations           =     93.532

EXIT: Optimal Solution Found.

                t_proc [s]   t_wall [s]    n_eval

ControlSolver         99.8         56.5         1

        nlp_f      0.00183      0.00183       111

        nlp_g         11.6         6.91       111

   nlp_grad_f      0.00311       0.0031       110

    nlp_jac_g         87.9         49.3       110

Solver Status: Solve_Succeeded

 

Any help would be appreciated!

Rugile 

[Max Hürlimann]

Hi everybody!

I have the exact same problem. The gradients of the constraints seem to take a long time to calculate.
Is there anyway to speed up solving when using a gpflow model for describing system dynamics or is it inherently always going to be slow?

[Joel Andersson]

I think Joris is doing some work on improving the speed in the interfaces, but in general, it's hard to avoid a lot of overhead. Does gpflow have some way to export models as C code? Interfacing models written in C code can be done with much less overhead.

Joel