NLP solver with JIT in python 3.7 and casadi 3.5.1

[mario.za...@gmail.com]

Hi guys,

I remember some time ago Joel adivsed me to use jit compilation with integrators in casadi to speed up the OCP solution time.  I tried to look up the documentation, but I am not really sure how I should proceed to do that. I see how I can compile a casadi function, but then the NLP solver has to take Jacobians and Hessians, so how can I do the compilation correctly?

By looking around it seemed to me that I could pass the option 'jit':True to the nlp solver, but then I get this error message

    Error code: dlopen(/Users/zmario/src/casadi-osx-py37-v3.5.1/casadi/libcasadi_importer_clang.dylib, 5): image not found

  Tried '' :

    Error code: dlopen(libcasadi_importer_clang.dylib, 5): image not found

  Tried '.' :

    Error code: dlopen(./libcasadi_importer_clang.dylib, 5): image not found

In fact that library is not present in the casadi folder.

Thanks a lot in advance!!

[Joris Gillis]

Dear Mario,

Combine with 'compiler' 'shell' option. The clang compiler is not currently shipped in casadi binaries for mac.

This will call a system compiler (gcc by default).

To switch to another compiler, use 'jit_options.compiler' 'clang' for example.

For compilation flags, set 'jit_options.flags' to a list of strings, ['-O3'] for example.

You can apply jit to nlpsol if your graph is not too big.

You could also apply it to underlying Functions, e.g. to integrator('intg','rk',{"simplify:True", "jit":True,"compiler":True})

Best regards,

 Joris

[mario.za...@gmail.com]

Thank you very much for the explanation Joris!!

My NLP is actually fairly large size, but since I use distributed multiple shooting, the integrators are small (4 states each, but 68 of them). Then I guess using the jit option on the nlp solver is out of the question, while using it on the integrators might be a good idea.

I am trying out these options, but my issue is that from what I see, the code generated for the integrator is only evaluating it and does not contain the Jacobian or "directional" Hessian. How are the derivatives evaluated then? This is my main concern, since they are the most time-consuming operation.

Thank you very much again!!

Btw, in both cases I get this error:

RuntimeError: .../casadi/core/options.cpp:246: Unknown option: simplify

Also, I suppose you meant "compiler":"shell".

[Joris Gillis]

Derivates of jitted functions are supposed to be jitted as well, automatically.

You can specify 'jit_cleanup' false to be able to check all generated source files.

"simplify" which works with "rk" integrator is there since CasADi 3.5.0 iirc

I'm happy to look at a minimal failing example.

  Joris

[mario.za...@gmail.com]

Hi Joris,

I don't know what I was doing wrong, now it seems to work.

Thanks a lot for your help!

I was wondering, I saw that there is the possibility to pass "simplify_options" as a dict. What does that do?

One more thing, since acados integrators are extremely efficient, is there any plan to make them available in casadi?

In my opinion that would make multiple shooting in casadi extremely appealing. Additionally, as far as I understood, they are generated using casadi expressions, so maybe interfacing them is not that hard?

Ciao,

Mario

[mario.za...@gmail.com]

One more thing,

once I have generated and compiled the code, if I close the shell and call the same code again, it will generate and compile the code once more. Is there a way to avoid that? This would be very practical, e.g., if I change the OCP formulation but not the integrator, so that the integrator code does not need to be generated and compiled again.

Thanks a lot in advance!

Ciao,

Mario

[Joris Gillis]

Dear Mario,

That's not currently possible with CasADi since we throw all generated functions into a single c file.

We'd need to break it down into several files that can be independently cached.

There's no fundamental stumbling block in the way for implementing, mostly work to be done.

If nothing changed in your formulation, you can profit from caching right now with jit_options.compiler = 'ccache gcc'

ccache is a well-known cached compiler which you should independently install.

You need to combine that with jit_options.temp_suffix  = false, to avoid random file names that induce a cache miss for default ccache settings.

Best regards,

  Joris