Large overhead of evaluating user-defined functions

[Truong Nghiem]

Hello,

I'm using CasADi 3.1-RC1 on Matlab 2015b and Mac OS.  A detailed explanation of my question is at the bottom.  The short version is as following.  I implemented several CasADi's user-defined functions by subclassing casadi.Callback.  It's necessary due to performance issues and that I need to supply user data to the functions.  I observed that, at least in Matlab, the overhead of evaluating a user-defined function is quite high, about 4ms on my system.  By profiling the code, I saw the majority of computation time was spent on CasADi's DM-related classes (e.g., DM.delete, DM.DM, GenDM, ExpDM, SpDM...).  That seems to me like a large overhead for a simple function evaluation.  I figure that by eliminating or reducing this overhead significantly, I may save 20-30% the time to solve my NLPs.  Is there a way to speed up the evaluation of user-defined functions in CasADi (in Matlab at least)?

Thanks.
Truong.

Detailed explanation of my question:
I'm working with NLPs that involve very large and complex nonlinear functions. Initially I used CasADi's symbolics to construct the nonlinear Functions, but it took over 2 minutes and gigabytes of memory for CasADi to construct each function, and the evaluation is not very fast (I believe due to the size of the functions). So I combined code generation and custom high-performance C code to implement these nonlinear functions in C, which are instantaneous to initialize and up to 10x faster to evaluate than CasADi.  I then implemented user-defined functions by subclassing casadi.Callback.  I had to do this rather than using external functions because I need to supply user data to my functions.  When using the user-defined functions, I observed large overhead in evaluating the functions as explained above.

[Joel Andersson]

Hi Truong,

First of all, it's good to understand what can can make CasADi functions slow to generate and compile. And memory-heavy to evaluate, especially if reverse mode AD is involved. When you create a CasADi expression, an operation will correspond to one or a couple of lines in the generated C file. For SX, it's a simple scalar operation and for MX it's a (scalar or vector or) matrix-valued operation. To avoid getting excessively large expressions, and hence excessively large number of lines of generated code, you need to structure your code into functions. A CasADi "Function" will correspond to a function in the generated code and can be called from other functions. Only the MX type supports function calls, so if you use SX, it will simply copy all of the lines of code into that location and you gain nothing.

Anyway, if you choose to go the route that you're doing now, and if speed is a concern, subclassing casadi.Callback is probably not the way to go. The most efficient way is probably to use external functions. You are right that external functions don't support user data, but it's not too hard to work around that. Just put the data in a text file and have it read in the first time the function is called. Your C code would contain something like:

#include <stdio.h>

...
static bool read_data = false;
static double data[1000];
...
int myfunction(...) {
 ...
 if (!read_data) {
  FILE* f = fopen ("mydata.txt","r");

  if (!f) return 1;
  for (int i=0; i<1000; ++i) {
    if (fscanf(f, "%lf", &data[i])==0) return 1;

  }
  fclose(f);

  read_data = true;
 }
 ...
}
...

Best regards,

Joel

[Truong Nghiem]

Hi Joel,

Thanks for your answer.  I will try the method you suggested. However, in my application, I also need to use multiple function instances of the same type but with different user data sets (in the same optimization problem).  I guess I may need to duplicate the source files, give them different names and compile them, then construct different casadi.external objects from them.

I also see the argument "mem" in the function evaluation, which is usually set to be 0, but I suspect it could be useful in my case to distinguish between different function instances.  Is there any document or example of using this "mem" argument, and how it is used by CasADi's external function?  Or does CasADi always call an external function with mem=0?

Best,
Truong

[Joel Andersson]

Hi!

CasADi function object are designed to be "pure", i.e. the outputs are uniquely determined by the inputs. Once you allow the function to depends on data, that no longer holds. If you do have data like that, the intended way is to pass it via the function inputs. If it's part of an optimization problem, the data sets could be parameters ("p" when constructing the NLP). Would that not work?

The "mem" parameter is intended to allow multithreaded/parallel evaluation, so it's not a replacement for this.

Joel