Example to use the Callback class to instantiate the objective of an NLP

[Andres Codas Duarte]

Hi,

   Thanks to continue the development and support of CasADi.

   I recently realized that CasADi 3.x was released and it came with the new Class Callback. I'm wondering if I can use Callback classes to instantiate NLP's.  I tried to build a simple example in python but I'm having some issues.  Could you point out what I'm doing wrong?  See the following code:

https://gist.github.com/anonymous/2553fb479570ca5d93ed829a101b1e74#file-ipoptexternal-py

from casadi import *

class MyCallback(Callback):
  def __init__(self, name, opts={}):
    Callback.__init__(self)

    # My plan is to implement this function externally.
    # For the sake of the example, lets use a simple MX    
    X = MX.sym('X',1,1)
    SQ = (X-5)**2    
    F = Function('Square',[X],[SQ],['x'],['y'])        
    self.F = F
   
    self.construct(name, opts)

  # Number of inputs and outputs
  def get_n_in(self): return 1
  def get_n_out(self): return 1

  # Initialize the object
  def init(self):
     print('Initializing object')

  def finalize(self):
     print('Finalizing object construction')    

  def has_jacobian(self): return True

  def get_jacobian(self):
      # I will also provide first order jacobians from external code
      return f.F.jacobian()

     
  # Evaluate numerically
  def eval(self, arg):
    x = arg[0]
    f = self.F(x)
    return [f]


x = MX.sym('x',1,1)
y = f(x)

# NLP
nlp = {'x':x, 'f':y}


# NLP solver options
opts = {}
opts["expand"] = True
opts["ipopt.max_iter"] = 1000
opts['ipopt.hessian_approximation'] = 'limited-memory'

# Allocate an NLP solver
solver = nlpsol("solver", "ipopt", nlp, opts)
arg = {}

# Initial condition
arg["x0"] = 10

# Bounds on x
arg["lbx"] = -100
arg["ubx"] = +100


# Solve the problem
res = solver(**arg)

# Print the optimal cost
print("optimal cost: ", float(res["f"]))

Thanks!
Andres

[Joel Andersson]

Hi Andres,
isn't there a typo where you wrote "f.F" instead of "self.F"? Joel

[Andres Codas Duarte]

Hi Joel,

   Thanks for your answer.

   My bad, I had a few typos, for some reason I submit the wrong code.  Here is my update, including your correction:

from casadi import *

class MyCallback(Callback):
  def __init__(self, name, opts={}):
    Callback.__init__(self)

    # My plan is to implement this function externally.
    # For the sake of the example, lets use a simple MX    
    X = MX.sym('X',1,1)
    SQ = (X-5)**2    
    F = Function('Square',[X],[SQ],['x'],['y'])        
    self.F = F
   
    self.construct(name, opts)

  # Number of inputs and outputs
  def get_n_in(self): return 1
  def get_n_out(self): return 1

  # Initialize the object
  def init(self):
     print('Initializing object')

  def finalize(self):
     print('Finalizing object construction')    

  def has_jacobian(self): return True

  def get_jacobian(self):

      # Will also provide first order jacobians from external code
      return self.F.jacobian()

     
  # Evaluate numerically
  def eval(self, arg):
    x = arg[0]
    f = self.F(x)
    return [f]

f = MyCallback('f')

x = MX.sym('x',1,1)
y = f(x)

# NLP
nlp = {'x':x, 'f':y}


# NLP solver options
opts = {}
opts["expand"] = True
opts["ipopt.max_iter"] = 1000
opts['ipopt.hessian_approximation'] = 'limited-memory'

# Allocate an NLP solver
solver = nlpsol("solver", "ipopt", nlp, opts)
arg = {}

# Initial condition
arg["x0"] = 10

# Bounds on x
arg["lbx"] = -100
arg["ubx"] = +100


# Solve the problem
res = solver(**arg)

# Print the optimal cost
print("optimal cost: ", float(res["f"]))

I get the error:
RuntimeError:  on line 1376 of file "/Users/travis/build/casadi/binaries/casadi/casadi/core/function/function_internal.cpp"
'eval_sx' not defined for N6casadi16CallbackInternalE

in the line:

solver = nlpsol("solver", "ipopt", nlp, opts)

Thanks!
Andres

[Joris Gillis]

Dear Andres,

One cannot perform an expand operation on a graph that contains a Callback; the Callback stands for an otherworthly chunk of code. From CasADi's point of view its a blackbox thing, expanding into scalars is impossible for this.
You still had two issues in your code:
  - get_jacobian is being called with more argument (check api docs)
  - The function that get_jacobian returns must have just one output (the jacobian). The (deprecated) F.jacobian syntax gives an object with 2 outputs.
    It is advised to work with expressions instead.

from casadi import *

class MyCallback(Callback):
  def __init__(self, name, opts={}):
    Callback.__init__(self)

    # My plan is to implement this function externally.
    # For the sake of the example, lets use a simple MX   
    X = MX.sym('X',1,1)
    SQ = (X-5)**2   
    F = Function('Square',[X],[SQ],['x'],['y'])       
    self.F = F

    self.X = X
    self.SQ = SQ

  
    self.construct(name, opts)

  # Number of inputs and outputs
  def get_n_in(self): return 1
  def get_n_out(self): return 1

  # Initialize the object
  def init(self):
     print('Initializing object')

  def finalize(self):
     print('Finalizing object construction')   

  def has_jacobian(self): return True

  def get_jacobian(self,name,options):

      # Will also provide first order jacobians from external code

      return Function('name',
        [self.X],
        [jacobian(self.SQ,self.X)],options)

    
  # Evaluate numerically
  def eval(self, arg):
    x = arg[0]
    f = self.F(x)

    print [x]

    return [f]



f = MyCallback('f')

x = MX.sym('x',1,1)
y = f(x)

print y

# NLP
nlp = {'x':x, 'f':y}


# NLP solver options
opts = {}

opts["ipopt.max_iter"] = 1000
opts['ipopt.hessian_approximation'] = 'limited-memory'

# Allocate an NLP solver
solver = nlpsol("solver", "ipopt", nlp, opts)
arg = {}

# Initial condition
arg["x0"] = 10

# Bounds on x
arg["lbx"] = -100
arg["ubx"] = +100


# Solve the problem
res = solver(**arg)

# Print the optimal cost
print("optimal cost: ", float(res["f"]))

Best regards,
  Joris

[Andres Codas Duarte]

Hi Joris,

   Thanks for your answer.

   Bottom line:  can I have a 'blackbox' node in the CasADi AD tree (or tape)?.  That is, I want to be able to interface CasADi to external functions.  To this end, I can interface the sparsity pattern, the function structure, function evaluations and Jacobian evaluations (including vector*Jacobian and Jacobian*vector) if necessary.  The code I provided was just a guess on how to do it.

   With this functionality I expect to be able to interface existing code, use it as a building block for other expressions, and let CasADi manage the Jacobian evaluations with graph coloring techniques and etc.  Is this on your road-map goals?

    Thanks,
    Andres

[Joel Andersson]

Hi!

Just to avoid misunderstanding. The error is due to your option:

opts["expand"] = True

This option is incompatible with the Callback class.

Just remove that line...

Joel

[Andres Codas Duarte]

Thanks Joel and Joris,

   This functionality, together with the capability of dealing with if_else statements, were in my wish-list for CasADi since 2012. Fantastic!

    Regards,
    Andres