AIB 2014-07: Algorithmic Differentiation of Numerical Methods: Second-Order Tangent and Adjoint Solvers for Systems of Parametrized Nonlinear Equations

[Thomas Ströder]

The following technical report is available from
http://aib.informatik.rwth-aachen.de:

Algorithmic Differentiation of Numerical Methods: Second-Order Tangent
and Adjoint Solvers for Systems of Parametrized Nonlinear Equations
Niloofar Safiran, Johannes Lotz, and Uwe Naumann
AIB 2014-07

Forward and reverse modes of algorithmic differentiation (AD) transform
implementations of multivariate vector functions as computer programs
into tangent and adjoint code, respectively. The reapplication of the
same ideas yields higher derivative code. In particular, second
derivatives play an important role in nonlinear programming.
Second-order methods based on Newton's algorithm promise faster
convergence in the neighbourhood of the minimum by taking into account
second derivative information. The adjoint mode is of particular
interest in large-scale nonlinear optimization due to the independence
of its computational cost on the number of free variables. Solvers for
parametrized system of n equations embedded into evaluation of objective
function for a (without loss of generality) unconstrained nonlinear
optimization problem. Require Hessian of objective with respect to free
variables implying need for second derivatives of the nonlinear solver.
The local computational overhead as well as the additional memory
requirement for the computation of second-order tangents/adjoints of the
solution vector with respect to parameters by a fully discrete method
(derived by AD) can quickly become prohibitive for large values of n.
Both can be reduced extremely by the second-order continuous approach to
differentiation of the underlying numerical method to be discussed in
this paper.