Using cross-validation for reducing risk of non-conservative surrogates in design optimization

[Felipe A. C. Viana]

Dear all,

Here it is a reference on how to use cross-validation to design
conservative surrogates:

F. A. C. Viana, V. Picheny, and R.T. Haftka, "USING CROSS-VALIDATION
FOR REDUCING RISK OF NON-CONSERVATIVE SURROGATES IN DESIGN
OPTIMIZATION," in: Engineering Risk Control and Optimization
Conference, Gainesville, FL, USA, February 22-23, 2009.

Conservative prediction refers to calculations or approximations that
most of the time safely estimate the actual response of the system. In
this work we consider the error associated with the use of surrogates,
and the use of a safety margin to compensate conservatively for these
errors. We propose the use of cross-validation for estimating the
safety margin needed to obtain target conservativeness level
(percentage of safe predictions). Additionally, we also check how well
cross-validation errors can be used to select surrogates that can
achieve high conservativeness with minimal loss of accuracy. The
approach was tested on two algebraic examples for ten basic surrogates
including different instances of kriging, polynomial response surface,
radial basis neural networks and support vector regression surrogates.
For these examples we found that cross-validation is effective for
selecting the safety margin. We also found that cross-validation
errors allow us to select a surrogate with the best combination of
accuracy and conservativeness.


You can find more about it online:
http://fchegury.googlepages.com

All the best,
Felipe Viana