Optimization techniques and their application.

The problem of optimizing a nonlinear function of one or more variables in the sense of locating the values of the variables which give the greatest or least value of the function, is considered from two points of view. First, the development of two new and improved techniques for optimization is described. Second, the ways in which the available techniques can be applied are discussed with reference to case studies of practical significance. The two new techniques are for unconstrained optimization problems of a type which frequently occur in curve-fitting and modelling applications and also in the solution of sets of nonlinear equations. The first of these is a new two-part algorithm for minimizing a sum of squares objective function; it uses a new descent method in combination with a modified Gauss-Newton search to give an algorithm which has proved extremely reliable even when applied to difficult problems. The second technique is a hybrid algorithm for minimizing a sum of moduli objective function; it makes novel use of the methods of parametric linear programming. Ten case studies of the application of optimization techniques are described, ranging from problems involving a single variable up to a problem with several hundred variables. The areas from which the applications are drawn include biochemistry, engineering, statistics and theoretical physics; the problems themselves are mainly concerned with curve-fitting or the solution of nonlinear equations.