The use of second derivatives in applied numerical optimisation.

Lofncal unconstrained numerical optimisation techniques are applied to a vast number of problems from various branches of engineering. The available methods may be divided into two classes: those that assume no special form for the objective function and those that require an objective function in the form of a sum of squares. While there exist a number of methods of the first class that use second derivatives, until now there has been a lack of second derivative methods of the second class. Although methods of the first class can be applied to an objective function in the form of a sum of squares, it is generally recognized that if the sum is zero at the solution methods of the second class exhibit better terminal convergence. This is demonstrated here using several examples, including a transistor model problem where the objective function is defined to be the sum of the squares of the residuals of a set of highly non-linear simultaneous equations. Problems of this type are prevalent in methods for the design of electrical circuits. The main objective of this research was to determine whether the use of second derivatives could be of benefit in the solution of these problems. A number of second derivative sum of squares optimisation algorithms were devised, investigated and assessed using the transistor model problem as a standard test case. The most successful methods were then incorporated into a program for the synthesis of three-terminal lumped linear networks comprising resistors and capacitors. The development of the algorithms and their performance on these and various other trials is described; based on the results obtained some conclusions are drawn regarding the areas where the new algorithms are likely to be of benefit.