A comparison of adaptive algorithms based on the methods of steepest descent and random search

TLDR: This paper compares the performance characteristics of three algorithms useful in adjusting the parameters of adaptive systems: the differential (DSD) and least-mean-square (LMS) algorithms, both based on the method of steepest descent, and the linear random search (LRS) algorithm, based on a random search procedure derived from the Darwinian concept of "natural selection.

Abstract: This paper compares the performance characteristics of three algorithms useful in adjusting the parameters of adaptive systems: the differential (DSD) and least-mean-square (LMS) algorithms, both based on the method of steepest descent, and the linear random search (LRS) algorithm, based on a random search procedure derived from the Darwinian concept of "natural selection." The LRS algorithm is presented here for the first time. Analytical expressions are developed that define the relationship between rate of adaptation and "misadjustment," a dimensionless measure of the difference between actual and optimal performance due to noise in the adaptive process. For a fixed rate of adaptation it is shown that the LMS algorithm, which is the most efficient, has a misadjustment proportional to the number of adaptive parameters, while the DSD and LRS algorithms have misadjustments proportional to the square of the number of adaptive parameters. The expressions developed are verified by computer simulations that demonstrate the application of the three algorithms to system modeling problems, of the LMS algorithm to the cancelling of broadband interference in the sidelobes of a receiving antenna array, and of the DSD and LRS algorithms to the phase control of a transmitting antenna array. The second application introduces a new method of constrained adaptive beamforming whose performance is not significantly affected by element nonuniformity. The third application represents a class of problems to which the LMS algorithm in the basic form described in this paper is not applicable.