The Hopfield neural networks are well suited to solving large-scale optimization problems, but their convergence characteristics are not theoretically known The author clarifies, by an eigenvalue analysis, conditions for converging to a vertex, a point on the edge, or an interior point of the hypercube Taking the traveling salesman problem as an example, the author shows how to determine the weighting factors of the constraints and the objective function in the energy Numerical calculations demonstrate that optimal or near-optimal solutions are obtained for 6, 10, and 13 cities >

Theories on the Hopfield neural networks

Assuming that the output function of neurons is monotonic and differentiable at any interior point in the output range, the condition necessary for a vertex of a hypercube to become a local minimum of the Hopfield neural networks and the form of the convergence region to that minimum are clarified. Based on this, a method for analyzing and suppressing spurious states in the networks is derived. It is shown that all the spurious states of the traveling salesman problem (TSP) for the Hopfield original energy function can be suppressed by the method, and the validity of the method is demonstrated by computer simulations. >

/pdf/global-convergence-and-suppression-of-spurious-states-of-the-2xce8243cs.pdf

Global convergence and suppression of spurious states of the Hopfield neural networks

Due to the rugged energy function of the original Hopfield networks, the output is usually one local minimum in the energy function. An analysis on the locations of local minima in Hopfield networks is presented, and a modified network architecture to eliminate such local minima is described. In particular, another amplifier is introduced at the processor nodes to give correction terms. This modified Hopfield network has been successfully applied to the construction of analog-to-digital converters with optimal solutions. Experimental results on the voltage transfer characteristics of data converters are presented. >

Modified Hopfield neural networks for retrieving the optimal solution

Three basic neural network schemes have been extensively studied by researchers: the iterative networks, the backpropagation networks, and the self-organizing networks. Simulated annealing is a probabilistic hill-climbing technique that accepts, with a nonzero but gradually decreasing probability, deterioration in the cost function of the optimization problems. Hardware annealing, which combines the simulated annealing technique with continuous-time electronic neural networks by changing the voltage gain of neurons, is discussed. The initial and final voltage gains for applying hardware annealing to Hopfield data-conversion networks are presented. In hardware annealing, the voltage gain of output neurons is increased from an initial low value to a final high value in a continuous fashion which helps to achieve the optimal solution for an optimization problem in one annealing cycle. Experimental results on the transfer function and transient response of electronic neural networks achieving the global minimum are also presented. >

Paralleled hardware annealing for optimal solutions on electronic neural networks

A simulated hardware annealing process for electronic neural circuits is derived from the analogy between the temperature in a Boltzmann machine and the amplifier gain in a VLSI chip. Here, varying the amplifier gain is equivalent to changing the temperature of the probability function in a Boltzmann machine. Decrease in the amplifier voltage gain is equivalent to temperature increase. The beginning and final annealing temperatures for the hardware annealing can be precisely determined. Theory and experimental results on a 4-b Hopfield analog-to-digital converters with simulated annealing are presented. >

Hardware annealing in electronic neural networks

The Hopfield architecture can be utilized in the VLSI implementation of several important optimization functions. A description is given of the properties of local minima in the energy function of Hopfield networks. A novel design technique to eliminate such local minima has been developed. The neural-based analog-to-digital converter is used as an example to demonstrate this design technique. Experimental results agree well with theoretical calculations on the output characteristics of the analog-to-digital converter. >

Lee

Papers

An investigation on local minima of a Hopfield network for optimization circuits