Showing papers by "Lawrence K. Saul published in 1994"
Proceedings Article•
01 Jan 1994TL;DR: A statistical mechanical framework for the modeling of discrete time series is proposed, and maximum likelihood estimation is done via Boltzmann learning in one-dimensional networks with tied weights, which motivates new architectures that address particular shortcomings of HMMs.
Abstract: We propose a statistical mechanical framework for the modeling of discrete time series. Maximum likelihood estimation is done via Boltzmann learning in one-dimensional networks with tied weights. We call these networks Boltzmann chains and show that they contain hidden Markov models (HMMs) as a special case. Our framework also motivates new architectures that address particular shortcomings of HMMs. We look at two such architectures: parallel chains that model feature sets with disparate time scales, and looped networks that model long-term dependencies between hidden states. For these networks, we show how to implement the Boltzmann learning rule exactly, in polynomial time, without resort to simulated or mean-field annealing. The necessary computations are done by exact decimation procedures from statistical mechanics.
81 citations
TL;DR: In this article, an exact integer algorithm was proposed to compute the partition function of a two-dimensional ±J Ising spin glass, which takes as input a set of quenched random bonds on the square lattice and returns the density of states as a function of energy.
55 citations
TL;DR: A large family of Boltzmann machines that can be trained by standard gradient descent, which can have one or more layers of hidden units, with tree-like connectivity, are introduced.
Abstract: We introduce a large family of Boltzmann machines that can be trained by standard gradient descent. The networks can have one or more layers of hidden units, with tree-like connectivity. We show how to implement the supervised learning algorithm for these Boltzmann machines exactly, without resort to simulated or mean-field annealing. The stochastic averages that yield the gradients in weight space are computed by the technique of decimation. We present results on the problems of N-bit parity and the detection of hidden symmetries.
38 citations