This tutorial provides an overview of the basic theory of hidden Markov models (HMMs) as originated by L.E. Baum and T. Petrie (1966) and gives practical details on methods of implementation of the theory along with a description of selected applications of the theory to distinct problems in speech recognition. Results from a number of original sources are combined to provide a single source of acquiring the background required to pursue further this area of research. The author first reviews the theory of discrete Markov chains and shows how the concept of hidden states, where the observation is a probabilistic function of the state, can be used effectively. The theory is illustrated with two simple examples, namely coin-tossing, and the classic balls-in-urns system. Three fundamental problems of HMMs are noted and several practical techniques for solving these problems are given. The various types of HMMs that have been studied, including ergodic as well as left-right models, are described. >

/pdf/a-tutorial-on-hidden-markov-models-and-selected-applications-1vlzbxbk68.pdf

A tutorial on hidden Markov models and selected applications in speech recognition

From the Publisher:
This is the first comprehensive treatment of feed-forward neural networks from the perspective of statistical pattern recognition. After introducing the basic concepts, the book examines techniques for modelling probability density functions and the properties and merits of the multi-layer perceptron and radial basis function network models. Also covered are various forms of error functions, principal algorithms for error function minimalization, learning and generalization in neural networks, and Bayesian techniques and their applications. Designed as a text, with over 100 exercises, this fully up-to-date work will benefit anyone involved in the fields of neural computation and pattern recognition.

/pdf/neural-networks-for-pattern-recognition-34puigiau1.pdf

Neural networks for pattern recognition

Publisher Summary This chapter provides an account of different neural network architectures for pattern recognition. A neural network consists of several simple processing elements called neurons. Each neuron is connected to some other neurons and possibly to the input nodes. Neural networks provide a simple computing paradigm to perform complex recognition tasks in real time. The chapter categorizes neural networks into three types: single-layer networks, multilayer feedforward networks, and feedback networks. It discusses the gradient descent and the relaxation method as the two underlying mathematical themes for deriving learning algorithms. A lot of research activity is centered on learning algorithms because of their fundamental importance in neural networks. The chapter discusses two important directions of research to improve learning algorithms: the dynamic node generation, which is used by the cascade correlation algorithm; and designing learning algorithms where the choice of parameters is not an issue. It closes with the discussion of performance and implementation issues.

Neural Networks for Pattern Recognition

In this paper we demonstrate that finite linear combinations of compositions of a fixed, univariate function and a set of affine functionals can uniformly approximate any continuous function ofn real variables with support in the unit hypercube; only mild conditions are imposed on the univariate function. Our results settle an open question about representability in the class of single hidden layer neural networks. In particular, we show that arbitrary decision regions can be arbitrarily well approximated by continuous feedforward neural networks with only a single internal, hidden layer and any continuous sigmoidal nonlinearity. The paper discusses approximation properties of other possible types of nonlinearities that might be implemented by artificial neural networks.

/pdf/approximation-by-superpositions-of-a-sigmoidal-function-1bvfeb1vf2.pdf

Approximation by superpositions of a sigmoidal function

In the first part of this paper, a regular recurrent neural network (RNN) is extended to a bidirectional recurrent neural network (BRNN). The BRNN can be trained without the limitation of using input information just up to a preset future frame. This is accomplished by training it simultaneously in positive and negative time direction. Structure and training procedure of the proposed network are explained. In regression and classification experiments on artificial data, the proposed structure gives better results than other approaches. For real data, classification experiments for phonemes from the TIMIT database show the same tendency. In the second part of this paper, it is shown how the proposed bidirectional structure can be easily modified to allow efficient estimation of the conditional posterior probability of complete symbol sequences without making any explicit assumption about the shape of the distribution. For this part, experiments on real data are reported.

/pdf/bidirectional-recurrent-neural-networks-3f0wsmrju7.pdf

Bidirectional recurrent neural networks

A high-performance low-complexity neural network wordspotter was developed using radial basis function (RBF) neutral networks in a hidden Markov model (HMM) framework. Two new complementary approaches substantially improve performance on the talker-independent Switchboard corpus. Figure of merit (FOM) training adapts wordspotter parameters to directly improve the FOM performance metric, and voice transformations generate additional training examples by warping the spectra of training data to mimic across-talker vocal tract length variability.

High-performance low-complexity wordspotting using neural networks

Artificial neural networks are of interest for two main reasons. First, they provide architectures to implement many algorithms used in speech rccognizcrs with fine grain massive parallelism. Second, they are leading to new computational algorithms and new approaches to speech recognition inspired by biological nervous systems. Neural net approaches to the problems of speech preprocessing, pattern classification, and time alignment are reviewed. Preprocessors using auditory nerve time‐synchrony models have provided improved recognition performance in noise [O. Ghitza, ICASSP 87, 2372–2375]. Highly parallel neural net architectures exist to implement many important traditional classification algorithms, such as k‐nearest neighbor and Gaussian classifiers [R. Lippmann, IEEE ASSP Mag. 4(2), 4–22 (1987)]. Newer multilayer perceptron classifiers trained with back propagation can form arbitrary decision regions, are robust, and train rapidly for convex decision regions. These nets performed as well as conventio...

Neural nets for speech recognition

Proceedings of the 10th international conference on Recent advances in intrusion detection

Host-Based Intrusion Detection.- Exploiting Execution Context for the Detection of Anomalous System Calls.- Understanding Precision in Host Based Intrusion Detection.- Anomaly-Based Intrusion Detection.- Comparing Anomaly Detection Techniques for HTTP.- Swaddler: An Approach for the Anomaly-Based Detection of State Violations in Web Applications.- Network-Based Intrusion Detection and Response.- Emulation-Based Detection of Non-self-contained Polymorphic Shellcode.- The NIDS Cluster: Scalable, Stateful Network Intrusion Detection on Commodity Hardware.- Cost-Sensitive Intrusion Responses for Mobile Ad Hoc Networks.- Insider Detection and Alert Correlation.- elicit: A System for Detecting Insiders Who Violate Need-to-Know.- On the Use of Different Statistical Tests for Alert Correlation - Short Paper.- Malicious Code Analysis.- Automated Classification and Analysis of Internet Malware.- "Out-of-the-Box" Monitoring of VM-Based High-Interaction Honeypots.- A Forced Sampled Execution Approach to Kernel Rootkit Identification.- Evasion.- Advanced Allergy Attacks: Does a Corpus Really Help?.- Alert Verification Evasion Through Server Response Forging.- Malicious Code Defense.- Hit-List Worm Detection and Bot Identification in Large Networks Using Protocol Graphs.- SpyShield: Preserving Privacy from Spy Add-Ons.- Vortex: Enabling Cooperative Selective Wormholing for Network Security Systems.

Recent advances in intrusion detection : 10th International Symposium, RAID 2007 Gold Coast, Australia, September 5-7, 2007 : proceedings

A physiological front-end preprocessor for speech recognition was evaluated using a large isolated-word database in quiet and noise. The front-end was based on the ensemble interval histogram (EIH) model developed by O. Ghitza. This model provides phase or synchrony information similar to that available on the auditory nerve. A modified EIH front-end was implemented and was tested using the Lincoln robust hidden Markov model isolated-word recognizer with a multistyle database at various signal-to-noise ratios (SNRs). The modified EIH front-end performed as well as a conventional mel-filter-bank front-end for normal speech. It provided a slight improvement in error rate at very low SNRs but required substantially more computation than the mel-filter-bank front-end

Richard P. Lippmann

Papers

High-performance low-complexity wordspotting using neural networks

Neural nets for speech recognition

Proceedings of the 10th international conference on Recent advances in intrusion detection

Recent advances in intrusion detection : 10th International Symposium, RAID 2007 Gold Coast, Australia, September 5-7, 2007 : proceedings

A physiologically motivated front-end for speech recognition