In September of 1976, four speech understanding systems were demonstrated, signifying the end of a five‐year program of research and development sponsored by the Advanced Research Projects Agency (ARPA). The best performance was displayed by the Harpy system developed at Carnegie–Mellon University. Harpy satisfied a set of design goals that were specified at the beginning of the program, including the gal of understanding over 90% of a set of naturally spoken sentences composed from a 1000‐word lexicon. After defining the nature of the speech understanding problem, the four systems are described and critically evaluated. Based on this review, a structure for a next‐generation speech understanding system is proposed and parts of it are considered as a possible model of the early stages of speech perception. The perceptual model addresses the issue of lexical access and includes a decoding network composed of expected spectral sequences for all word strings of English.

Review of the ARPA speech understanding project

A recognizer of meaningful sequences of isolated words spoken in the Italian language is presented. The system has been designed in a modular fashion and the current version has been obtained by adding the syntactic and semantic levels to the isolated-word recognizer previously developed by the same authors. The lexical, syntactic and semantic levels are integrated into a hierarchical knowledge source represented by a transition network grammar. The parsing is directed by the control unit and is performed by means of a combined strategy of bottom-up and top-down techniques, using a modified version of the Earley algorithm. Lexical hypotheses are verified at the phonemic level assigning them a score based on a distance measure evaluated by an error-correcting parsing algorithm. The protocol used in the preliminary experi - ments is a robot command language.

Syntax and semantics in a word-sequence recognition system

In this paper, we propose a direct method for speech rate estimation from acoustic features without requiring any automatic speech transcription. We compare various spectral and temporal signal analysis and smoothing strategies to better characterize the underlying syllable structure to derive speech rate. The proposed algorithm extends the methods of spectral sub- band correlation by including temporal correlation and the use of prominent spectral subbands for improving the signal correlation essential for syllable detection. Furthermore, to address some of the practical robustness issues in previously proposed methods, we introduce some novel components into the algorithm such as the use of pitch confidence for filtering spurious syllable envelope peaks, magnifying window for tackling neighboring syllable smearing, and relative peak measure thresholds for pseudo peak rejection. We also describe an automated approach for learning algorithm parameters from data, and find the optimal settings through Monte Carlo simulations and parameter sensitivity analysis. Final experimental evaluations are conducted based on a portion of the Switchboard corpus for which manual phonetic segmentation information, and published results for direct comparison are available. The results show a correlation coefficient of 0.745 with respect to the ground truth based on manual segmentation. This result is about a 17% improvement compared to the current best single estimator and a 11% improvement over the multiestimator evaluated on the same Switchboard database.

/pdf/robust-speech-rate-estimation-for-spontaneous-speech-4mnjfk13tq.pdf

Robust Speech Rate Estimation for Spontaneous Speech

The paper continues the authors' investigation of machines that adaptively acquire language through interaction with a complex environment. In particular, the present work focuses on the problem of spoken word acquisition, using the authors' proposed principles to motivate a method to govern the emergence of word symbols from the speech signal. The mechanism involves a connectionist network embedded in a feedback control system. The resulting system has two unique characteristics. First, no text is utilized by the device, in contrast to all other speech understanding systems. Second, the vocabulary and grammar is unconstrained, being acquired by the device during the course of performing its task. This is also in contrast to all other systems, in which the salient vocabulary and grammar are preprogrammed. A rudimentary baseline experiment is described, involving 1105 natural language utterances in an automated call routing application scenario. >

An experiment in spoken language acquisition

Lexical Access From Features (LAFF) is a proposed knowledge-based speech recognition system which uses landmarks to guide the search for distinctive features. The first stage in LAFF must find Vowel landmarks. This task is similar to automatic detection of syllable nuclei (ASD). 
This thesis adapts and extends ASD algorithms for Vowel landmark detection. In addition to existing work on ASD, the acoustic theory of speech production was used to predict characteristics of vowels, and studies were done on a speech database to test the predictions. The resulting data guided the development of an improved Vowel landmark detector (VLD). 
Studies of the TIMIT database showed that about 94% of vowels have a peak of energy in the F1 region, and that about 89% of vowels have a peak in F1 frequency. Energy and frequency peaks were fairly highly correlated, with both peaks tending to appear before the midpoint of the vowel duration (as labeled), and frequency peaks tending to appear before energy peaks. Landmark based vowel classification was not found to be sensitive to the precise location of the landmark. Energy in a fixed frequency band (300 to 900 Hz) was found to be as good for finding landmarks as the energy at F1, enabling a simple design for a VLD without the complexity of formant tracking. 
The VLD was based on a peak picking technique, using a recursive convex hull algorithm. Three acoustic cues (peak-to-dip depth, duration, and level) were combined using a multilayer perceptron with two hidden units. The perceptron was trained by matching landmarks to syllabic nuclei derived from the TIMIT aligned phonetic transcription. Pairs of abutting vowels were allowed to match either one or two landmarks without penalty. The perceptron was trained first by back propagation using mean squared error, and then by gradient descent using error rate. The final VLD's error rate was about 12%, with about 3.5% insertions and 8.5% deletions, which compares favorably to the 6% of vowels without peaks. Most errors occurred in predictable circumstances, such as high vowels adjacent to semivowels, or very reduced schwas. Further work should include improvements to the output confidence score, and error correction as part of vowel quality detection. (Copies available exclusively from MIT Libraries, Rm. 14-0551, Cambridge, MA 02139-4307. Ph. 617-253-5668; Fax 617-253-1690.)

Automatic syllable detection for vowel landmarks

Sperry Univac is developing a linguistically oriented computer system which recognizes natural spoken phrases and sentences without requiring extensive adjustments for individual users and without the need for each user to repeat every vocabulary word for system training. The recognition system produces a linguistic description of the unknown utterance by determining its component sound segments. Words from a dictionary, or lexicon, (represented in terms of the same linguistic segments) are then strung together and compared to the analysis segments to determine which syntactically allowed sequence of vocabulary words best matches the unknown utterance. Two data bases have been used recently for development and testing, including alphanumeric sequences constructed from a vocabulary of 36 words, and data management commands from a vocabulary of 64 words. For both data bases, average correct recognition scores were 95% for individual words, and 88% for complete sentences. Current plans for enhancing the recognition system include enlarging the vocabulary and number of recognizable sentence types, making additional use of prosodic information, and introducing phonetic verification rules and a word verification component. The ultimate goal of this development is to provide an effective sentence recognition system for natural and practical speech input to computers.

http://ieeexplore.ieee.org/iel6/8365/26347/01170372.pdf

A system for the recognition of spoken connected word sequences

Sperry Univac is developing a linguistically oriented system for locating important words in conversational speech. The system uses acoustic, prosodic, and phonetic analyses to produce a phonetic description of the incoming speech. Next, phonetic dictionary representations of the keywords to be found are compared to all portions of the phonetic analysis. High scoring matches are then verified by aligning prestored spectral patterns with the spectral information found during analysis, and resulting good matches are announced as likely keyword occurrences. Current results are presented for this system, which is being developed and tested on bandlimited, conversational speech from a large, diverse speaker population.

An automatic word spotting system for conversational speech

The Speech Communications Group at SPERRY UNIVAC Defense Systems is developing a linguistically-oriented procedure for recognizing words, phrases, and natural sentences by computer. The major components of the current speech recognition system perform acoustic and phonetic analysis, phonetic segmentation, and lexical matching and scoring. The acoustic processing is based on a linear-predictive spectral analysis of the speech signal. Sounds are classified by manner, place, and voicing using formant frequencies and other spectral functions, as well as information about syllable boundaries and nuclei. A linear sequence of analysis segments is created, and matched against the lexicon using a scoring matrix that ranks analysis-lexical segment pairs by their expected confusions. Word sequences are progressively formed and ranked against the entire input to determine the most likely phrases spoken. When the recognition system was tested on a 31-word vocabulary from two male speakers, single word recognition scores of 95% correct were obtained when the task syntax was used. Preliminary results for recognizing connected word sequences from three male speakers range from 54 to 74% for a task with constrained word order. Current plans for enhancing the recognition system include the incorporation of components for phonological rules, speaker normalization, and prosodic guidelines. By adding more powerful procedures for syntactic and semantic analysis, the system will be extended from the recognition of several-word noun phrases to the understanding of more natural sentences.

A speech recognition system for connected word sequences

Sperry Univac is developing a linguistically oriented system for locating key information-carrying words in conversational speech. The system first analyzes the speech acoustically, providing information on bandlimited energies and fundamental and formant frequencies. A prosodic analysis then indicates the location of syllables. Finally, a phonetic word verification technique is exercised on each syllable. Prestored acoustic properties characteristic of each phonetic segment of each keyword are compared against the incoming acoustic properties found at appropriate locations anchored within and around the syllables. Acoustic-phonetic parameter alignments exhibiting great similarity are announced as likely keyword occurrences. Current results are presented for this system, which is being developed and tested on noisy, bandlimited, conversational speech from a large and diverse speaker population.

D. Kloker

Papers

A system for the recognition of spoken connected word sequences

An automatic word spotting system for conversational speech

A speech recognition system for connected word sequences

Word spotting in conversational speech