This study was designed as a replication and extension of the classic study of vowel acoustics by Peterson and Barney (PB) [J. Acoust. Soc. Am. 24, 175–184 (1952)]. Recordings were made of 50 men, 50 women, and 50 children producing the vowels /i, i, eh, ae, hooked backward eh, inverted vee), a, open oh, u, u/ in h–V–d syllables. Formant contours for F1–F4 were measured from LPC spectra using a custom interactive editing tool. For comparison with the PB data, formant patterns were sampled at a time that was judged by visual inspection to be maximally steady. Preliminary analysis shows numerous differences between the present data and those of PB, both in terms of average formant frequencies for vowels, and the degree of overlap among adjacent vowels. As with the original study, listening tests showed that the signals were nearly always identified as the vowel intended by the talker.

Acoustic characteristics of American English vowels

The temporal properties of speech appear to play a more important role in linguistic contrasts than has hitherto been appreciated. Therefore, a new framework for describing the acoustic structure of speech based purely on temporal aspects has been developed. From this point of view, speech can be said to be comprised of three main temporal features, based on dominant fluctuation rates: envelope, periodicity, and fine-structure. Each feature has distinct acoustic manifestations, auditory and perceptual correlates, and roles in linguistic contrasts. The applicability of this three-featured temporal system is discussed in relation to hearing-impaired and normal listeners.

Temporal Information in Speech: Acoustic, Auditory and Linguistic Aspects

Speech separation is the task of separating target speech from background interference. Traditionally, speech separation is studied as a signal processing problem. A more recent approach formulates speech separation as a supervised learning problem, where the discriminative patterns of speech, speakers, and background noise are learned from training data. Over the past decade, many supervised separation algorithms have been put forward. In particular, the recent introduction of deep learning to supervised speech separation has dramatically accelerated progress and boosted separation performance. This paper provides a comprehensive overview of the research on deep learning based supervised speech separation in the last several years. We first introduce the background of speech separation and the formulation of supervised separation. Then, we discuss three main components of supervised separation: learning machines, training targets, and acoustic features. Much of the overview is on separation algorithms where we review monaural methods, including speech enhancement (speech-nonspeech separation), speaker separation (multitalker separation), and speech dereverberation, as well as multimicrophone techniques. The important issue of generalization, unique to supervised learning, is discussed. This overview provides a historical perspective on how advances are made. In addition, we discuss a number of conceptual issues, including what constitutes the target source.

Supervised Speech Separation Based on Deep Learning: An Overview

The effect of smearing the temporal envelope on the speech-reception threshold (SRT) for sentences in noise and on phoneme identification was investigated for normal-hearing listeners. For this purpose, the speech signal was split up into a series of frequency bands (width of 1/4, 1/2, or 1 oct) and the amplitude envelope for each band was low-pass filtered at cutoff frequencies of 0, 1/2, 1, 2, 4, 8, 16, 32, or 64 Hz. Results for 36 subjects show (1) a severe reduction in sentence intelligibility for narrow processing bands at low cutoff frequencies (0-2 Hz); and (2) a marginal contribution of modulation frequencies above 16 Hz to the intelligibility of sentences (provided that lower modulation frequencies are completely present). For cutoff frequencies above 4 Hz, the SRT appears to be independent of the frequency bandwidth upon which envelope filtering takes place. Vowel and consonant identification with nonsense syllables were studied for cutoff frequencies of 0, 2, 4, 8, or 16 Hz in 1/4-oct bands. Results for 24 subjects indicate that consonants are more affected than vowels. Errors in vowel identification mainly consist of reduced recognition of diphthongs and of confusions between long and short vowels. In case of consonant recognition, stops appear to suffer most, with confusion patterns depending on the position in the syllable (initial, medial, or final).

Effect of temporal envelope smearing on speech reception

The yield of alkenols and cycloalkenols is substantially improved by carrying out the reaction of olefins with formaldehyde in the presence of selected catalysts. In accordance with one embodiment, alk-3-en-1-ols are produced in good yields from isobutylene and formaldehyde in the presence of organic carboxylic acid salts of Group IB metals.

The Organization of the Cochlear Receptor

If two vowels with different fundamental frequencies ( f0’s) are presented simultaneously and monaurally, listeners often hear two talkers producing different vowels on different pitches. This paper describes the evaluation of four computational models of the auditory and perceptual processes which may underlie this ability. Each model involves four stages: (i) frequency analysis using an ‘‘auditory’’ filter bank, (ii) determination of the pitches present in the stimulus, (iii) segregation of the competing speech sources by grouping energy associated with each pitch to create two derived spectral patterns, and (iv) classification of the derived spectral patterns to predict the probabilities of listeners’ vowel‐identification responses. The ‘‘place’’ models carry out the operations of pitch determination and spectral segregation by analyzing the distribution of rms levels across the channels of the filter bank. The ‘‘place–time’’ models carry out these operations by analyzing the periodicities in the wavef...

Modeling the perception of concurrent vowels: vowels with different fundamental frequencies.

Speech recognition performance was measured in normal-hearing and cochlear-implant listeners with maskers consisting of either steady-state speech-spectrum-shaped noise or a competing sentence. Target sentences from a male talker were presented in the presence of one of three competing talkers (same male, different male, or female) or speech-spectrum-shaped noise generated from this talker at several target-to-masker ratios. For the normal-hearing listeners, target-masker combinations were processed through a noise-excited vocoder designed to simulate a cochlear implant. With unprocessed stimuli, a normal-hearing control group maintained high levels of intelligibility down to target-to-masker ratios as low as 0 dB and showed a release from masking, producing better performance with single-talker maskers than with steady-state noise. In contrast, no masking release was observed in either implant or normal-hearing subjects listening through an implant simulation. The performance of the simulation and implant groups did not improve when the single-talker masker was a different talker compared to the same talker as the target speech, as was found in the normal-hearing control. These results are interpreted as evidence for a significant role of informational masking and modulation interference in cochlear implant speech recognition with fluctuating maskers. This informational masking may originate from increased target-masker similarity when spectral resolution is reduced.

Cochlear implant speech recognition with speech maskers.

Statistical analysis of F1 and F2 measurements from nucleus and offglide sections of isolated Canadian English vowels shows significant formant frequency change not only for the ‘‘phonetic diphthongs’’ /e/ and /o/, but also for the ‘‘monophthongs’’ /ι/, /q/, and /1/. In a perceptual experiment, brief sections were extracted from ‘‘nucleus’’ and ‘‘offglide’’ portions of naturally produced vowels. Two sections from each vowel were presented to listeners in each of three conditions: (1) natural order (nucleus followed by offglide); (2) repeated nucleus (nucleus followed by itself); and (3) reverse (offglide followed by nucleus). Listeners’ error rates for the natural order condition were comparable to those for unmodified full vowels (averaging 14% and 13%, respectively). Significantly higher error rates were found for the repeated nucleus (32%) and reverse (38%) conditions. Observed confusion matrices were strongly correlated with predictions from a pattern recognition model incorporating the formant measur...

Modeling the role of inherent spectral change in vowel identification

Speech is the primary vehicle of human social interaction. In everyday life, speech communication occurs under an enormous range of different environmental conditions. The demands placed on the process of speech communication are great, but nonetheless it is generally successful. Powerful selection pressures have operated to maximize its effectiveness. The adaptability of speech is illustrated most clearly in its resistance to distortion. In transit from speaker to listener, speech signals are often altered by background noise and other interfering signals, such as reverberation, as well as by imperfections of the frequency or temporal response of the communication channel. Adaptations for robust speech transmission include adjustments in articulation to offset the deleterious effects of noise and interference (Lombard 1911; Lane and Tranel 1971); efficient acousticphonetic coupling, which allows evidence of linguistic units to be conveyed in parallel (Hockett 1955; Liberman et al. 1967; Greenberg 1996; see Diehl and Lindblom, Chapter 3); and specializations of auditory perception and selective attention (Darwin and Carlyon 1995). Speech is a highly efficient and robust medium for conveying information under adverse conditions because it combines strategic forms of redundancy to minimize the loss of information. Coker and Umeda (1974, p. 349) define redundancy as “any characteristic of the language that forces spoken messages to have, on average, more basic elements per message, or more cues per basic element, than the barest minimum [necessary for conveying the linguistic message].” This definition does not address the function of redundancy in speech communication, however. Coker and Umeda note that “redundancy can be used effectively; or it can be squandered on uneven repetition of certain data, leaving other crucial items very vulnerable to noise. . . . But more likely, if a redundancy is a property of a language and has to be learned, then it has a purpose.” Coker and Umeda conclude that the purpose of redundancy in speech communication is to provide a basis for error correction and resistance to noise.

The Perception of Speech Under Adverse Conditions

This study investigates conditions under which vowels are well recognized and relates perceptual identification of individual tokens to acoustic characteristics. Results support recent findings that isolated vowels may be readily identified by listeners. Two experiments provided evidence that certain response tasks result in inflated error rates. Subsequent experiments showed improved identification in a fixed speaker context, compared with randomized speakers, for isolated vowels and gated centers. Performance was worse for gated vowels, suggesting that dynamic properties (such as duration and diphthongization) supplement steady‐state cues. However, even‐speaker‐randomized gated vowels were well identified (14% errors). Measures of ’’steady‐state information’’ (formant frequencies and f0), ’’dynamic information’’ (formant slopes and duration), and ’’speaker information’’ (normalization) were adopted. Discriminant analyses of acoustic measurements indicated relatively little overlap between vowel categories. Using a new technique for relating acoustic measurements of individual tokens with identification by listeners, it is shown that (a) identification performance is clearly related to acoustic characteristics; (b) improvement in the fixed speaker context is correlated with improved statistical separation resulting from formant normalization, for the gated vowels; and (c) ’’dynamic information’’ is related to identification differences between full and gated isolated vowels.

Peter F. Assmann

Papers

Modeling the perception of concurrent vowels: vowels with different fundamental frequencies.

Cochlear implant speech recognition with speech maskers.

Modeling the role of inherent spectral change in vowel identification

The Perception of Speech Under Adverse Conditions

Vowel identification: Orthographic, perceptual, and acoustic aspects