An algorithm is presented for the estimation of the fundamental frequency (F0) of speech or musical sounds. It is based on the well-known autocorrelation method with a number of modifications that combine to prevent errors. The algorithm has several desirable features. Error rates are about three times lower than the best competing methods, as evaluated over a database of speech recorded together with a laryngograph signal. There is no upper limit on the frequency search range, so the algorithm is suited for high-pitched voices and music. The algorithm is relatively simple and may be implemented efficiently and with low latency, and it involves few parameters that must be tuned. It is based on a signal model (periodic signal) that may be extended in several ways to handle various forms of aperiodicity that occur in particular applications. Finally, interesting parallels may be drawn with models of auditory processing.

/pdf/yin-a-fundamental-frequency-estimator-for-speech-and-music-53voam3nec.pdf

YIN, a fundamental frequency estimator for speech and music

Voice quality variations include a set of voicing sound source modifications ranging from laryngealized to normal to breathy phonation. Analysis of reiterant imitations of two sentences by ten female and six male talkers has shown that the potential acoustic cues to this type of voice quality variation include: (1) increases to the relative amplitude of the fundamental frequency component as open quotient increases; (2) increases to the amount of aspiration noise that replaces higher frequency harmonics as the arytenoids become more separated; (3) increases to lower formant bandwidths; and (4) introduction of extra pole zeros in the vocal-tract transfer function associated with tracheal coupling. Perceptual validation of the relative importance of these cues for signaling a breathy voice quality has been accomplished using a new voicing source model for synthesis of more natural male and female voices. The new formant synthesizer, KLSYN88, is fully documented here. Results of the perception study indicate that, contrary to previous research which emphasizes the importance of increased amplitude of the fundamental component, aspiration noise is perceptually most important. Without its presence, increases to the fundamental component may induce the sensation of nasality in a high-pitched voice. Further results of the acoustic analysis include the observations that: (1) over the course of a sentence, the acoustic manifestations of breathiness vary considerably--tending to increase for unstressed syllables, in utterance-final syllables, and at the margins of voiceless consonants; (2) on average, females are more breathy than males, but there are very large differences between subjects within each gender; (3) many utterances appear to end in a "breathy-laryngealized" type of vibration; and (4) diplophonic irregularities in the timing of glottal periods occur frequently, especially at the end of an utterance. Diplophonia and other deviations from perfect periodicity may be important aspects of naturalness in synthesis.

/pdf/analysis-synthesis-and-perception-of-voice-quality-1q0nbgfn1a.pdf

Analysis, synthesis, and perception of voice quality variations among female and male talkers

1. The temporal discharge patterns of auditory nerve fibers in Dial-anesthetized cats were studied in response to periodic complex acoustic waveforms that evoke pitches at their fundamental frequencies. Single-formant vowels, amplitude-modulated (AM) and quasi-frequency-modulated tones. AM noise, click trains, and other complex tones were utilized. Distributions of intervals between successive spikes ("1st-order intervals") and between both successive and nonsuccessive spikes ("all-order intervals") were computed from spike trains. Intervals from many fibers were pooled to estimate interspike interval distributions for the entire auditory nerve. Properties of these "pooled interspike interval distributions," such as the positions of interval peaks and their relative heights, were examined for correspondence to the psychophysical data on pitch frequency and pitch salience. 2. For a diverse set of complex stimuli and levels, the most frequent all-order interspike interval present in the pooled distribution corresponded to the pitch heard in psychophysical experiments. Pitch estimates based on pooled interval distributions (30-85 fibers, 100 stimulus presentations per fiber) were highly accurate (within 1%) for harmonic stimuli that produce strong pitches at 60 dB SPL. 3. Although the most frequent intervals in pooled all-order interval distributions were very stable with respect to sound intensity level (40, 60, and 80 dB total SPL), this was not necessarily the case for first-order interval distributions. Because the low pitches of complex tones are largely invariant with respect to level, pitches estimated from all-order interval distributions correspond better to perception. 4. Spectrally diverse stimuli that evoke similar low pitches produce pooled interval distributions with similar most-frequent intervals. This suggests that the pitch equivalence of these different stimuli could result from central auditory processing mechanisms that analyze interspike interval patterns. 5. Complex stimuli that evoke strong or "salient" pitches produce pooled interval distributions with high peak-to-mean ratios. Those stimuli that evoke weak pitches produce pooled interval distributions with low peak-to-mean ratios. 6. Pooled interspike interval distributions for stimuli consisting of low-frequency components generally resembled the short-time auto-correlation function of stimulus waveforms. Pooled interval distributions for stimuli consisting of high-frequency components resembled the short-time autocorrelation function of the waveform envelope. 7. Interval distributions in populations of neurons constitute a general, distributed means of encoding, transmitting, and representing information. Existence of a central processor capable of analyzing these interval patterns could provide a unified explanation for many different aspects of pitch perception.

Neural correlates of the pitch of complex tones. I. Pitch and pitch salience

A Sawtooth Waveform Inspired Pitch Estimator (SWIPE) has been developed for processing speech and music. SWIPE is shown to outperform existing algorithms on several publicly available speech/musical-instruments databases and a disordered speech database. SWIPE estimates the pitch as the fundamental frequency of the sawtooth waveform whose spectrum best matches the spectrum of the input signal. A decaying cosine kernel provides an extension to older frequency-based, sieve-type estimation algorithms by providing smooth peaks with decaying amplitudes to correlate with the harmonics of the signal. An improvement on the algorithm is achieved by using only the first and prime harmonics, which significantly reduces subharmonic errors commonly found in other pitch estimation algorithms.

/pdf/swipe-a-sawtooth-waveform-inspired-pitch-estimator-for-rx8xljapsb.pdf

Swipe: a sawtooth waveform inspired pitch estimator for speech and music

Speech is the primary vehicle of human social interaction. In everyday life, speech communication occurs under an enormous range of different environmental conditions. The demands placed on the process of speech communication are great, but nonetheless it is generally successful. Powerful selection pressures have operated to maximize its effectiveness. The adaptability of speech is illustrated most clearly in its resistance to distortion. In transit from speaker to listener, speech signals are often altered by background noise and other interfering signals, such as reverberation, as well as by imperfections of the frequency or temporal response of the communication channel. Adaptations for robust speech transmission include adjustments in articulation to offset the deleterious effects of noise and interference (Lombard 1911; Lane and Tranel 1971); efficient acousticphonetic coupling, which allows evidence of linguistic units to be conveyed in parallel (Hockett 1955; Liberman et al. 1967; Greenberg 1996; see Diehl and Lindblom, Chapter 3); and specializations of auditory perception and selective attention (Darwin and Carlyon 1995). Speech is a highly efficient and robust medium for conveying information under adverse conditions because it combines strategic forms of redundancy to minimize the loss of information. Coker and Umeda (1974, p. 349) define redundancy as “any characteristic of the language that forces spoken messages to have, on average, more basic elements per message, or more cues per basic element, than the barest minimum [necessary for conveying the linguistic message].” This definition does not address the function of redundancy in speech communication, however. Coker and Umeda note that “redundancy can be used effectively; or it can be squandered on uneven repetition of certain data, leaving other crucial items very vulnerable to noise. . . . But more likely, if a redundancy is a property of a language and has to be learned, then it has a purpose.” Coker and Umeda conclude that the purpose of redundancy in speech communication is to provide a basis for error correction and resistance to noise.

The Perception of Speech Under Adverse Conditions

Recent developments in hearing theory have resulted in the rather general acceptance of the idea that the perception of pitch of complex sounds is the result of the psychological pattern recognition process. The pitch is supposedly mediated by the fundamental of the harmonic spectrum which fits the spectrum of the complex sound optimally. The problem of finding the pitch is then equivalent to finding the best harmonic match. Goldstein [J. Acoust. Soc. Am. 54, 1496-1516 (1973)] has described an objective procedure for finding the best fit for stimuli containing relatively few spectral components. He uses maximum likelihood criterion. Application of this procedure to various data on the pitch of complex sounds yielded good results. This motivated our efforts to apply the pattern recognition theory of pitch to the problem of measuring pitch in speech. Although we were able to follow the main line of Goldstein's procedure, some essential changes had to be made. The most important is that in our implementation not all spectral components of the complex sound have to be classified as belonging to the harmonic pattern. We introduced a harmonics sieve to determine whether components are rejected or accepted at a candidate pitch. A simple criterion, based on the components accepted and rejected, led to the decision on which candidate pitch was to be finally selected. The performance and reliability of this psychoacoustically based pitch meter were tested in a LPC-vocoder system.

/pdf/measurement-of-pitch-in-speech-an-implementation-of-4zf8wmorf0.pdf

Measurement of pitch in speech : an implementation of Goldstein's theory of pitch perception

For speech synthesis applications a formant description of the speech signal has a number of advantages over oUter parametrizatiolls. The analysis of formant frequencies and bandwidths from the LPC coefficients has two drawbacks: sometimes the number of formants detected is smaller than is needed for the synthesizer alld sometimes due to numerical instability, the analysis fails completely. A method is described to first derive the formant frequencies by means of the Split Levinson Algorithm and second, to find optimal bandwidth values from a table.

Lei Lf Willems

Papers

Measurement of pitch in speech : an implementation of Goldstein's theory of pitch perception

Robust formant analysis

The formator: a speech analysis-synthesis system based on formant extraction from linear prediction coefficients

Speech synthesis by rule; Why, what and how?

IPOVOX II: A speech synthesizer