An algorithm is presented for the estimation of the fundamental frequency (F0) of speech or musical sounds. It is based on the well-known autocorrelation method with a number of modifications that combine to prevent errors. The algorithm has several desirable features. Error rates are about three times lower than the best competing methods, as evaluated over a database of speech recorded together with a laryngograph signal. There is no upper limit on the frequency search range, so the algorithm is suited for high-pitched voices and music. The algorithm is relatively simple and may be implemented efficiently and with low latency, and it involves few parameters that must be tuned. It is based on a signal model (periodic signal) that may be extended in several ways to handle various forms of aperiodicity that occur in particular applications. Finally, interesting parallels may be drawn with models of auditory processing.

/pdf/yin-a-fundamental-frequency-estimator-for-speech-and-music-53voam3nec.pdf

YIN, a fundamental frequency estimator for speech and music

In the development process of noise-reduction algorithms, an objective machine-driven intelligibility measure which shows high correlation with speech intelligibility is of great interest. Besides reducing time and costs compared to real listening experiments, an objective intelligibility measure could also help provide answers on how to improve the intelligibility of noisy unprocessed speech. In this paper, a short-time objective intelligibility measure (STOI) is presented, which shows high correlation with the intelligibility of noisy and time-frequency weighted noisy speech (e.g., resulting from noise reduction) of three different listening experiments. In general, STOI showed better correlation with speech intelligibility compared to five other reference objective intelligibility models. In contrast to other conventional intelligibility models which tend to rely on global statistics across entire sentences, STOI is based on shorter time segments (386 ms). Experiments indeed show that it is beneficial to take segment lengths of this order into account. In addition, a free Matlab implementation is provided.

An Algorithm for Intelligibility Prediction of Time–Frequency Weighted Noisy Speech

福祉関連職におけるMaslach Burnout Inventoryの因子構造の比較

(NOTE: Each chapter begins with an introduction and concludes with a Summary, Exercises and Bibliography.) 1. Introduction. Discrete-Time Speech Signal Processing. The Speech Communication Pathway. Analysis/Synthesis Based on Speech Production and Perception. Applications. Outline of Book. 2. A Discrete-Time Signal Processing Framework. Discrete-Time Signals. Discrete-Time Systems. Discrete-Time Fourier Transform. Uncertainty Principle. z-Transform. LTI Systems in the Frequency Domain. Properties of LTI Systems. Time-Varying Systems. Discrete-Fourier Transform. Conversion of Continuous Signals and Systems to Discrete Time. 3. Production and Classification of Speech Sounds. Anatomy and Physiology of Speech Production. Spectrographic Analysis of Speech. Categorization of Speech Sounds. Prosody: The Melody of Speech. Speech Perception. 4. Acoustics of Speech Production. Physics of Sound. Uniform Tube Model. A Discrete-Time Model Based on Tube Concatenation. Vocal Fold/Vocal Tract Interaction. 5. Analysis and Synthesis of Pole-Zero Speech Models. Time-Dependent Processing. All-Pole Modeling of Deterministic Signals. Linear Prediction Analysis of Stochastic Speech Sounds. Criterion of "Goodness". Synthesis Based on All-Pole Modeling. Pole-Zero Estimation. Decomposition of the Glottal Flow Derivative. Appendix 5.A: Properties of Stochastic Processes. Random Processes. Ensemble Averages. Stationary Random Process. Time Averages. Power Density Spectrum. Appendix 5.B: Derivation of the Lattice Filter in Linear Prediction Analysis. 6. Homomorphic Signal Processing. Concept. Homomorphic Systems for Convolution. Complex Cepstrum of Speech-Like Sequences. Spectral Root Homomorphic Filtering. Short-Time Homomorphic Analysis of Periodic Sequences. Short-Time Speech Analysis. Analysis/Synthesis Structures. Contrasting Linear Prediction and Homomorphic Filtering. 7. Short-Time Fourier Transform Analysis and Synthesis. Short-Time Analysis. Short-Time Synthesis. Short-Time Fourier Transform Magnitude. Signal Estimation from the Modified STFT or STFTM. Time-Scale Modification and Enhancement of Speech. Appendix 7.A: FBS Method with Multiplicative Modification. 8. Filter-Bank Analysis/Synthesis. Revisiting the FBS Method. Phase Vocoder. Phase Coherence in the Phase Vocoder. Constant-Q Analysis/Synthesis. Auditory Modeling. 9. Sinusoidal Analysis/Synthesis. Sinusoidal Speech Model. Estimation of Sinewave Parameters. Synthesis. Source/Filter Phase Model. Additive Deterministic-Stochastic Model. Appendix 9.A: Derivation of the Sinewave Model. Appendix 9.B: Derivation of Optimal Cubic Phase Parameters. 10. Frequency-Domain Pitch Estimation. A Correlation-Based Pitch Estimator. Pitch Estimation Based on a "Comb Filter<170. Pitch Estimation Based on a Harmonic Sinewave Model. Glottal Pulse Onset Estimation. Multi-Band Pitch and Voicing Estimation. 11. Nonlinear Measurement and Modeling Techniques. The STFT and Wavelet Transform Revisited. Bilinear Time-Frequency Distributions. Aeroacoustic Flow in the Vocal Tract. Instantaneous Teager Energy Operator. 12. Speech Coding. Statistical Models of Speech. Scaler Quantization. Vector Quantization (VQ). Frequency-Domain Coding. Model-Based Coding. LPC Residual Coding. 13. Speech Enhancement. Introduction. Preliminaries. Wiener Filtering. Model-Based Processing. Enhancement Based on Auditory Masking. Appendix 13.A: Stochastic-Theoretic parameter Estimation. 14. Speaker Recognition. Introduction. Spectral Features for Speaker Recognition. Speaker Recognition Algorithms. Non-Spectral Features in Speaker Recognition. Signal Enhancement for the Mismatched Condition. Speaker Recognition from Coded Speech. Appendix 14.A: Expectation-Maximization (EM) Estimation. Glossary.Speech Signal Processing.Units.Databases.Index.About the Author.

Discrete-Time Speech Signal Processing: Principles and Practice

Phase patterns of neuronal responses reliably discriminate speech in human auditory cortex.

This article presents a quantitative binaural signal detection model which extends the monaural model described by Dau et al. [J. Acoust. Soc. Am. 99, 3615–3622 (1996)]. The model is divided into three stages. The first stage comprises peripheral preprocessing in the right and left monaural channels. The second stage is a binaural processor which produces a time-dependent internal representation of the binaurally presented stimuli. This stage is based on the Jeffress delay line extended with tapped attenuator lines. Through this extension, the internal representation codes both interaural time and intensity differences. In contrast to most present-day models, which are based on excitatory–excitatory interaction, the binaural interaction in the present model is based on contralateral inhibition of ipsilateral signals. The last stage, a central processor, extracts a decision variable that can be used to detect the presence of a signal in a detection task, but could also derive information about the position...

/pdf/binaural-processing-model-based-on-contralateral-inhibition-2n3bpr6rca.pdf

Binaural processing model based on contralateral inhibition. I. Model structure.

When an audio—visual event is perceived in the natural environment, a physical delay will always occur between the arrival of the leading visual component and that of the trailing auditory component. This natural timing relationship suggests that the point of subjective simultaneity (PSS) should occur at an auditory delay greater than or equal to 0 msec. A review of the literature suggests that PSS estimates derived from a temporal order judgment (TOJ) task differ from those derived from a synchrony judgment (SJ) task, with (unnatural) auditory-leading PSS values reported mainly for the TOJ task. We report data from two stimulus types that differed in terms of complexity— namely, (1) a flash and a click and (2) a bouncing ball and an impact sound. The same participants judged the temporal order and synchrony of both stimulus types, using three experimental methods: (1) a TOJ task with two response categories (“audio first” or “video first”), (2) an SJ task with two response categories (“synchronous” or “asynchronous”; SJ2), and (3) an SJ task with three response categories (“audio first,” “synchronous,” or “video first”; SJ3). Both stimulus types produced correlated PSS estimates with the SJ tasks, but the estimates from the TOJ procedure were uncorrelated with those obtained from the SJ tasks. These results suggest that the SJ task should be preferred over the TOJ task when the primary interest is in perceived audio—visual synchrony.

/pdf/audiovisual-synchrony-and-temporal-order-judgments-effects-8fjt8ikxm9.pdf

Audiovisual synchrony and temporal order judgments: Effects of experimental method and stimulus type

This and the accompanying paper [Dau et al., J. Acoust. Soc. Am. 99, ••–•• (1996)] describe a quantitative model for signal processing in the auditory system. The model combines several stages of preprocessing with a decision device that has the properties of an optimal detector. The present paper compares model predictions for a variety of experimental conditions with the performance of human observers. Simulated and psychophysically determined thresholds were estimated with a three‐interval forced‐choice adaptive procedure. All model parameters were kept constant for all simulations discussed in this paper. For frozen‐noise maskers, the effects of the following stimulus parameters were examined: signal frequency, signal phase, temporal position and duration of the signal within the masker under conditions of simultaneous masking, masker level, and masker duration under conditions of forward masking, and backward masking. The influence of signal phase and the temporal position of the signal, including positions at masker onset, was determined for a random‐noise masker and compared with corresponding results obtained for a frozen noise. The model describes all the experimental data with an accuracy of a few dB with the following exceptions: forward‐masked thresholds obtained with brief maskers are too high and the change in threshold with a change in signal duration is too small. Both discrepancies have their origin in the adaptation stages in the preprocessing part of the model. On the basis of the wide range of simulated conditions we conclude that the present model is a successful approach to describing the detection process in the human auditory system.

https://pure.tue.nl/ws/files/1307552/622104.pdf

A quantitative model of the effective signal processing in the auditory system. ii. simulations and measurements

Although extensive research has been done in the field of machine-based localization, the degrading effect of reverberation and the presence of multiple sources on localization performance has remained a major problem. Motivated by the ability of the human auditory system to robustly analyze complex acoustic scenes, the associated peripheral stage is used in this paper as a front-end to estimate the azimuth of sound sources based on binaural signals. One classical approach to localize an acoustic source in the horizontal plane is to estimate the interaural time difference (ITD) between both ears by searching for the maximum in the cross-correlation function. Apart from ITDs, the interaural level difference (ILD) can contribute to localization, especially at higher frequencies where the wavelength becomes smaller than the diameter of the head, leading to ambiguous ITD information. The interdependency of ITD and ILD on azimuth is a complex pattern that depends also on the room acoustics, and is therefore learned by azimuth-dependent Gaussian mixture models (GMMs). Multiconditional training is performed to take into account the variability of the binaural features which results from multiple sources and the effect of reverberation. The proposed localization model outperforms state-of-the-art localization techniques in simulated adverse acoustic conditions.

/pdf/a-probabilistic-model-for-robust-localization-based-on-a-cfwrcnw3tm.pdf

A Probabilistic Model for Robust Localization Based on a Binaural Auditory Front-End

A new experimental technique for studying binaural processing at high frequencies is introduced. Binaural masking level differences (BMLDs) for the conditions N0Sπ and NπS0 were measured for a tonal signal in narrow-band noise at 125, 250, and 4000 Hz. In addition, “transposed” stimuli were generated, which were centered at 4000 Hz, but were designed to preserve within the envelope the temporal “fine-structure” information available at the two lower frequencies. The BMLDs measured with the 125-Hz transposed stimuli were essentially the same as BMLDs from the regular 125-Hz condition. The transposed 250-Hz stimuli generally produced smaller BMLDs than the stimuli centered at 250 Hz, but the pattern of results as a function of masker bandwidth was the same. The patterns of results from the transposed stimuli are different from those of the 4000-Hz condition and, consistent with the low-frequency masker data, generally show higher BMLDs. The results indicate that the mechanisms underlying binaural processing at low and high frequencies are similar, and that frequency-dependent differences in BMLDs probably reflect the inability of the auditory system to encode the temporal fine structure of high-frequency stimuli.

/pdf/a-new-approach-to-comparing-binaural-masking-level-3j9oazkcdx.pdf

AG Armin Kohlrausch

Papers

Binaural processing model based on contralateral inhibition. I. Model structure.

Audiovisual synchrony and temporal order judgments: Effects of experimental method and stimulus type

A quantitative model of the effective signal processing in the auditory system. ii. simulations and measurements

A Probabilistic Model for Robust Localization Based on a Binaural Auditory Front-End

A new approach to comparing binaural masking level differences at low and high frequencies.