In this paper, we evaluate the performance of several objective measures in terms of predicting the quality of noisy speech enhanced by noise suppression algorithms. The objective measures considered a wide range of distortions introduced by four types of real-world noise at two signal-to-noise ratio levels by four classes of speech enhancement algorithms: spectral subtractive, subspace, statistical-model based, and Wiener algorithms. The subjective quality ratings were obtained using the ITU-T P.835 methodology designed to evaluate the quality of enhanced speech along three dimensions: signal distortion, noise distortion, and overall quality. This paper reports on the evaluation of correlations of several objective measures with these three subjective rating scales. Several new composite objective measures are also proposed by combining the individual objective measures using nonparametric and parametric regression analysis techniques.

Evaluation of Objective Quality Measures for Speech Enhancement

The notion of perceptual coding, which is based on the concept of distortion masking by the signal being compressed, is developed. Progress in this field as a result of advances in classical coding theory, modeling of human perception, and digital signal processing, is described. It is proposed that fundamental limits in the science can be expressed by the semiquantitative concepts of perceptual entropy and the perceptual distortion-rate function, and current compression technology is examined in that framework. Problems and future research directions are summarized. >

Signal compression based on models of human perception

The author proposes a perceptually motivated modification to the cepstral distance measure (CD) based on the mel frequency scale and critical-band filtering. The new objective parameter is referred to as the mel cepstral distance (MCD). The author measures and compares the performance of the CD and MCD algorithms by applying them to a dataset representing low-bit-rate code-excited linear prediction (CELP)-coded speech with simulated channel conditions. The improvement in correlation with subjective DAM scores indicates that critical band filtering (and frequency warping) allows better modeling of perceived quality. >

Mel-cepstral distance measure for objective speech quality assessment

The articulation index (AI), speech-transmission index (STI), and coherence-based intelligibility metrics have been evaluated primarily in steady-state noisy conditions and have not been tested extensively in fluctuating noise conditions. The aim of the present work is to evaluate the performance of new speech-based STI measures, modified coherence-based measures, and AI-based measures operating on short-term (30 ms) intervals in realistic noisy conditions. Much emphasis is placed on the design of new band-importance weighting functions which can be used in situations wherein speech is corrupted by fluctuating maskers. The proposed measures were evaluated with intelligibility scores obtained by normal-hearing listeners in 72 noisy conditions involving noise-suppressed speech (consonants and sentences) corrupted by four different maskers (car, babble, train, and street interferences). Of all the measures considered, the modified coherence-based measures and speech-based STI measures incorporating signal-specific band-importance functions yielded the highest correlations (r=0.89-0.94). The modified coherence measure, in particular, that only included vowel/consonant transitions and weak consonant information yielded the highest correlation (r=0.94) with sentence recognition scores. The results from this study clearly suggest that the traditional AI and STI indices could benefit from the use of the proposed signal- and segment-dependent band-importance functions.

/pdf/objective-measures-for-predicting-speech-intelligibility-in-2nk6xcxi0v.pdf

Objective measures for predicting speech intelligibility in noisy conditions based on new band-importance functions.

A perceptually motivated objective measure for evaluating speech quality is presented. The measure, computed from the original and coded versions of an utterance, exhibits statistically a monotonic relationship with the mean opinion score, a widely used criterion for speech coder assessment. For each 10-ms segment of an utterance, a weighted spectral vector is computed via 15 critical band filters for telephone bandwidth speech. The overall distortion, called Bark spectral distortion (BSD), is the average squared Euclidean distance between spectral vectors of the original and coded utterances. The BSD takes into account auditory frequency warping, critical band integration, amplitude sensitivity variations with frequency, and subjective loudness. >

An objective measure for predicting subjective quality of speech coders

The effect of transmission delay on speech quality in telecommunications is described, with human factors such as conversational mode and the talker's knowledge of the cause of delay taken into account. Objective quality estimation methods for delay effects are proposed, and these methods are applied in an actual communications network. In connection with delay perception in a telephone conversation, the assumption was verified that a talker expects a particular response time from a partner, and that delay that is outside this expectation time window is noticed. Taking this information into account, a subjective conversational experiment is controlled by six kinds of tasks by varying the temporal characteristics. Thus, a subjective assessment of delay effects is obtained by laboratory tests in relation to the detectability threshold, opinion rating, and conversational efficiency. Objective quality measures for each test were defined as a linear combination of temporal parameters that correspond closely to subjective qualities. >

Pure delay effects on speech quality in telecommunications

An LPC (linear predictive coding) cepstrum distance measure (CD) is introduced as an objective measure for estimating the subjective quality of speech signals. Good correspondence between LPC CD and the subjective quality, expressed in terms of both opinion equivalent Q and mean opinion score, are shown. Good repeatability of objective quality evaluation using LPC CD is also shown. A method for generating an artificial voice signal that reflects the characteristics of real speech signals is described. The LPC CD values calculated using this artificial voice are almost the same as those calculated using real speech signals. The speaker-dependency of the coded-speech quality is shown to be an important factor in low-bit-rate speech coding. Even taking this factor into consideration, LPC CD is shown to be effective for estimating the subjective quality. >

Objective quality evaluation for low-bit-rate speech coding systems

In an acoustic noise suppressor, a power spectrum component and a phase component are extracted from an input signal by a frequency analysis part, while at the same time a check is made in a speech/non-speech identification part to see if the input signal is a speech signal or noise. Only when the input signal is noise, its spectrum is stored in a storage part and is weighted by a psychoacoustic weighting function W(f), and the weighted spectrum is subtracted from the power spectrum of the input signal and is reconverted to a time-domain signal by making its inverse analysis.

Acoustic noise suppressor

Speech-quality assessment methods for speech-coding systems

This paper describes objective quality measures to evaluate speech quality for various kinds of voiceband CODECs in common. The voiceband CODECs studied were PCM, ADM, ADPCM, ATC (Adaptive Transform Coding) and APC-AB (Adaptive Predictive Coding with Adaptive Bit Allocation). First, several objective quality measures in time and frequency domain were defined. They were SNR, Segmental SNR, Spectral Distortion, LPC Cepstrum Distance, COSH, Likelihood Ratio and Weighted Likelihood Ratio. Second, speech quality for voiceband CODECs were evaluated by subjective and objective quality measures. The subjective measures used were based on opinion test and articulation test. Finally, the relationship between objective measures and subjectively evaluated values was studied. It was concluded that the LPC Cepstrum Distance measure had best correspondence to Mean Opinion Score, among the objective measures studied. It was also concluded that the Wighted Likelihood Ratio measure had best correspondence to Articulation Score.

Kenzo Itoh

Papers

Pure delay effects on speech quality in telecommunications

Objective quality evaluation for low-bit-rate speech coding systems

Acoustic noise suppressor

Speech-quality assessment methods for speech-coding systems

Comparison of objective speech quality measures for voiceband CODECs