Today, a large number of audio features exists in audio retrieval for different purposes, such as automatic speech recognition, music information retrieval, audio segmentation, and environmental sound retrieval. The goal of this chapter is to review latest research in the context of audio feature extraction and to give an application-independent overview of the most important existing techniques. We survey state-of-the-art features from various domains and propose a novel taxonomy for the organization of audio features. Additionally, we identify the building blocks of audio features and propose a scheme that allows for the description of arbitrary features. We present an extensive literature survey and provide more than 200 references to relevant high-quality publications.

Features for Content-Based Audio Retrieval

In this paper, an approach for robust singing signal detection in speech/music discrimination is proposed and applied to applications of audio indexing. Conventional approaches in speech/music discrimination can provide reasonable performance with regular music signals but often perform poorly with singing segments. This is due mainly to the fact that speech and singing signals are extremely close and traditional features used in speech recognition do not provide a reliable cue for speech and singing signal discrimination. In order to improve the robustness of speech/music discrimination, a new set of features derived from the harmonic coefficient and its 4 Hz modulation values are developed in this paper, and these new features provide additional and reliable cues to separate speech from singing. In addition, a rule-based post-filtering scheme is also described which leads to further improvements in speech/music discrimination. Source-independent audio indexing experiments on the PBS Skills database indicate that the proposed approach can greatly reduce the classification error rate on singing segments in the audio stream. Comparing with existing approaches, the overall segmentation error rate is reduced by more than 30%, averaged over all shows in the database.

/pdf/robust-singing-detection-in-speech-music-discriminator-14yoene307.pdf

Robust singing detection in speech/music discriminator design

Perceptual harmonic cepstral coefficients (PHCC) are proposed as features to extract from speech for recognition in noisy environments. A weighting function, which depends on the prominence of the harmonic structure, is applied to the power spectrum to ensure accurate representation of the voiced speech spectral envelope. The harmonics' weighted power spectrum undergoes mel-scaled band-pass filtering, and the log-energy of the filters' output is discrete cosine transformed to produce cepstral coefficients. Lower spectral clipping is applied to the power spectrum, followed by within-filter root-power amplitude compression to reduce amplitude variation without compromise of the gain invariance properties. Experiments show significant recognition gains of PHCC over MFCC, with 23% and 36% error rate reduction for the Mandarin digit database in white and babble noise environments.

Perceptual harmonic cepstral coefficients for speech recognition in noisy environment

A speech coding system and associated method rely on a speech encoder and a speech decoder. The speech decoder includes a Linear Predictive Coding (LPC) filter having an input and an output. The LPC filter provides synthesized speech at the output in response to voiced and unvoiced excitation provided at the input. A harmonic generator for providing voiced excitation to the input of the LPC filter includes a spectral formant enhancer for attenuating the amplitude of harmonics generate by the harmonic generator in spectral valleys between format peaks of respective frames of voiced speech. The system and method reduce perceived buzziness while increasing perceived spectral depth of synthesized speech at the output of the LPC filter.

Speech coding system and method including spectral formant enhancer

A new method for quantization of the LPC coefficients in a speech coder includes an improved form of switched predictive multi-stage vector quantization. The switch predictive quantization includes at least a pair of codebook sets in a MSVQ quantizer and a first and second prediction matrix 24a and 24b with the first prediction matrix 1 used with codebook set 1 and prediction matrix 2 used with codebook set 2 and the encoder determines which prediction matrix/codebooks set produces the minimum quantization error at detector 35 and control 29 gates the indices with the minimum error out of the speech coder.

Method for switched-predictive quantization

Sinusoidal speech coders have been widely studied for low bit rate coding around 4 kbit/s. However, the estimation error of the sinusoidal model parameters would seriously degrade the speech quality. In general, the estimation errors are caused by the effects of various types of speech signal or background noise. In this paper we propose a sinusoidal speech coder with robust parameter determination methods. They consist of spectro-temporal autocorrelation method for robust pitch determination, frequency shifting method for robust voicing level measurement, and residual-spectrum magnitude coding method for spectral magnitude compensation. From the experimental results, we can find the robustnesses of the proposed techniques. In addition, informal listening test of the synthesized speech confirms the effectiveness of the incorporated schemes.

A spectrally mixed excitation (SMX) vocoder with robust parameter determination

To represent the spectral envelope, the LSF (line spectrum frequencies) can be used for its good properties in quantization. The interframe correlation of the LSF improves the performance of static quantization, but it is sensitive to channel errors. The proposed quantizer, noise robust-switched prediction-linked split vector quantizer (NR-SP-LSVQ), combines switched prediction with three static and a predictive quantizers. It limits the propagation of channel errors in a few frames by switching to the static ones. Moreover, each of the static ones is designed for the robustness against background noise, while the predictive one is for the steady region of speech. It increases the performance in noise corrupted as well as clean speech.

https://ieeexplore.ieee.org/iel3/4917/13569/00623904.pdf

Spectral envelope quantization with noise robustness

The synthesized speech quality of the multiband excitation (MBE) vocoder highly depends on the accuracy of the pitch estimation, and it is uses pitch tracking and spectral analysis-by-synthesis (AbS) procedures to determine the fine pitch with reduced gross pitch errors. However, pitch tracking has limits in practice because it requires a large lookahead. Also, the spectral AbS, if used alone, results in frequent gross pitch errors. So we propose a weighted spectral AbS method which can reduce gross pitch errors without extra-lookahead in comparison with spectral AbS method. The weight in the proposed method is defined by the covariance of excitation spectrum of speech signal to compensate for the gross pitch error. From the comparison of the pitch contours between the spectral AbS and the weighted spectral AbS methods, it is confirmed that the proposed method considerably reduces gross pitch errors rather than spectral AbS method.

http://ieeexplore.ieee.org/iel3/4917/13569/00623876.pdf

Pitch estimation using spectral covariance method for low-delay MBE vocoder

This paper proposes a new 4 kbit/s speech coder based on CELP structure with 45 ms total codec delay. The coder is mainly featured by the renewal codebook of the excitation signal and the linked split-vector quantizer of line spectrum pair parameters which enable the coder to get high quality speech at low bit rate. In addition, techniques of formant enhancement in the spectral envelop and harmonic recovery in the transient region are also introduced to reduce buzzy and hoarse sounds, respectively. From the intensive listening test with intermediated response system (IRS) speech, we obtained a comparable subjective quality to 32 kbit/s ADPCM (ITU Recommendation G.726) under a nominal speech input level of -26 dB overload.

Sang Ryong Kim

Papers

A spectrally mixed excitation (SMX) vocoder with robust parameter determination

Spectral envelope quantization with noise robustness

Pitch estimation using spectral covariance method for low-delay MBE vocoder

A 4 kbit/s renewal code excited linear prediction speech coder

Joint estimation of pitch, band magnitudes, and v\UV decisions for MBE vocoder.