A method and system use an alternative sensor signal received from a sensor other than an air conduction microphone to estimate a clean speech value. The estimation uses either the alternative sensor signal alone, or in conjunction with the air conduction microphone signal. The clean speech value is estimated without using a model trained from noisy training data collected from an air conduction microphone. Under one embodiment, correction vectors are added to a vector formed from the alternative sensor signal in order to form a filter, which is applied to the air conductive microphone signal to produce the clean speech estimate. In other embodiments, the pitch of a speech signal is determined from the alternative sensor signal and is used to decompose an air conduction microphone signal. The decomposed signal is then used to determine a clean signal estimate.

Method and apparatus for multi-sensory speech enhancement

A mobile device includes an air conduction microphone and an alternative sensor that provides an alternative sensor signal indicative of speech. A communication interface permits the mobile device to communicate directly with other mobile devices.

Method and apparatus for multi-sensory speech enhancement on a mobile device

Air- and bone-conductive integrated microphones have been introduced by the authors [Y. Zheng, et al., 2003, Z. Zhang et al., 2004] for speech enhancement in noisy environments. In this paper, we present a novel technique, called direct filtering, to combine the two channels from the air- and bone-conductive microphone for speech enhancement. Compared to the previous technique, the advantage of the direct filtering is that it does not require any training, and it is speaker independent. Experiments show that this technique effectively removes noises and significantly improves speech recognition accuracy even in highly non-stationary noisy environments.

/pdf/direct-filtering-for-air-and-bone-conductive-microphones-24kc6q327o.pdf

Direct filtering for air- and bone-conductive microphones

A mobile computing device, adapted to be held in the palm of a user's hand, includes an antenna for transmission of information from the mobile computing device. A first microphone, adapted to convert audible speech from the user into speech signals, is positioned at a distal end of the antenna. The antenna is rotatable, while the mobile computing device is held by the user, into a position which directs the first microphone toward the mouth of the user. A speech sensor outputs a sensor signal indicative of whether the user is speaking.

Personal mobile computing device having antenna microphone and speech detection for improved speech recognition

Continuing our previous work (Zhang et al. (2004), Liu et al. (2004)) on using air- and bone-conductive integrated microphones, and in particular on using the direct filtering approach (Liu et al. (2004)) for speech enhancement in noisy environments, we present in this paper a refined version of the direct filtering algorithm. The new algorithm takes into account explicitly the leakage of background noise into the bone channel. We also present a new algorithm that detects and removes an artifact known as teeth clacks. Experiments show that the addition of the above algorithms improves system performance to a large extent even in highly nonstationary noisy environments.

/pdf/leakage-model-and-teeth-clack-removal-for-air-and-bone-3kxlqjt1cl.pdf

Leakage model and teeth clack removal for air- and bone-conductive integrated microphones

This paper presents recognition results for a novel microphone solution for ASR in adverse acoustic environments. PARAT is an intelligent hearing protector and communication terminal implemented as a lightweight earplug. The inner microphone in PARAT captures the speech signal behind a seal in the auditory canal, thus impeding contamination from external noise sources. Experiments on a stereo speech database contaminated with noise show that the PARAT microphone compares favourably with a high-quality noise-cancelling close-talking microphone.

http://ieeexplore.ieee.org/iel5/9212/29213/01318460.pdf

T. Holter

Papers

On the feasibility of ASR in extreme noise using the PARAT earplug communication terminal