Talker localization with microphone arrays has received significant attention lately as a means for the automated tracking of individuals in an enclosure and as a necessary component of any general purpose speech capture system. Several algorithmic approaches are available for speech source localization with multi-channel data. This chapter summarizes the current field and comments on the general merits and shortcomings of each genre. A new localization method is then presented in detail. By utilizing key features of existing methods, this new algorithm is shown to be significantly more robust to acoustical conditions, particularly reverberation effects, than the traditional localization techniques in use today.

Robust Localization in Reverberant Rooms

The effectiveness of probabilistic tracking of objects in image sequences has been revolutionized by the development of particle filtering. Whereas Kalman filters are restricted to Gaussian distributions, particle filters can propagate more general distributions, albeit only approximately. This is of particular benefit in visual tracking because of the inherent ambiguity of the visual world that stems from its richness and complexity. One important advantage of the particle filtering framework is that it allows the information from different measurement sources to be fused in a principled manner. Although this fact has been acknowledged before, it has not been fully exploited within a visual tracking context. Here we introduce generic importance sampling mechanisms for data fusion and discuss them for fusing color with either stereo sound, for teleconferencing, or with motion, for surveillance with a still camera. We show how each of the three cues can be modeled by an appropriate data likelihood function, and how the intermittent cues (sound or motion) are best handled by generating proposal distributions from their likelihood functions. Finally, the effective fusion of the cues by particle filtering is demonstrated on real teleconference and surveillance data.

/pdf/data-fusion-for-visual-tracking-with-particles-3rjpt37twm.pdf

Data fusion for visual tracking with particles

/pdf/a-high-accuracy-low-latency-technique-for-talker-2n390ioovn.pdf

A High-Accuracy, Low-Latency Technique for Talker Localization in Reverberant Environments Using Microphone Arrays

To find the position of an acoustic source in a room, the relative delay between two (or more) microphone signals for the direct sound must be determined. The generalized cross-correlation method is the most popular technique to do so and is well explained in a landmark paper by Knapp and Carter. In this paper, a new approach is proposed that is based on eigenvalue decomposition. Indeed, the eigenvector corresponding to the minimum eigenvalue of the covariance matrix of the microphone signals contains the impulse responses between the source and the microphone signals (and therefore all the information we need for time delay estimation). In experiments, the proposed algorithm performs well and is very accurate.

Adaptive eigenvalue decomposition algorithm for passive acoustic source localization

A practical methodology for speech source localization with microphone arrays

A two-stage algorithm for determining talker location from linear microphone array data

In this paper, baseline speech recognition performance is determined both for a single remote microphone and for a signal derived from a delay-and-sum beamformer using an eight-microphone linear array. An HMM-based, connected-speech, 38-word vocabulary (alphabet, digits, 'space', 'period'), talker-independent speech recognition system is used for testing performance. Normal performance, with no language model, i.e., raw word-level performance, is currently about 81% for a set of talkers not in the training set and about 91% for training set data. The system has been trained and tested using a close-talking head-mounted microphone. Since a meaningful comparison requires using the same speech, the existing speech database was appropriately pre-filtered, played out through a transducer (speaker) in the room environment, picked-up by the microphone array, and re-stored as a digital file. The resulting file was post-processed and used as input to the recognizer; the recognition performance indicates the effect of the input device. The baseline experiment showed that both a single remote microphone and the beamformed signal reduced performance by 12% in a room with no other talkers. For the array tested, the error is generally attributable to reverberation off the floor and ceiling.

Stuart E. Kirtman

Papers

A two-stage algorithm for determining talker location from linear microphone array data

Experimental results for baseline speech recognition performance using input acquired from a linear microphone array