Showing papers on "Pointwise mutual information published in 1999"

Optimum quantization for detector fusion: some proofs, examples, and pathology

Abstract: We consider optimal decentralized (or equivalently, quantized) detection for the Neyman– Pearson, Bayes, Ali–Silvey distance, and mutual (Shannon) information criteria. In all cases, it is shown that the optimal sensor decision rules are quantizers that operate on the likelihood ratio of the observations. We further show that randomized fusion rules are suboptimal for the mutual information criterion. We also show that if the processes observed at the sensors are conditionally independent and identically distributed, and the criterion for optimization is either a member of a subclass of the Ali–Silvey distances or local mutual information, then the quantizers used at all of the sensors are identical. We give an example to show that for the Neyman–Pearson and Bayes criteria this is not generally true. We go into some detail with respect to this last, and derive necessary conditions for an assumptions of identical sensor quantizer maps to be reasonable.

Relevancy of time-frequency features for phonetic classification measured by mutual information

Abstract: In this paper we use mutual information to study the distribution in time and frequency of information relevant for phonetic classification. A large database of hand-labeled fluent speech is used to (a) compute the mutual information between phoneme labels and a point of logarithmic energy in the time-frequency plane and (b) compute the joint mutual information between phoneme labels and two points of logarithmic energy in the time-frequency plane.

Unsupervised and supervised learning: Mutual information between parameters and observations

Abstract: learning tasks. The parameter is a possibly, but not necessarily, high-dimensional vector. We derive exact bounds and asymptotic behaviors for the mutual information as a function of the data size and of some properties of the probability of the data given the parameter. We compare these exact results with the predictions of replica calculations. We briefly discuss the universal properties of the mutual information as a function of data size. @S1063-651X~99!00403-1#

Search for Information Bearing Components in Speech

Abstract: In this paper, we use mutual information to characterize the distributions of phonetic and speaker/channel information in a time-frequency space. The mutual information (MI) between the phonetic label and one feature, and the joint mutual information (JMI) between the phonetic label and two or three features are estimated. The Miller's bias formulas for entropy and mutual information estimates are extended to include higher order terms. The MI and the JMI for speaker/channel recognition are also estimated. The results are complementary to those for phonetic classification. Our results show how the phonetic information is locally spread and how the speaker/channel information is globally spread in time and frequency.