https://hal.inria.fr/inria-00075657/document

Distance measures for signal processing and pattern recognition

The performance lost to data quantization in signal detection and estimation procedures is considered. The performance is measured by f-divergences, which are useful indices of discrimination information between statistical hypotheses. Some properties of the f-divergences are briefly discussed, and a general result for the loss in divergence due to uniform data quantization is obtained. Several applications of this result in specific problems of signal detection and estimation being developed, and some numerical results that illustrate the asymptotic behavior of the divergence in these applications are given. The divergence lost to finite nonuniform quantization through the use of a companding model is considered, and the asymptotically optimum nonuniform quantizer within this class is derived. Some interesting problems that remain open are discussed. >

Fine quantization in signal detection and estimation

The standard procedure in numerical classification and identification of micro-organisms based on binary features is given a justification based on the principle of maximum entropy. This principle also strongly supports the assumption that all characteristics upon which the classification is based are equally important and the use of polythetic taxa. The relevance of the principle of maximum entropy in connection with taxonomic structures based on clustering and maximal predictivity is discussed. A result on asymptotic separateness of maximum entropy distributions has implications for minimizing identification errors.

Numerical taxonomy and the principle of maximum entropy

Asymptotic analysis using companding approximations has proven to be a very useful technique for the performance analysis and design of minimum-distortion data quantizers. However, when quantized data is to be used for inferential (e.g., detection or estimation) purposes, the use of distortion-based performance criteria is inappropriate since the measures of quality in such problems are usually based on quantities such as error probability or mean-square estimation error. In this paper we use companding approximations to derive asymptotic criteria for the evaluation of quantizer performance and the design of optimum quantizers based on statistical measures of divergence at the output of the quantizer. Several applications of these results to signal detection and parameter estimation are considered.

A companding approximation for the statistical divergence of quantized data

This paper considers optimization of distributed detectors under the Bayes criterion. A distributed detector consists of multiple local detectors and a fusion center that combines the local decision results to obtain a final decision. Introduced first are distributional distance measures, the mutual information (MI) and the conditional mutual information (CMI), that are obtained by applying information theoretic concepts to detection problems. Error bound analyses show that these distance measures approximate the Bayesian probability of error better than the conventional ones regardless of the operational environments. Then, a new optimization technique is proposed for distributed Bayes detectors. The method uses the distributional distances instead of the original Bayes criterion to avoid the complexity barrier of the optimization problem. Numerical examples show that the proposed distance measures yield solutions far better than the existing ones.

J. C. Ruitenbeek

Papers

A class of lower bounds on the Bayesian probability of error