Showing papers in "arXiv: Data Analysis, Statistics and Probability in 2001"

Stochastic models which separate fractal dimension and Hurst effect

Abstract: Fractal behavior and long-range dependence have been observed in an astonishing number of physical systems. Either phenomenon has been modeled by self-similar random functions, thereby implying a linear relationship between fractal dimension, a measure of roughness, and Hurst coefficient, a measure of long-memory dependence. This letter introduces simple stochastic models which allow for any combination of fractal dimension and Hurst exponent. We synthesize images from these models, with arbitrary fractal properties and power-law correlations, and propose a test for self-similarity.

On a quantitative method to analyse dynamical and measurement noise

Abstract: This letter reports on a new method of analysing experimentally gained time series with respect to different types of noise involved, namely, we show that it is possible to differentiate between dynamical and measurement noise. This method does not depend on previous knowledge of model equations. For the complicated case of a chaotic dynamics spoiled at the same time by dynamical and measurement noise, we even show how to extract from data the magnitude of both types of noise. As a further result, we present a new criterion to verify the correct embedding for chaotic dynamics with dynamical noise.

Reconstruction of dynamical equations for traffic flow

Abstract: Traffic flow data collected by an induction loop detector on the highway close to Koeln-Nord are investigated with respect to their dynamics including the stochastic content. In particular we present a new method, with which the flow dynamics can be extracted directly from the measured data. As a result a Langevin equation for the traffic flow is obtained. From the deterministic part of the flow dynamics, stable fixed points are extracted and set into relation with common features of the fundamental diagram.

Deconvolution problems in x-ray absorption fine structure

Abstract: A Bayesian method application to the deconvolution of EXAFS spectra is considered. It is shown that for purposes of EXAFS spectroscopy, from the infinitely large number of Bayesian solutions it is possible to determine an optimal range of solutions, any one from which is appropriate. Since this removes the requirement for the uniqueness of solution, it becomes possible to exclude the instrumental broadening and the lifetime broadening from EXAFS spectra. In addition, we propose several approaches to the determination of optimal Bayesian regularization parameter. The Bayesian deconvolution is compared with the deconvolution which uses the Fourier transform and optimal Wiener filtering. It is shown that XPS spectra could be in principle used for extraction of a one-electron absorptance. The amplitude correction factors obtained after deconvolution are considered and discussed.

alphaPDE: A New Multivariate Technique for Parameter Estimation

Abstract: We present alphaPDE, a new multivariate analysis technique for parameter estimation. The method is based on a direct construction of joint probability densities of known variables and the parameters to be estimated. We show how posterior densities and best-value estimates are then obtained for the parameters of interest by a straightforward manipulation of these densities. The method is essentially non-parametric and allows for an intuitive graphical interpretation. We illustrate the method by outlining how it can be used to estimate the mass of the top quark, and we explain how the method is applied to an ensemble of events containing background.

Penalized maximum likelihood for multivariate Gaussian mixture

Abstract: In this paper, we first consider the parameter estimation of a multivariate random process distribution using multivariate Gaussian mixture law. The labels of the mixture are allowed to have a general probability law which gives the possibility to modelize a temporal structure of the process under study. We generalize the case of univariate Gaussian mixture in [Ridolfi99] to show that the likelihood is unbounded and goes to infinity when one of the covariance matrices approaches the boundary of singularity of the non negative definite matrices set. We characterize the parameter set of these singularities. As a solution to this degeneracy problem, we show that the penalization of the likelihood by an Inverse Wishart prior on covariance matrices results to a penalized or maximum a posteriori criterion which is bounded. Then, the existence of positive definite matrices optimizing this criterion can be guaranteed. We also show that with a modified EM procedure or with a Bayesian sampling scheme, we can constrain covariance matrices to belong to a particular subclass of covariance matrices. Finally, we study degeneracies in the source separation problem where the characterization of parameter singularity set is more complex. We show, however, that Inverse Wishart prior on covariance matrices eliminates the degeneracies in this case too.

A Comparison of Two Approaches: Maximum Entropy on the Mean (MEM) and Bayesian Estimation (BAYES) for Inverse Problems

Abstract: To handle with inverse problems, two probabilistic approaches have been proposed: the maximum entropy on the mean (MEM) and the Bayesian estimation (BAYES). The main object of this presentation is to compare these two approaches which are in fact two different inference procedures to define the solution of an inverse problem as the optimizer of a compound criterion. Keywords: Inverse problems, Maximum Entropy on the Mean, Bayesian inference, Convex analysis.

Strange Attractors in Multipath propagation: Detection and characterisation

Abstract: Multipath propagation of radio waves in indoor/outdoor environments shows a highly irregular behavior as a function of time. Typical modeling of this phenomenon assumes the received signal is a stochastic process composed of the superposition of various altered replicas of the transmitted one, their amplitudes and phases being drawn from specific probability densities. We set out to explore the hypothesis of the presence of deterministic chaos in signals propagating inside various buildings at the University of Calgary. The correlation dimension versus embedding dimension saturates to a value between 3 and 4 for various antenna polarizations. The full Liapunov spectrum calculated contains two positive exponents and yields through the Kaplan-Yorke conjecture the same dimension obtained from the correlation sum. The presence of strange attractors in multipath propagation hints to better ways to predict the behaviour of the signal and better methods to counter the effects of interference. The use of Neural Networks in non linear prediction will be illustrated in an example and potential applications of same will be highlighted.

Entropy in Signal Processing (Entropie en Traitement du Signal)

Abstract: R\'esum\'e: Le principal objet de cette communication est de faire une r\'etro perspective succincte de l'utilisation de l'entropie et du principe du maximum d'entropie dans le domaine du traitement du signal Apr\`es un bref rappel de quelques d\'efinitions et du principe du maximum d'entropie, nous verrons successivement comment l'entropie est utilis\'ee en s\'eparation de sources, en mod\'elisation de signaux, en analyse spectrale et pour la r\'esolution des probl\`emes inverses lin\'eaires Mots cl\'es : Entropie, Entropie crois\'ee, Distance de Kullback, Information mutuelle, Estimation spectrale, Probl\`emes inverses Abstract: The main object of this work is to give a brief overview of the different ways the entropy has been used in signal and image processing After a short introduction of different quantities related to the entropy and the maximum entropy principle, we will study their use in different fields of signal processing such as: source separation, model order selection, spectral estimation and, finally, general linear inverse problems Keywords : Entropy, Relative entropy, Kullback distance, Mutual information, Spectral estimation, Inverse problems

A Bayesian Approach to Shape Reconstruction of a Compact Object from a Few Number of Projections

Abstract: Image reconstruction in X ray tomography consists in determining an object from its projections. In many applications such as non destructive testing, we look for an image who has a constant value inside a region (default) and another constant value outside that region (homogeneous region surrounding the default). The image reconstruction problem becomes then the determination of the shape of that region. In this work we model the object (the default region) as a polygonal disc and propose a new method for the estimation of the coordinates of its vertices directly from a very limited number of its projections. Keywords: Computed Imaging, Tomography, Shape reconstruction, Non destructive testing, Regularization, Bayesian estimation, Deformable contours.

Bayesian inference for inverse problems

Abstract: Traditionally, the MaxEnt workshops start by a tutorial day. This paper summarizes my talk during 2001'th workshop at John Hopkins University. The main idea in this talk is to show how the Bayesian inference can naturally give us all the necessary tools we need to solve real inverse problems: starting by simple inversion where we assume to know exactly the forward model and all the input model parameters up to more realistic advanced problems of myopic or blind inversion where we may be uncertain about the forward model and we may have noisy data. Starting by an introduction to inverse problems through a few examples and explaining their ill posedness nature, I briefly presented the main classical deterministic methods such as data matching and classical regularization methods to show their limitations. I then presented the main classical probabilistic methods based on likelihood, information theory and maximum entropy and the Bayesian inference framework for such problems. I show that the Bayesian framework, not only generalizes all these methods, but also gives us natural tools, for example, for inferring the uncertainty of the computed solutions, for the estimation of the hyperparameters or for handling myopic or blind inversion problems. Finally, through a deconvolution problem example, I presented a few state of the art methods based on Bayesian inference particularly designed for some of the mass spectrometry data processing problems.

A Good Measure for Bayesian Inference

Abstract: The Gaussian theory of errors has been generalized to situations, where the Gaussian distribution and, hence, the Gaussian rules of error propagation are inadequate. The generalizations are based on Bayes' theorem and a suitable measure. The following text sketches some chapters of a monograph that is presently prepared. We concentrate on the material that is - to the best of our knowledge - not yet in the statistical literature. See especially the extension of form invariance to discrete data in section 4, the criterion on the compatibility between a proposed distribution and sparse data in section 7 and the ``discovery'' of probability amplitudes in section 9.

Statistical inference and modeling with the S distribution

Abstract: We consider the problem of statistical inference for the S distribution and introduce new minimum distance estimators for the four parameters of the S distribution using Kolmogorov-Smirnov, Cramer-von Mises and related distance metrics. Approximate goodness-of-fit and confidence intervals for parameters are calculated using bootstrap methods. We discuss further how the S distribution can be used to solve various problems of statistical modeling associated with parameter inference, including goodness-of-fit tests, Monte Carlo simulations and modeling trends in the distributions.

New Advances in Bayesian Calculation for Linear and Nonlinear Inverse Problems

Abstract: The Bayesian approach has proved to be a coherent approach to handle ill posed Inverse problems. However, the Bayesian calculations need either an optimization or an integral calculation. The maximum a posteriori (MAP) estimation requires the minimization of a compound criterion which, in general, has two parts: a data fitting part and a prior part. In many situations the criterion to be minimized becomes multimodal. The cost of the Simulated Annealing (SA) based techniques is in general huge for inverse problems. Recently a deterministic optimization technique, based on Graduated Non Convexity (GNC), have been proposed to overcome this difficulty. The objective of this paper is to show two specific implementations of this technique for the following situations: -- Linear inverse problems where the solution is modeled as a piecewise continuous function. The non convexity of the criterion is then due to the special choice of the prior; -- A nonlinear inverse problem which arises in inverse scattering where the non convexity of the criterion is due to the likelihood part. Keywords: Inverse problems, Regularization, Bayesian calculation, Global optimization, Graduated Non Convexity.

A Bayesian Approach for the Determination of the Charge Density from Elastic Electron Scattering Data

Abstract: The problem of the determination of the charge density from limited information about the charge form factor is an ill-posed inverse problem. A Bayesian probabilistic approach to this problem which permits to take into account both errors and prior information about the solution is presented. We will show that many classical methods can be considered as special cases of the proposed approach. We address also the problem of the basis function choice for the discretization and the uncertainty of the solution. Some numerical results for an analytical model are presented to show the performance of the proposed method.

Similarity transformations approach for a generalized Fokker-Planck equation

Abstract: By using similarity transformations approach, the exact propagator for a generalized one-dimensional Fokker-Planck equation, with linear drift force and space-time dependent diffusion coefficient, is obtained. The method is simple and enables us to recover and generalize special cases studied through the Lie algebraic approach and the Green function technique.

Quasi-optimal observables: Attaining the quality of maximal likelihood in parameter estimation when only a MC event generator is available

Abstract: A new method of quasi-optimal observables allows one to approach the quality of data processing usually associated with the method of maximal likelihood within the simpler algorithmic context of generalized moments.

Probabilistic methods for data fusion

Abstract: The main object of this paper is to show how we can use classical probabilistic methods such as Maximum Entropy (ME), maximum likelihood (ML) and/or Bayesian (BAYES) approaches to do microscopic and macroscopic data fusion. Actually ME can be used to assign a probability law to an unknown quantity when we have macroscopic data (expectations) on it. ML can be used to estimate the parameters of a probability law when we have microscopic data (direct observation). BAYES can be used to update a prior probability law when we have microscopic data through the likelihood. When we have both microscopic and macroscopic data we can use first ME to assign a prior and then use BAYES to update it to the posterior law thus doing the desired data fusion. However, in practical data fusion applications, we may still need some engineering feeling to propose realistic data fusion solutions. Some simple examples in sensor data fusion and image reconstruction using different kind of data are presented to illustrate these ideas. Keywords: Data fusion, Maximum entropy, Maximum likelihood, Bayesian data fusion, EM algorithm.

Shape reconstruction in X-ray tomography from a small number of projections using deformable models

Abstract: X-ray tomographic image reconstruction consists of determining an object function from its projections. In many applications such as non-destructive testing, we look for a fault region (air) in a homogeneous, known background (metal). The image reconstruction problem then becomes the determination of the shape of the default region. Two approaches can be used: modeling the image as a binary Markov random field and estimating the pixels of the image, or modeling the shape of the fault and estimating it directly from the projections. In this work we model the fault shape by a deformable polygonal disc or a deformable polyhedral volume and propose a new method for directly estimating the coordinates of its vertices from a very limited number of its projections. The basic idea is not new, but in other competing methods, in general, the fault shape is modeled by a small number of parameters (polygonal shapes with very small number of vertices, snakes and deformable templates) and these parameters are estimated either by least squares or by maximum likelihood methods. We propose modeling the shape of the fault region by a polygon with a large number of vertices, allowing modeling of nearly any shape and estimation of its vertices' coordinates directly from the projections by defining the solution as the minimizer of an appropriate regularized criterion. This formulation can also be interpreted as a maximum a posteriori (MAP) estimate in a Bayesian estimation framework. To optimize this criterion we use either a simulated annealing or a special purpose deterministic algorithm based on iterated conditional modes (ICM). The simulated results are very encouraging, especially when the number and the angles of projections are very limited.

The effect of increasing the rate of repetitions of classical reactions

Abstract: Using quantum theory operator methods we discuss the general reversible reactions $A_1+A_2+... A_r \leftrightarrow B_1+B_2+... +B_s$, where $r$ and $s$ are arbitrary natural positive numbers. We show that if either direction of the reaction is repeated a large number of times $N$ in a finite total time $T$ then in the limit of very large $N$, keeping $T$ constant, one remains with the initial reacting particles only. We also show that if the reaction evolves through different possible paths of evolution, each of them beginning at the same side of the reaction, proceeds through different intermediate consecutive reactions and ends at the other side, then one may ``realize'' any such path by performing in a dense manner the set of reactions along it. The same results are also numerically demonstrated for the specific reversible reaction $A+B \leftrightarrow A+C$. We note that similar results have been shown to hold also in the quantum regime.