Showing papers on "Maximum a posteriori estimation published in 2007"

Introduction to Bayesian Statistics

Abstract: The Introduction to Bayesian Statistics (2nd Edition) presents Bayes theorem, the estimation of unknown parameters, the determination of confidence regions and the derivation of tests of hypotheses for the unknown parameters, in a manner that is simple, intuitive and easy to comprehend. The methods are applied to linear models, in models for a robust estimation, for prediction and filtering and in models for estimating variance components and covariance components. Regularization of inverse problems and pattern recognition are also covered while Bayesian networks serve for reaching decisions in systems with uncertainties. If analytical solutions cannot be derived, numerical algorithms are presented such as the Monte Carlo integration and Markov Chain Monte Carlo methods.

Bayesian Regularization for Normal Mixture Estimation and Model-Based Clustering

Abstract: Normal mixture models are widely used for statistical modeling of data, including cluster analysis. However maximum likelihood estimation (MLE) for normal mixtures using the EM algorithm may fail as the result of singularities or degeneracies. To avoid this, we propose replacing the MLE by a maximum a posteriori (MAP) estimator, also found by the EM algorithm. For choosing the number of components and the model parameterization, we propose a modified version of BIC, where the likelihood is evaluated at the MAP instead of the MLE. We use a highly dispersed proper conjugate prior, containing a small fraction of one observation's worth of information. The resulting method avoids degeneracies and singularities, but when these are not present it gives similar results to the standard method using MLE, EM and BIC.

Feedforward Carrier Recovery for Coherent Optical Communications

Abstract: We study a carrier-synchronization scheme for coherent optical communications that uses a feedforward architecture that can be implemented in digital hardware without a phase-locked loop. We derive the equations for maximum a posteriori joint detection of the transmitted symbols and the carrier phase. The result is a multidimensional optimization problem that we approximate with a two-stage iterative algorithm: The first stage is a symbol-by-symbol soft detector of the carrier phase, and the second stage is a hard-decision phase estimator that uses prior and subsequent soft-phase decisions to obtain a minimum mean-square-error phase estimate by exploiting the temporal correlation in the phase-noise process. The received symbols are then derotated by the hard-decision phase estimates, and maximum- likelihood sequence detection of the symbols follows. As each component in the carrier-recovery unit can be separately optimized, the resulting system is highly flexible. We show that the optimum hard-decision phase estimator is a linear filter whose impulse response consists of a causal and an anticausal exponential sequence, which we can truncate and implement as an finite-impulse- response filter. We derive equations for the phase-error variance and the system bit-error ratio (BER). Our results show that 4, 8, and 16 quadrature-amplitude-modulation (QAM) transmissions at 1 dB above sensitivity for BER = 10-3 is possible with laser beat linewidths of DeltanuTb = 1.3 X 10-4, 1.3 X 10-4, and 1.5 x 105 when a decision-directed soft-decision phase estimator is employed.

Single Image Motion Deblurring Using Transparency

Abstract: One of the key problems of restoring a degraded image from motion blur is the estimation of the unknown shift-invariant linear blur filter. Several algorithms have been proposed using image intensity or gradient information. In this paper, we separate the image deblurring into filter estimation and image deconvolution processes, and propose a novel algorithm to estimate the motion blur filter from a perspective of alpha values. The relationship between the object boundary transparency and the image motion blur is investigated. We formulate the filter estimation as solving a maximum a posteriori (MAP) problem with the defined likelihood and prior on transparency. Our unified approach can be applied to handle both the camera motion blur and the object motion blur.

Towards a coherent statistical framework for dense deformable template estimation

Abstract: Summary. The problem of estimating probabilistic deformable template models in the field of computer vision or of probabilistic atlases in the field of computational anatomy has not yet received a coherent statistical formulation and remains a challenge. We provide a careful definition and analysis of a well-defined statistical model based on dense deformable templates for grey level images of deformable objects. We propose a rigorous Bayesian framework for which we prove asymptotic consistency of the maximum a posteriori estimate and which leads to an effective iterative estimation algorithm of the geometric and photometric parameters in the small sample setting. The model is extended to mixtures of finite numbers of such components leading to a fine description of the photometric and geometric variations of an object class. We illustrate some of the ideas with images of handwritten digits and apply the estimated models to classification through maximum likelihood.

A MAP Approach for Joint Motion Estimation, Segmentation, and Super Resolution

Abstract: Super resolution image reconstruction allows the recovery of a high-resolution (HR) image from several low-resolution images that are noisy, blurred, and down sampled. In this paper, we present a joint formulation for a complex super-resolution problem in which the scenes contain multiple independently moving objects. This formulation is built upon the maximum a posteriori (MAP) framework, which judiciously combines motion estimation, segmentation, and super resolution together. A cyclic coordinate descent optimization procedure is used to solve the MAP formulation, in which the motion fields, segmentation fields, and HR images are found in an alternate manner given the two others, respectively. Specifically, the gradient-based methods are employed to solve the HR image and motion fields, and an iterated conditional mode optimization method to obtain the segmentation fields. The proposed algorithm has been tested using a synthetic image sequence, the "Mobile and Calendar" sequence, and the original "Motorcycle and Car" sequence. The experiment results and error analyses verify the efficacy of this algorithm

Clinical DT-MRI Estimation, Smoothing, and Fiber Tracking With Log-Euclidean Metrics

Abstract: Diffusion tensor magnetic resonance imaging (DT-MRI or DTI) is an imaging modality that is gaining importance in clinical applications. However, in a clinical environment, data have to be acquired rapidly, often at the expense of the image quality. This often results in DTI datasets that are not suitable for complex postprocessing like fiber tracking. We propose a new variational framework to improve the estimation of DT-MRI in this clinical context. Most of the existing estimation methods rely on a log-Gaussian noise (Gaussian noise on the image logarithms), or a Gaussian noise, that do not reflect the Rician nature of the noise in MR images with a low signal-to-noise ratio (SNR). With these methods, the Rician noise induces a shrinking effect: the tensor volume is underestimated when other noise models are used for the estimation. In this paper, we propose a maximum likelihood strategy that fully exploits the assumption of a Rician noise. To further reduce the influence of the noise, we optimally exploit the spatial correlation by coupling the estimation with an anisotropic prior previously proposed on the spatial regularity of the tensor field itself, which results in a maximum a posteriori estimation. Optimizing such a nonlinear criterion requires adapted tools for tensor computing. We show that Riemannian metrics for tensors, and more specifically the log-Euclidean metrics, are a good candidate and that this criterion can be efficiently optimized. Experiments on synthetic data show that our method correctly handles the shrinking effect even with very low SNR, and that the positive definiteness of tensors is always ensured. Results on real clinical data demonstrate the truthfulness of the proposed approach and show promising improvements of fiber tracking in the brain and the spinal cord.

Variational Bayesian Approach to Movie Rating Prediction

Abstract: Singular value decomposition (SVD) is a matrix decomposition algorithm that returns the optimal (in the sense of squared error) low-rank decomposition of a matrix. SVD has found widespread use across a variety of machine learning applications, where its output is interpreted as compact and informative representations of data. The Netflix Prize challenge, and collaborative filtering in general, is an ideal application for SVD, since the data is a matrix of ratings given by users to movies. It is thus not surprising to observe that most currently successful teams use SVD, either with an extension, or to interpolate with results returned by other algorithms. Unfortunately SVD can easily overfit due to the extreme data sparsity of the matrix in the Netflix Prize challenge, and care must be taken to regularize properly. In this paper, we propose a Bayesian approach to alleviate overfitting in SVD, where priors are introduced and all parameters are integrated out using variational inference. We show experimentally that this gives significantly improved results over vanilla SVD. For truncated SVDs of rank 5, 10, 20, and 30, our proposed Bayesian approach achieves 2.2% improvement over a naive approach, 1.6% improvement over a gradient descent approach dealing with unobserved entries properly, and 0.9% improvement over a maximum a posteriori (MAP) approach.

A Bayesian Approach to Deformed Pattern Matching of Iris Images

Abstract: We describe a general probabilistic framework for matching patterns that experience in-plane nonlinear deformations, such as iris patterns. Given a pair of images, we derive a maximum a posteriori probability (MAP) estimate of the parameters of the relative deformation between them. Our estimation process accomplishes two things simultaneously: it normalizes for pattern warping and it returns a distortion-tolerant similarity metric which can be used for matching two nonlinearly deformed image patterns. The prior probability of the deformation parameters is specific to the pattern-type and, therefore, should result in more accurate matching than an arbitrary general distribution. We show that the proposed method is very well suited for handling iris biometrics, applying it to two databases of iris images which contain real instances of warped patterns. We demonstrate a significant improvement in matching accuracy using the proposed deformed Bayesian matching methodology. We also show that the additional computation required to estimate the deformation is relatively inexpensive, making it suitable for real-time applications

A Class-Adaptive Spatially Variant Mixture Model for Image Segmentation

Abstract: We propose a new approach for image segmentation based on a hierarchical and spatially variant mixture model. According to this model, the pixel labels are random variables and a smoothness prior is imposed on them. The main novelty of this work is a new family of smoothness priors for the label probabilities in spatially variant mixture models. These Gauss-Markov random field-based priors allow all their parameters to be estimated in closed form via the maximum a posteriori (MAP) estimation using the expectation-maximization methodology. Thus, it is possible to introduce priors with multiple parameters that adapt to different aspects of the data. Numerical experiments are presented where the proposed MAP algorithms were tested in various image segmentation scenarios. These experiments demonstrate that the proposed segmentation scheme compares favorably to both standard and previous spatially constrained mixture model-based segmentation

Map-based underdetermined blind source separation of convolutive mixtures by hierarchical clustering and l 1 -norm minimization

Abstract: We address the problem of underdetermined BSS. While most previous approaches are designed for instantaneous mixtures, we propose a time-frequency-domain algorithm for convolutive mixtures. We adopt a two-step method based on a general maximum a posteriori (MAP) approach. In the first step, we estimate the mixing matrix based on hierarchical clustering, assuming that the source signals are suciently sparse. The algorithm works directly on the complex-valued data in the time-frequency domain and shows better convergence than algorithms based on self-organizing maps. The assumption of Laplacian priors for the source signals in the second step leads to an algorithm for estimating the source signals. It involves the l1-norm minimization of complex numbers because of the use of the time-frequency-domain approach. We compare a combinatorial approach initially designed for real numbers with a second-order cone programming (SOCP) approach designed for complex numbers. We found that although the former approach is not theoretically justified for complex numbers, its results are comparable to, or even better than, the SOCP solution. The advantage is a lower computational cost for problems with low input/output dimensions.

Bayesian adaptive lassos with non-convex penalization

Abstract: The lasso (Tibshirani,1996) has sparked interest in the use of penalization of the log-likelihood for variable selection, as well as shrinkage. Recently, there have been attempts to propose penalty functions which improve upon the Lassos properties for variable selection and prediction, such as SCAD (Fan and Li, 2001) and the Adaptive Lasso (Zou, 2006). We adopt the Bayesian interpretation of the Lasso as the maximum a posteriori (MAP) estimate of the regression coefficients, which have been given independent, double exponential prior distributions. Generalizing this prior provides a family of adaptive lasso penalty functions, which includes the quasi-cauchy distribution (Johnstone and Silverman, 2005) as a special case. The properties of this approach are explored. We are particularly interested in the more variables than observations case of characteristic importance for data arising in chemometrics, genomics and proteomics - to name but three. Our methodology can give rise to multiple modes of the posterior distribution and we show how this may occur even with the convex lasso. These multiple modes do no more than reflect the indeterminacy of the model. We give fast algorithms and suggest a strategy of using a set of perfectly fitting random starting values to explore different regions of the parameter space with substantial posterior support. Simulations show that our procedure provides significant improvements on a range of established procedures and we provide an example from chemometrics.

Lagrangian Relaxation for MAP Estimation in Graphical Models

Abstract: We develop a general framework for MAP es- timation in discrete and Gaussian graphical models using Lagrangian relaxation techniques. The key idea is to refor- mulate an intractable estimation problem as one defined on a more tractable graph, but subject to additional constraints. Relaxing these constraints gives a tractable dual problem, one defined by a thin graph, which is then optimized by an iterative procedure. When this iterative optimization leads to a consistent estimate, one which also satisfies the constraints, then it corresponds to an optimal MAP estimate of the original model. Otherwise there is a "duality gap", and we obtain a bound on the optimal solution. Thus, our approach combines convex optimization with dynamic programming techniques applicable for thin graphs. The popular tree-reweighted max- product (TRMP) method may be seen as solving a particular class of such relaxations, where the intractable graph is relaxed to a set of spanning trees. We also consider relaxations to a set of small induced subgraphs, thin subgraphs (e.g. loops), and a connected tree obtained by "unwinding" cycles. In addition, we propose a new class of multiscale relaxations that introduce "summary" variables. The potential benefits of such generalizations include: reducing or eliminating the "duality gap" in hard problems, reducing the number of Lagrange multipliers in the dual problem, and accelerating convergence of the iterative optimization procedure.

Knowledge-Aided Bayesian Detection in Heterogeneous Environments

Abstract: We address the problem of detecting a signal of interest in the presence of noise with unknown covariance matrix, using a set of training samples. We consider a situation where the environment is not homogeneous, i.e., when the covariance matrices of the primary and the secondary data are different. A knowledge-aided Bayesian framework is proposed, where these covariance matrices are considered as random, and some information about the covariance matrix of the training samples is available. Within this framework, the maximum a priori (MAP) estimate of the primary data covariance matrix is derived. It is shown that it amounts to colored loading of the sample covariance matrix of the secondary data. The MAP estimate is in turn used to yield a Bayesian version of the adaptive matched filter. Numerical simulations illustrate the performance of this detector, and compare it with the conventional adaptive matched filter

Super-Resolution Based on Fast Registration and Maximum a Posteriori Reconstruction

Abstract: In this paper, we propose a maximum a posteriori framework for the super-resolution problem, i.e., reconstructing high-resolution images from shifted, rotated, low-resolution degraded observations. The main contributions of this work are two; first, the use of a new locally adaptive edge preserving prior for the super-resolution problem. Second an efficient two-step reconstruction methodology that includes first an initial registration using only the low-resolution degraded observations. This is followed by a fast iterative algorithm implemented in the discrete Fourier transform domain in which the restoration, interpolation and the registration subtasks of this problem are preformed simultaneously. We present examples with both synthetic and real data that demonstrate the advantages of the proposed framework.

Convolutive Blind Source Separation in the Frequency Domain Based on Sparse Representation

Abstract: Convolutive blind source separation (CBSS) that exploits the sparsity of source signals in the frequency domain is addressed in this paper. We assume the sources follow complex Laplacian-like distribution for complex random variable, in which the real part and imaginary part of complex-valued source signals are not necessarily independent. Based on the maximum a posteriori (MAP) criterion, we propose a novel natural gradient method for complex sparse representation. Moreover, a new CBSS method is further developed based on complex sparse representation. The developed CBSS algorithm works in the frequency domain. Here, we assume that the source signals are sufficiently sparse in the frequency domain. If the sources are sufficiently sparse in the frequency domain and the filter length of mixing channels is relatively small and can be estimated, we can even achieve underdetermined CBSS. We illustrate the validity and performance of the proposed learning algorithm by several simulation examples.

Distributed estimation via dual decomposition

Abstract: The focus of this paper is to develop a framework for distributed estimation via convex optimization. We deal with a network of complex sensor subsystems with local estimation and signal processing. More specifically, the sensor subsystems locally solve a maximum likelihood (or maximum a posteriori probability) estimation problem by maximizing a (strictly) concave log-likelihood function subject to convex constraints. These local implementations are not revealed outside the subsystem. The subsystems interact with one another via convex coupling constraints. We discuss a distributed estimation scheme to fuse the local subsystem estimates into a globally optimal estimate that satisfies the coupling constraints. The approach uses dual decomposition techniques in combination with the subgradient method to develop a simple distributed estimation algorithm. Many existing methods of data fusion are suboptimal, i.e., they do not maximize the log-likelihood exactly but rather ‘fuse’ partial results from many processors. For linear gaussian formulation, least mean square (LMS) consensus provides optimal (maximum likelihood) solution. The main contribution of this work is to provide a new approach for data fusion which is based on distributed convex optimization. It applies to a class of problems, described by concave log-likelihood functions, which is much broader than the LMS consensus setup.

Novel estimation methods for unsupervised discovery of latent structure in natural language text

Abstract: This thesis is about estimating probabilistic models to uncover useful hidden structure in data; specifically, we address the problem of discovering syntactic structure in natural language text. We present three new parameter estimation techniques that generalize the standard approach, maximum likelihood estimation, in different ways. Contrastive estimation maximizes the conditional probability of the observed data given a "neighborhood" of implicit negative examples. Skewed deterministic annealing locally maximizes likelihood using a cautious parameter search strategy that starts with an easier optimization problem than likelihood, and iteratively moves to harder problems, culminating in likelihood. Structural annealing is similar, but starts with a heavy bias toward simple syntactic structures and gradually relaxes the bias. Our estimation methods do not make use of annotated examples. We consider their performance in both an unsupervised model selection setting, where models trained under different initialization and regularization settings are compared by evaluating the training objective on a small set of unseen, unannotated development data, and supervised model selection, where the most accurate model on the development set (now with annotations) is selected. The latter is far superior, but surprisingly few annotated examples are required. The experimentation focuses on a single dependency grammar induction task, in depth. The aim is to give strong support for the usefulness of the new techniques in one scenario. It must be noted, however, that the task (as defined here and in prior work) is somewhat artificial, and improved performance on this particular task is not a direct contribution to the greater field of natural language processing. The real problem the task seeks to simulate---the induction of syntactic structure in natural language text---is certainly of interest to the community, but this thesis does not directly approach the problem of exploiting induced syntax in applications. We also do not attempt any realistic simulation of human language learning, as our newspaper text data do not resemble the data encountered by a child during language acquisition. Further, our iterative learning algorithms assume a fixed batch of data that can be repeatedly accessed, not a long stream of data observed over time in tandem with acquisition. (Of course, the cognitive criticisms apply to virtually all existing learning methods in natural language processing, not just the new ones presented here.) Nonetheless, the novel estimation methods presented are, we will argue, better suited to adaptation for real engineering tasks than the maximum likelihood baseline. Our new methods are shown to achieve significant improvements over maximum likelihood estimation and maximum a posteriori estimation, using the EM algorithm, for a state-of-the-art probabilistic model used in dependency grammar induction (Klein and Manning, 2004). The task is to induce dependency trees from part-of-speech tag sequences; we follow standard practice and train and test on sequences of ten tags or fewer. Our results are the best published to date for six languages, with supervised model selection: English (improvement from 41.6% directed attachment accuracy to 66.7%, a 43% relative error rate reduction), German (54.4 → 71.8%, a 38% error reduction), Bulgarian (45.6% → 58.3%, a 23% error reduction), Mandarin (50.0% → 58.0%, a 16% error reduction), Turkish (48.0% → 62.4%, a 28% error reduction, but only 2% error reduction from a left-branching baseline, which gives 61.8%), and Portuguese (42.5% → 71.8%, a 51% error reduction). We also demonstrate the success of contrastive estimation at learning to disambiguate part-of-speech tags (from unannotated English text): 78.0% to 88.7% tagging accuracy on a known-dictionary task (a 49% relative error rate reduction), and 66.5% to 78.4% on a more difficult task with less dictionary knowledge (a 35% error rate reduction). The experiments presented in this thesis give one of the most thorough explorations to date of unsupervised parameter estimation for models of discrete structures. Two sides of the problem are considered in depth: the choice of objective function to be optimized during training, and the method of optimizing it. We find that both are important in unsupervised learning. Our best results on most of the six languages involve both improved objectives and improved search. The methods presented in this thesis were originally presented in Smith and Eisner (2004, 2005a,b, 2006). The thesis gives a more thorough exposition, relating the methods to other work, presents more experimental results and error analysis, and directly compares the methods to each other.

Model distortions in Bayesian MAP reconstruction

Abstract: The Bayesian approach and especially the maximum a posteriori (MAP) estimator is most widely used to solve various problems in signal and image processing, such as denoising and deblurring, zooming, and reconstruction. The reason is that it provides a coherent statistical framework to combine observed (noisy) data with prior information on the unknown signal or image which is optimal in a precise statistical sense. This paper presents an objective critical analysis of the MAP approach. It shows that the MAP solutions substantially deviate from both the data-acquisition model and the prior model that underly the MAP estimator. This is explained theoretically using several analytical properties of the MAP solutions and is illustrated using examples and experiments. It follows that the MAP approach is not relevant in the applications where the data-observation and the prior models are accurate. The construction of solutions (estimators) that respect simultaneously two such models remains an open question.

Batch and Sequential Bayesian Estimators of the Number of Active Terminals in an IEEE 802.11 Network

Abstract: The performance of the IEEE 802.11 protocol based on the distributed coordination function (DCF) has been shown to be dependent on the number of competing terminals and the backoff parameters. Better performance can be expected if the parameters are adapted to the number of active users. In this paper we develop both off-line and online Bayesian signal processing algorithms to estimate the number of competing terminals. The estimation is based on the observed use of the channel and the number of competing terminals is modeled as a Markov chain with unknown transition matrix. The off-line estimator makes use of the Gibbs sampler whereas the first online estimator is based on the sequential Monte Carlo (SMC) technique. A deterministic variant of the SMC estimator is then developed, which is simpler to implement and offers superior performance. Finally a novel approximate maximum a posteriori (MAP) algorithm for hidden Markov models (HMM) with unknown transition matrix is proposed. Realistic IEEE 802.11 simulations using the ns-2 network simulator are provided to demonstrate the excellent performance of the proposed estimators

Estimation receiver operating characteristic curve and ideal observers for combined detection/estimation tasks.

Abstract: The localization receiver operating characteristic (LROC) curve is a standard method to quantify performance for the task of detecting and locating a signal. This curve is generalized to arbitrary detection/estimation tasks to give the estimation ROC (EROC) curve. For a two-alternative forced-choice study, where the observer must decide which of a pair of images has the signal and then estimate parameters pertaining to the signal, it is shown that the average value of the utility on those image pairs where the observer chooses the correct image is an estimate of the area under the EROC curve (AEROC). The ideal LROC observer is generalized to the ideal EROC observer, whose EROC curve lies above those of all other observers for the given detection/estimation task. When the utility function is nonnegative, the ideal EROC observer is shown to share many mathematical properties with the ideal observer for the pure detection task. When the utility function is concave, the ideal EROC observer makes use of the posterior mean estimator. Other estimators that arise as special cases include maximum a posteriori estimators and maximum-likelihood estimators.

Confident Interpretation of Bayesian Decision Tree Ensembles for Clinical Applications

Abstract: Bayesian averaging (BA) over ensembles of decision models allows evaluation of the uncertainty of decisions that is of crucial importance for safety-critical applications such as medical diagnostics. The interpretability of the ensemble can also give useful information for experts responsible for making reliable decisions. For this reason, decision trees (DTs) are attractive decision models for experts. However, BA over such models makes an ensemble of DTs uninterpretable. In this paper, we present a new approach to probabilistic interpretation of Bayesian DT ensembles. This approach is based on the quantitative evaluation of uncertainty of the DTs, and allows experts to find a DT that provides a high predictive accuracy and confident outcomes. To make the BA over DTs feasible in our experiments, we use a Markov Chain Monte Carlo technique with a reversible jump extension. The results obtained from clinical data show that in terms of predictive accuracy, the proposed method outperforms the maximum a posteriori (MAP) method that has been suggested for interpretation of DT ensembles

Row-Column Soft-Decision Feedback Algorithm for Two-Dimensional Intersymbol Interference

Abstract: We present a novel iterative row-column soft decision feedback algorithm (IRCSDFA) for detection of binary images corrupted by 2-D intersymbol interference and additive white Gaussian noise. The algorithm exchanges weighted soft information between row and column maximum a posteriori (MAP) detectors. Each MAP detector exploits soft-decision feedback from previously processed rows or columns. The new algorithm gains about 0.3 dB over the previously best published results for the 2times2 averaging mask. For a non-separable 3times3 mask, the IRCSDFA gains 0.8 dB over a previous soft-input/soft-output iterative algorithm which decomposes the 2-D convolution into 1-D row and column operations.

Statistical maritime radar duct estimation using hybrid genetic algorithm–Markov chain Monte Carlo method

Abstract: [1] This paper addresses the problem of estimating the lower atmospheric refractivity ( M profile) under nonstandard propagation conditions frequently encountered in low-altitude maritime radar applications. This is done by statistically estimating the duct strength (range- and height-dependent atmospheric index of refraction) from the sea surface reflected radar clutter. These environmental statistics can then be used to predict the radar performance. In previous work, genetic algorithms (GA) and Markov chain Monte Carlo (MCMC) samplers were used to calculate the atmospheric refractivity from returned radar clutter. Although GA is fast and estimates the maximum a posteriori ( MAP) solution well, it poorly calculates the multidimensional integrals required to obtain the means, variances, and underlying posterior probability distribution functions of the estimated parameters. More accurate distributions and integral calculations can be obtained using MCMC samplers, such as the Metropolis-Hastings and Gibbs sampling (GS) algorithms. Their drawback is that they require a large number of samples relative to the global optimization techniques such as GA and become impractical with an increasing number of unknowns. A hybrid GA-MCMC method based on the nearest neighborhood algorithm is implemented in this paper. It is an improved GA method which improves integral calculation accuracy through hybridization with a MCMC sampler. Since the number of forward models is determined by GA, it requires fewer forward model samples than a MCMC, enabling inversion of atmospheric models with a larger number of unknowns.

Log transformation benefits parameter estimation in microwave tomographic imaging.

Abstract: Microwave tomographic imaging falls under a broad category of nonlinear parameter estimation methods when a Gauss-Newton iterative reconstruction technique is used. A fundamental requirement in using these approaches is evaluating the appropriateness of the regression model. While there have been numerous investigations of regularization techniques to improve overall image quality, few, if any, studies have explored the underlying statistical properties of the model itself. The ordinary least squares (OLS) approach is used most often, but there are other options such as the weighted least squares (WLS), maximum likelihood (ML), and maximum a posteriori (MAP) that may be more appropriate. In addition, a number of variance stabilizing transformations can be applied to make the inversion intrinsically more linear. In this paper, a statistical analysis is performed of the properties of the residual errors from the reconstructed images utilizing actual measured data and it is demonstrated that the OLS algorithm with a log transformation ( OLS log ) is clearly advantageous relative to the more commonly used OLS approach by itself. In addition, several high contrastimaging experiments are performed, which demonstrate that different subsets of data are emphasized in each method and may contribute to the overall image quality differences.

Simplified Soft-Output Detection of CPM Signals Over Coherent and Phase Noise Channels

Abstract: We consider continuous phase modulations (CPMs) in iteratively decoded serially concatenated schemes. Although the overall receiver complexity mainly depends on that of the CPM detector, almost all papers in the literature consider the optimal maximum a posteriori (MAP) symbol detection algorithm and only a few attempts have been made to design low-complexity suboptimal schemes. This problem is faced in this paper by first considering the case of an ideal coherent detection, then extending it to the more interesting case of a transmission over a typical satellite channel affected by phase noise. In both cases, we adopt a simplified representation of an M-ary CPM signal based on the principal pulses of its Laurent decomposition. Since it is not possible to derive the exact detection rule by means of a probabilistic reasoning, the framework of factor graphs (FGs) and the sum-product algorithm (SPA) is used. In the case of channels affected by phase noise, continuous random variables representing the phase samples are explicitly introduced in the FG. By pursuing the principal approach to manage continuous random variables in a FG, i.e., the canonical distribution approach, two algorithms are derived which do not require the presence of known (pilot) symbols, thanks to the intrinsic differential encoder embedded in the CPM modulator.