Showing papers on "Maximum a posteriori estimation published in 1990"
TL;DR: It is shown theoretically and experimentally that the outputs of the MLP approximate the probability distribution over output classes conditioned on the input, i.e. the maximum a posteriori probabilities.
Abstract: The statistical use of a particular classic form of a connectionist system, the multilayer perceptron (MLP), is described in the context of the recognition of continuous speech. A discriminant hidden Markov model (HMM) is defined, and it is shown how a particular MLP with contextual and extra feedback input units can be considered as a general form of such a Markov model. A link between these discriminant HMMs, trained along the Viterbi algorithm, and any other approach based on least mean square minimization of an error function (LMSE) is established. It is shown theoretically and experimentally that the outputs of the MLP (when trained along the LMSE or the entropy criterion) approximate the probability distribution over output classes conditioned on the input, i.e. the maximum a posteriori probabilities. Results of a series of speech recognition experiments are reported. The possibility of embedding MLP into HMM is described. Relations with other recurrent networks are also explained. >
400 citations
TL;DR: Property of the EM algorithm in such contexts are discussed, concentrating on rates of conver- gence, and an alternative that is usually more practical and converges at least as quickly is presented.
Abstract: SUMMARY The EM algorithm is a popular approach to maximum likelihood estimation but has not been much used for penalized likelihood or maximum a posteriori estimation This paper discusses properties of the EM algorithm in such contexts, concentrating on rates of conver- gence, and presents an alternative that is usually more practical and converges at least as quickly The EM algorithm is a general approach to maximum likelihood estimation, rather than a specific algorithm Dempster et al (1977) discussed the method and derived basic properties, demonstrating that a variety of procedures previously developed rather informally could be unified The common strand to problems where the approach is applicable is a notion of 'incomplete data'; this includes the conventional sense of 'missing data' but is much broader than that The EM algorithm demon- strates its strength in situations where some hypothetical experiment yields data from which estimation is particularly convenient and economical: the 'incomplete' data actually at hand are regarded as observable functions of these 'complete' data The resulting algorithms, while usually slow to converge, are often extremely simple and remain practical in large problems where no other approaches may be feasible Dempster et al (1977) briefly refer to the use of the same approach to the problem of finding the posterior mode (maximum a posteriori estimate) in a Bayesian estima-
385 citations
20 May 1990
289 citations
03 Apr 1990
TL;DR: A phoneme based, speaker-dependent continuous-speech recognition system embedding a multilayer perceptron (MLP) into a hidden Markov model (HMM) approach is described, which appears to be somewhat better when MLP methods are used to estimate the probabilities.
Abstract: A phoneme based, speaker-dependent continuous-speech recognition system embedding a multilayer perceptron (MLP) (i.e. a feedforward artificial neural network) into a hidden Markov model (HMM) approach is described. Contextual information from a sliding window on the input frames is used to improve frame or phoneme classification performance over the corresponding performance for simple maximum-likelihood probabilities, or even maximum a posteriori (MAP) probabilities which are estimated without the benefit of context. Performance for a simple discrete density HMM system appears to be somewhat better when MLP methods are used to estimate the probabilities. >
197 citations
TL;DR: Several texture segmentation algorithms based on deterministic and stochastic relaxation principles, and their implementation on parallel networks, are described, and results of the various schemes in classifying some real textured images are presented.
Abstract: Several texture segmentation algorithms based on deterministic and stochastic relaxation principles, and their implementation on parallel networks, are described. The segmentation process is posed as an optimization problem and two different optimality criteria are considered. The first criterion involves maximizing the posterior distribution of the intensity field given the label field (maximum a posteriori estimate). The posterior distribution of the texture labels is derived by modeling the textures as Gauss Markov random fields (GMRFs) and characterizing the distribution of different texture labels by a discrete multilevel Markov model. A stochastic learning algorithm is proposed. This iterated hill-climbing algorithm combines fast convergence of deterministic relaxation with the sustained exploration of the stochastic algorithms, but is guaranteed to find only a local minimum. The second optimality criterion requires minimizing the expected percentage of misclassification per pixel by maximizing the posterior marginal distribution, and the maximum posterior marginal algorithm is used to obtain the corresponding solution. All these methods implemented on parallel networks can be easily extended for hierarchical segmentation; results of the various schemes in classifying some real textured images are presented. >
136 citations
TL;DR: Several aspects of the application of regularization theory in image restoration are presented, extended by extending the applicability of the stabilizing functional approach to 2-D ill-posed inverse problems by proposing a variety of regularizing filters and iterative regularizing algorithms.
Abstract: Several aspects of the application of regularization theory in image restoration are presented. This is accomplished by extending the applicability of the stabilizing functional approach to 2-D ill-posed inverse problems. Inverse restoration is formulated as the constrained minimization of a stabilizing functional. The choice of a particular quadratic functional to be minimized is related to the a priori knowledge regarding the original object through a formulation of image restoration as a maximum a posteriori estimation problem. This formulation is based on image representation by certain stochastic partial differential equation image models. The analytical study and computational treatment of the resulting optimization problem are subsequently presented. As a result, a variety of regularizing filters and iterative regularizing algorithms are proposed. A relationship between the regularized solutions proposed and optimal Wiener estimation is identified. The filters and algorithms proposed are evaluated through several experimental results. >
98 citations
02 Dec 1990
TL;DR: Suboptimal, low-complexity nonlinear equalizer structures derived from the probabilistic symbol-by-symbol maximum a posteriori probability (MAP) algorithm and the M-ary soft-output Viterbi algorithm (SOVA) are investigated.
Abstract: Suboptimal, low-complexity nonlinear equalizer structures derived from the probabilistic symbol-by-symbol maximum a posteriori probability (MAP) algorithm and the M-ary soft-output Viterbi algorithm (SOVA) are investigated. Both algorithms deliver reliability information for each symbol. The complexity of both algorithms in their simplest form is of the order of the conventional reduced-state Viterbi equalizer with hard outputs. Simulation results for a terrestrial time-varying frequency-selective fading channel are given. Realistic channel estimation at high Doppler speeds is included. The coding gain of the investigated 8-state trellis-coded 8-PSK scheme is about 6 dB at a BER (bit error rate) of 10/sup -3/. The gain by making use of soft-decisions is about 4-5 dB at a BER of 10/sup -3/. >
93 citations
TL;DR: A theory of consistent support lines which serves as a basis for reconstruction algorithms that take the form of constrained optimization algorithms is developed, revealing a rich geometry that makes it possible to include prior information about object position and boundary smoothness in the estimation of object shape.
Abstract: Algorithms are proposed for reconstructing convex sets given noisy support line measurements. It is observed that a set of measured support lines may not be consistent with any set in the plane. A theory of consistent support lines which serves as a basis for reconstruction algorithms that take the form of constrained optimization algorithms is developed. The formal statement of the problem and constraints reveals a rich geometry that makes it possible to include prior information about object position and boundary smoothness. The algorithms, which use explicit noise models and prior knowledge, are based on maximum-likelihood and maximum a posteriori estimation principles and are implemented using efficient linear and quadratic programming codes. Experimental results are presented. This research sets the stage for a more general approach to the incorporation of prior information concerning the estimation of object shape. >
92 citations
TL;DR: In this article, a method is developed to directly obtain maximum likelihood estimates of symmetric stable distribution parameters, which is a difficult estimation problem since the likelihood function is expressed as an integral.
Abstract: A method is developed to directly obtain maximum likelihood estimates of symmetric stable distribution parameters. This is a difficult estimation problem since the likelihood function is expressed as an integral. The estimation routine is tested on a Monte Carlo sample and produces reasonable estimates.
80 citations
TL;DR: In this article, Bayes estimators of the parameters of the Marshall-Olkin exponential distribution are obtained when random samples from series and parallel systems are available, with respect to the quadratic loss function, and the prior distribution allows for prior dependence among the components of the parameter vector.
Abstract: SUMMARY Bayes estimators of the parameters of the Marshall-Olkin exponential distribution are obtained when random samples from series and parallel systems are available. The estimators are with respect to the quadratic loss function, and the prior distribution allows for prior dependence among the components of the parameter vector. Exact and approximate highest posterior density credible ellipsoids for the parameters are also obtained. In contrast with series sampling, the Bayes estimators under parallel sampling are not in closed form, and numerical procedures are required to obtain estimates. Bayes estimators of the reliability functions are also given. The gain in asymptotic precision of parallel estimates over series estimates is also ascertained theoretically.
70 citations
TL;DR: It is shown that in the maximum likelihood estimator (MLE) and FMAPE algorithms, the only correct choice of initial image for the iterative procedure in the absence of a priori knowledge about the image configuration is a uniform field.
Abstract: The development and tests of an iterative reconstruction algorithm for emission tomography based on Bayesian statistical concepts are described. The algorithm uses the entropy of the generated image as a prior distribution, can be accelerated by the choice of an exponent, and converges uniformly to feasible images by the choice of one adjustable parameter. A feasible image has been defined as one that is consistent with the initial data (i.e. it is an image that, if truly a source of radiation in a patient, could have generated the initial data by the Poisson process that governs radioactive disintegration). The fundamental ideas of Bayesian reconstruction are discussed, along with the use of an entropy prior with an adjustable contrast parameter, the use of likelihood with data increment parameters as conditional probability, and the development of the new fast maximum a posteriori with entropy (FMAPE) Algorithm by the successive substitution method. It is shown that in the maximum likelihood estimator (MLE) and FMAPE algorithms, the only correct choice of initial image for the iterative procedure in the absence of a priori knowledge about the image configuration is a uniform field. >
TL;DR: In this article, the location and scale parameters of an exponential distribution based on singly and doubly censored samples are estimated using the maximum likelihood (ML) estimation method, which does not admit explicit solutions.
Abstract: The maximum likelihood (ML) estimation of the location and scale parameters of an exponential distribution based on singly and doubly censored samples is given When the sample is multiply censored (some middle observations being censored), however, the ML method does not admit explicit solutions In this case we present a simple approximation to the likelihood equation and derive explicit estimators which are linear functions of order statistics Finally, we present some examples to illustrate this method of estimation
TL;DR: In this paper, an asymptotic solution to Zakai's equation for the unnormalized conditional probability density of the signal, given the noisy measurements, was constructed, and the expansion of the minimum error variance filter and its mean square estimation error (MSEE) was used to construct approximate filters whose MSEE agreed with that of the optimal one.
Abstract: We consider the problem of filtering one-dimensional diffusions with nonlinear drift coefficients, transmitted through a nonlinear fow noise channel. We construct an asymptotic solution to Zakai’s equation for the unnormalized conditional probability density of the signal, given the noisy measurements. This expansion is used to find the asymptotic expansion of the minimum error variance filter and its mean square estimation error (MSEE). We construct approximate filters whose MSEE agrees with that of the optimal one to a given degree of accuracy. The dimension of the approximate filter increases with the required degree of accuracy. Similarly, we expand the maximum a posteriori probability estimator and the minimum energy estimator and compare their performance. We also discuss some extended Kalman filters and present some examples.
TL;DR: In the present method, there are essentially no practical restrictions in modeling the prior and the likelihood, and the Bayes estimate or the posterior mean is used mainly here in addition to the posterior mode.
Abstract: This paper describes a method for an objective selection of the optimal prior distribution, or for adjusting its hyper-parameter, among the competing priors for a variety of Bayesian models. In order to implement this method, the integration of very high dimensional functions is required to get the normalizing constants of the posterior and even of the prior distribution. The logarithm of the high dimensional integral is reduced to the one-dimensional integration of a cerain function with respect to the scalar parameter over the range of the unit interval. Having decided the prior, the Bayes estimate or the posterior mean is used mainly here in addition to the posterior mode. All of these are based on the simulation of Gibbs distributions such as Metropolis' Monte Carlo algorithm. The improvement of the integration's accuracy is substantial in comparison with the conventional crude Monte Carlo integration. In the present method, we have essentially no practical restrictions in modeling the prior and the likelihood. Illustrative artificial data of the lattice system are given to show the practicability of the present procedure.
04 Dec 1990
TL;DR: The authors propose a segmentation algorithm which handles both jump and crease edges and has been integrated with a region-based segmentation scheme resulting in a robust surface segmentation method.
Abstract: Consideration is given to the application of Markov random field (MRF) models to the problem of edge labeling in range images. The authors propose a segmentation algorithm which handles both jump and crease edges. The jump and crease edge likelihoods at each edge site are computed using special local operators. These likelihoods are then combined in a Bayesian framework with a MRF prior distribution on the edge labels to derive the a posterior distribution of labels. An approximation to the maximum a posteriori estimate is used to obtain the edge labelings. The edge-based segmentation has been integrated with a region-based segmentation scheme resulting in a robust surface segmentation method. >
TL;DR: Maximum a posteriori (MAP) estimation is used, together with statistical models for the speckle noise and for the curve-generation process, to find the most probable estimate of the feature, given the image data.
Abstract: A method for finding curves in digital images with speckle noise is described. The solution method differs from standard linear convolutions followed by thresholds in that it explicitly allows curvature in the features. Maximum a posteriori (MAP) estimation is used, together with statistical models for the speckle noise and for the curve-generation process, to find the most probable estimate of the feature, given the image data. The estimation process is first described in general terms. Then, incorporation of the specific neighborhood system and a multiplicative noise model for speckle allows derivation of the solution, using dynamic programming, of the estimation problem. The detection of curvilinear features is considered separately. The detection results allow the determination of the minimal size of detectable feature. Finally, the estimation of linear features, followed by a detection step, is shown for computer-simulated images and for a SAR image of sea ice.
16 Jun 1990
TL;DR: When constraints on the interactions between line processes are removed, the deterministic, graduated nonconvexity (GNC) algorithm has been shown to find close to optimum solutions.
Abstract: An energy function for maximum a posteriori (MAP) image estimation is presented. The energy function is highly nonconvex, and finding the global minimum is a nontrival problem. When constraints on the interactions between line processes are removed, the deterministic, graduated nonconvexity (GNC) algorithm has been shown to find close to optimum solutions. The GNC model is generalized. Any number of constraints on the line processes can be added as a result of using the adiabatic approximation. The resulting algorithm is a combination of the conjugate gradient (CG) and the iterated conditional modes (ICM) algorithms and is completely deterministic. Since the GNC algorithm can be obtained as a special case of this approach, the algorithm is called the generalized GNC or G/sup 2/NC algorithm. It is executed on two aerial images. Results are presented along with comparisons to the GNC algorithm. >
22 Oct 1990
TL;DR: In this article, the effects of several Gibbs prior distributions in terms of noise characteristics, edge sharpness, and overall quantitative accuracy of the final estimates obtained from an iterative maximum a posteriori (MAP) procedure applied to data from a realistic chest phantom are demonstrated.
Abstract: The effects of several of Gibbs prior distributions in terms of noise characteristics, edge sharpness, and overall quantitative accuracy of the final estimates obtained from an iterative maximum a posteriori (MAP) procedure applied to data from a realistic chest phantom are demonstrated. The effects of the adjustable parameters built into the prior distribution on these properties are examined. It is found that these parameter values influence the noise and edge characteristics of the final estimate and can generate reconstructions closer to the actual solution than maximum likelihood (ML). In addition, it is found that the choice of the shape of the prior distribution affects the noise characteristics and edge sharpness in the final estimate. >
TL;DR: A comparison between stochastic and deterministic relaxation algorithms for maximum a posteriori estimation of gray-level images modeled by noncausal Gauss-Markov random fields and corrupted by film grain noise is presented.
Abstract: The authors present a comparison between stochastic and deterministic relaxation algorithms for maximum a posteriori estimation of gray-level images modeled by noncausal Gauss-Markov random fields (GMRF) and corrupted by film grain noise. The degradation involves nonlinear transformation and multiplicative noise. Parameters for the GMRF model were estimated from the original image using maximum-likelihood techniques. To overcome modeling errors, a constraint minimization approach is suggested for estimating the parameters to ensure the positivity of the power spectral density function. Real image experiments with various noise variances and magnitudes of the nonlinear transformation are presented. >
TL;DR: The estimation of 2D motion from spatio-temporally sampled image sequences is discussed, concentrating on the optimization aspect of the problem formulated through a Bayesian framework based on Markov random field models.
TL;DR: In this article, a primal-dual constrained optimization procedure is proposed to reconstruct images from finite sets of noisy projections that may be available only over limited or sparse angles using a Markov random field (MRF) that includes information about the mass, center of mass and convex hull of the object.
TL;DR: In this article, a Bayes estimation procedure is introduced that allows the nature and strength of prior beliefs to be easily specified and modal posterior estimates to be obtained as easily as maximum likelihood estimates.
Abstract: A Bayes estimation procedure is introduced that allows the nature and strength of prior beliefs to be easily specified and modal posterior estimates to be obtained as easily as maximum likelihood estimates. The procedure is based on constructing posterior distributions that are formally identical to likelihoods, but are based on sampled data as well as artificial data reflecting prior information. Improvements in performance of modal Bayes procedures relative to maximum likelihood estimation are illustrated for Rasch-type models. Improvements range from modest to dramatic, depending on the model and the number of items being considered.
03 Apr 1990
TL;DR: A class of discrete image-reconstruction and restoration problems is addressed and the maximum a posteriori (MAP) Bayesian approach with maximum entropy (ME) priors to solve the linear system of equations which arises after the discretization of the integral equations.
Abstract: A class of discrete image-reconstruction and restoration problems is addressed. A brief description is given of the maximum a posteriori (MAP) Bayesian approach with maximum entropy (ME) priors to solve the linear system of equations which is obtained after the discretization of the integral equations which arises in various tomographic image restoration and reconstruction problems. The main problems of choosing an a priori probability law for the image and determining its parameters from the data is discussed. A method simultaneously estimating the parameters of the ME a priori probability density function and the pixel values of the image is proposed, and some simulations which compare this method with some classical ones are given. >
IBM1
TL;DR: It is shown that neural network architectures may offer a valuable alternative to the Bayesian classifier and it is demonstrated that the a posteriori class probabilities can be efficiently computed by a deterministic feedforward network which is called the Boltzmann perceptron classifier (BPC).
Abstract: It is shown that neural network architectures may offer a valuable alternative to the Bayesian classifier. With neural networks, the a posteriori probabilities are computed with no a priori assumptions about the probability distribution functions (PDFs) that generate the data. Rather than assuming certain types of PDFs for the input data, the neural classifier uses a general type of input-output mapping which is then designed to optimally comply with a given set of examples called the training set. It is demonstrated that the a posteriori class probabilities can be efficiently computed by a deterministic feedforward network which is called the Boltzmann perceptron classifier (BPC). Maximum a posteriori (MAP) classifiers are also constructed as a special case of the BPC. Structural relationships between the BPC and a conventional multilayer perceptron (MLP) are given, and it is demonstrated that rather intricate boundaries between classes can be formed even with a relatively modest number of network units. Simulation results show that the BPC is comparable in performance to a Bayesian classifier. >
16 Jun 1990
TL;DR: A Bayesian approach is proposed for stereo matching to derive the maximum a posteriori estimation of depth by using the invariant property of image intensity and modeling the disparity as a Markov random field (MRF).
Abstract: A Bayesian approach is proposed for stereo matching to derive the maximum a posteriori estimation of depth. How a pyramid data structure can be combined with simulated annealing to speed up convergence in stereo matching is described. Using the invariant property of image intensity and modeling the disparity as a Markov random field (MRF), the pyramid structure is followed from high (coarse) level to low (fine) level to derive the maximum a posteriori estimates. Simulation results on both random dot diagrams and synthesized images show the promise of this multiresolution stereo approach. >
01 Apr 1990
TL;DR: A new approach to image segmentation is presented that integrates region and boundary information within a great framework of Maximum a Posteriori (MAP) estimation and decision theory and is shown to be more effective than previous methods in capturing complex region shapes.
Abstract: A new approach to image segmentation is presented that integrates region and boundary information within a great framework of Maximum a Posteriori (MAP) estimation and decision theory. The algorithm employs iterative, decision-directed estimation performed on a spatially localised basis but within a multiresolution representation. The use of a multiresolution technique ensures both robustness in noise and efficiency of computation, while the model-based estimation and decision process is both flexible and spatially local, thus avoiding assumptions about global homogeneity or size and number of regions. The method gives accurate segmentations at low signal-to-noise ratios and is shown to be more effective than previous methods in capturing complex region shapes.
01 Jul 1990
TL;DR: A sensor fusion approach to tissue classification and segmentation in which each of the three images are treated as the output of different sensors and a new deterministic relaxation scheme that updates the belief intervals is presented.
Abstract: Multi-spectral image data fusion techniques for tissue classification of magnetic resonance (MR) images are presented. Using MR it is possible to obtain imagesof proton density the spin-lattice relaxation time constant ( T1) and the spin-spin relaxation time constant (T2) of the same anatomical section of the human body. In this paper we adopt a sensor fusion approach to tissue classification and segmentation in which each of the three images are treated as the output of different sensors. Regions of the images are modeled as noncausal Gaussian Markov random fields (GMRFs) and the underlying tissue label image is also assumed to follow a Gibbs distribution. Two different multi-spectral tissue labeling algorithms maximum a posteriori (MAP) estimation and the Dempster-Shafer evidential reasoning technique are presented. In the Bayesian MAP approach we use an independent opinion pool for data fusion and a deterministic relaxation to obtain the MAP solution. In practice the Bayesian approach may be too restrictive and a likelihood represented by a point probability value is usually an overstatement of what is actually known. In the Dempster-Shafer approach we adopt Dempster''s rule of combination for data fusion using belief intervals and ignorance to represent our confidence in a particular labeling and we present a new deterministic relaxation scheme that updates the belief intervals. Results obtained from real MR images are presented. 1.© (1990) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.
05 Nov 1990
TL;DR: An approximation to the Maximum A Posteriori (MAP) and the Maximum Posterior Marginal (MPM) estimates of the region labels is computed and implemented using a parallel optimization network.
Abstract: Several algorithms for segmenting single look Synthetic Aperture Radar (SAR) complex data into regions of similar and homogeneous statistical characteristics are presented. The image model is composed of two models, one for the speckled complex amplitudes, and the other for the region labels. Speckle is modeled from the physics of the SAR imaging and processing system, and region labels are represented as a Markov random field. Based on this composite image model, an approximation to the Maximum A Posteriori (MAP) and the Maximum Posterior Marginal (MPM) estimates of the region labels is computed and implemented using a parallel optimization network. The performance of this algorithm on highly speckled fine resolution SAR data is discussed and illustrated using both simulated and actual SAR complex data.
TL;DR: In this paper, an iterative identification method for a linear state-space model with outliers and missing data is proposed by applying the Expectation-Maximization (EM) algorithm.