scispace - formally typeset
Search or ask a question

Showing papers on "Markov random field published in 2011"


Journal ArticleDOI
TL;DR: This work describes the technical and implementation aspects of Atropos, an ITK-based multivariate n-class open source segmentation algorithm distributed with ANTs and evaluates its performance on two different ground-truth datasets.
Abstract: We introduce Atropos, an ITK-based multivariate n-class open source segmentation algorithm distributed with ANTs (http://www.picsl.upenn.edu/ANTs). The Bayesian formulation of the segmentation problem is solved using the Expectation Maximization (EM) algorithm with the modeling of the class intensities based on either parametric or non-parametric finite mixtures. Atropos is capable of incorporating spatial prior probability maps (sparse), prior label maps and/or Markov Random Field (MRF) modeling. Atropos has also been efficiently implemented to handle large quantities of possible labelings (in the experimental section, we use up to 69 classes) with a minimal memory footprint. This work describes the technical and implementation aspects of Atropos and evaluates its performance on two different ground-truth datasets. First, we use the BrainWeb dataset from Montreal Neurological Institute to evaluate three-tissue segmentation performance via (1) K-means segmentation without use of template data; (2) MRF segmentation with initialization by prior probability maps derived from a group template; (3) Prior-based segmentation with use of spatial prior probability maps derived from a group template. We also evaluate Atropos performance by using spatial priors to drive a 69-class EM segmentation problem derived from the Hammers atlas from University College London. These evaluation studies, combined with illustrative examples that exercise Atropos options, demonstrate both performance and wide applicability of this new platform-independent open source segmentation tool.

471 citations


Journal ArticleDOI
TL;DR: This paper proposes a novel, nonparametric approach for object recognition and scene parsing using a new technology the authors name label transfer, which is easy to implement, has few parameters, and embeds contextual information naturally in the retrieval/alignment procedure.
Abstract: While there has been a lot of recent work on object recognition and image understanding, the focus has been on carefully establishing mathematical models for images, scenes, and objects. In this paper, we propose a novel, nonparametric approach for object recognition and scene parsing using a new technology we name label transfer. For an input image, our system first retrieves its nearest neighbors from a large database containing fully annotated images. Then, the system establishes dense correspondences between the input image and each of the nearest neighbors using the dense SIFT flow algorithm [28], which aligns two images based on local image structures. Finally, based on the dense scene correspondences obtained from SIFT flow, our system warps the existing annotations and integrates multiple cues in a Markov random field framework to segment and recognize the query image. Promising experimental results have been achieved by our nonparametric scene parsing system on challenging databases. Compared to existing object recognition approaches that require training classifiers or appearance models for each object category, our system is easy to implement, has few parameters, and embeds contextual information naturally in the retrieval/alignment procedure.

431 citations


Book
07 Oct 2011
TL;DR: The uncertainty modeling techniques that are developed, and the utility of these techniques in various applications, support the claim that Bayesian modeling is a powerful and practical framework for low-level vision.
Abstract: Over the last decade, many low-level vision algorithms have been devised for extracting depth from one or more intensity images. The output of such algorithms usually contains no indication of the uncertainty associated with the scene reconstruction. In other areas of computer vision and robotics, the need for such error modeling is becoming recognized, both because of the uncertainty inherent in sensing and because of the desire to integrate information from different sensors or viewpoints. In this thesis, we develop a new Bayesian model for the dense fields that are commonly used in low-level vision. The Bayesian model consists of three components: a prior model, a sensor model, and a posterior model. The prior model captures any a priori information about the structure of the dense field. We construct this model by using the smoothness constraints for regularization to define a Markov Random Field. The sensor model describes the behaviour and noise characteristics of our measurement system. We develop a number of sensor models for both sparse depth measurements and dense flow or intensity measurements. The posterior model combines the information from the prior and sensor models using Bayes' Rule, and can be used as the input to later stages of processing. We show how to compute optimal estimates from the posterior model, and also how to compute the uncertainty (variance) in these estimates. This thesis applies Bayesian modeling to a number of low-level vision problems. The main application is the on-line extraction of depth from motion. For this application, we use a two-dimensional generalization of the Kalman filter to convert the current posterior model into a prior model for the next estimate. The resulting incremental algorithm provides a dense on-line estimate of depth whose uncertainty and error are reduced over time. Other applications of Bayesian modeling, include the choice of optimal smoothing parameter for interpolation; the determination of observer motion from sparse depth measurements without correspondence; and the construction of multiresolution relative surface representations. The approach to uncertainty modeling which we develop, and the utility of this approach in various applications, support our thesis that Bayesian modeling is a useful and practical framework for low-level vision.

341 citations


Journal ArticleDOI
TL;DR: A context-sensitive technique for unsupervised change detection in multitemporal remote sensing images based on fuzzy clustering approach and takes care of spatial correlation between neighboring pixels of the difference image produced by comparing two images acquired on the same geographical area at different times.

307 citations


Proceedings ArticleDOI
20 Jun 2011
TL;DR: This work presents an alternative formulation for SfM based on finding a coarse initial solution using a hybrid discrete-continuous optimization, and then improving that solution using bundle adjustment, and shows that it can produce models that are similar to or better than those produced with incremental bundles adjustment, but more robustly and in a fraction of the time.
Abstract: Recent work in structure from motion (SfM) has successfully built 3D models from large unstructured collections of images downloaded from the Internet. Most approaches use incremental algorithms that solve progressively larger bundle adjustment problems. These incremental techniques scale poorly as the number of images grows, and can drift or fall into bad local minima. We present an alternative formulation for SfM based on finding a coarse initial solution using a hybrid discrete-continuous optimization, and then improving that solution using bundle adjustment. The initial optimization step uses a discrete Markov random field (MRF) formulation, coupled with a continuous Levenberg-Marquardt refinement. The formulation naturally incorporates various sources of information about both the cameras and the points, including noisy geotags and vanishing point estimates. We test our method on several large-scale photo collections, including one with measured camera positions, and show that it can produce models that are similar to or better than those produced with incremental bundle adjustment, but more robustly and in a fraction of the time.

283 citations


Book
22 Jul 2011
TL;DR: This volume demonstrates the power of the Markov random field in vision, treating the MRF both as a tool for modeling image data and, utilizing recently developed algorithms, as a means of making inferences about images.
Abstract: This volume demonstrates the power of the Markov random field (MRF) in vision, treating the MRF both as a tool for modeling image data and, utilizing recently developed algorithms, as a means of making inferences about images. These inferences concern underlying image and scene structure as well as solutions to such problems as image reconstruction, image segmentation, 3D vision, and object labeling. It offers key findings and state-of-the-art research on both algorithms and applications. After an introduction to the fundamental concepts used in MRFs, the book reviews some of the main algorithms for performing inference with MRFs; presents successful applications of MRFs, including segmentation, super-resolution, and image restoration, along with a comparison of various optimization methods; discusses advanced algorithmic topics; addresses limitations of the strong locality assumptions in the MRFs discussed in earlier chapters; and showcases applications that use MRFs in more complex ways, as components in bigger systems or with multiterm energy functions. The book will be an essential guide to current research on these powerful mathematical tools.

272 citations


Proceedings ArticleDOI
20 Jun 2011
TL;DR: This paper proposes to recognize collective activities using the crowd context and introduces a new scheme for learning it automatically, constructed upon a Random Forest structure which randomly samples variable volume spatio-temporal regions to pick the most discriminating attributes for classification.
Abstract: In this paper we present a framework for the recognition of collective human activities. A collective activity is defined or reinforced by the existence of coherent behavior of individuals in time and space. We call such coherent behavior ‘Crowd Context’. Examples of collective activities are “queuing in a line” or “talking”. Following [7], we propose to recognize collective activities using the crowd context and introduce a new scheme for learning it automatically. Our scheme is constructed upon a Random Forest structure which randomly samples variable volume spatio-temporal regions to pick the most discriminating attributes for classification. Unlike previous approaches, our algorithm automatically finds the optimal configuration of spatio-temporal bins, over which to sample the evidence, by randomization. This enables a methodology for modeling crowd context. We employ a 3D Markov Random Field to regularize the classification and localize collective activities in the scene. We demonstrate the flexibility and scalability of the proposed framework in a number of experiments and show that our method outperforms state-of-the art action classification techniques [7, 19].

266 citations


Proceedings ArticleDOI
06 Nov 2011
TL;DR: The model can be seen as a Markov random field of topic models, which connects the documents based on their similarity, and the topics learned with the model are shared across connected documents, thus encoding the relations between different modalities.
Abstract: Many applications involve multiple-modalities such as text and images that describe the problem of interest. In order to leverage the information present in all the modalities, one must model the relationships between them. While some techniques have been proposed to tackle this problem, they either are restricted to words describing visual objects only, or require full correspondences between the different modalities. As a consequence, they are unable to tackle more realistic scenarios where a narrative text is only loosely related to an image, and where only a few image-text pairs are available. In this paper, we propose a model that addresses both these challenges. Our model can be seen as a Markov random field of topic models, which connects the documents based on their similarity. As a consequence, the topics learned with our model are shared across connected documents, thus encoding the relations between different modalities. We demonstrate the effectiveness of our model for image retrieval from a loosely related text.

203 citations


Journal ArticleDOI
TL;DR: A novel deformable image registration paradigm that exploits Markov random field formulation and powerful discrete optimization algorithms is introduced, leading to a modular, powerful, and flexible formulation that can account for arbitrary image-matching criteria, various local deformation models, and regularization constraints.
Abstract: This review introduces a novel deformable image registration paradigm that exploits Markov random field formulation and powerful discrete optimization algorithms. We express deformable registration as a minimal cost graph problem, where nodes correspond to the deformation grid, a node's connectivity corresponds to regularization constraints, and labels correspond to 3D deformations. To cope with both iconic and geometric (landmark-based) registration, we introduce two graphical models, one for each subproblem. The two graphs share interconnected variables, leading to a modular, powerful, and flexible formulation that can account for arbitrary image-matching criteria, various local deformation models, and regularization constraints. To cope with the corresponding optimization problem, we adopt two optimization strategies: a computationally efficient one and a tight relaxation alternative. Promising results demonstrate the potential of this approach. Discrete methods are an important new trend in medical image registration, as they provide several improvements over the more traditional continuous methods. This is illustrated with several key examples where the presented framework outperforms existing general-purpose registration methods in terms of both performance and computational complexity. Our methods become of particular interest in applications where computation time is a critical issue, as in intraoperative imaging, or where the huge variation in data demands complex and application-specific matching criteria, as in large-scale multimodal population studies. The proposed registration framework, along with a graphical interface and corresponding publications, is available for download for research purposes (for Windows and Linux platforms) from http://www.mrf-registration.net.

202 citations


Proceedings ArticleDOI
20 Jun 2011
TL;DR: This paper presents a method for joint stereo matching and object segmentation that is able to recover the depth of regions that are fully occluded in one input view, which to the knowledge is new for stereo matching.
Abstract: This paper presents a method for joint stereo matching and object segmentation. In our approach a 3D scene is represented as a collection of visually distinct and spatially coherent objects. Each object is characterized by three different aspects: a color model, a 3D plane that approximates the object's disparity distribution, and a novel 3D connectivity property. Inspired by Markov Random Field models of image segmentation, we employ object-level color models as a soft constraint, which can aid depth estimation in powerful ways. In particular, our method is able to recover the depth of regions that are fully occluded in one input view, which to our knowledge is new for stereo matching. Our model is formulated as an energy function that is optimized via fusion moves. We show high-quality disparity and object segmentation results on challenging image pairs as well as standard benchmarks. We believe our work not only demonstrates a novel synergy between the areas of image segmentation and stereo matching, but may also inspire new work in the domain of automatic and interactive object-level scene manipulation.

182 citations


Journal ArticleDOI
TL;DR: An adaptive Markov random field approach is proposed for classification of hyperspectral imagery with the introduction of a relative homogeneity index for each pixel and the use of this index to determine an appropriate weighting coefficient for the spatial contribution in the MRF classification.
Abstract: An adaptive Markov random field (MRF) approach is proposed for classification of hyperspectral imagery in this letter. The main feature of the proposed method is the introduction of a relative homogeneity index for each pixel and the use of this index to determine an appropriate weighting coefficient for the spatial contribution in the MRF classification. In this way, overcorrection of spatially high variation areas can be avoided. Support vector machines are implemented for improved class modeling and better estimate of spectral contribution to this approach. Experimental results of a synthetic hyperspectral data set and a real hyperspectral image demonstrate that the proposed method works better on both homogeneous regions and class boundaries with improved classification accuracy.

Journal ArticleDOI
TL;DR: This study model the phasor angles across the buses as a Markov random field (MRF) and devise a multiscale network inference algorithm that carries out fault detection and localization in a decentralized manner.
Abstract: Fault diagnosis in power grids is known to be challenging, due to the massive scale and spatial coupling therein. In this study, we explore multiscale network inference for fault detection and localization. Specifically, we model the phasor angles across the buses as a Markov random field (MRF), where the conditional correlation coefficients of the MRF are quantified in terms of the physical parameters of power systems. Based on the MRF model, we then study decentralized network inference for fault diagnosis, through change detection and localization in the conditional correlation matrix of the MRF. Particularly, based on the hierarchical topology of practical power systems, we devise a multiscale network inference algorithm that carries out fault detection and localization in a decentralized manner. Simulation results are used to demonstrate the effectiveness of the proposed approach.

Journal ArticleDOI
TL;DR: A Bayesian model is introduced to exploit the possible spatial correlations between the pixels in hyperspectral image unmixing and a Markov chain Monte Carlo method is considered that generates samples asymptotically distributed according to the posterior.
Abstract: This paper describes a new algorithm for hyperspectral image unmixing. Most unmixing algorithms proposed in the literature do not take into account the possible spatial correlations between the pixels. In this paper, a Bayesian model is introduced to exploit these correlations. The image to be unmixed is assumed to be partitioned into regions (or classes) where the statistical properties of the abundance coefficients are homogeneous. A Markov random field, is then proposed to model the spatial dependencies between the pixels within any class. Conditionally upon a given class, each pixel is modeled by using the classical linear mixing model with additive white Gaussian noise. For this model, the posterior distributions of the unknown parameters and hyperparameters allow the parameters of interest to be inferred. These parameters include the abundances for each pixel, the means and variances of the abundances for each class, as well as a classification map indicating the classes of all pixels in the image. To overcome the complexity of the posterior distribution, we consider a Markov chain Monte Carlo method that generates samples asymptotically distributed according to the posterior. The generated samples are then used for parameter and hyperparameter estimation. The accuracy of the proposed algorithms is illustrated on synthetic and real data.

Journal ArticleDOI
TL;DR: This work introduces a transformation of general higher-order Markov random field with binary labels into a first-order one that has the same minima as the original, and formalizes a framework for approximately minimizing higher- order multilabel MRF energies that combines the new reduction with the fusion-move and QPBO algorithms.
Abstract: We introduce a transformation of general higher-order Markov random field with binary labels into a first-order one that has the same minima as the original. Moreover, we formalize a framework for approximately minimizing higher-order multilabel MRF energies that combines the new reduction with the fusion-move and QPBO algorithms. While many computer vision problems today are formulated as energy minimization problems, they have mostly been limited to using first-order energies, which consist of unary and pairwise clique potentials, with a few exceptions that consider triples. This is because of the lack of efficient algorithms to optimize energies with higher-order interactions. Our algorithm challenges this restriction that limits the representational power of the models so that higher-order energies can be used to capture the rich statistics of natural scenes. We also show that some minimization methods can be considered special cases of the present framework, as well as comparing the new method experimentally with other such techniques.

Journal ArticleDOI
TL;DR: This work presents an extensive validation of Nyul's approach for intensity normalization in real clinical domain where even after intensity inhomogeneity correction that accounts for scanner-specific artifacts, the MRI volumes can be affected from variations such as data heterogeneity resulting from multi-site multi-scanner acquisitions, the presence of multiple sclerosis lesions and the stage of disease progression in the brain.

Journal ArticleDOI
TL;DR: The experiments that were performed on a bitemporal TerraSAR-X StripMap data set from South West England during and after a large-scale flooding in 2007 confirm the effectiveness of the proposed change detection method and show an increased classification accuracy of the hybrid MRF model in comparison to the sole application of the HMAP estimation.
Abstract: The near real-time provision of precise information about flood dynamics from synthetic aperture radar (SAR) data is an essential task in disaster management. A novel tile-based parametric thresholding approach under the generalized Gaussian assumption is applied on normalized change index data to automatically solve the three-class change detection problem in large-size images with small class a priori probabilities. The thresholding result is used for the initialization of a hybrid Markov model which integrates scale-dependent and spatiocontextual information into the labeling process by combining hierarchical with noncausal Markov image modeling. Hierarchical maximum a posteriori (HMAP) estimation using the Markov chains in scale, originally developed on quadtrees, is adapted to hierarchical irregular graphs. To reduce the computational effort of the iterative optimization process that is related to noncausal Markov models, a Markov random field (MRF) approach is defined, which is applied on a restricted region of the lowest level of the graph, selected according to the HMAP labeling result. The experiments that were performed on a bitemporal TerraSAR-X StripMap data set from South West England during and after a large-scale flooding in 2007 confirm the effectiveness of the proposed change detection method and show an increased classification accuracy of the hybrid MRF model in comparison to the sole application of the HMAP estimation. Additionally, the impact of the graph structure and the chosen model parameters on the labeling result as well as on the performance is discussed.

Journal ArticleDOI
TL;DR: In this paper, a Fisher-Markov Selector is proposed to select the most discriminating subset of variables by solving a very simple unconstrained objective function which in fact can be obtained with an explicit expression.
Abstract: Selecting features for multiclass classification is a critically important task for pattern recognition and machine learning applications. Especially challenging is selecting an optimal subset of features from high-dimensional data, which typically have many more variables than observations and contain significant noise, missing components, or outliers. Existing methods either cannot handle high-dimensional data efficiently or scalably, or can only obtain local optimum instead of global optimum. Toward the selection of the globally optimal subset of features efficiently, we introduce a new selector - which we call the Fisher-Markov selector - to identify those features that are the most useful in describing essential differences among the possible groups. In particular, in this paper we present a way to represent essential discriminating characteristics together with the sparsity as an optimization objective. With properly identified measures for the sparseness and discriminativeness in possibly high-dimensional settings, we take a systematic approach for optimizing the measures to choose the best feature subset. We use Markov random field optimization techniques to solve the formulated objective functions for simultaneous feature selection. Our results are noncombinatorial, and they can achieve the exact global optimum of the objective function for some special kernels. The method is fast; in particular, it can be linear in the number of features and quadratic in the number of observations. We apply our procedure to a variety of real-world data, including mid--dimensional optical handwritten digit data set and high-dimensional microarray gene expression data sets. The effectiveness of our method is confirmed by experimental results. In pattern recognition and from a model selection viewpoint, our procedure says that it is possible to select the most discriminating subset of variables by solving a very simple unconstrained objective function which in fact can be obtained with an explicit expression.

Journal ArticleDOI
TL;DR: Experiments on real-life surveillance videos demonstrate that the proposed sequential technique for static background estimation obtains considerably better background estimates (both qualitatively and quantitatively) than median filtering and the recently proposed "intervals of stable intensity" method.
Abstract: For the purposes of foreground estimation, the true background model is unavailable in many practical circumstances and needs to be estimated from cluttered image sequences. We propose a sequential technique for static background estimation in such conditions, with low computational and memory requirements. Image sequences are analysed on a block-by-block basis. For each block location a representative set is maintained which contains distinct blocks obtained along its temporal line. The background estimation is carried out in a Markov Random Field framework, where the optimal labelling solution is computed using iterated conditional modes. The clique potentials are computed based on the combined frequency response of the candidate block and its neighbourhood. It is assumed that the most appropriate block results in the smoothest response, indirectly enforcing the spatial continuity of structures within a scene. Experiments on real-life surveillance videos demonstrate that the proposedmethod obtains considerably better background estimates (both qualitatively and quantitatively) than median filtering and the recently proposed "intervals of stable intensity" method. Further experiments on the Wallflower dataset suggest that the combination of the proposed method with a foreground segmentation algorithm results in improved foreground segmentation.

Journal ArticleDOI
TL;DR: This paper demonstrates that near-optimal performance in MIMO-ISI channels with large dimensions can be achieved at low complexities through simple yet effective simplifications/approximations, and shows that these message passing algorithms can be used in an iterative manner with local neighborhood search algorithms to improve the reliability/performance of M-QAM symbol detection.
Abstract: In this paper, we deal with low-complexity near-optimal detection/equalization in large-dimension multiple-input multiple-output inter-symbol interference (MIMO-ISI) channels using message passing on graphical models. A key contribution in the paper is the demonstration that near-optimal performance in MIMO-ISI channels with large dimensions can be achieved at low complexities through simple yet effective simplifications/approximations, although the graphical models that represent MIMO-ISI channels are fully/densely connected (loopy graphs). These include 1) use of Markov random field (MRF)-based graphical model with pairwise interaction, in conjunction with message damping, and 2) use of factor graph (FG)-based graphical model with Gaussian approximation of interference (GAI). The per-symbol complexities are O(K2nt2) and O(Knt) for the MRF and the FG with GAI approaches, respectively, where K and nt denote the number of channel uses per frame, and number of transmit antennas, respectively. These low-complexities are quite attractive for large dimensions, i.e., for large Knt. From a performance perspective, these algorithms are even more interesting in large-dimensions since they achieve increasingly closer to optimum detection performance for increasing Knt. Also, we show that these message passing algorithms can be used in an iterative manner with local neighborhood search algorithms to improve the reliability/performance of M-QAM symbol detection.

Journal ArticleDOI
TL;DR: This work proposes a contextual and probabilistic method for detection of tree crowns in urban areas using a Markov random field based super resolution mapping (SRM) approach in very high resolution images.
Abstract: Identification of tree crowns from remote sensing requires detailed spectral information and submeter spatial resolution imagery. Traditional pixel-based classification techniques do not fully exploit the spatial and spectral characteristics of remote sensing datasets. We propose a contextual and probabilistic method for detection of tree crowns in urban areas using a Markov random field based super resolution mapping (SRM) approach in very high resolution images. Our method defines an objective energy function in terms of the conditional probabilities of panchromatic and multispectral images and it locally optimizes the labeling of tree crown pixels. Energy and model parameter values are estimated from multiple implementations of SRM in tuning areas and the method is applied in QuickBird images to produce a 0.6 m tree crown map in a city of The Netherlands. The SRM output shows an identification rate of 66% and commission and omission errors in small trees and shrub areas. The method outperforms tree crown identification results obtained with maximum likelihood, support vector machines and SRM at nominal resolution (2.4 m) approaches.

Journal ArticleDOI
TL;DR: The authors formulate background subtraction as minimizing a penalized instantaneous risk functional-yielding a local online discriminative algorithm that can quickly adapt to temporal changes and develop an implementation that can run efficiently on the highly parallel graphics processing unit (GPU).
Abstract: The authors examine the problem of segmenting foreground objects in live video when background scene textures change over time. In particular, we formulate background subtraction as minimizing a penalized instantaneous risk functional-yielding a local online discriminative algorithm that can quickly adapt to temporal changes. We analyze the algorithm's convergence, discuss its robustness to nonstationarity, and provide an efficient nonlinear extension via sparse kernels. To accommodate interactions among neighboring pixels, a global algorithm is then derived that explicitly distinguishes objects versus background using maximum a posteriori inference in a Markov random field (implemented via graph-cuts). By exploiting the parallel nature of the proposed algorithms, we develop an implementation that can run efficiently on the highly parallel graphics processing unit (GPU). Empirical studies on a wide variety of datasets demonstrate that the proposed approach achieves quality that is comparable to state-of-the-art offline methods, while still being suitable for real-time video analysis (≥ 75nfps on a mid-range GPU).

Proceedings ArticleDOI
22 May 2011
TL;DR: This paper presents a Markov Random Field (MRF)-based approach for depth map super-resolution and enhancement, based on a novel data term formulation that fits well to the unique characteristics of depth maps.
Abstract: This paper presents a Markov Random Field (MRF)-based approach for depth map super-resolution and enhancement. Given a low-resolution or moderate quality depth map, we study the problem of enhancing its resolution or quality with a registered high-resolution color image. Different from the previous methods, this MRF-based approach is based on a novel data term formulation that fits well to the unique characteristics of depth maps. We also discuss a few important design choices that boost the performance of general MRF-based methods. Experimental results show that our proposed approach achieves high-resolution depth maps at more desirable quality, both qualitatively and quantitatively. It can also be applied to enhance the depth maps derived with state-of-the-art stereo methods, resulting in the raised ranking based on the Middlebury benchmark.

Proceedings ArticleDOI
20 Jun 2011
TL;DR: A novel global stereo model that makes use of constraints from points with known depths, i.e., the Ground Control Points (GCPs) as referred to in stereo literature, is presented.
Abstract: We present a novel global stereo model that makes use of constraints from points with known depths, i.e., the Ground Control Points (GCPs) as referred to in stereo literature. Our formulation explicitly models the influences of GCPs in a Markov Random Field. A novel GCPs-based regu-larization term is naturally integrated into our global optimization framework in a principled way using the Bayes rule. The optimal solution of the inference problem can be approximated via existing energy minimization techniques such as graph cuts used in this paper. Our generic probabilistic framework allows GCPs to be obtained from various modalities and provides a natural way to integrate the information from multiple sensors. Quantitative evaluations demonstrate the effectiveness of the proposed formulation for regularizing the ill-posed stereo matching problem and improving reconstruction accuracy.

Journal ArticleDOI
TL;DR: A new message passing scheme named tile-based BP that reduces the memory and bandwidth to a fraction of the ordinary BP algorithms without performance degradation by splitting the MRF into many tiles and only storing the messages across the neighboring tiles is proposed.
Abstract: Loopy belief propagation (BP) is an effective solution for assigning labels to the nodes of a graphical model such as the Markov random field (MRF), but it requires high memory, bandwidth, and computational costs. Furthermore, the iterative, pixel-wise, and sequential operations of BP make it difficult to parallelize the computation. In this paper, we propose two techniques to address these issues. The first technique is a new message passing scheme named tile-based BP that reduces the memory and bandwidth to a fraction of the ordinary BP algorithms without performance degradation by splitting the MRF into many tiles and only storing the messages across the neighboring tiles. The tile-wise processing also enables data reuse and pipeline, resulting in efficient hardware implementation. The second technique is an O(L) fast message construction algorithm that exploits the properties of robust functions for parallelization. We apply these two techniques to a very large-scale integration circuit for stereo matching that generates high-resolution disparity maps in near real-time. We also implement the proposed schemes on graphics processing unit (GPU) which is four-time faster than standard BP on GPU.

Journal ArticleDOI
TL;DR: The proposed proximity-based BM25 model, called BM25P, is compared to strong state-of-the-art evaluation baselines, and shows that the use of term proximity considerably enhances the retrieval effectiveness of the classical probabilistic models.

Journal ArticleDOI
TL;DR: This paper investigates the problem of fusion of remote sensing images, e.g., multispectral image fusion, based on MRf models and incorporates the contextual constraints via MRF models into the fusion model and develops fusion algorithms under the maximum a posteriori criterion.
Abstract: Markov random field (MRF) models are powerful tools to model image characteristics accurately and have been successfully applied to a large number of image processing applications. This paper investigates the problem of fusion of remote sensing images, e.g., multispectral image fusion, based on MRF models and incorporates the contextual constraints via MRF models into the fusion model. Fusion algorithms under the maximum a posteriori criterion are developed to search for solutions. Our algorithm is applicable to both multiscale decomposition (MD)-based image fusion and non-MD-based image fusion. Experimental results are provided to demonstrate the improvement of fusion performance by our algorithms.

Journal ArticleDOI
TL;DR: A multivariate hierarchical approach is developed, at the heart of which is a new representation of a multivariate Markov random field that allows for flexible modeling of the multivariate spatial dependencies, including the cross-dependencies between variables.
Abstract: Climate models have become an important tool in the study of climate and climate change, and ensemble experiments consisting of multiple climate-model runs are used in studying and quantifying the uncertainty in climate-model output. However, there are often only a limited number of model runs available for a particular experiment, and one of the statistical challenges is to characterize the distribution of the model output. To that end, we have developed a multivariate hierarchical approach, at the heart of which is a new representation of a multivariate Markov random field. This approach allows for flexible modeling of the multivariate spatial dependencies, including the cross-dependencies between variables. We demonstrate this statistical model on an ensemble arising from a regional-climate-model experiment over the western United States, and we focus on the projected change in seasonal temperature and precipitation over the next 50 years.

Proceedings ArticleDOI
20 Jun 2011
TL;DR: A higher-order Markov Random Field illumination model, which combines low-level shadow evidence with high-level prior knowledge for the joint estimation of cast shadows and the illumination environment, is described.
Abstract: In this paper, we propose a novel framework to jointly recover the illumination environment and an estimate of the cast shadows in a scene from a single image, given coarse 3D geometry. We describe a higher-order Markov Random Field (MRF) illumination model, which combines low-level shadow evidence with high-level prior knowledge for the joint estimation of cast shadows and the illumination environment. First, a rough illumination estimate and the structure of the graphical model in the illumination space is determined through a voting procedure. Then, a higher order approach is considered where illumination sources are coupled with the observed image and the latent variables corresponding to the shadow detection. We examine two inference methods in order to effectively minimize the MRF energy of our model. Experimental evaluation shows that our approach is robust to rough knowledge of geometry and reflectance and inaccurate initial shadow estimates. We demonstrate the power of our MRF illumination model on various datasets and show that we can estimate the illumination in images of objects belonging to the same class using the same coarse 3D model to represent all instances of the class.

Journal ArticleDOI
TL;DR: A unified framework which uses a generative model of the imaging process and can address spatial super-resolution, space-time super- resolution, image deconvolution, single-image expansion, removal of noise, and image restoration is presented.
Abstract: We address the problem of super-resolution-obtaining high-resolution images and videos from multiple low-resolution inputs. The increased resolution can be in spatial or temporal dimensions, or even in both. We present a unified framework which uses a generative model of the imaging process and can address spatial super-resolution, space-time super-resolution, image deconvolution, single-image expansion, removal of noise, and image restoration. We model a high-resolution image or video as a Markov random field and use maximum a posteriori estimate as the final solution using graph-cut optimization technique. We derive insights into what super-resolution magnification factors are possible and the conditions necessary for super-resolution. We demonstrate spatial super-resolution reconstruction results with magnifications higher than predicted limits of magnification. We also formulate a scheme for selective super-resolution reconstruction of videos to obtain simultaneous increase of resolutions in both spatial and temporal directions. We show that it is possible to achieve space-time magnification factors beyond what has been suggested in the literature by selectively applying super-resolution constraints. We present results on both synthetic and real input sequences.

Journal ArticleDOI
25 Aug 2011-PLOS ONE
TL;DR: Multicoil2 is presented, an algorithm that predicts both the location and oligomerization state (two versus three helices) of coiled coils in protein sequences and significantly improves both coiled-coil detection and dimer versus trimer state prediction over the original Multicoil algorithm retrained on a newly-constructed database of coil sequences.
Abstract: The alpha-helical coiled coil can adopt a variety of topologies, among the most common of which are parallel and antiparallel dimers and trimers. We present Multicoil2, an algorithm that predicts both the location and oligomerization state (two versus three helices) of coiled coils in protein sequences. Multicoil2 combines the pairwise correlations of the previous Multicoil method with the flexibility of Hidden Markov Models (HMMs) in a Markov Random Field (MRF). The resulting algorithm integrates sequence features, including pairwise interactions, through multinomial logistic regression to devise an optimized scoring function for distinguishing dimer, trimer and non-coiled-coil oligomerization states; this scoring function is used to produce Markov Random Field potentials that incorporate pairwise correlations localized in sequence. Multicoil2 significantly improves both coiled-coil detection and dimer versus trimer state prediction over the original Multicoil algorithm retrained on a newly-constructed database of coiled-coil sequences. The new database, comprised of 2,105 sequences containing 124,088 residues, includes reliable structural annotations based on experimental data in the literature. Notably, the enhanced performance of Multicoil2 is evident when tested in stringent leave-family-out cross-validation on the new database, reflecting expected performance on challenging new prediction targets that have minimal sequence similarity to known coiled-coil families. The Multicoil2 program and training database are available for download from http://multicoil2.csail.mit.edu.