Showing papers on "Markov random field published in 2014"

Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation

Abstract: This paper proposes a new hybrid architecture that consists of a deep Convolutional Network and a Markov Random Field. We show how this architecture is successfully applied to the challenging problem of articulated human pose estimation in monocular images. The architecture can exploit structural domain constraints such as geometric relationships between body joint locations. We show that joint training of these two model paradigms improves performance and allows us to significantly outperform existing state-of-the-art techniques.

Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation

Abstract: This paper proposes a new hybrid architecture that consists of a deep Convolu-tional Network and a Markov Random Field We show how this architecture is successfully applied to the challenging problem of articulated human pose estimation in monocular images The architecture can exploit structural domain constraints such as geometric relationships between body joint locations We show that joint training of these two model paradigms improves performance and allows us to significantly outperform existing state-of-the-art techniques

Image processing of multiphase images obtained via X-ray microtomography: A review

Abstract: Easier access to X-ray microtomography (μCT) facilities has provided much new insight from high-resolution imaging for various problems in porous media research. Pore space analysis with respect to functional properties usually requires segmentation of the intensity data into different classes. Image segmentation is a nontrivial problem that may have a profound impact on all subsequent image analyses. This review deals with two issues that are neglected in most of the recent studies on image segmentation: (i) focus on multiclass segmentation and (ii) detailed descriptions as to why a specific method may fail together with strategies for preventing the failure by applying suitable image enhancement prior to segmentation. In this way, the presented algorithms become very robust and are less prone to operator bias. Three different test images are examined: a synthetic image with ground-truth information, a synchrotron image of precision beads with three different fluids residing in the pore space, and a μCT image of a soil sample containing macropores, rocks, organic matter, and the soil matrix. Image blur is identified as the major cause for poor segmentation results. Other impairments of the raw data like noise, ring artifacts, and intensity variation can be removed with current image enhancement methods. Bayesian Markov random field segmentation, watershed segmentation, and converging active contours are well suited for multiclass segmentation, yet with different success to correct for partial volume effects and conserve small image features simultaneously.

Contextual classification of lidar data and building object detection in urban areas

Abstract: In this work we address the task of the contextual classification of an airborne LiDAR point cloud. For that purpose, we integrate a Random Forest classifier into a Conditional Random Field (CRF) framework. It is a flexible approach for obtaining a reliable classification result even in complex urban scenes. In this way, we benefit from the consideration of context on the one hand and from the opportunity to use a large amount of features on the other hand. Considering the interactions in our experiments increases the overall accuracy by 2%, though a larger improvement becomes apparent in the completeness and correctness of some of the seven classes discerned in our experiments. We compare the Random Forest approach to linear models for the computation of unary and pairwise potentials of the CRF, and investigate the relevance of different features for the LiDAR points as well as for the interaction of neighbouring points. In a second step, building objects are detected based on the classified point cloud. For that purpose, the CRF probabilities for the classes are plugged into a Markov Random Field as unary potentials, in which the pairwise potentials are based on a Potts model. The 2D binary building object masks are extracted and evaluated by the benchmark ISPRS Test Project on Urban Classification and 3D Building Reconstruction. The evaluation shows that the main buildings (larger than 50 m 2 ) can be detected very reliably with a correctness larger than 96% and a completeness of 100%.

Fuzzy Clustering With a Modified MRF Energy Function for Change Detection in Synthetic Aperture Radar Images

Abstract: In this paper, we put forward a novel approach for change detection in synthetic aperture radar (SAR) images. The approach classifies changed and unchanged regions by fuzzy c-means (FCM) clustering with a novel Markov random field (MRF) energy function. In order to reduce the effect of speckle noise, a novel form of the MRF energy function with an additional term is established to modify the membership of each pixel. In addition, the degree of modification is determined by the relationship of the neighborhood pixels. The specific form of the additional term is contingent upon different situations, and it is established ultimately by utilizing the least-square method. There are two aspects to our contributions. First, in order to reduce the effect of speckle noise, the proposed approach focuses on modifying the membership instead of modifying the objective function. It is computationally simple in all the steps involved. Its objective function can just return to the original form of FCM, which leads to its consuming less time than that of some obviously recently improved FCM algorithms. Second, the proposed approach modifies the membership of each pixel according to a novel form of the MRF energy function through which the neighbors of each pixel, as well as their relationship, are concerned. Theoretical analysis and experimental results on real SAR datasets show that the proposed approach can detect the real changes as well as mitigate the effect of speckle noises. Theoretical analysis and experiments also demonstrate its low time complexity.

A Bayesian-MRF Approach for PRNU-Based Image Forgery Detection

Abstract: Graphics editing programs of the last generation provide ever more powerful tools, which allow for the retouching of digital images leaving little or no traces of tampering. The reliable detection of image forgeries requires, therefore, a battery of complementary tools that exploit different image properties. Techniques based on the photo-response non-uniformity (PRNU) noise are among the most valuable such tools, since they do not detect the inserted object but rather the absence of the camera PRNU, a sort of camera fingerprint, dealing successfully with forgeries that elude most other detection strategies. In this paper, we propose a new approach to detect image forgeries using sensor pattern noise. Casting the problem in terms of Bayesian estimation, we use a suitable Markov random field prior to model the strong spatial dependences of the source, and take decisions jointly on the whole image rather than individually for each pixel. Modern convex optimization techniques are then adopted to achieve a globally optimal solution and the PRNU estimation is improved by resorting to nonlocal denoising. Large-scale experiments on simulated and real forgeries show that the proposed technique largely improves upon the current state of the art, and that it can be applied with success to a wide range of practical situations.

Hyperspectral Image Classification Using Gaussian Mixture Models and Markov Random Fields

Abstract: The Gaussian mixture model is a well-known classification tool that captures non-Gaussian statistics of multivariate data. However, the impractically large size of the resulting parameter space has hindered widespread adoption of Gaussian mixture models for hyperspectral imagery. To counter this parameter-space issue, dimensionality reduction targeting the preservation of multimodal structures is proposed. Specifically, locality-preserving nonnegative matrix factorization, as well as local Fisher's discriminant analysis, is deployed as preprocessing to reduce the dimensionality of data for the Gaussian-mixture-model classifier, while preserving multimodal structures within the data. In addition, the pixel-wise classification results from the Gaussian mixture model are combined with spatial-context information resulting from a Markov random field. Experimental results demonstrate that the proposed classification system significantly outperforms other approaches even under limited training data.

Real-time exemplar-based face sketch synthesis

Abstract: This paper proposes a simple yet effective face sketch synthesis method. Similar to existing exemplar-based methods, a training dataset containing photo-sketch pairs is required, and a K-NN photo patch search is performed between a test photo and every training exemplar for sketch patch selection. Instead of using the Markov Random Field to optimize global sketch patch selection, this paper formulates face sketch synthesis as an image denoising problem which can be solved efficiently using the proposed method. Real-time performance can be obtained on a state-of-the-art GPU. Meanwhile quantitative evaluations on face sketch recognition and user study demonstrate the effectiveness of the proposed method. In addition, the proposed method can be directly extended to the temporal domain for consistent video sketch synthesis, which is of great importance in digital entertainment.

Visual Tracking via Probability Continuous Outlier Model

Abstract: In this paper, we present a novel online visual tracking method based on linear representation. First, we present a novel probability continuous outlier model (PCOM) to depict the continuous outliers that occur in the linear representation model. In the proposed model, the element of the noisy observation sample can be either represented by a PCA subspace with small Guassian noise or treated as an arbitrary value with a uniform prior, in which the spatial consistency prior is exploited by using a binary Markov random field model. Then, we derive the objective function of the PCOM method, the solution of which can be iteratively obtained by the outlier-free least squares and standard max-flow/min-cut steps. Finally, based on the proposed PCOM method, we design an effective observation likelihood function and a simple update scheme for visual tracking. Both qualitative and quantitative evaluations demonstrate that our tracker achieves very favorable performance in terms of both accuracy and speed.

Spectral–Spatial Classification of Hyperspectral Data Using Local and Global Probabilities for Mixed Pixel Characterization

Abstract: Remotely sensed hyperspectral image classification is a very challenging task. This is due to many different aspects, such as the presence of mixed pixels in the data or the limited information available a priori. This has fostered the need to develop techniques able to exploit the rich spatial and spectral information present in the scenes while, at the same time, dealing with mixed pixels and limited training samples. In this paper, we present a new spectral–spatial classifier for hyperspectral data that specifically addresses the issue of mixed pixel characterization. In our presented approach, the spectral information is characterized both locally and globally, which represents an innovation with regard to previous approaches for probabilistic classification of hyperspectral data. Specifically, we use a subspace-based multinomial logistic regression method for learning the posterior probabilities and a pixel-based probabilistic support vector machine classifier as an indicator to locally determine the number of mixed components that participate in each pixel. The information provided by local and global probabilities is then fused and interpreted in order to characterize mixed pixels. Finally, spatial information is characterized by including a Markov random field (MRF) regularizer. Our experimental results, conducted using both synthetic and real hyperspectral images, indicate that the proposed classifier leads to state-of-the-art performance when compared with other approaches, particularly in scenarios in which very limited training samples are available.

Robust deformable and occluded object tracking with dynamic graph.

Abstract: While some efforts have been paid to handle deformation and occlusion in visual tracking, they are still great challenges. In this paper, a dynamic graph-based tracker (DGT) is proposed to address these two challenges in a unified framework. In the dynamic target graph, nodes are the target local parts encoding appearance information, and edges are the interactions between nodes encoding inner geometric structure information. This graph representation provides much more information for tracking in the presence of deformation and occlusion. The target tracking is then formulated as tracking this dynamic undirected graph, which is also a matching problem between the target graph and the candidate graph. The local parts within the candidate graph are separated from the background with Markov random field, and spectral clustering is used to solve the graph matching. The final target state is determined through a weighted voting procedure according to the reliability of part correspondence, and refined with recourse to a foreground/background segmentation. An effective online updating mechanism is proposed to update the model, allowing DGT to robustly adapt to variations of target structure. Experimental results show improved performance over several state-of-the-art trackers, in various challenging scenarios.

Fusion of airborne laserscanning point clouds and images for supervised and unsupervised scene classification

Abstract: Automatic urban object detection from airborne remote sensing data is essential to process and efficiently interpret the vast amount of airborne imagery and Laserscanning (ALS) data available today. This paper combines ALS data and airborne imagery to exploit both: the good geometric quality of ALS and the spectral image information to detect the four classes buildings, trees, vegetated ground and sealed ground. A new segmentation approach is introduced which also makes use of geometric and spectral data during classification entity definition. Geometric, textural, low level and mid level image features are assigned to laser points which are quantified into voxels. The segment information is transferred to the voxels and those clusters of voxels form the entity to be classified. Two classification strategies are pursued: a supervised method, using Random Trees and an unsupervised approach, embedded in a Markov Random Field framework and using graph-cuts for energy optimization. A further contribution of this paper concerns the image-based point densification for building roofs which aims to mitigate the accuracy problems related to large ALS point spacing. Results for the ISPRS benchmark test data show that to rely on color information to separate vegetation from non-vegetation areas does mostly lead to good results, but in particular in shadow areas a confusion between classes might occur. The unsupervised classification strategy is especially sensitive in this respect. As far as the point cloud densification is concerned, we observe similar sensitivity with respect to color which makes some planes to be missed out, or false detections still remain. For planes where the densification is successful we see the expected enhancement of the outline.

Beat the MTurkers: Automatic Image Labeling from Weak 3D Supervision

Abstract: Labeling large-scale datasets with very accurate object segmentations is an elaborate task that requires a high degree of quality control and a budget of tens or hundreds of thousands of dollars. Thus, developing solutions that can automatically perform the labeling given only weak supervision is key to reduce this cost. In this paper, we show how to exploit 3D information to automatically generate very accurate object segmentations given annotated 3D bounding boxes. We formulate the problem as the one of inference in a binary Markov random field which exploits appearance models, stereo and/or noisy point clouds, a repository of 3D CAD models as well as topological constraints. We demonstrate the effectiveness of our approach in the context of autonomous driving, and show that we can segment cars with the accuracy of 86% intersection-over-union, performing as well as highly recommended MTurkers!

Edge guided single depth image super resolution

Abstract: Recently, consumer depth cameras have gained significant popularity due to their affordable cost. However, the limited resolution and quality of the depth map generated by these cameras are still problems for several applications. In this paper, we propose a novel framework for single depth image super resolution guided by a high resolution edge map constructed from the edges in the low resolution depth image via a Markov Random Field (MRF) optimization. With the guidance of the high resolution edge map, the high resolution depth image is up-sampled via a joint bilateral filter. The edge guidance not only helps avoid artifacts introduced by direct texture prediction, but also reduces the jagged artifacts and preserves the sharp edges. Experimental results demonstrate the effectiveness of our proposed algorithm compared to previously reported methods.

Improving SAR-Based Urban Change Detection by Combining MAP-MRF Classifier and Nonlocal Means Similarity Weights

Abstract: In remote sensing change detection, Markov random field (MRF) has been used successfully to model the prior probability using class-labels dependencies MRF has played an important role in the detection of complex urban changes using optical images However, the preservation of details in urban change analysis turns out to be a highly complex task if multitemporal SAR images with their speckle are to be used Here, the ability of MRF to preserve geometric details and to combat speckle effect at the same time becomes questionable Blob-region phenomenon and fine structures removal are common consequences of the application of traditional MRF-based change detection algorithm To overcome these limitations, the iterated conditional modes (ICM) framework for the optimization of the maximum a posteriori (MAP-MRF) criterion function is extended to include a nonlocal probability maximization step This probability model, which characterizes the relationship between pixels’ class-labels in a nonlocal scale, has the potential to preserve spatial details and to reduce speckle effects Two multitemporal SAR datasets were used to assess the proposed algorithm Experimental results using three density functions [ie, the log normal (LN), generalized Gaussian (GG), and normal distributions (ND)] have demonstrated the efficiency of the proposed approach in terms of detail preservation and noise suppression Compared with the traditional MRF algorithm, the proposed approach proved to be less-sensitive to the value of the contextual parameter and the chosen density function The proposed approach has also shown less sensitivity to the quality of the initial change map when compared with the ICM algorithm

Prostate MRI Segmentation Using Learned Semantic Knowledge and Graph Cuts

Abstract: We propose a fully automated method for prostate segmentation using random forests (RFs) and graph cuts. A volume of interest (VOI) is automatically selected using supervoxel segmentation, and its subsequent classification using image features and RF classifiers. The VOIs probability map is generated using image and context features, and a second set of RF classifiers. The negative log-likelihood of the probability maps acts as the penalty cost in a second-order Markov random field cost function. Semantic information from the second set of RF classifiers is an important measure of each feature to the classification task, which contributes to formulating the smoothness cost. The cost function is optimized using graph cuts to get the final segmentation of the prostate. With average dice metric (DM) (on the training set) and DM (on the test set), our experimental results show that inclusion of the context and semantic information contributes to higher segmentation accuracy than other methods.

Analysis, Evaluation, and Comparison of Polarimetric SAR Speckle Filtering Techniques

Abstract: Speckle noise filtering on polarimetric SAR (PolSAR) images remains a challenging task due to the difficulty to reduce a scatterer-dependent noise while preserving the polarimetric information and the spatial information. This challenge is particularly acute on single look complex images, where little information about the scattering process can be derived from a rank-1 covariance matrix. This paper proposes to analyze and to evaluate the performances of a set of PolSAR speckle filters. The filter performances are measured by a set of ten different indicators, including relative errors on incoherent target decomposition parameters, coherences, polarimetric signatures, point target, and edge preservation. The result is a performance profile for each individual filter. The methodology consists of simulating a set of artificial PolSAR images on which the various filters will be evaluated. The image morphology is stochastic and determined by a Markov random field and the number of scattering classes is allowed to vary so that we can explore a large range of image configurations. Evaluation on real PolSAR images is also considered. Results show that filters performances need to be assessed using a complete set of indicators, including distributed scatterer parameters, radiometric parameters, and spatial information preservation.

Rear-View Vehicle Detection and Tracking by Combining Multiple Parts for Complex Urban Surveillance

Abstract: Traffic surveillance is an important topic in intelligent transportation systems. Robust vehicle detection and tracking is one challenging problem for complex urban traffic surveillance. This paper proposes a rear-view vehicle detection and tracking method based on multiple vehicle salient parts using a stationary camera. We show that spatial modeling of these vehicle parts is crucial for overall performance. First, the vehicle is treated as an object composed of multiple salient parts, including the license plate and rear lamps. These parts are localized using their distinctive color, texture, and region feature. Furthermore, the detected parts are treated as graph nodes to construct a probabilistic graph using a Markov random field model. After that, the marginal posterior of each part is inferred using loopy belief propagation to get final vehicle detection. Finally, the vehicles' trajectories are estimated using a Kalman filter, and a tracking-based detection technique is realized. Experiments in practical urban scenarios are carried out under various weather conditions. It can be shown that our method adapts to partial occlusion and various lighting conditions. Experiments also show that our method can achieve real-time performance.

Characterness: An Indicator of Text in the Wild

Abstract: Text in an image provides vital information for interpreting its contents, and text in a scene can aid a variety of tasks from navigation to obstacle avoidance and odometry. Despite its value, however, detecting general text in images remains a challenging research problem. Motivated by the need to consider the widely varying forms of natural text, we propose a bottom-up approach to the problem, which reflects the characterness of an image region. In this sense, our approach mirrors the move from saliency detection methods to measures of objectness. In order to measure the characterness, we develop three novel cues that are tailored for character detection and a Bayesian method for their integration. Because text is made up of sets of characters, we then design a Markov random field model so as to exploit the inherent dependencies between characters. We experimentally demonstrate the effectiveness of our characterness cues as well as the advantage of Bayesian multicue integration. The proposed text detector outperforms state-of-the-art methods on a few benchmark scene text detection data sets. We also show that our measurement of characterness is superior than state-of-the-art saliency detection models when applied to the same task.

Residual Component Analysis of Hyperspectral Images—Application to Joint Nonlinear Unmixing and Nonlinearity Detection

Abstract: This paper presents a nonlinear mixing model for joint hyperspectral image unmixing and nonlinearity detection. The proposed model assumes that the pixel reflectances are linear combinations of known pure spectral components corrupted by an additional nonlinear term, affecting the end members and contaminated by an additive Gaussian noise. A Markov random field is considered for nonlinearity detection based on the spatial structure of the nonlinear terms. The observed image is segmented into regions where nonlinear terms, if present, share similar statistical properties. A Bayesian algorithm is proposed to estimate the parameters involved in the model yielding a joint nonlinear unmixing and nonlinearity detection algorithm. The performance of the proposed strategy is first evaluated on synthetic data. Simulations conducted with real data show the accuracy of the proposed unmixing and nonlinearity detection strategy for the analysis of hyperspectral images.

Learning representations for weakly supervised natural language processing tasks

Abstract: Finding the right representations for words is critical for building accurate NLP systems when domain-specific labeled data for the task is scarce. This article investigates novel techniques for extracting features from n-gram models, Hidden Markov Models, and other statistical language models, including a novel Partial Lattice Markov Random Field model. Experiments on part-of-speech tagging and information extraction, among other tasks, indicate that features taken from statistical language models, in combination with more traditional features, outperform traditional representations alone, and that graphical model representations outperform n-gram models, especially on sparse and polysemous words.

Semi-Global Matching: A Principled Derivation in Terms of Message Passing

Abstract: Semi-global matching, originally introduced in the context of dense stereo, is a very successful heuristic to minimize the energy of a pairwise multi-label Markov Random Field defined on a grid. We offer the first principled explanation of this empirically successful algorithm, and clarify its exact relation to belief propagation and tree-reweighted message passing. One outcome of this new connection is an uncertainty measure for the MAP label of a variable in a Markov Random Field.

Filter Forests for Learning Data-Dependent Convolutional Kernels

Abstract: We propose 'filter forests' (FF), an efficient new discriminative approach for predicting continuous variables given a signal and its context. FF can be used for general signal restoration tasks that can be tackled via convolutional filtering, where it attempts to learn the optimal filtering kernels to be applied to each data point. The model can learn both the size of the kernel and its values, conditioned on the observation and its spatial or temporal context. We show that FF compares favorably to both Markov random field based and recently proposed regression forest based approaches for labeling problems in terms of efficiency and accuracy. In particular, we demonstrate how FF can be used to learn optimal denoising filters for natural images as well as for other tasks such as depth image refinement, and 1D signal magnitude estimation. Numerous experiments and quantitative comparisons show that FFs achieve accuracy at par or superior to recent state of the art techniques, while being several orders of magnitude faster.

Similarity-Aware Patchwork Assembly for Depth Image Super-resolution

Abstract: This paper describes a patchwork assembly algorithm for depth image super-resolution. An input low resolution depth image is disassembled into parts by matching similar regions on a set of high resolution training images, and a super-resolution image is then assembled using these corresponding matched counterparts. We convert the super resolution problem into a Markov Random Field (MRF) labeling problem, and propose a unified formulation embedding (1) the consistency between the resolution enhanced image and the original input, (2) the similarity of disassembled parts with the corresponding regions on training images, (3) the depth smoothness in local neighborhoods, (4) the additional geometric constraints from self-similar structures in the scene, and (5) the boundary coincidence between the resolution enhanced depth image and an optional aligned high resolution intensity image. Experimental results on both synthetic and real-world data demonstrate that the proposed algorithm is capable of recovering high quality depth images with X4 resolution enhancement along each coordinate direction, and that it outperforms state-of-the-arts [14] in both qualitative and quantitative evaluations.

Super-Resolution Mapping of Forests With Bitemporal Different Spatial Resolution Images Based on the Spatial-Temporal Markov Random Field

Abstract: High deforestation rates necessitate satellite images for the timely updating of forest maps. Coarse spatial resolution remotely sensed images have wide swath and high temporal resolution. However, the mixed pixel problem lowers the mapping accuracy and hampers the application of these images. The development of remote sensing technology has enabled the storage of a great amount of medium spatial resolution images that recorded the historical conditions of the earth. The combination of timely updated coarse spatial resolution images and previous medium spatial resolution images is a promising technique for mapping forests in large areas with instant updating at low expense. Super-resolution mapping (SRM) is a method for mapping land cover classes with a finer spatial resolution than the input coarse resolution image. This method can reduce the mixed pixel problem of coarse spatial resolution images to a certain extent. In this paper, a novel spatial-temporal SRM based on a Markov random field, called STMRF_SRM, is proposed using a current coarse spatial resolution Moderate-Resolution Imaging Spectroradiometer image and a previous medium spatial resolution Landsat Thematic Mapper image as input. The proposed model encourages the spatial smoothing of land cover classes for spatially neighboring subpixels and keeps temporal links between temporally neighboring subpixels in bitemporal images. Results show that the proposed STMRF_SRM model can generate forest maps with higher overall accuracy and kappa value.

Detection and inpainting of facial wrinkles using texture orientation fields and Markov random field modeling.

Abstract: Facial retouching is widely used in media and entertainment industry. Professional software usually require a minimum level of user expertise to achieve the desirable results. In this paper, we present an algorithm to detect facial wrinkles/imperfection. We believe that any such algorithm would be amenable to facial retouching applications. The detection of wrinkles/imperfections can allow these skin features to be processed differently than the surrounding skin without much user interaction. For detection, Gabor filter responses along with texture orientation field are used as image features. A bimodal Gaussian mixture model (GMM) represents distributions of Gabor features of normal skin versus skin imperfections. Then, a Markov random field model is used to incorporate the spatial relationships among neighboring pixels for their GMM distributions and texture orientations. An expectation-maximization algorithm then classifies skin versus skin wrinkles/imperfections. Once detected automatically, wrinkles/imperfections are removed completely instead of being blended or blurred. We propose an exemplar-based constrained texture synthesis algorithm to inpaint irregularly shaped gaps left by the removal of detected wrinkles/imperfections. We present results conducted on images downloaded from the Internet to show the efficacy of our algorithms.