Showing papers on "Image gradient published in 2016"

Deep multi-scale video prediction beyond mean square error

Abstract: Learning to predict future images from a video sequence involves the construction of an internal representation that models the image evolution accurately, and therefore, to some degree, its content and dynamics. This is why pixel-space video prediction may be viewed as a promising avenue for unsupervised feature learning. In addition, while optical flow has been a very studied problem in computer vision for a long time, future frame prediction is rarely approached. Still, many vision applications could benefit from the knowledge of the next frames of videos, that does not require the complexity of tracking every pixel trajectories. In this work, we train a convolutional network to generate future frames given an input sequence. To deal with the inherently blurry predictions obtained from the standard Mean Squared Error (MSE) loss function, we propose three different and complementary feature learning strategies: a multi-scale architecture, an adversarial training method, and an image gradient difference loss function. We compare our predictions to different published results based on recurrent neural networks on the UCF101 dataset

Semantic Image Segmentation with Task-Specific Edge Detection Using CNNs and a Discriminatively Trained Domain Transform

Abstract: Deep convolutional neural networks (CNNs) are the backbone of state-of-art semantic image segmentation systems. Recent work has shown that complementing CNNs with fully-connected conditional random fields (CRFs) can significantly enhance their object localization accuracy, yet dense CRF inference is computationally expensive. We propose replacing the fully-connected CRF with domain transform (DT), a modern edge-preserving filtering method in which the amount of smoothing is controlled by a reference edge map. Domain transform filtering is several times faster than dense CRF inference and we show that it yields comparable semantic segmentation results, accurately capturing object boundaries. Importantly, our formulation allows learning the reference edge map from intermediate CNN features instead of using the image gradient magnitude as in standard DT filtering. This produces task-specific edges in an end-to-end trainable system optimizing the target semantic segmentation quality.

Blind image quality assessment by relative gradient statistics and adaboosting neural network

Abstract: The image gradient is a commonly computed image feature and a potentially predictive factor for image quality assessment (IQA). Indeed, it has been successfully used for both full- and no- reference image quality prediction. However, the gradient orientation has not been deeply explored as a predictive source of information for image quality assessment. Here we seek to amend this by studying the quality relevance of the relative gradient orientation, viz., the gradient orientation relative to the surround. We also deploy a relative gradient magnitude feature which accounts for perceptual masking and utilize an AdaBoosting back-propagation (BP) neural network to map the image features to image quality. The generalization of the AdaBoosting BP neural network results in an effective and robust quality prediction model. The new model, called Oriented Gradients Image Quality Assessment (OG-IQA), is shown to deliver highly competitive image quality prediction performance as compared with the most popular IQA approaches. Furthermore, we show that OG-IQA has good database independence properties and a low complexity. OG-IQA extracts a 6-dimensional relative gradient feature vector from the inputs.OG-IQA utilizes an AdaBoosting BP neural network to map the image features to image quality.OG-IQA delivers highly competitive image quality prediction performance and has a relatively low time complexity.

Fusion of Infrared and Visible Sensor Images Based on Anisotropic Diffusion and Karhunen-Loeve Transform

Abstract: Image fusion is a process of generating a more informative image from a set of source images. Major applications of image fusion are in navigation and military. Here, infrared and visible sensors are used to capture complementary images of the targeted scene. The complementary information of these source images has to be integrated into a single image using some fusion algorithms. The aim of any fusion method is to transfer maximum information from the source images to the fused image with a minimum information loss. It has to minimize the artifacts in the fused image. In this paper, we propose a new edge preserving image fusion method for infrared and visible sensor images. Anisotropic diffusion is used to decompose the source images into approximation and detail layers. Final detail and approximation layers are calculated with the help of Karhunen-Loeve transform and weighted linear superposition, respectively. A fused image is generated from the linear combination of final detail and approximation layers. Performance of the proposed algorithm is assessed with the help of petrovic metrics. The results of the proposed algorithm are compared with the traditional and recent image fusion algorithms. Results reveal that the proposed method outperforms the existing methods.

Multi-Exposure and Multi-Focus Image Fusion in Gradient Domain

Abstract: A multi-exposure and multi-focus image fusion algorithm is proposed. The algorithm is developed for color images and is based on blending the gradients of the luminance components of the input images using the maximum gradient magnitude at each pixel location and then obtaining the fused luminance using a Haar wavelet-based image reconstruction technique. This image reconstruction algorithm is of O(N) complexity and includes a Poisson solver at each resolution to eliminate artifacts that may appear due to the nonconservative nature of the resulting gradient. The fused chrominance, on the other hand, is obtained as a weighted mean of the chrominance channels. The particular case of grayscale images is treated as luminance fusion. Experimental results and comparison with other fusion techniques indicate that the proposed algorithm is fast and produces similar or better results than existing techniques for both multi-exposure as well as multi-focus images.

Medical Image Synthesis with Context-Aware Generative Adversarial Networks

Abstract: Computed tomography (CT) is critical for various clinical applications, e.g., radiotherapy treatment planning and also PET attenuation correction. However, CT exposes radiation during acquisition, which may cause side effects to patients. Compared to CT, magnetic resonance imaging (MRI) is much safer and does not involve any radiations. Therefore, recently, researchers are greatly motivated to estimate CT image from its corresponding MR image of the same subject for the case of radiotherapy planning. In this paper, we propose a data-driven approach to address this challenging problem. Specifically, we train a fully convolutional network to generate CT given an MR image. To better model the nonlinear relationship from MRI to CT and to produce more realistic images, we propose to use the adversarial training strategy and an image gradient difference loss function. We further apply AutoContext Model to implement a context-aware generative adversarial network. Experimental results show that our method is accurate and robust for predicting CT images from MRI images, and also outperforms three state-of-the-art methods under comparison.

Augmenting Layer-Based Object Detection With Deep Convolutional Neural Networks

Abstract: By way of example, the technology disclosed by this document receives image data; extracts a depth image and a color image from the image data; creates a mask image by segmenting the depth image; determines a first likelihood score from the depth image and the mask image using a layered classifier; determines a second likelihood score from the color image and the mask image using a deep convolutional neural network; and determines a class of at least a portion of the image data based on the first likelihood score and the second likelihood score. Further, the technology can pre-filter the mask image using the layered classifier and then use the pre-filtered mask image and the color image to calculate a second likelihood score using the deep convolutional neural network to speed up processing.

Shading-Aware Multi-view Stereo

Abstract: We present a novel multi-view reconstruction approach that effectively combines stereo and shape-from-shading energies into a single optimization scheme. Our method uses image gradients to transition between stereo-matching (which is more accurate at large gradients) and Lambertian shape-from-shading (which is more robust in flat regions). In addition, we show that our formulation is invariant to spatially varying albedo without explicitly modeling it. We show that the resulting energy function can be optimized efficiently using a smooth surface representation based on bicubic patches, and demonstrate that this algorithm outperforms both previous multi-view stereo algorithms and shading based refinement approaches on a number of datasets.

Gradient Direction for Screen Content Image Quality Assessment

Abstract: In this letter, we make the first attempt to explore the usage of the gradient direction to conduct the perceptual quality assessment of the screen content images (SCIs). Specifically, the proposed approach first extracts the gradient direction based on the local information of the image gradient magnitude, which not only preserves gradient direction consistency in local regions, but also demonstrates sensitivities to the distortions introduced to the SCI. A deviation-based pooling strategy is subsequently utilized to generate the corresponding image quality index. Moreover, we investigate and demonstrate the complementary behaviors of the gradient direction and magnitude for SCI quality assessment. By jointly considering them together, our proposed SCI quality metric outperforms the state-of-the-art quality metrics in terms of correlation with human visual system perception.

A Decomposition Framework for Image Denoising Algorithms

Abstract: In this paper, we consider an image decomposition model that provides a novel framework for image denoising. The model computes the components of the image to be processed in a moving frame that encodes its local geometry (directions of gradients and level lines). Then, the strategy we develop is to denoise the components of the image in the moving frame in order to preserve its local geometry, which would have been more affected if processing the image directly. Experiments on a whole image database tested with several denoising methods show that this framework can provide better results than denoising the image directly, both in terms of Peak signal-to-noise ratio and Structural similarity index metrics.

Blind Image Deconvolution by Automatic Gradient Activation

Abstract: Blind image deconvolution is an ill-posed inverse problem which is often addressed through the application of appropriate prior. Although some priors are informative in general, many images do not strictly conform to this, leading to degraded performance in the kernel estimation. More critically, real images may be contaminated by nonuniform noise such as saturation and outliers. Methods for removing specific image areas based on some priors have been proposed, but they operate either manually or by defining fixed criteria. We show here that a subset of the image gradients are adequate to estimate the blur kernel robustly, no matter the gradient image is sparse or not. We thus introduce a gradient activation method to automatically select a subset of gradients of the latent image in a cutting-plane-based optimization scheme for kernel estimation. No extra assumption is used in our model, which greatly improves the accuracy and flexibility. More importantly, the proposed method affords great convenience for handling noise and outliers. Experiments on both synthetic data and real-world images demonstrate the effectiveness and robustness of the proposed method in comparison with the state-of-the-art methods.

Robust Edge-Stop Functions for Edge-Based Active Contour Models in Medical Image Segmentation

Abstract: Edge-based active contour models are effective in segmenting images with intensity inhomogeneity but often fail when applied to images containing poorly defined boundaries, such as in medical images. Traditional edge-stop functions (ESFs) utilize only gradient information, which fails to stop contour evolution at such boundaries because of the small gradient magnitudes. To address this problem, we propose a framework to construct a group of ESFs for edge-based active contour models to segment objects with poorly defined boundaries. In our framework, which incorporates gradient information as well as probability scores from a standard classifier, the ESF can be constructed from any classification algorithm and applied to any edge-based model using a level set method. Experiments on medical images using the distance regularized level set for edge-based active contour models as well as the k-nearest neighbours and the support vector machine confirm the effectiveness of the proposed approach.

On image gradients in digital image correlation

Abstract: In digital image correlation (DIC), the unknown displacement field is typically identified by minimizing the linearized form of the brightness conservation equation, while the minimization scheme also involves a linearization, yielding a two-step linearization with four implicit assumptions. These assumptions become apparent by minimizing the non-linear brightness conservation equation in a consistent mathematical setting, yielding a one-step linearization allowing a thorough study of the DIC tangent operator. Through this analysis, eight different image gradient operators are defined, and the impact of these alternative image gradients on the accuracy, efficiency, and initial guess robustness is discussed on the basis of a number of academic examples and representative test cases. The main conclusion is that for most cases, the image gradient most common in literature is recommended, except for cases with: (1) large rotations; (2) initial guess instabilities; and (3) costly iterations due to other reasons (e.g., integrated DIC), where a large deformation corrected mixed gradient is recommended instead.

Fast depth image denoising and enhancement using a deep convolutional network

Abstract: We propose a depth image denoising and enhancement framework using a light convolutional network. The network contains three layers for high dimension projection, missing data completion and image reconstruction. We jointly use both depth and visual images as inputs. For the gray image, we design a pre-processing procedure to enhance the edges and remove unnecessary detail. For the depth image, we propose a data augmentation strategy to regenerate and increase essential training data. Further, we propose a weighted loss function for network training to adaptively improve the learning efficiency. We tested our algorithm on benchmark data and obtained very promising visual and quantitative results at real-time speed.

Feature selection for surface defect classification of extruded aluminum profiles

Abstract: This research investigates detection and classification of two types of the surface defects in extruded aluminium profiles; blisters and scratches An experimental system is used to capture images and appropriate statistical features from a novel technique based on gradient-only co-occurrence matrices (GOCM) are proposed to detect and classify three distinct classes; non-defective, blisters and scratches The developed methodology makes use of the Sobel edge detector to obtain the gradient magnitude of the image (GOCM) A comparison is made between the statistical features extracted from the original image (GLCM) and those extracted from the gradient magnitude (GOCM) This paper describes in detail every step of the image processing with example pictures illustrating the methodology The features extracted from the image processing are classified by a two-layer feed-forward artificial neural network The artificial neural network training is tested using different combinations of statistical features with different topologies Features are compared individually and grouped Results are discussed, achieving up to 986 % total testing accuracy

Single-Image Super-Resolution via an Iterative Reproducing Kernel Hilbert Space Method

Abstract: Image super-resolution (SR), a process to enhance image resolution, has important applications in satellite imaging, high-definition television, medical imaging, and so on. Many existing approaches use multiple low-resolution (LR) images to recover one high-resolution (HR) image. In this paper, we present an iterative scheme to solve single -image SR problems. It recovers a high-quality HR image from solely one LR image without using a training data set. We solve the problem from image intensity function estimation perspective and assume that the image contains smooth and edge components. We model the smooth components of an image using a thin-plate reproducing kernel Hilbert space and the edges using approximated Heaviside functions. The proposed method is applied to image patches, aiming to reduce computation and storage. Visual and quantitative comparisons with some competitive approaches show the effectiveness of the proposed method.

Enhanced Local Gradient Order Features and Discriminant Analysis for Face Recognition

Abstract: Robust descriptor-based subspace learning with complex data is an active topic in pattern analysis and machine intelligence. A few researches concentrate the optimal design on feature representation and metric learning. However, traditionally used features of single-type, e.g., image gradient orientations (IGOs), are deficient to characterize the complete variations in robust and discriminant subspace learning. Meanwhile, discontinuity in edge alignment and feature match are not been carefully treated in the literature. In this paper, local order constrained IGOs are exploited to generate robust features. As the difference-based filters explicitly consider the local contrasts within neighboring pixel points, the proposed features enhance the local textures and the order-based coding ability, thus discover intrinsic structure of facial images further. The multimodal features are automatically fused in the most discriminant subspace. The utilization of adaptive interaction function suppresses outliers in each dimension for robust similarity measurement and discriminant analysis. The sparsity-driven regression model is modified to adapt the classification issue of the compact feature representation. Extensive experiments are conducted by using some benchmark face data sets, e.g., of controlled and uncontrolled environments, to evaluate our new algorithm.

Robust image hashing based on color vector angle and Canny operator

Abstract: Image hashing is a novel technology of multimedia processing, and finds many applications, such as image forensics, image retrieval and image indexing. Conventional image hashing algorithms have limitations in reaching desirable classification performances between rotation robustness and discrimination. Aiming at this issue, we propose a robust image hashing based on color vector angle and Canny operator. Specifically, our hashing firstly converts input image to a normalized image by interpolation and Gaussian low-pass filtering. And then, color vector angles and image edges are both extracted from the normalized image. Finally, statistical features incorporating color vector angles and image edges are calculated to form image hash. We conduct experiments with 2762 images to validate efficiency of our hashing. The experimental results show that our hashing is robust against normal digital processing, such as image rotation, brightness/contrast adjustment and JPEG compression, and reaches good discrimination. Receiver operating characteristics (ROC) curve comparisons with some state-of-the-art algorithms indicate that our hashing outperforms these compared algorithms in classification performances between robustness and discriminative capability.

Non-local total-variation (NLTV) minimization combined with reweighted L1-norm for compressed sensing CT reconstruction.

Abstract: The compressed sensing (CS) technique has been employed to reconstruct CT/CBCT images from fewer projections as it is designed to recover a sparse signal from highly under-sampled measurements. Since the CT image itself cannot be sparse, a variety of transforms were developed to make the image sufficiently sparse. The total-variation (TV) transform with local image gradient in L1-norm was adopted in most cases. This approach, however, which utilizes very local information and penalizes the weight at a constant rate regardless of different degrees of spatial gradient, may not produce qualified reconstructed images from noise-contaminated CT projection data. This work presents a new non-local operator of total-variation (NLTV) to overcome the deficits stated above by utilizing a more global search and non-uniform weight penalization in reconstruction. To further improve the reconstructed results, a reweighted L1-norm that approximates the ideal sparse signal recovery of the L0-norm is incorporated into the NLTV reconstruction with additional iterates. This study tested the proposed reconstruction method (reweighted NLTV) from under-sampled projections of 4 objects and 5 experiments (1 digital phantom with low and high noise scenarios, 1 pelvic CT, and 2 CBCT images). We assessed its performance against the conventional TV, NLTV and reweighted TV transforms in the tissue contrast, reconstruction accuracy, and imaging resolution by comparing contrast-noise-ratio (CNR), normalized root-mean square error (nRMSE), and profiles of the reconstructed images. Relative to the conventional NLTV, combining the reweighted L1-norm with NLTV further enhanced the CNRs by 2-4 times and improved reconstruction accuracy. Overall, except for the digital phantom with low noise simulation, our proposed algorithm produced the reconstructed image with the lowest nRMSEs and the highest CNRs for each experiment.

Guided Image Contrast Enhancement Based on Retrieved Images in Cloud

Abstract: We propose a guided image contrast enhancement framework based on cloud images, in which the context- sensitive and context-free contrast is jointly improved via solving a multi-criteria optimization problem. In particular, the context-sensitive contrast is improved by performing advanced unsharp masking on the input and edge-preserving filtered images, while the context-free contrast enhancement is achieved by the sigmoid transfer mapping. To automatically determine the contrast enhancement level, the parameters in the optimization process are estimated by taking advantages of the retrieved images with similar content. For the purpose of automatically avoiding the involvement of low-quality retrieved images as the guidance, a recently developed no-reference image quality metric is adopted to rank the retrieved images from the cloud. The image complexity from the free-energy-based brain theory and the surface quality statistics in salient regions are collaboratively optimized to infer the parameters. Experimental results confirm that the proposed technique can efficiently create visually-pleasing enhanced images which are better than those produced by the classical techniques in both subjective and objective comparisons.

Underwater image restoration based on minimum information loss principle and optical properties of underwater imaging

Abstract: Restoring underwater image from a single image is known to be an ill-posed problem. Some assumptions made in previous methods are not suitable in many situations. In this paper, an effective method is proposed to restore underwater images. Using the quad-tree subdivision and graph-based segmentation, the global background light can be robustly estimated. The medium transmission map is estimated based on minimum information loss principle and optical properties of underwater imaging. Qualitative experiments show that our results are characterized by relatively genuine color, natural appearance, and improved contrast and visibility. Quantitative comparisons demonstrate that the proposed method can achieve better quality of underwater images when compared with several other methods.

Edge detection in medical ultrasound images using adjusted Canny edge detection algorithm

Abstract: Ultrasound medical images are very important component of the diagnostics process. They are widely used since ultrasound is a non-invasive and non-ionizing diagnostics method. As a part of image analysis, edge detection is often used for further segmentation or more precise measurements of elements in the picture. Edges represent high frequency components of an image. Unfortunately, ultrasound images are subject to degradations, especially speckle noise which is also a high frequency component. That poses a problem for edge detection algorithms since filters for noise removal also degrade edges. Canny operator is widely used as an excellent edge detector, however it also includes Gaussian smoothing element that may significantly soften edges. In this paper we propose a modified Canny algorithm where Gaussian smoothing is replaced by modified median filter that successfully removes speckle noise with little degradation of edges followed by weak weighted smoothing filter that in a controlled way removes other noise, again with insignificant damage to the edges. Our proposed algorithm was tested on standard benchmark image and compared to other approaches from literature where it proved to be successful in precisely determining edges of internal organs.

Quaternion-Michelson Descriptor for Color Image Classification

Abstract: In this paper, we develop a simple yet powerful framework called quaternion-Michelson descriptor (QMD) to extract local features for color image classification. Unlike traditional local descriptors extracted directly from the original (raw) image space, QMD is derived from the Michelson contrast law and the quaternionic representation (QR) of color images. The Michelson contrast is a stable measurement of image contents from the viewpoint of human perception, while QR is able to handle all the color information of the image holisticly and to preserve the interactions among different color channels. In this way, QMD integrates both the merits of Michelson contrast and QR. Based on the QMD framework, we further propose two novel quaternionic Michelson contrast binary pattern descriptors from different perspectives. Experiments and comparisons on different color image classification databases demonstrate that the proposed framework and descriptors outperform several state-of-the-art methods.

Noise suppression for dual-energy CT via penalized weighted least-square optimization with similarity-based regularization.

Abstract: Purpose: Dual-energy CT (DECT) expands applications of CTimaging in its capability to decompose CTimages into material images. However, decomposition via direct matrix inversion leads to large noise amplification and limits quantitative use of DECT. Their group has previously developed a noise suppression algorithm via penalized weighted least-square optimization with edge-preservation regularization (PWLS-EPR). In this paper, the authors improve method performance using the same framework of penalized weighted least-square optimization but with similarity-based regularization (PWLS-SBR), which substantially enhances the quality of decomposed images by retaining a more uniform noise power spectrum (NPS). Methods: The design of PWLS-SBR is based on the fact that averaging pixels of similar materials gives a low-noise image. For each pixel, the authors calculate the similarity to other pixels in its neighborhood by comparing CT values. Using an empirical Gaussian model, the authors assign high/low similarity value to one neighboring pixel if its CT value is close/far to the CT value of the pixel of interest. These similarity values are organized in matrix form, such that multiplication of the similarity matrix to the image vector reduces imagenoise. The similarity matrices are calculated on both high- and low-energy CTimages and averaged. In PWLS-SBR, the authors include a regularization term to minimize the L-2 norm of the difference between the images without and with noise suppression via similarity matrix multiplication. By using all pixel information of the initial CTimages rather than just those lying on or near edges, PWLS-SBR is superior to the previously developed PWLS-EPR, as supported by comparison studies on phantoms and a head-and-neck patient. Results: On the line-pair slice of the Catphan©600 phantom, PWLS-SBR outperforms PWLS-EPR and retains spatial resolution of 8 lp/cm, comparable to the original CTimages, even at 90% reduction in noise standard deviation (STD). Similar performance on spatial resolution is observed on an anthropomorphic head phantom. In addition, results of PWLS-SBR show substantially improved image quality due to preservation of image NPS. On the Catphan©600 phantom, NPS using PWLS-SBR has a correlation of 93% with that via direct matrix inversion, while the correlation drops to −52% for PWLS-EPR. Electron density measurement studies indicate high accuracy of PWLS-SBR. On seven different materials, the measured electron densities calculated from the decomposed material images using PWLS-SBR have a root-mean-square error (RMSE) of 1.20%, while the results of PWLS-EPR have a RMSE of 2.21%. In the study on a head-and-neck patient, PWLS-SBR is shown to reduce noise STD by a factor of 3 on material images with image qualities comparable to CTimages, whereas fine structures are lost in the PWLS-EPR result. Additionally, PWLS-SBR better preserves low contrast on the tissueimage. Conclusions: The authors propose improvements to the regularization term of an optimization framework which performs iterative image-domain decomposition for DECT with noise suppression. The regularization term avoids calculation of image gradient and is based on pixel similarity. The proposed method not only achieves a high decomposition accuracy, but also improves over the previous algorithm on NPS as well as spatial resolution.

Edge detection of license plate using Sobel operator

Abstract: Edge detection is one of the important operations in image processing and computer vision. It is the process that is used to locate the boundaries of objects or textures depicted in an image. To know the positions of these boundaries is a critical task in the process of image enhancement, recognition, restoration and compression. The edges of image are considered to be the most important attributes of image that provide valuable information for human image perception. As the data of edge detection is very large, therefore the speed of image processing becomes a difficult problem. The sobel operator is used for edge detection. In the edge function, the Sobel method uses the derivative approximation to find edges of the image. So, it returns edges at those points where the gradient of the considered image is maximum. The horizontal and vertical gradient matrices are used for the Sobel method whose dimensions are 3×3 in the edge detection operations. It has been proven by the results we have obtained, that the edge detection mathematical method by simulation using MATLAB software is very good method for the analyzing the image. After reading the pixels of an image the algorithm is applied in VERILOG. The entire simulation of the above process is done VERILOG using “XILINX-14.1”. And to display input and output image MATLAB is used. This paper focuses on software used to detect edges of image employing mainly the MATLAB program for solving this problem.

Ghosting-free multi-exposure image fusion in gradient domain

Abstract: This paper presents an algorithm to produce ghosting-free High Dynamic Range (HDR) image by fusing set of multiple exposed images in gradient domain. Recently proposed Gradient domain based exposure fusion method provides high quality result but the scope of which is limited to static camera without foreground object motion. The presence of moving objects/hand shake produces a set of misaligned images. The result of gradient domain approach on misaligned images suffers from ghosting artifacts. In order to produce better HDR image without image registration, we propose to create an aligned image set from input image set by photometric calibration. The gradient of aligned image set is then used to reconstruct the fused final image. The proposed algorithm tested on several publicly available dynamic image sets shows that resultant HDR image is ghosting-free and well exposed. Additionally, the proposed method is fast and thus can be used in consumer appliances such as mobile phones, portable devices with digital cameras.