Showing papers on "Subpixel rendering published in 2021"

Super-Resolution Mapping Based on Spatial–Spectral Correlation for Spectral Imagery

Abstract: Due to the influences of imaging conditions, spectral imagery can be coarse and contain a large number of mixed pixels. These mixed pixels can lead to inaccuracies in the land-cover class (LC) mapping. Super-resolution mapping (SRM) can be used to analyze such mixed pixels and obtain the LC mapping information at the subpixel level. However, traditional SRM methods mostly rely on spatial correlation based on linear distance, which ignores the influences of nonlinear imaging conditions. In addition, spectral unmixing errors affect the accuracy of utilized spectral properties. In order to overcome the influence of linear and nonlinear imaging conditions and utilize more accurate spectral properties, the SRM based on spatial–spectral correlation (SSC) is proposed in this work. Spatial correlation is obtained using the mixed spatial attraction model (MSAM) based on the linear Euclidean distance. Besides, a spectral correlation that utilizes spectral properties based on the nonlinear Kullback–Leibler distance (KLD) is proposed. Spatial and spectral correlations are combined to reduce the influences of linear and nonlinear imaging conditions, which results in an improved mapping result. The utilized spectral properties are extracted directly by spectral imagery, thus avoiding the spectral unmixing errors. Experimental results on the three spectral images show that the proposed SSC yields better mapping results than state-of-the-art methods.

Super-resolution-based Change Detection Network with Stacked Attention Module for Images with Different Resolutions

Abstract: Change detection, which aims to distinguish surface changes based on bi-temporal images, plays a vital role in ecological protection and urban planning. Since high resolution (HR) images cannot be typically acquired continuously over time, bi-temporal images with different resolutions are often adopted for change detection in practical applications. Traditional subpixel-based methods for change detection using images with different resolutions may lead to substantial error accumulation when HR images are employed; this is because of intraclass heterogeneity and interclass similarity. Therefore, it is necessary to develop a novel method for change detection using images with different resolutions, that is more suitable for HR images. To this end, we propose a super-resolution-based change detection network (SRCDNet) with a stacked attention module. The SRCDNet employs a super resolution (SR) module containing a generator and a discriminator to directly learn SR images through adversarial learning and overcome the resolution difference between bi-temporal images. To enhance the useful information in multi-scale features, a stacked attention module consisting of five convolutional block attention modules (CBAMs) is integrated to the feature extractor. The final change map is obtained through a metric learning-based change decision module, wherein a distance map between bi-temporal features is calculated. The experimental results demonstrate the superiority of the proposed method, which not only outperforms all baselines -with the highest F1 scores of 87.40% on the building change detection dataset and 92.94% on the change detection dataset -but also obtains the best accuracies on experiments performed with images having a 4x and 8x resolution difference. The source code of SRCDNet will be available at this https URL.

Deep Subpixel Mapping Based on Semantic Information Modulated Network for Urban Land Use Mapping

Abstract: Mixed pixel problem is omnipresent in remote sensing images for urban land use interpretation due to the hardware limitations. Subpixel mapping (SPM) is a usual way to solve this problem by improving the observation scale and realizing a finer spatial resolution land cover mapping. Recently, deep learning-based subpixel mapping network (DLSMNet) was proposed, benefited from its strong representation and learning ability, to restore a visually pleasing finer mapping. However, the spatial context features of artifacts are usually aggregated and progressively lost during the forward pass of the network without sufficient representation, which make it difficult to be learned and restored. In this article, a semantic information modulated (SIM) deep subpixel mapping network (SIMNet) is proposed, which uses low-resolution semantic images as prior, to reinforce the representation of spatial context features. In SIMNet, SIM module is proposed to parametrically incorporate the semantic prior into the state-of-the-art (SOTA) feed forward network architecture in an end-to-end training fashion. Furthermore, stacked SIM module with residual blocks (SIM_ResBlock) is adopted to pass the representation of spatial context feature to the deep layers, to get it fully learned during backpropagation. Experiments have been implemented on three public urban scenario data sets, and the SIMNet generates a clearer outline of artificial facilities with sufficient spatial context, and is distinctive for even individual building, which is challenging for other SOTA DLSMNet. The results demonstrate that the proposed SIMNet is a promising way for high-resolution urban land use mapping from easily available lower resolution remote sensing images.Mixed pixel problem is omnipresent in remote sensing images for urban land use interpretation due to the hardware limitations. Subpixel mapping (SPM) is a usual way to solve this problem by improving the observation scale and realizing a finer spatial resolution land cover mapping. Recently, deep learning-based subpixel mapping network (DLSMNet) was proposed, benefited from its strong representation and learning ability, to restore a visually pleasing finer mapping. However, the spatial context features of artifacts are usually aggregated and progressively lost during the forward pass of the network without sufficient representation, which make it difficult to be learned and restored. In this article, a semantic information modulated (SIM) deep subpixel mapping network (SIMNet) is proposed, which uses low-resolution semantic images as prior, to reinforce the representation of spatial context features. In SIMNet, SIM module is proposed to parametrically incorporate the semantic prior into the state-of-the-art (SOTA) feed forward network architecture in an end-to-end training fashion. Furthermore, stacked SIM module with residual blocks (SIM_ResBlock) is adopted to pass the representation of spatial context feature to the deep layers, to get it fully learned during backpropagation. Experiments have been implemented on three public urban scenario data sets, and the SIMNet generates a clearer outline of artificial facilities with sufficient spatial context, and is distinctive for even individual building, which is challenging for other SOTA DLSMNet. The results demonstrate that the proposed SIMNet is a promising way for high-resolution urban land use mapping from easily available lower resolution remote sensing images.

Unsupervised Pansharpening Based on Self-Attention Mechanism

Abstract: Pansharpening is to fuse a multispectral image (MSI) of low-spatial-resolution (LR) but rich spectral characteristics with a panchromatic image (PAN) of high spatial resolution (HR) but poor spectral characteristics. Traditional methods usually inject the extracted high-frequency details from PAN into the upsampled MSI. Recent deep learning endeavors are mostly supervised assuming that the HR MSI is available, which is unrealistic especially for satellite images. Nonetheless, these methods could not fully exploit the rich spectral characteristics in the MSI. Due to the wide existence of mixed pixels in satellite images where each pixel tends to cover more than one constituent material, pansharpening at the subpixel level becomes essential. In this article, we propose an unsupervised pansharpening (UP) method in a deep-learning framework to address the abovementioned challenges based on the self-attention mechanism (SAM), referred to as UP-SAM. The contribution of this article is threefold. First, the SAM is proposed where the spatial varying detail extraction and injection functions are estimated according to the attention representations indicating spectral characteristics of the MSI with subpixel accuracy. Second, such attention representations are derived from mixed pixels with the proposed stacked attention network powered with a stick-breaking structure to meet the physical constraints of mixed pixel formulations. Third, the detail extraction and injection functions are spatial varying based on the attention representations, which largely improves the reconstruction accuracy. Extensive experimental results demonstrate that the proposed approach is able to reconstruct sharper MSI of different types, with more details and less spectral distortion compared with the state-of-the-art.

Deep Convolutional Neural Network Framework for Subpixel Mapping

Abstract: Subpixel mapping (SPM) is an effective way to solve the mixed pixel problem, which is a ubiquitous phenomenon in remotely sensed imagery, by characterizing subpixel distribution within the mixed pixels. In fact, the majority of the classical and state-of-the-art SPM algorithms can be viewed as a convolution process, but these methods rely heavily on fixed and handcrafted kernels that are insufficient in characterizing a geographically realistic distribution image. In addition, the traditional SPM approach is based on the prerequisite of abundance images derived from spectral unmixing (SU), during which process uncertainty inherently exists and is propagated to the SPM. In this article, a kernel-learnable convolutional neural network (CNN) framework for subpixel mapping (SPMCNN-F) is proposed. In SPMCNN-F, the kernel is learnable during the training stage based on the given training sample pairs of low- and high-resolution patches for learning a geographically realistic prior, instead of fixed priors. The end-to-end mapping structure enables direct subpixel information extraction from the original coarse image, avoiding the uncertainty propagation from the SU. In the experiments undertaken in this study, two state-of-the-art super-resolution networks were selected as application demonstrations of the proposed SPMCNN-F method. In experiment part, three hyperspectral image data sets were adopted, two in a synthetic coarse image approach and one in a real coarse image approach, for the validation. Additionally, a new data set with pairs of Moderate-resolution Imaging Spectroradiometer (MODIS) and Landsat images were adopted in a real coarse image approach, for further validation of SPMCNN-F in large-scale area. The restored fine distribution images obtained in all the experiments showed a perceptually better reconstruction quality, both qualitatively and quantitatively, confirming the superiority of the proposed SPM framework.

Robust Face Alignment by Multi-Order High-Precision Hourglass Network

Abstract: Heatmap regression (HR) has become one of the mainstream approaches for face alignment and has obtained promising results under constrained environments. However, when a face image suffers from large pose variations, heavy occlusions and complicated illuminations, the performances of HR methods degrade greatly due to the low resolutions of the generated landmark heatmaps and the exclusion of important high-order information that can be used to learn more discriminative features. To address the alignment problem for faces with extremely large poses and heavy occlusions, this paper proposes a heatmap subpixel regression (HSR) method and a multi-order cross geometry-aware (MCG) model, which are seamlessly integrated into a novel multi-order high-precision hourglass network (MHHN). The HSR method is proposed to achieve high-precision landmark detection by a well-designed subpixel detection loss (SDL) and subpixel detection technology (SDT). At the same time, the MCG model is able to use the proposed multi-order cross information to learn more discriminative representations for enhancing facial geometric constraints and context information. To the best of our knowledge, this is the first study to explore heatmap subpixel regression for robust and high-precision face alignment. The experimental results from challenging benchmark datasets demonstrate that our approach outperforms state-of-the-art methods in the literature.

Fingerprint identification display device and driving method thereof

Abstract: A fingerprint identification display device, includes: a display panel having a plurality of display units defined therein, and at least some of the plurality of display units are located in a fingerprint identification region, each of which comprises display subpixels and an identification subpixel; a light guide plate; one or more light emitting components configured to make detection light emitted therefrom travel in the light guide plate in a total reflection manner; a first polarizer comprising a first region and a second region with perpendicular polarization directions, and a second polarizer comprising a third region and a fourth region with perpendicular polarization directions, wherein the first region and the third region correspond to the display subpixels and have perpendicular polarization directions, and the second region and the fourth region correspond to the identification subpixel; and a photosensitive sensing unit disposed at the second polarizer, and corresponding to the identification subpixel.

A deep learning semantic template matching framework for remote sensing image registration

Abstract: We propose a deep learning framework by the probability of the predicting semantic spatial position distribution for remote sensing image registration. Traditional matching methods optimize similarity metrics with pixel-by-pixel searching, which is time consuming and sensitive to radiometric differences. Driven by learning-based methods, we take the reference and template images as inputs and map them to the semantic distribution position of the corresponding reference image. We apply the affine invariant to obtain a correspondence between the overlap of the barycenter position of the semantic template and the center pixel point, which transforms the semantic boundary alignment into a point-to-point matching problem. Additionally, two loss functions are proposed, one for optimizing the subpixel matching position and the other for determining the semantic spatial probability distribution of the matching template. In this work, we explore the influence of the template radius size, the filling form of training labels, and the weighted combination of loss function on the matching accuracy. Our experiments show that the trained model is robust to template deformation while still operating orders of magnitude faster. Furthermore, our proposed method implements high matching accuracy in four large scene images with radiometric differences. This ensures the improved speed of remote sensing image analysis and pipeline processing while facilitating novel directions in learning-based registration. Our code is freely available at https://github.com/liliangzhi110/semantictemplatematching .

Infrared Small UAV Target Detection Based on Residual Image Prediction via Global and Local Dilated Residual Networks

Abstract: Thermal infrared imaging possesses the ability to monitor unmanned aerial vehicles (UAVs) in both day and night conditions. However, long-range detection of the infrared UAVs often suffers from small/dim targets, heavy clutter, and noise in the complex background. The conventional local prior-based and the nonlocal prior-based methods commonly have a high false alarm rate and low detection accuracy. In this letter, we propose a model that converts small UAV detection into a problem of predicting the residual image (i.e., background, clutter, and noise). Such novel reformulation allows us to directly learn a mapping from the input infrared image to the residual image. The constructed image-to-image network integrates the global and the local dilated residual convolution blocks into the U-Net, which can capture local and contextual structure information well and fuse the features at different scales both for image reconstruction. Additionally, subpixel convolution is utilized to upscale the image and avoid image distortion during upsampling. Finally, the small UAV target image is obtained by subtracting the residual image from the input infrared image. The comparative experiments demonstrate that the proposed method outperforms state-of-the-art ones in detecting real-world infrared images with heavy clutter and dim targets.

Research on Change Detection Method of High-Resolution Remote Sensing Images Based on Subpixel Convolution

Abstract: Remote sensing image change detection method plays a great role in land cover research, disaster assessment, medical diagnosis, video surveillance, and other fields, so it has attracted wide attention. Based on a small sample dataset from SZTAKI AirChange Benchmark Set, in order to solve the problem that the deep learning network needs a large number of samples, this work first uses nongenerative sample augmentation method and generative sample augmentation method based on deep convolutional generative adversarial networks, and then, constructs a remote sensing image change detection model based on an improved DeepLabv3+ network. This model can realize end-to-end training and prediction of remote sensing image change detection with subpixel convolution. Finally, Landsat 8, Google Earth, and Onera satellite change detection datasets are used to verify the generalization performance of this network. The experimental results show that the improved network accuracy is 95.1% and the generalization performance is acceptable.

Detection of Subpixel Targets on Hyperspectral Remote Sensing Imagery Based on Background Endmember Extraction

Abstract: The low spatial resolution associated with imaging spectrometers has caused subpixel target detection to become a special problem in hyperspectral image (HSI) processing that poses considerable challenges In subpixel target detection, the size of the target is smaller than that of a pixel, making the spatial information of the target almost useless so that a detection algorithm must rely on the spectral information of the image To address this problem, this article proposes a subpixel target detection algorithm for hyperspectral remote sensing imagery based on background endmember extraction First, we propose a background endmember extraction algorithm based on robust nonnegative dictionary learning to obtain the background endmember spectrum of the image Next, we construct a hyperspectral subpixel target detector based on pixel reconstruction (HSPRD) to perform pixel-by-pixel target detection on the image to be tested using the background endmember spectral matrix and the spectra of known ground targets Finally, the subpixel target detection results are obtained The experimental results show that, compared with other existing subpixel target detection methods, the algorithm proposed here can provide the optimum target detection results for both synthetic and real-world data sets

A novel industrial chip parameters identification method based on cascaded region segmentation for surface mount equipment

Abstract: It is necessary to obtain the geometric parameters of the chip in surface mount technology(SMT). The traditional manual measurement method has low efficiency and poor accuracy. Therefore, this paper proposes a robust,accurate, and fast automatic learning algorithm for chip parameters. The adaptive threshold FAST feature points extraction based on the local and global gray difference is proposed and the coarse positioning of the chip is achieved by enclosing the extracted FAST feature points.The cascaded region segmentation, including pins group extraction and pin extraction, is then proposed to separate the pins group and each pin and the interference suppression strategy is applied to enhance the segmentation robustness. The accurate contour of each pin is subsequently extracted by using the sub-pixel edge extraction method and the geometric dimensions of the pin and body of the chip are calculated. The defect detection is finally per-formed based on the calculated geometric dimensions. We compared proposed method with the Hanwha SM481Plusmachine, the pin clustering method, and the pin line fitting method on SMT hardware platform. The results show that proposed method has high robustness, high precision and high efficiency.

Collaborative Coupled Hyperspectral Unmixing Based Subpixel Change Detection for Analyzing Coastal Wetlands

Abstract: Owing to the complicated and heterogeneous distribution characteristics of wetland features, the existing hyperspectral technology is difficult to investigate the inner-pixel subtle changes. In this article, we present a subpixel change detection method based on collaborative coupled unmixing (SCDUM) for monitoring coastal wetlands. A novel multitemporal and spatial scale collaborative endmember extraction method based on joint spatial and spectral information is proposed. In the proposed method, the multitemporal hyperspectral images are first jointly clustered and segmented based on multifeature fusion of spectral features, texture features, and shape features. Then, a different spatial scale nonnegative matrix factorization based on original and downsampled multitemporal hyperspectral images is proposed to accurately extract the pure endmembers of each segmented images. Finally, the global abundance of the multitemporal image is effectively estimated for change detection. In addition, in order to verify the accuracy of the change detection results without reference, an accuracy verification strategy by using high spatial resolution Sentinel-2A image as auxiliary data is implemented. The Yellow River Estuary coastal wetland was selected as the research area, and the Gaofen-5 and ZY-1 02D hyperspectral images were used as the research data. In particular, the proposed method not only provides the overall change information, but also obtains the component of change direction and intensity of each kind of endmember, and the experimental results show that the SCDUM gives more accurate detection results, with closer to the endmember spectral curves of real objects, compared with other state-of-the-art methods.

Unsupervised Bayesian Subpixel Mapping of Hyperspectral Imagery Based on Band-Weighted Discrete Spectral Mixture Model and Markov Random Field

Abstract: Although accurate training and initialization information is difficult to acquire, unsupervised hyperspectral subpixel mapping (SPM) without relying on this predefined information is an insufficiently addressed research issue. This letter presents a novel Bayesian approach for unsupervised SPM of hyperspectral imagery (HSI) based on the Markov random field (MRF) and a band-weighted discrete spectral mixture model (BDSMM), with the following key characteristics. First, this is an unsupervised approach that allows adjustment of abundance and endmember information adaptively for less relying on algorithm initialization. Second, this approach consists of the BDSMM for accommodating the noise heterogeneity and the hidden label field of subpixels in HSI. The BDSMM also integrates SPM into the spectral mixture analysis and allows enhanced SPM by fully exploring the endmember-abundance patterns in HSI. Third, the MRF and BDSMM are integrated into a Bayesian framework to use both the spatial and spectral information efficiently, and an expectation-maximization (EM) approach is designed to solve the model by iteratively estimating the endmembers and the label field. Experiments on both simulated and real HSI demonstrate that the proposed algorithm can yield better performance than traditional methods.

Subpixel Matching Using Double-Precision Gradient-Based Method for Digital Image Correlation.

Abstract: Digital image correlation (DIC) for displacement and strain measurement has flourished in recent years. There are integer pixel and subpixel matching steps to extract displacement from a series of images in the DIC approach, and identification accuracy mainly depends on the latter step. A subpixel displacement matching method, named the double-precision gradient-based algorithm (DPG), is proposed in this study. After, the integer pixel displacement is identified using the coarse-fine search algorithm. In order to improve the accuracy and anti-noise capability in the subpixel extraction step, the traditional gradient-based method is used to analyze the data on the speckle patterns using the computer, and the influence of noise is considered. These two nearest integer pixels in one direction are both utilized as an interpolation center. Then, two subpixel displacements are extracted by the five-point bicubic spline interpolation algorithm using these two interpolation centers. A novel combination coefficient considering contaminated noises is presented to merge these two subpixel displacements to obtain the final identification displacement. Results from a simulated speckle pattern and a painted beam bending test show that the accuracy of the proposed method can be improved by four times that of the traditional gradient-based method that reaches the same high accuracy as the Newton–Raphson method. The accuracy of the proposed method efficiently reaches at 92.67%, higher than the Newton-Raphon method, and it has better anti-noise performance and stability.

Optical-numerical method based on a convolutional neural network for full-field subpixel displacement measurements

Abstract: The subpixel displacement estimation is an important step to calculation of the displacement between two digital images in optics and image processing. Digital image correlation (DIC) is an effective method for measuring displacement due to its high accuracy. Various DIC algorithms to compare images and to obtain displacement have been implemented. However, there are some drawbacks to DIC. It can be computationally expensive when processing a sequence of continuously deformed images. To simplify the subpixel displacement estimation and to explore a different measurement scheme, a convolutional neural network with a transfer learning based subpixel displacement measurement method (CNN-SDM) is proposed in this paper. The basic idea of the method is to compare images of an object decorated with speckle patterns before and after deformation by CNN, and thereby to achieve a coarse-to-fine subpixel displacement estimation. The proposed CNN is a classification model consisting of two convolutional neural networks in series. The results of simulated and real experiments are shown that the proposed CNN-SDM method is feasibly effective for subpixel displacement measurement due its high efficiency, robustness, simple structure and few parameters.

Hardware-Oriented Algorithm for High-Speed Laser Centerline Extraction Based on Hessian Matrix

Abstract: Centerline extraction is the basic and key procedure in line-structured laser 3-D scanners. In this article, we propose a hardware-oriented algorithm for fast and accurate extraction of a laser centerline based on the Hessian matrix. The algorithm is divided into three low-coupling modules that can be processed in parallel—coarse positioning, linewidth estimation, and precise positioning module. In the coarse positioning module, a window slider is used to traverse all image pixels in the raster-scan mode for collecting local image features, based on which the potential region of interest (ROI) containing laser lines is detected. In the linewidth estimation module, the second-order moment features in the detected ROI are calculated to estimate the linewidth according to the local rectangular similarity characteristics of the laser line. In the precise positioning module, the optimal Gaussian template estimated by the linewidth is convolved with the ROI to obtain the Hessian matrix. Based on the Hessian matrix, the normal direction of the laser line is obtained, and the second-order Taylor expansion is performed in that direction to determine the subpixel position of the center point. Finally, non-maximum suppression is used to remove noise points and obtain the most reliable single-pixel centerline. The proposed algorithm is evaluated using thousands of sample images with different materials, exposure times, and laser line shapes, and it presents high robustness for subpixel precision extraction. By implementing the proposed algorithm on a field-programmable gate array (FPGA), a high-speed, high-precision laser centerline extraction system is achieved, which operates at 1350 frames/s for a 1-million-pixel video stream.

Precision single-particle localization using radial variance transform

Abstract: We introduce an image transform designed to highlight features with high degree of radial symmetry for identification and subpixel localization of particles in microscopy images. The transform is based on analyzing pixel value variations in radial and angular directions. We compare the subpixel localization performance of this algorithm to other common methods based on radial or mirror symmetry (such as fast radial symmetry transform, orientation alignment transform, XCorr, and quadrant interpolation), using both synthetic and experimentally obtained data. We find that in all cases it achieves the same or lower localization error, frequently reaching the theoretical limit.

Bayesian Detection of Solid Subpixel Targets

Abstract: We implement and evaluate a Bayesian detector for opaque subpixel hyperspectral targets of unknown abundance. Using both simulated and real hyperspectral backgrounds, we compare this detector to the more conventional generalized likelihood ratio test (GLRT) approach, identifying theoretical differences and observing numerical similarities. Among the theoretical advantages provided by the Bayesian detector is admissibility, which means that no detector can be uniformly superior to it. Potential disadvantages include the need to choose a prior distribution, and the computation required to integrate that distribution. For solid subpixel targets, the uniform prior is a natural choice, and we find that adequately-accurate numerical integration can be achieved with only a few evaluations of the likelihood function. We show results for targets implanted in both simulated and real data.

A Framework for Hexagonal Image Processing Using Hexagonal Pixel-Perfect Approximations in Subpixel Resolution

Abstract: Image processing in hexagonal lattice has many advantages rather than square lattice. Researchers have addressed benefits of hexagonal structure in applications such as binarization, rotation, scaling and edge detection. Approximately all existing hardwares for capturing and displaying images are based on square lattice. Therefore, the best way for using advantages of hexagonal lattice is to find a proper software approach to convert square pixels to hexagonal ones. This paper presents a hexagonal platform based on interpolation which addresses three existing hexagonal challenges including imperfect hexagonal shape, inaccurate intensity level of hexagonal pixels and lower resolution in hexagonal space. The proposed interpolation is computed by overlaps between square and hexagonal pixels. Overlap types are formulated mathematically in 8 separate cases. Each overlap case is detected automatically and used to compute final gray-level intensity of hexagonal pixels. It is mathematically and experimentally shown that the proposed method satisfies necessary conditions for square-to-hexagonal conversion. The proposed scheme is evaluated on synthetic and real images with 10 different levels of noise in interpolation and edge detection applications. In synthetic images, the proposed method achieves the best figure of merit (FOM) 99.92% and 98.67% in high and low SNRs 100 and 20, respectively. Also, the proposed method outperforms existing state of the art hexagonal lattices with interclass correlation coefficient (ICC) 84.18% and mean rating 7.7 (out of 9) in real images.

Subpixel sampling moiré method for in-plane displacement measurement considering the symmetric errors induced by interpolation

Abstract: Sampling moire method (SMM) is a highly accurate vision-based deformation measurement method, whose measurement error is minimized when an integer that is closest to the grating pitch is selected as the sampling pitch. We propose a subpixel SMM by applying subpixel interpolation operation before the downsampling in traditional SMM; the sampling pitch was selected as a noninteger nearest to the grating pitch in the subpixel resolution image. Meanwhile, the average filter method was used to eliminate the symmetric error cause by interpolation. As a result, the period of moire fringe was enlarged greatly, and the measurement accuracy was also increased. To investigate the efficiency of the subpixel SMM, a computer simulation was applied to analyze the accuracy of the subpixel SMM. Then a simple tensile experiment was conducted to validate the efficiency of this method, and the result of the subpixel SMM accorded well with the fiber Bragg grating. In summary, the proposed novel subpixel SMM is capable of being used for high accuracy in-plane deformation measuring.

Subpixel and On-Line Motion Correction for Photoacoustic Dermoscopy

Abstract: Photoacoustic dermoscopy (PAD) has been proven to visualize the microvascular network within the dermis noninvasively, and has great application potential and advantages. In practice, some subjects are children or unwell and may have difficulty in controlling their breathing or trembling. The artifacts caused by the uncontrollable trembling will degrade the image quality, resulting in deviation of the pathophysiological features. This work introduces a subpixel and on-line motion correction method for clinical application of PAD. The high-accuracy motion correction was realized by the subpixel motion estimation, which was achieved by five-fold upsampling the cross-correlation matrix between A-lines or B-scan images. Since the motion correction algorithm only takes the acquired data as a priori, it can be processed immediately when a B-scan is input, so it can be embedded in the acquisition program, leading to the motion correction during the data acquisition. Moreover, the algorithm was verified by the experimental data of human skin. The results demonstrated that the images after sub-pixel motion correction are obviously improved visually, and the structural similarity index measurement (SSIM) and peak signal-to-noise ratio (PSNR) between adjacent B-scan images after correction are increased by 31.6% and 47.7% compared to the uncorrected images. To conclude, the proposed algorithm can make it robust in terms of accuracy and reliability and will help PAD fulfill its clinical promise.

Subpixel target implantation to assess pansharpening performance on hyperspectral datasets

Abstract: Target detection is one of the most important applications utilizing the rich spectral information from hyperspectral imaging systems. Data fusion algorithms applied on hyperspectral datasets address the inherent spatial-spectral resolution tradeoff in these imaging systems by combining spectral information from hyperspectral data with spatial information from hi-res panchromatic or multispectral images (e.g., hi-res RGB). This paper presents the first attempt at using an iterative target implantation technique as a modification to Wald's protocol to assess the performance of data fusion algorithms in target detection tasks. More specifically, this paper looks at how the sharpening process localizes and discriminates the subpixel target from its background, and characterizes an image-wide detectability of any single subpixel target independent of location in the image. We used NNDi use as our pansharpener to perform HRPAN+LRHSI data fusion and the adaptive coherence estimator (ACE) as our target detector. Results show that our methodology is effective at assessing (1) how the sharpening process enhances target-background separability within any 5x5 window anywhere on the image and (2) how the sharpening process enhances the detectability of a single subpixel target over the entire hyperspectral image.

A Fast Image Alignment Approach for 2D Classification of Cryo-EM Images Using Spectral Clustering

Abstract: Three-dimensional (3D) reconstruction in single-particle cryo-electron microscopy (cryo-EM) is a significant technique for recovering the 3D structure of proteins or other biological macromolecules from their two-dimensional (2D) noisy projection images taken from unknown random directions. Class averaging in single-particle cryo-EM is an important procedure for producing high-quality initial 3D structures, where image alignment is a fundamental step. In this paper, an efficient image alignment algorithm using 2D interpolation in the frequency domain of images is proposed to improve the estimation accuracy of alignment parameters of rotation angles and translational shifts between the two projection images, which can obtain subpixel and subangle accuracy. The proposed algorithm firstly uses the Fourier transform of two projection images to calculate a discrete cross-correlation matrix and then performs the 2D interpolation around the maximum value in the cross-correlation matrix. The alignment parameters are directly determined according to the position of the maximum value in the cross-correlation matrix after interpolation. Furthermore, the proposed image alignment algorithm and a spectral clustering algorithm are used to compute class averages for single-particle 3D reconstruction. The proposed image alignment algorithm is firstly tested on a Lena image and two cryo-EM datasets. Results show that the proposed image alignment algorithm can estimate the alignment parameters accurately and efficiently. The proposed method is also used to reconstruct preliminary 3D structures from a simulated cryo-EM dataset and a real cryo-EM dataset and to compare them with RELION. Experimental results show that the proposed method can obtain more high-quality class averages than RELION and can obtain higher reconstruction resolution than RELION even without iteration.

Efficient Multisource Remote Sensing Image Matching Using Dominant Orientation of Gradient

Abstract: Image matching is the key step for image registration Due to the existing nonlinear intensity differences between multisource images, their matching is still a challenging task A fast matching approach based on dominant orientation of gradient (DOG) is proposed in this article, which is robust to nonlinear intensity variations The DOG feature maps are constructed by extracting DOG feature of each pixel in the images in the first place A template matching method is used to determine correspondences between images based on the feature representations We define a similarity measurement, referred to as sum of cosine differences, which can be accelerated by fast Fourier transform Subsequently, the subpixel accuracy can be achieved by fitting the similarity measurement using a quadratic polynomial modal A new variable template matching (VTM) method has been developed to improve the matching performance Experimental results confirm that the proposed matching approach is robust to nonlinear intensity differences and has time efficiency The VTM method additionally improves the matching precision effectively