Showing papers on "Subpixel rendering published in 2018"

ActiveStereoNet: End-to-End Self-Supervised Learning for Active Stereo Systems

Abstract: In this paper we present ActiveStereoNet, the first deep learning solution for active stereo systems. Due to the lack of ground truth, our method is fully self-supervised, yet it produces precise depth with a subpixel precision of 1 / 30th of a pixel; it does not suffer from the common over-smoothing issues; it preserves the edges; and it explicitly handles occlusions. We introduce a novel reconstruction loss that is more robust to noise and texture-less patches, and is invariant to illumination changes. The proposed loss is optimized using a window-based cost aggregation with an adaptive support weight scheme. This cost aggregation is edge-preserving and smooths the loss function, which is key to allow the network to reach compelling results. Finally we show how the task of predicting invalid regions, such as occlusions, can be trained end-to-end without ground-truth. This component is crucial to reduce blur and particularly improves predictions along depth discontinuities. Extensive quantitatively and qualitatively evaluations on real and synthetic data demonstrate state of the art results in many challenging scenes.

Road Surface 3D Reconstruction Based on Dense Subpixel Disparity Map Estimation

Abstract: Various 3D reconstruction methods have enabled civil engineers to detect damage on a road surface. To achieve the millimeter accuracy required for road condition assessment, a disparity map with subpixel resolution needs to be used. However, none of the existing stereo matching algorithms are specially suitable for the reconstruction of the road surface. Hence in this paper, we propose a novel dense subpixel disparity estimation algorithm with high computational efficiency and robustness. This is achieved by first transforming the perspective view of the target frame into the reference view, which not only increases the accuracy of the block matching for the road surface but also improves the processing speed. The disparities are then estimated iteratively using our previously published algorithm, where the search range is propagated from three estimated neighboring disparities. Since the search range is obtained from the previous iteration, errors may occur when the propagated search range is not sufficient. Therefore, a correlation maxima verification is performed to rectify this issue, and the subpixel resolution is achieved by conducting a parabola interpolation enhancement. Furthermore, a novel disparity global refinement approach developed from the Markov random fields and fast bilateral stereo is introduced to further improve the accuracy of the estimated disparity map, where disparities are updated iteratively by minimizing the energy function that is related to their interpolated correlation polynomials. The algorithm is implemented in C language with a near real-time performance. The experimental results illustrate that the absolute error of the reconstruction varies from 0.1 to 3 mm.

PMSC: PatchMatch-Based Superpixel Cut for Accurate Stereo Matching

Abstract: Estimating the disparity and normal direction of one pixel simultaneously, instead of only disparity, also known as 3D label methods, can achieve much higher subpixel accuracy in the stereo matching problem. However, it is extremely difficult to assign an appropriate 3D label to each pixel from the continuous label space $\mathbb {R}^{3}$ while maintaining global consistency because of the infinite parameter space. In this paper, we propose a novel algorithm called PatchMatch-based superpixel cut to assign 3D labels of an image more accurately. In order to achieve robust and precise stereo matching between local windows, we develop a bilayer matching cost, where a bottom–up scheme is exploited to design the two layers. The bottom layer is employed to measure the similarity between small square patches locally by exploiting a pretrained convolutional neural network, and then, the top layer is developed to assemble the local matching costs in large irregular windows induced by the tangent planes of object surfaces. To optimize the spatial smoothness of local assignments, we propose a novel strategy to update 3D labels. In the procedure of optimization, both segmentation information and random refinement of PatchMatch are exploited to update candidate 3D label set for each pixel with high probability of achieving lower loss. Since pairwise energy of general candidate label sets violates the submodular property of graph cut, we propose a novel multilayer superpixel structure to group candidate label sets into candidate assignments, which thereby can be efficiently fused by $\alpha $ -expansion graph cut. Extensive experiments demonstrate that our method can achieve higher subpixel accuracy in different data sets, and currently ranks first on the new challenging Middlebury 3.0 benchmark among all the existing methods.

Enhancing the Spatial Resolution of Stereo Images Using a Parallax Prior

Abstract: We present a novel method that can enhance the spatial resolution of stereo images using a parallax prior. While traditional stereo imaging has focused on estimating depth from stereo images, our method utilizes stereo images to enhance spatial resolution instead of estimating disparity. The critical challenge for enhancing spatial resolution from stereo images: how to register corresponding pixels with subpixel accuracy. Since disparity in traditional stereo imaging is calculated per pixel, it is directly inappropriate for enhancing spatial resolution. We, therefore, learn a parallax prior from stereo image datasets by jointly training two-stage networks. The first network learns how to enhance the spatial resolution of stereo images in luminance, and the second network learns how to reconstruct a high-resolution color image from high-resolution luminance and chrominance of the input image. Our two-stage joint network enhances the spatial resolution of stereo images significantly more than single-image super-resolution methods. The proposed method is directly applicable to any stereo depth imaging methods, enabling us to enhance the spatial resolution of stereo images.

Road surface 3d reconstruction based on dense subpixel disparity map estimation

Abstract: Various 3D reconstruction methods have enabled civil engineers to detect damage on a road surface. To achieve the millimetre accuracy required for road condition assessment, a disparity map with subpixel resolution needs to be used. However, none of the existing stereo matching algorithms are specially suitable for the reconstruction of the road surface. Hence in this paper, we propose a novel dense subpixel disparity estimation algorithm with high computational efficiency and robustness. This is achieved by first transforming the perspective view of the target frame into the reference view, which not only increases the accuracy of the block matching for the road surface but also improves the processing speed. The disparities are then estimated iteratively using our previously published algorithm where the search range is propagated from three estimated neighbouring disparities. Since the search range is obtained from the previous iteration, errors may occur when the propagated search range is not sufficient. Therefore, a correlation maxima verification is performed to rectify this issue, and the subpixel resolution is achieved by conducting a parabola interpolation enhancement. Furthermore, a novel disparity global refinement approach developed from the Markov Random Fields and Fast Bilateral Stereo is introduced to further improve the accuracy of the estimated disparity map, where disparities are updated iteratively by minimising the energy function that is related to their interpolated correlation polynomials. The algorithm is implemented in C language with a near real-time performance. The experimental results illustrate that the absolute error of the reconstruction varies from 0.1 mm to 3 mm.

DGC-Net: Dense Geometric Correspondence Network

Abstract: This paper addresses the challenge of dense pixel correspondence estimation between two images. This problem is closely related to optical flow estimation task where ConvNets (CNNs) have recently achieved significant progress. While optical flow methods produce very accurate results for the small pixel translation and limited appearance variation scenarios, they hardly deal with the strong geometric transformations that we consider in this work. In this paper, we propose a coarse-to-fine CNN-based framework that can leverage the advantages of optical flow approaches and extend them to the case of large transformations providing dense and subpixel accurate estimates. It is trained on synthetic transformations and demonstrates very good performance to unseen, realistic, data. Further, we apply our method to the problem of relative camera pose estimation and demonstrate that the model outperforms existing dense approaches.

ActiveStereoNet: End-to-End Self-Supervised Learning for Active Stereo Systems

Abstract: In this paper we present ActiveStereoNet, the first deep learning solution for active stereo systems. Due to the lack of ground truth, our method is fully self-supervised, yet it produces precise depth with a subpixel precision of $1/30th$ of a pixel; it does not suffer from the common over-smoothing issues; it preserves the edges; and it explicitly handles occlusions. We introduce a novel reconstruction loss that is more robust to noise and texture-less patches, and is invariant to illumination changes. The proposed loss is optimized using a window-based cost aggregation with an adaptive support weight scheme. This cost aggregation is edge-preserving and smooths the loss function, which is key to allow the network to reach compelling results. Finally we show how the task of predicting invalid regions, such as occlusions, can be trained end-to-end without ground-truth. This component is crucial to reduce blur and particularly improves predictions along depth discontinuities. Extensive quantitatively and qualitatively evaluations on real and synthetic data demonstrate state of the art results in many challenging scenes.

Object-Based Superresolution Land-Cover Mapping From Remotely Sensed Imagery

Abstract: Superresolution mapping (SRM) is a widely used technique to address the mixed pixel problem in pixel-based classification. Advanced object-based classification will face a similar mixed phenomenon—a mixed object that contains different land-cover classes. Currently, most SRM approaches focus on estimating the spatial location of classes within mixed pixels in pixel-based classification. Little if any consideration has been given to predicting where classes spatially distribute within mixed objects. This paper, therefore, proposes a new object-based SRM strategy (OSRM) to deal with mixed objects in object-based classification. First, it uses the deconvolution technique to estimate the semivariograms at target subpixel scale from the class proportions of irregular objects. Then, an area-to-point kriging method is applied to predict the soft class values of subpixels within each object according to the estimated semivariograms and the class proportions of objects. Finally, a linear optimization model at object level is built to determine the optimal class labels of subpixels within each object. Two synthetic images and a real remote sensing image were used to evaluate the performance of OSRM. The experimental results demonstrated that OSRM generated more land-cover details within mixed objects than did the traditional object-based hard classification and performed better than an existing pixel-based SRM method. Hence, OSRM provides a valuable solution to mixed objects in object-based classification.

Improving Hyperspectral Subpixel Target Detection Using Hybrid Detection Space

Abstract: Abstract. A hyperspectral image (HSI) has high-spectral and low-spatial resolution. As a result, most targets exist as subpixels, which pose challenges during target detection. Moreover, limitations of target and background samples always hinder the detection performance. In this study, a hybrid method for subpixel target detection of an HSI is developed. The scores of matched filter (MF) and adaptive cosine estimator (ACE) are used to construct a hybrid detection space. The reference target spectrum and background covariance matrix are improved iteratively based on the distribution property of targets, using the hybrid detection space. As the iterative process proceeds, the reference target spectra get closer to the central line, which connects the centers of the target and the background, resulting in a noticeable improvement in target detection. One synthetic dataset and two real datasets are used in the experiments. The results are evaluated based on the mean detection rate (DR), receiver operating characteristic curve, and observations of the detection results. For the synthetic experiment, the hybrid method improves more than 10 times with regard to the average DR compared with that of the traditional MF and ACE algorithms, which use N-FINDR target extraction and Reed–Xiaoli detector for background estimation.

Autostereoscopic 3D display using directional subpixel rendering

Abstract: In this paper we present an autostereoscopic 3D display using a directional subpixel rendering algorithm in which clear left-right images are expressed in real time based on a viewer's 3D eye positions. In order to maintain the 3D image quality over a wide viewing range, we designed an optical layer that generates a uniformly distributed light field. The proposed 3D rendering method is simple, and each pixel processing can be performed independently in parallel computing environments. To prove the effectiveness of our display system, we implemented 31.5" 3D monitor and 10.1" 3D tablet prototypes in which the 3D rendering is processed in the GPU and FPGA board, respectively.

Real-Time Stereo Vision for Road Surface 3-D Reconstruction

Abstract: Stereo vision techniques have been widely used in civil engineering to acquire 3-D road data. The two important factors of stereo vision are accuracy and speed. However, it is very challenging to achieve both of them simultaneously and therefore the main aim of developing a stereo vision system is to improve the trade-off between these two factors. In this paper, we present a real-time stereo vision system used for road surface 3- D reconstruction. The proposed system is developed from our previously published 3-D reconstruction algorithm where the perspective view of the target image is first transformed into the reference view, which not only increases the disparity accuracy but also improves the processing speed. Then, the correlation cost between each pair of blocks is computed and stored in two 3-D cost volumes. To adaptively aggregate the matching costs from neighbourhood systems, bilateral filtering is performed on the cost volumes. This greatly reduces the ambiguities during stereo matching and further improves the precision of the estimated disparities. Finally, the subpixel resolution is achieved by conducting a parabola interpolation and the subpixel disparity map is used to reconstruct the 3-D road surface. The proposed algorithm is implemented on an NVIDIA GTX 1080 GPU for the real-time purpose. The experimental results illustrate that the reconstruction accuracy is around 3 mm.

A Subpixel Registration Algorithm for Low PSNR Images

Abstract: This paper presents a fast algorithm for obtaining high-accuracy subpixel translation of low PSNR images Instead of locating the maximum point on the upsampled images or fitting the peak of correlation surface, the proposed algorithm is based on the measurement of centroid on the cross correlation surface by Modified Moment method Synthetic images, real solar images and standard testing images with white Gaussian noise added were tested, and the results show that the accuracies of our algorithm are comparable with other subpixel registration techniques and the processing speed is higher The drawback is also discussed at the end of this paper

Subpixellic Methods for Sidelobes Suppression and Strong Targets Extraction in Single Look Complex SAR Images

Abstract: Synthetic aperture radar (SAR) images display very high dynamic ranges. Man-made structures (like buildings or power towers) produce echoes that are several orders of magnitude stronger than echoes from diffusing areas (vegetated areas) or from smooth surfaces (e.g., roads). The impulse response of the SAR imaging system is, thus, clearly visible around the strongest targets: sidelobes spread over several pixels, masking the much weaker echoes from the background. To reduce the sidelobes of the impulse response, images are generally spectrally apodized, trading resolution for a reduction of the sidelobes. This apodization procedure (global or shift-variant) introduces spatial correlations in the speckle-dominated areas that complicates the design of estimation methods. This paper describes strategies to cancel sidelobes around point-like targets while preserving the spatial resolution and the statistics of speckle-dominated areas. An irregular sampling grid is built to compensate the subpixel shifts and turn cardinal sines into discrete Diracs. A statistically grounded approach for point-like target extraction is also introduced, thereby providing a decomposition of a single look complex image into two components: a speckle-dominated image and the point-like targets. This decomposition can be exploited to produce images with improved quality (full resolution and suppressed sidelobes) suitable both for visual inspection and further processing (multitemporal analysis, despeckling, interferometry).

A New Spectral-Spatial Sub-Pixel Mapping Model for Remotely Sensed Hyperspectral Imagery

Abstract: In this paper, a new joint spectral–spatial subpixel mapping model is proposed for hyperspectral remotely sensed imagery. Conventional approaches generally use an intermediate step based on the derivation of fractional abundance maps obtained after a spectral unmixing process, and thus the rich spectral information contained in the original hyperspectral data set may not be utilized fully. In this paper, a concept of subpixel abundance map, which calculates the abundance fraction of each subpixel to belong to a given class, was introduced. This allows us to directly connect the original (coarser) hyperspectral image with the final subpixel result. Furthermore, the proposed approach incorporates the spectral information contained in the original hyperspectral imagery and the concept of spatial dependence to generate a final subpixel mapping result. The proposed approach has been experimentally evaluated using both synthetic and real hyperspectral images, and the obtained results demonstrate that the method achieves better results when compared to other seven subpixel mapping methods. The numerical comparisons are based on different indexes such as the overall accuracy and the CPU time. Moreover, the obtained results are statistically significant at 95% confidence.

Subpixel Surface Water Extraction (SSWE) Using Landsat 8 OLI Data

Abstract: Surface water extraction from remote sensing imagery has been a very active research topic in recent years, as this problem is essential for monitoring the environment, ecosystems, climate, and so on. In order to extract surface water accurately, we developed a new subpixel surface water extraction (SSWE) method, which includes three steps. Firstly, a new all bands water index (ABWI) was developed for pure water pixel extraction. Secondly, the mixed water–land pixels were extracted by a morphological dilation operation. Thirdly, the water fractions within the mixed water–land pixels were estimated by local multiple endmember spectral mixture analysis (MESMA). The proposed ABWI and SSWE have been evaluated by using three data sets collected by the Landsat 8 Operational Land Imager (OLI). Results show that the accuracy of ABWI is higher than that of the normalized difference water index (NDWI). According to the obtained surface water maps, the proposed SSWE shows better performance than the automated subpixel water mapping method (ASWM). Specifically, the root-mean-square error (RMSE) obtained by our SSWE for the data sets considered in experiments is 0.117, which is better than that obtained by ASWM (0.143). In conclusion, the SSWE can be used to extract surface water with high accuracy, especially in areas with optically complex aquatic environments.

Deep Generative Matching Network for Optical and SAR Image Registration

Abstract: Multimodal remote sensing images contain complementary information, thus, could potentially benefit many remote sensing applications. To this end, the image registration is a common requirement for utilizing the multimodal images. However, due to the rather different imaging mechanisms, multimodal image registration becomes much more challenging than ordinary registration, particular for optical and synthetic aperture radar (SAR) images. In this work, we design a deep matching network to exploit the latent and coherent features between multimodal patch pairs for inferring their matching labels. But, the network requires immense data for training, which is not usually met. To address this issue, we propose a generative matching network (GMN) to generate the coupled optical and SAR images, hence, improve the quantity and diversity of the training data. The experimental results show that our proposal significantly improves the registration performance of optical and SAR image registration, and achieves subpixel or close to subpixel error.

Heterogeneous parallel computing accelerated iterative subpixel digital image correlation

Abstract: Parallel computing techniques have been introduced into digital image correlation (DIC) in recent years and leads to a surge in computation speed. The graphics processing unit (GPU)-based parallel computing demonstrated a surprising effect on accelerating the iterative subpixel DIC, compared with CPU-based parallel computing. In this paper, the performances of the two kinds of parallel computing techniques are compared for the previously proposed path-independent DIC method, in which the initial guess for the inverse compositional Gauss-Newton (IC-GN) algorithm at each point of interest (POI) is estimated through the fast Fourier transform-based cross-correlation (FFT-CC) algorithm. Based on the performance evaluation, a heterogeneous parallel computing (HPC) model is proposed with hybrid mode of parallelisms in order to combine the computing power of GPU and multicore CPU. A scheme of trial computation test is developed to optimize the configuration of the HPC model on a specific computer. The proposed HPC model shows excellent performance on a middle-end desktop computer for real-time subpixel DIC with high resolution of more than 10000 POIs per frame.

Multiobjective Subpixel Land-Cover Mapping

Abstract: The hyperspectral subpixel mapping (SPM) technique can generate a land-cover map at the subpixel scale by modeling the relationship between the abundance map and the spatial distribution image of the subpixels. However, this is an inverse ill-posed problem. The most widely used way to resolve the problem is to introduce additional information as a regularization term and acquire the unique optimal solution. However, the regularization parameter either needs to be determined manually or it cannot be determined in a fully adaptive manner. Thus, in this paper, the multiobjective subpixel land-cover mapping (MOSM) framework for hyperspectral remote sensing imagery is proposed, in which the two function terms [the fidelity term and the prior term (i.e., the regularization term)] can be optimized simultaneously, and there is no need to determine the regularization parameter explicitly. In order to achieve this goal, two strategies are designed in MOSM: 1) a high-resolution distribution image-based individual encoding strategy is designed in order to calculate the prior term accurately and 2) a subfitness-based individual comparison strategy is designed in order to generate subpixel land-cover mapping solutions with a high quality to update the population. Four data sets (one simulated, two synthetic, and one real hyperspectral image) were used to test the proposed method. The experimental results show that MOSM can perform better than the other subpixel land-cover mapping methods, demonstrating the effectiveness of MOSM in balancing the fidelity term and prior term in the SPM model.

Subpixel-Based Precipitation Nowcasting with the Pyramid Lucas–Kanade Optical Flow Technique

Abstract: Short-term high-resolution quantitative precipitation forecasting (QPF) is very important for flash-flood warning, navigation safety, and other hydrological applications. This paper proposes a subpixel-based QPF algorithm using a pyramid Lucas–Kanade optical flow technique (SPLK) for short-time rainfall forecast. The SPLK tracks the storm on the subpixel level by using the optical flow technique and then extrapolates the precipitation using a linear method through redistribution and interpolation. The SPLK compares with object-based and pixel-based nowcasting algorithms using eight thunderstorm events to assess its performance. The results suggest that the SPLK can perform better nowcasting of precipitation than the object-based and pixel-based algorithms with higher adequacy in tracking and predicting severe storms in 0–2 h lead-time forecasting.

Display substrate and display device

Abstract: The invention discloses a display substrate and a display device to improve the screen-to-body ratio of the display device and to achieve a better full-screen display effect. A display area of the display substrate comprises a light-transmitting display part, wherein the light-transmitting display part comprises a plurality of subpixels arranged in an array; and each row of subpixels include a first color subpixel, a second color subpixel, a third color subpixel and a transparent subpixel arranged in cycle. The display substrate disclosed by the embodiment of the invention can be applied to afull-screen display device; lighting devices, such as a camera and a light sensor are arranged at the back side of the display substrate and are opposite to the position of the light-transmitting display part of the display substrate; the light-transmitting display part and the other parts of a screen can carrying out normal display; and light can also be projected on the lighting devices in a transmitting manner, so that, compared with the prior art, the display substrate has the advantage that the screen-to-body ratio is greatly improved, thereby making the full-screen display effect more excellent.

Subpixel Land Cover Mapping Using Multiscale Spatial Dependence

Abstract: This paper proposes a new subpixel mapping (SPM) method based on multiscale spatial dependence (MSD). At the beginning, it adopts object-based and pixel-based soft classifications to generate the class proportions within each object and each pixel, respectively. Then, the object-scale spatial dependence of land cover classes is extracted from the class proportions of objects, and the combined spatial dependence at both pixel scale and subpixel scale is obtained from the class proportions of pixels. Furthermore, these spatial dependences are fused as the MSD for each subpixel. Last, a linear optimization model on each object is built to determine where the land cover classes spatially distribute within each mixed object at subpixel scales. Three experiments on two synthetic images and a real remote sensing image are carried out to evaluate the effectiveness of MSD. The experimental results show that MSD performed better than four existing SPM methods by generating less isolated classified pixels than those generated by three pixel-based SPM methods and more land cover local details than that generated by an object-based SPM method. Hence, MSD provides a valuable solution to producing land cover maps at subpixel scales.

Subpixel three-dimensional laser imaging with a downscaled avalanche photodiode array using code division multiple access

Abstract: Scannerless time-of-flight three-dimensional imaging, the successor to raster scanning three-dimensional imaging, relies on large-scale detector arrays to obtain high pixel resolution; however, manufacturing limitations lead to a bottleneck in imaging resolution. Here, we report a methodology for laser imaging using code division multiple access, which involves three key steps. Optical encoding is carried out for a pulsed laser to generate space-time encoded beams for projection. Optical multiplexing subdivides the backscattered light signals and multiplexes them to a downscaled avalanche-photodiode array. Subpixel decoding decodes the digitized encoded full waveforms and decomposes the features of all subpixels. Accordingly, we design a prototype based on a 64-order encoder and a 4-element avalanche-photodiode array and conduct an outdoor experiment. We demonstrate that the system is capable of obtaining 256 pixels per frame in push-broom mode and reconstruct a three-dimensional image with centimetre-level lateral resolution and range precision at a distance of ∼112 m. Scannerless time of flight three-dimensional devices can produce high-quality images from the ground or in space and provide information on light detection and ranging. The authors design and demonstrate a downscaled subpixel 3D laser imaging device which uses pulse-encoded illumination to encode the pixels.

Scientific image rendering for space scenes with the SurRender software.

Abstract: Spacecraft autonomy can be enhanced by vision-based navigation (VBN) techniques. Applications range from manoeuvers around Solar System objects and landing on planetary surfaces, to in-orbit servicing or space debris removal. The development and validation of VBN algorithms relies on the availability of physically accurate relevant images. Yet archival data from past missions can rarely serve this purpose and acquiring new data is often costly. The SurRender software is an image simulator that addresses the challenges of realistic image rendering, with high representativeness for space scenes. Images are rendered by raytracing, which implements the physical principles of geometrical light propagation, in physical units. A macroscopic instrument model and scene objects reflectance functions are used. SurRender is specially optimized for space scenes, with huge distances between objects and scenes up to Solar System size. Raytracing conveniently tackles some important effects for VBN algorithms: image quality, eclipses, secondary illumination, subpixel limb imaging, etc. A simulation is easily setup (in MATLAB, Python, and more) by specifying the position of the bodies (camera, Sun, planets, satellites) over time, 3D shapes and material surface properties. SurRender comes with its own modelling tool enabling to go beyond existing models for shapes, materials and sensors (projection, temporal sampling, electronics, etc.). It is natively designed to simulate different kinds of sensors (visible, LIDAR, etc.). Tools are available for manipulating huge datasets to store albedo maps and digital elevation models, or for procedural (fractal) texturing that generates high-quality images for a large range of observing distances (from millions of km to touchdown). We illustrate SurRender performances with a selection of case studies, placing particular emphasis on a 900-km Moon flyby simulation.

Back to Basics in Ultrasound Velocimetry: Tracking Speckles by Using a Standard Piv Algorithm

Abstract: In this IUS proceeding, we describe a classical block-matching approach that we used during the 2018 SA-VFI (Synthetic Aperture 2-D Vector Flow imaging) challenge. To estimate frame-to-frame displacements, we used blockwise FFT-based ensemble cross-correlations. Subpixel displacements were obtained by parabolic peak fitting. We opted for a coarse-to-fine multiscale scheme to increase the resolution and precision. Robustness and accuracy were improved by including a robust unsupervised smoother in the estimation process. 2-D velocity vector fields were computed in several flow phantoms (from both simulations and experiments) provided by the organizers of the SAV-FI challenge. Our results showed that the standard block-matching approach provided reliable 2-D velocity vector fields, in terms of magnitude and angle. Complex flow patterns, like those occurring in the carotid bifurcation, were also estimated accurately. In summary, the long-standing block-matching technique by normalized cross-correlation is effective for flow estimation by ultrasound imaging. The final estimates returned by the method will be uploaded on the SA-VFI platform, and the results will be compared with those obtained by all the competitors during the challenge session at IEEE IUS 2018 in Kobe (Japan).

Real-Time Stereo Vision for Road Surface 3-D Reconstruction.

Abstract: Stereo vision techniques have been widely used in civil engineering to acquire 3-D road data. The two important factors of stereo vision are accuracy and speed. However, it is very challenging to achieve both of them simultaneously and therefore the main aim of developing a stereo vision system is to improve the trade-off between these two factors. In this paper, we present a real-time stereo vision system used for road surface 3-D reconstruction. The proposed system is developed from our previously published 3-D reconstruction algorithm where the perspective view of the target image is first transformed into the reference view, which not only increases the disparity accuracy but also improves the processing speed. Then, the correlation cost between each pair of blocks is computed and stored in two 3-D cost volumes. To adaptively aggregate the matching costs from neighbourhood systems, bilateral filtering is performed on the cost volumes. This greatly reduces the ambiguities during stereo matching and further improves the precision of the estimated disparities. Finally, the subpixel resolution is achieved by conducting a parabola interpolation and the subpixel disparity map is used to reconstruct the 3-D road surface. The proposed algorithm is implemented on an NVIDIA GTX 1080 GPU for the real-time purpose. The experimental results illustrate that the reconstruction accuracy is around 3 mm.

Towards breaking the spatial resolution barriers: An optical flow and super-resolution approach for sea ice motion estimation

Abstract: Estimation of sea ice motion at fine scales is important for a number of regional and local level applications, including modeling of sea ice distribution, ocean-atmosphere and climate dynamics, as well as safe navigation and sea operations. In this study, we propose an optical flow and super-resolution approach to accurately estimate motion from remote sensing images at a higher spatial resolution than the original data. First, an external example learning-based super-resolution method is applied on the original images to generate higher resolution versions. Then, an optical flow approach is applied on the higher resolution images, identifying sparse correspondences and interpolating them to extract a dense motion vector field with continuous values and subpixel accuracies. Our proposed approach is successfully evaluated on passive microwave, optical, and Synthetic Aperture Radar data, proving appropriate for multi-sensor applications and different spatial resolutions. The approach estimates motion with similar or higher accuracy than the original data, while increasing the spatial resolution of up to eight times. In addition, the adopted optical flow component outperforms a state-of-the-art pattern matching method. Overall, the proposed approach results in accurate motion vectors with unprecedented spatial resolutions of up to 1.5 km for passive microwave data covering the entire Arctic and 20 m for radar data, and proves promising for numerous scientific and operational applications.