scispace - formally typeset
Search or ask a question

Showing papers on "Upsampling published in 2014"


Journal ArticleDOI
TL;DR: This paper describes an application framework to perform high-quality upsampling and completion on noisy depth maps by combining the additional high-resolution RGB input when upsampled a low-resolution depth map together with a weighting scheme that favors structure details.
Abstract: This paper describes an application framework to perform high-quality upsampling and completion on noisy depth maps. Our framework targets a complementary system setup, which consists of a depth camera coupled with an RGB camera. Inspired by a recent work that uses a nonlocal structure regularization, we regularize depth maps in order to maintain fine details and structures. We extend this regularization by combining the additional high-resolution RGB input when upsampling a low-resolution depth map together with a weighting scheme that favors structure details. Our technique is also able to repair large holes in a depth map with consideration of structures and discontinuities utilizing edge information from the RGB input. Quantitative and qualitative results show that our method outperforms existing approaches for depth map upsampling and completion. We describe the complete process for this system, including device calibration, scene warping for input alignment, and even how our framework can be extended for video depth-map completion with the consideration of temporal coherence.

115 citations


Book ChapterDOI
03 Nov 2014
TL;DR: This work presents a computationally efficient architecture for image super-resolution that achieves state-of-the-art results on images with large spatial extend and empirically shows that upsampling methods work much better on latent representations than in the original spatial domain.
Abstract: We present a computationally efficient architecture for image super-resolution that achieves state-of-the-art results on images with large spatial extend. Apart from utilizing Convolutional Neural Networks, our approach leverages recent advances in fast approximate inference for sparse coding. We empirically show that upsampling methods work much better on latent representations than in the original spatial domain. Our experiments indicate that the proposed architecture can serve as a basis for additional future improvements in image super-resolution.

77 citations


Journal ArticleDOI
TL;DR: A fast image upsampling method within a two-scale framework to ensure the sharp construction of upsampled image for both large-scale edges and small-scale structures that outperforms current state-of-the-art approaches based on quantitative and qualitative evaluations, as well as perceptual evaluation by a user study.
Abstract: In this paper, we present a fast image upsampling method within a two-scale framework to ensure the sharp construction of upsampled image for both large-scale edges and small-scale structures. In our approach, the low-frequency image is recovered via a novel sharpness preserving interpolation technique based on a well-constructed displacement field, which is estimated by a cross-resolution sharpness preserving model. Within this model, the distances of pixels on edges are preserved, which enables the recovery of sharp edges in the high-resolution result. Likewise, local high-frequency structures are reconstructed via a sharpness preserving reconstruction algorithm. Extensive experiments show that our method outperforms current state-of-the-art approaches, based on quantitative and qualitative evaluations, as well as perceptual evaluation by a user study. Moreover, our approach is very fast so as to be practical for real applications.

62 citations


Journal ArticleDOI
TL;DR: A new non-local means feature-based technique that uses structural information of a high resolution (HR) image with a different contrast and interpolates the low resolution (LR) image, resulting in a more accurate similarity measure in comparison with conventional patch-based approach.
Abstract: In magnetic resonance imaging (MRI), spatial resolution is limited by several factors such as acquisition time, short physiological phenomena, and organ motion. The acquired image usually has higher resolution in two dimensions (the acquisition plane) in comparison with the third dimension, resulting in highly anisotropic voxel size. Interpolation of these low resolution (LR) images using standard techniques, such as linear or spline interpolation, results in distorted edges in the planes perpendicular to the acquisition plane. This poses limitation on conducting quantitative analyses of LR images, particularly on their voxel-wise analysis and registration. We have proposed a new non-local means feature-based technique that uses structural information of a high resolution (HR) image with a different contrast and interpolates the LR image. In this approach, the similarity between voxels is estimated using a feature vector that characterizes the laminar pattern of the brain structures, resulting in a more accurate similarity measure in comparison with conventional patch-based approach. This technique can be applied to LR images with both anisotropic and isotropic voxel sizes. Experimental results conducted on brain MRI scans of patients with brain tumors, multiple sclerosis, epilepsy, as well as schizophrenic patients and normal controls show that the proposed method is more accurate, requires fewer computations, and thus is significantly faster than a previous state-of-the-art patch-based technique. We also show how the proposed method may be used to upsample regions of interest drawn on LR images.

54 citations


Journal Article
TL;DR: The key reason why the designed multiresolution imaging camera can provide us with real images of different resolutions is that it builds a solid foundation for evaluating various algorithms and analyzing the images with different resolutions, which is very important for vision.
Abstract: Imaging resolution has been standing as a core parameter in various applications of vision. Mostly, high resolutions are desirable or essential for many applications, e.g., in most remote sensing systems, and therefore much has been done to achieve a higher resolution of an image based on one or a series of images of relatively lower resolutions. On the other hand, lower resolutions are also preferred in some cases, e.g., for displaying images in a very small screen or interface. Accordingly, algorithms for image upsampling or downsampling have also been proposed. In the above algorithms, the downsampled or upsampled (super-resolution) versions of the original image are often taken as test images to evaluate the performance of the algorithms. However, there is one important question left unanswered: whether the downsampled or upsampled versions of the original image can represent the low-resolution or high-resolution real images from a camera? To tackle this point, the following works are carried out: 1) a multiresolution camera is designed to simultaneously capture images in three different resolutions; 2) at a given resolution (i.e., image size), the relationship between a pair of images is studied, one gained via either downsampling or super-resolution, and the other is directly captured at this given resolution by an imaging device; and 3) the performance of the algorithms of super-resolution and image downsampling is evaluated by using the given image pairs. The key reason why we can effectively tackle the aforementioned issues is that the designed multiresolution imaging camera can provide us with real images of different resolutions, which builds a solid foundation for evaluating various algorithms and analyzing the images with different resolutions, which is very important for vision.

50 citations


Patent
14 Jul 2014
TL;DR: In this article, a method of processing a depth image includes receiving a high-resolution color image and a low-resolution depth image corresponding to the high resolution color image, generating a feature vector based on a depth distribution of the low resolution depth image, and selecting a filter to upsample the low level depth image by classifying a generated feature vector according to a previously learnt classifier.
Abstract: A method of processing a depth image includes receiving a high-resolution color image and a low-resolution depth image corresponding to the high-resolution color image, generating a feature vector based on a depth distribution of the low-resolution depth image, selecting a filter to upsample the low-resolution depth image by classifying a generated feature vector according to a previously learnt classifier, upsampling the low-resolution depth image by using a selected filter, and outputting an upsampled high-resolution depth image.

48 citations


Journal ArticleDOI
TL;DR: Two pan-sharpening methods based on the non-subsampled contourlet transform (NSCT) are proposed that preserves both spectral and spatial qualities while decreasing computation time and results demonstrate the efficiency of the proposed methods.
Abstract: Two pan-sharpening methods based on the non-subsampled contourlet transform (NSCT) are proposed. NSCT is very efficient in representing the directional information and capturing intrinsic geometrical structures of the objects. It has characteristics of high resolution, shift-invariance, and high directionality. In the proposed methods, a given number of decomposition levels are used for multispectral (MS) images while a higher number of decomposition levels are used for Pan images relatively to the ratio of the Pan pixel size to the MS pixel size. This preserves both spectral and spatial qualities while decreasing computation time. Moreover, upsampling of MS images is performed after NSCT and not before. By applying upsampling after NSCT, structures and detail information of the MS images are more likely to be preserved and thus stay more distinguishable. Hence, we propose to exploit this property in pan-sharpening by fusing it with detail information provided by the Pan image at the same fine level. The proposed methods are tested on WorldView-2 datasets and compared with the standard pan-sharpening technique. Visual and quantitative results demonstrate the efficiency of the proposed methods. Both spectral and spatial qualities have been improved.

45 citations


Proceedings ArticleDOI
04 May 2014
TL;DR: Experimental results show that the proposed depth enhancement and up-sampling techniques produce slightly more accurate depth at the full resolution with improved rendering quality of intermediate views.
Abstract: Depth images are often presented at a lower spatial resolution, either due to limitations in the acquisition of the depth or to increase compression efficiency. As a result, upsampling low-resolution depth images to a higher spatial resolution is typically required prior to depth image based rendering. In this paper, depth enhancement and up-sampling techniques are proposed using a graph-based formulation. In one scheme, the depth is first upsampled using a conventional method, then followed by a graph-based joint bilateral filtering to enhance edges and reduce noise. A second scheme avoids the two-step processing and upsamples the depth directly using the proposed graph-based joint bilateral upsampling. Both filtering and interpolation problems are formulated as regularization problems and the solutions are different from conventional approaches. Further, we also studied operations on different graph structures such as star graph and 8-connected graph. Experimental results show that the proposed methods produce slightly more accurate depth at the full resolution with improved rendering quality of intermediate views.

42 citations


Proceedings ArticleDOI
01 Oct 2014
TL;DR: This paper proposes a road detection approach based solely on dense 3D-LIDAR data that is suitable for applications on rural areas and inner-city with unmarked roads, and obtaining promising results when compared to other state-of-art approaches.
Abstract: This paper proposes a road detection approach based solely on dense 3D-LIDAR data. The approach is built up of four stages: (1) 3D-LIDAR points are projected to a 2D reference plane; then, (2) dense height maps are computed using an upsampling method; (3) applying a sliding-window technique in the upsampled maps, probability distributions of neighbouring regions are compared according to a similarity measure; finally, (4) morphological operations are used to enhance performance against disturbances. Our detection approach does not depend on road marks, thus it is suitable for applications on rural areas and inner-city with unmarked roads. Experiments have been carried out in a wide variety of scenarios using the recent KITTI-ROAD benchmark, obtaining promising results when compared to other state-of-art approaches.

39 citations


Proceedings ArticleDOI
23 Jun 2014
TL;DR: This paper presents a novel guided image filtering method using multipoint local polynomial approximation (LPA) with range guidance and develops a scheme with constant computational complexity for generating a spatial adaptive support region around a point.
Abstract: This paper presents a novel guided image filtering method using multipoint local polynomial approximation (LPA) with range guidance. In our method, the LPA is extended from a pointwise model into a multipoint model for reliable filtering and better preserving image spatial variation which usually contains the essential information in the input image. In addition, we develop a scheme with constant computational complexity (invariant to the size of filtering kernel) for generating a spatial adaptive support region around a point. By using the hybrid of the local polynomial model and color/intensity based range guidance, the proposed method not only preserves edges but also does a much better job in preserving spatial variation than existing popular filtering methods. Our method proves to be effective in a number of applications: depth image upsampling, joint image denoising, details enhancement, and image abstraction. Experimental results show that our method produces better results than state-of-the-art methods and it is also computationally efficient.

38 citations


Journal ArticleDOI
Jie Dong1, Yan Ye1
TL;DR: Simulations show this algorithm improves the R-D performance over a wide range of bit rates, and given the optimal sampling ratio, dedicated filters for down- and upsampling are also designed.
Abstract: Previous research has shown that downsampling prior to encoding and upsampling after decoding can improve the rate-distortion (R-D) performance compared with directly coding the original video using standard technologies, e.g., JPEG and H.264/AVC, especially at low bit rates. This paper proposes a practical algorithm to find the optimal downsampling ratio that balances the distortions caused by downsampling and coding, thus achieving the overall optimal R-D performance. Given the optimal sampling ratio, dedicated filters for down- and upsampling are also designed. Simulations show this algorithm improves the R-D performance over a wide range of bit rates.

Journal ArticleDOI
TL;DR: A method for increasing spatial resolution of a depth map using its corresponding high-resolution (HR) color image as a guide that improves depth map upsampling both quantitatively and qualitatively and can be extended to handle real data with occluded regions caused by the displacement between color and depth sensors.
Abstract: This paper presents a method for increasing spatial resolution of a depth map using its corresponding high-resolution (HR) color image as a guide Most of the previous methods rely on the assumption that depth discontinuities are highly correlated with color boundaries, leading to artifacts in the regions where the assumption is broken To prevent scene texture from being erroneously transferred to reconstructed scene surfaces, we propose a framework for dividing the color image into different regions and applying different methods tailored to each region type For the region classification, we first segment the low-resolution (LR) depth map into regions of smooth surfaces, and then use them to guide the segmentation of the color image Using the consensus of multiple image segmentations obtained by different super-pixel generation methods, the color image is divided into continuous and discontinuous regions: in the continuous regions, their HR depth values are interpolated from LR depth samples without exploiting the color information In the discontinuous regions, their HR depth values are estimated by sequentially applying more complicated depth-histogram-based methods Through experiments, we show that each step of our method improves depth map upsampling both quantitatively and qualitatively We also show that our method can be extended to handle real data with occluded regions caused by the displacement between color and depth sensors

Journal ArticleDOI
TL;DR: The likelihood for each signal model is developed based on the entire data set and used in an information theoretic framework to achieve reliable order estimation performance for dependent samples.
Abstract: Estimation of the dimension of the signal subspace, or order detection, is one of the key issues in many signal processing problems. Information theoretic criteria are widely used to estimate the order under the independently and identically distributed (i.i.d.) sampling assumption. However, in many applications, the i.i.d. sampling assumption does not hold. Previous approaches address the dependent sample issue by downsampling the data set so that existing order detection methods can be used. By discarding data, the sample size is decreased causing degradation in the accuracy of the order estimation. In this paper, we introduce two likelihood estimators for dependent samples based on two signal models. The likelihood for each signal model is developed based on the entire data set and used in an information theoretic framework to achieve reliable order estimation performance for dependent samples. Experimental results show the desirable performance of the new method.

Journal ArticleDOI
TL;DR: This paper study how the detectability of embedding changes is affected when the cover image is downsampled prior to embedding, and analytically compute the Fisher information rate for any mutually independent embedding operation and derive the proper scaling of the secure payload with resizing.
Abstract: The accuracy of steganalysis in digital images primarily depends on the statistical properties of neighboring pixels, which are strongly affected by the image acquisition pipeline as well as any processing applied to the image. In this paper, we study how the detectability of embedding changes is affected when the cover image is downsampled prior to embedding. This topic is important for practitioners because the vast majority of images posted on websites, image sharing portals, or attached to e-mails are downsampled. It is also relevant to researchers as the security of steganographic algorithms is commonly evaluated on databases of downsampled images. In the first part of this paper, we investigate empirically how the steganalysis results depend on the parameters of the resizing algorithm-the choice of the interpolation kernel, the scaling factor (resize ratio), antialiasing, and the downsampled pixel grid alignment. We report on several novel phenomena that appear valid universally across the tested cover sources, steganographic methods, and steganalysis features. This paper continues with a theoretical analysis of the simplest interpolation kernel - the box kernel. By fitting a Markov chain model to pixel rows, we analytically compute the Fisher information rate for any mutually independent embedding operation and derive the proper scaling of the secure payload with resizing. For least significant bit (LSB) matching and a limited range of downscaling, the theory fits experiments rather well, which indicates the existence of a new scaling law expressing the length of the secure payload when the cover size is modified by subsampling.

Journal ArticleDOI
TL;DR: A computational approach to generate realistic DoF effects for mobile devices such as tablets by calibrating the rear-facing stereo cameras and rectifying the stereo image pairs through FCam API, and generating a low-res disparity map using graph cuts stereo matching and subsequently upsample it via joint bilateral upsampling.
Abstract: The depth of field (DoF) effect is a useful tool in photography and cinematography because of its aesthetic value. However, capturing and displaying dynamic DoF effect were until recently a quality unique to expensive and bulky movie cameras. A computational approach to generate realistic DoF effects for mobile devices such as tablets is proposed. We first calibrate the rear-facing stereo cameras and rectify the stereo image pairs through FCam API, then generate a low-res disparity map using graph cuts stereo matching and subsequently upsample it via joint bilateral upsampling. Next, we generate a synthetic light field by warping the raw color image to nearby viewpoints, according to the corresponding values in the upsampled high-resolution disparity map. Finally, we render dynamic DoF effect on the tablet screen with light field rendering. The user can easily capture and generate desired DoF effects with arbitrary aperture sizes or focal depths using the tablet only, with no additional hardware or software required. The system has been examined in a variety of environments with satisfactory results, according to the subjective evaluation tests.

Patent
07 Oct 2014
TL;DR: In this paper, a video coding system may perform inter-layer processing by simultaneously performing inverse tone mapping and color gamut conversion scalability processes on a base layer of a video signal.
Abstract: A video coding system may perform inter-layer processing by simultaneously performing inverse tone mapping and color gamut conversion scalability processes on a base layer of a video signal. The video coding system may then perform upsampling on the processed base layer. The processed base layer may be used to code an enhancement layer. Bit depth may be considered for color gamut conversion modules. Luma and/or chroma bit depths may be aligned with respective larger or smaller bit depth values of luma and/or chroma.

Patent
Kuang-Tsan Wu1, John D. McNicol1, David F. Welch1, Stephen G. Grubb1, Pierre Mertz1 
28 Aug 2014
TL;DR: In this paper, an apparatus including a photodiode, a low pass filter, an analog-to-digital converter, an interpolation circuit and a digital signal processor is disclosed.
Abstract: An apparatus including a photodiode, a low pass filter, an analog-to-digital converter, an interpolation circuit and a digital signal processor is disclosed. The photodiode receives a portion of a plurality of optical signals, each of which is modulated in accordance with a corresponding one of a plurality of data streams, and each having a corresponding one of a plurality of wavelengths. The photodiode supplies an electrical output. The low-pass filter supplies a filtered output in response to the electrical output. The analog-to-digital converter is configured to sample the filtered output at a first sampling rate to generate a plurality of first data samples. The interpolation circuit is configured to receive the plurality of first data samples and supply a plurality of second data samples at a second sampling rate less the first sampling rate. The digital signal processor circuit is configured to receive the plurality of second data samples.

Proceedings ArticleDOI
TL;DR: Intellectual property cores, containing switch controlled Finite Impulse Response (FIR) filters, are developed and applied to the Field Programmable Gate Array (FPGA) XC6SLX45 from Xilinx to enable the hardware design to work economically.
Abstract: A novel and low-cost embedded hardware architecture for real-time refocusing based on a standard plenoptic camera is presented in this study. The proposed layout design synthesizes refocusing slices directly from micro images by omitting the process for the commonly used sub-aperture extraction. Therefore, intellectual property cores, containing switch controlled Finite Impulse Response (FIR) filters, are developed and applied to the Field Programmable Gate Array (FPGA) XC6SLX45 from Xilinx. Enabling the hardware design to work economically, the FIR filters are composed of stored product as well as upsampling and interpolation techniques in order to achieve an ideal relation between image resolution, delay time, power consumption and the demand of logic gates. The video output is transmitted via High-Definition Multimedia Interface (HDMI) with a resolution of 720p at a frame rate of 60 fps conforming to the HD ready standard. Examples of the synthesized refocusing slices are presented.

Patent
25 Jul 2014
TL;DR: In this article, a method for image enhancement is described, which includes accessing two or more sets of first pixel data representative of two images of a physical object, and then combining these sets of pixel data to generate an enhanced image of the physical object.
Abstract: Embodiments of a method and apparatus for image enhancement are generally described herein. In some embodiments, the method includes accessing two or more sets of first pixel data representative of two or more images of a physical object. The method can further include transforming the two or more sets of pixel data to generate two or more sets of frequency domain data. The method can further include upsampling each of the two or more sets of frequency domain data to generate a set of upsampled frequency domain data. The method can further include re-transforming the set of upsampled frequency domain data to generate two or more sets of second pixel data. The method can further include combining two or more sets of second pixel data to generate an enhanced image of the physical object. Other example methods, systems, and apparatuses are described.

Proceedings ArticleDOI
Xiaojin Gong1, Jianqiang Ren1, Baisheng Lai1, Chaohua Yan1, Hui Qian1 
23 Jun 2014
TL;DR: A new approach to upsample depth maps when aligned high-resolution color images are given, which exploits the cosparsity of analytic analysis operators performed on a depth map, together with data fidelity and color guided smoothness constraints for upsampling.
Abstract: This paper proposes a new approach to upsample depth maps when aligned high-resolution color images are given. Such a task is referred to as guided depth upsampling in our work. We formulate this problem based on the recently developed sparse representation analysis models. More specifically, we exploit the cosparsity of analytic analysis operators performed on a depth map, together with data fidelity and color guided smoothness constraints for upsampling. The formulated problem is solved by the greedy analysis pursuit algorithm. Since our approach relies on the analytic operators such as the Wavelet transforms and the finite difference operators, it does not require any training data but a single depth-color image pair. A variety of experiments have been conducted on both synthetic and real data. Experimental results demonstrate that our approach outperforms the specialized state-of-the-art algorithms.

01 Jan 2014
TL;DR: The effectiveness of the proposed technique in overcoming traditional angular aliasing and corruption artifacts is validated with 3D ranging data acquired from internal and external surfaces of exhumed water pipes, and the resulting 2.5D maps can be more accurately and completely computed to higher resolutions.
Abstract: This paper presents a novel robust processing methodology for computing 2.5D thickness maps from dense 3D collocated surfaces. The proposed pipeline is suitable to faithfully adjust data representation detailing as required, from preserving fine surface features to coarse interpretations. The foundations of the proposed technique exploit spatial point-based filtering, ray tracing techniques and the Robust Implicit Moving Least Squares (RIMLS) algorithm applied to dense 3D datasets, such as those acquired from laser scanners. The effectiveness of the proposed technique in overcoming traditional angular aliasing and corruption artifacts is validated with 3D ranging data acquired from internal and external surfaces of exhumed water pipes. It is shown that the resulting 2.5D maps can be more accurately and completely computed to higher resolutions, while significantly reducing the number of raytracing errors when compared with 2.5D thickness maps derived from our current approach.

Patent
26 Feb 2014
TL;DR: In this paper, a video and image lossy compression method for image coding is proposed, which combines traditional JPEG, JPEG2000, H264 and HEVC-code standard algorithms with super-resolution image reconstruction.
Abstract: The invention discloses a video and image lossy compression method for image coding The method combines traditional JPEG, JPEG2000, H264 and HEVC-code standard algorithms with super-resolution image reconstruction and designs an image compression method combining the super-resolution reconstruction on the basis Downsampling is carried out on an input video and image, wherein a downsampling method adopts a Bicubic algorithm and a downsampling multiple is 2 The number of dot arrays of a downsampling image is only 1/4 of that of an original image The encoding rate of the downsampling image is far lower than that of an original input image so that the encoding rate is reduced At the same time, on the basis that robustness differences of a residual image and a general image are analyzed, a negative-feedback step is introduced in the design so that part of high-frequency detail information lost in a super-resolution image reconstruction step is remedied and reconstructed video or image quality is improved Compared with the JPEG and the H264 standard algorithms, the compression method reduces the encoding rate greatly under a situation that image quality is the same

Journal ArticleDOI
TL;DR: A new depth upsampler in which an upsampled depth map is computed at each pixel as the average of neighboring pixels, weighted by color and depth intensity filters is presented.
Abstract: In this paper, we present a new depth upsampler in which an upsampled depth map is computed at each pixel as the average of neighboring pixels, weighted by color and depth intensity filters. The proposed method features two parameters, an adaptive smoothing parameter and a control parameter. The adaptive smoothing parameter is determined based on the ratio between a depth map and its corresponding color image. The adaptive smoothing parameter is used to control the dynamic range of the color-range filter. The control parameter assigns a larger weighting factor to pixels in the object to which a missing pixel belongs. In a comparison with five existing upsamplers, the proposed method outperforms all five in terms of both objective and subjective quality.

Patent
19 Mar 2014
TL;DR: In this paper, an online fault detection method for a reduced set-based downsampling unbalance SVM (Support Vector Machine) transformer is proposed. But the method is not suitable for the case where the upsampling algorithm is selected improperly sometimes, and thus the poor classifying effect is caused.
Abstract: The invention relates to an online fault detection method for a reduced set-based downsampling unbalance SVM (Support Vector Machine) transformer. At present, the research of improving the performance of an unbalance data downsampling SVM algorithm comprises upsampling and downsampling. The SVM model calculating cost of the upsampling algorithm is increased. The downsampling algorithm is selected improperly sometimes, and thus the poor classifying effect is caused. The online fault detection method comprises the following steps: (1), acquiring a vibration signal of a transformer; (2), obtaining a noise reduction vibration signal; (3), obtaining multiple groups of fault detection feature data; (4), clustering by using a K-mean algorithm; (5), figuring out a weight value of each sample; (6), establishing a majority sample reduction vector solution optimization model; (7), obtaining an SVM fault diagnosis model; and (8), inputting a sample to be tested to an unbalance SVM detector trained in the step 7, analyzing a result output from the detector to obtain a working state of the transformer, and realizing online fault detection of the transformer. The online fault detection method is used for detecting the fault of the transformer online.

Journal ArticleDOI
TL;DR: An adaptive directional wavelet transform is constructed, which has shown improved image coding performance over these adaptiveirectional wavelet transforms and objective and subjective improvements when compared with the directionlets applied independently on each image segment.
Abstract: Directionlets allow a construction of perfect reconstruction and critically sampled multidirectional anisotropic basis, yet retaining the separable filtering of standard wavelet transform. However, due to the spatially varying filtering and downsampling direction, it is forced to apply spatial segmentation and process each segment independently. Because of this independent processing of the image segments, directionlets suffer from the following two major limitations when applied to, say, image coding. First, failure to exploit the correlation across block boundaries degrades the coding performance and also induces blocking artifacts, thus making it mandatory to use de-blocking filter at low bit rates. Second, spatial scalability, i.e., minimum segment size or the number of levels of the transform, is limited due to independent processing of segments. We show that, with simple modifications in the block boundaries, we can overcome these limitations by, what we call, in-phase lifting implementation of directionlets. In the context of directionlets using in-phase lifting, we identify different possible groups of downsampling matrices that would allow the construction of a multilevel transform without forcing independent processing of segments both with and without any modifications in the segment boundary. Experimental results in image coding show objective and subjective improvements when compared with the directionlets applied independently on each image segment. As an application, using both the in-phase lifting implementation of directionlets and the adaptive directional lifting, we have constructed an adaptive directional wavelet transform, which has shown improved image coding performance over these adaptive directional wavelet transforms.

Patent
10 Apr 2014
TL;DR: In this article, a sampling filter process for scalable video coding is provided for re-sampling using video data obtained from an encoder or decoder process of a base layer (BL) in a multi-layer system using adaptive phase shifting to improve quality in Scalable High efficiency Video Coding (SHVC).
Abstract: A sampling filter process is provided for scalable video coding. The process provides for re-sampling using video data obtained from an encoder or decoder process of a base layer (BL) in a multi-layer system using adaptive phase shifting to improve quality in Scalable High efficiency Video Coding (SHVC). In order to compensate for phase offsets introduced by downsampling an appropriate phase offset adjustment is made for upsampling in SHVC with an appropriate offset included for proper luma/chroma color space positions. In one approach the luma/chroma phase offset is specified and a filter is selected to apply the appropriate phase change.

Journal ArticleDOI
TL;DR: An interactive volumetric image manipulation framework that can enable the rapid deployment and instant utility of patient‐specific medical images in virtual surgery simulation while requiring little user involvement is systematically advocated.
Abstract: This paper systematically advocates an interactive volumetric image manipulation framework, which can enable the rapid deployment and instant utility of patient-specific medical images in virtual surgery simulation while requiring little user involvement. We seamlessly integrate multiple technical elements to synchronously accommodate physics-plausible simulation and high-fidelity anatomical structures visualization. Given a volumetric image, in a user-transparent way, we build a proxy to represent the geometrical structure and encode its physical state without the need of explicit 3-D reconstruction. On the basis of the dynamic update of the proxy, we simulate large-scale deformation, arbitrary cutting, and accompanying collision response driven by a non-linear finite element method. By resorting to the upsampling of the sparse displacement field resulted from non-linear finite element simulation, the cut/deformed volumetric image can evolve naturally and serves as a time-varying 3-D texture to expedite direct volume rendering. Moreover, our entire framework is built upon CUDA Beihang University, Beijing, China and thus can achieve interactive performance even on a commodity laptop. The implementation details, timing statistics, and physical behavior measurements have shown its practicality, efficiency, and robustness. Copyright © 2013 John Wiley & Sons, Ltd.

Patent
03 Mar 2014
TL;DR: In this article, a method of coding video data includes upsampling at least a portion of a reference layer picture to an upsampled picture having an upsamspled picture size.
Abstract: A method of coding video data includes upsampling at least a portion of a reference layer picture to an upsampled picture having an upsampled picture size. The upsampled picture size has a horizontal upsampled picture size and a vertical upsampled picture size. At least one of the horizontal or vertical upsampled picture sizes may be different than a horizontal picture size or vertical picture size, respectively, of an enhancement layer picture. In addition, position information associated with the upsampled picture may be signaled. An inter-layer reference picture may be generated based on the upsampled picture and the position information.

Proceedings ArticleDOI
02 Jul 2014
TL;DR: A depth measurement reliability function based on optimizing free parameters to test data and verifying it with independent test cases is determined, decreasing the depth upsampling error by almost 40% in comparison to competing proposals.
Abstract: Accurate scene depth capture is essential for the success of three-dimensional television (3DTV), e.g. for high quality view synthesis in autostereoscopic multiview displays. Unfortunately, scene depth is not easily obtained and often of limited quality. Dedicated Time-of-Flight (ToF) sensors can deliver reliable depth readings where traditional methods, such as stereovision analysis, fail. However, since ToF sensors provide only limited spatial resolution and suffer from sensor noise, sophisticated upsampling methods are sought after. A multitude of ToF solutions have been proposed over the recent years. Most of them achieve ToF superresolution (TSR) by sensor fusion between ToF and additional sources, e.g. video. We recently proposed a weighted error energy minimization approach for ToF super-resolution, incorporating texture, sensor noise and temporal information. For this article, we take a closer look at the sensor noise weighting related to the Time-of-Flight active brightness signal. We determine a depth measurement reliability function based on optimizing free parameters to test data and verifying it with independent test cases. In the presented double-weighted TSR proposal, depth readings are weighted into the upsampling process with regard to their reliability, removing erroneous influences in the final result. Our evaluations prove the desired effect of depth measurement reliability weighting, decreasing the depth upsampling error by almost 40% in comparison to competing proposals.

Patent
07 Jan 2014
TL;DR: In this article, the authors describe an encoding and decoding system for the provision of high quality digital representations of audio signals with particular attention to the correct perceptual rendering of fast transients at modest sample rates.
Abstract: Encoding and decoding systems are described for the provision of high quality digital representations of audio signals with particular attention to the correct perceptual rendering of fast transients at modest sample rates. This is achieved by optimising downsampling and upsampling filters to minimise the length of the impulse response while adequately attenuating alias products that have been found perceptually harmful.