Showing papers on "Upsampling published in 2011"

High quality depth map upsampling for 3D-TOF cameras

Abstract: This paper describes an application framework to perform high quality upsampling on depth maps captured from a low-resolution and noisy 3D time-of-flight (3D-ToF) camera that has been coupled with a high-resolution RGB camera. Our framework is inspired by recent work that uses nonlocal means filtering to regularize depth maps in order to maintain fine detail and structure. Our framework extends this regularization with an additional edge weighting scheme based on several image features based on the additional high-resolution RGB input. Quantitative and qualitative results show that our method outperforms existing approaches for 3D-ToF upsampling. We describe the complete process for this system, including device calibration, scene warping for input alignment, and even how the results can be further processed using simple user markup.

Physics-inspired upsampling for cloth simulation in games

Abstract: We propose a method for learning linear upsampling operators for physically-based cloth simulation, allowing us to enrich coarse meshes with mid-scale details in minimal time and memory budgets, as required in computer games. In contrast to classical subdivision schemes, our operators adapt to a specific context (e.g. a flag flapping in the wind or a skirt worn by a character), which allows them to achieve higher detail. Our method starts by pre-computing a pair of coarse and fine training simulations aligned with tracking constraints using harmonic test functions. Next, we train the upsampling operators with a new regularization method that enables us to learn mid-scale details without overfitting. We demonstrate generalizability to unseen conditions such as different wind velocities or novel character motions. Finally, we discuss how to re-introduce high frequency details not explainable by the coarse mesh alone using oscillatory modes.

A constrained, total-variation minimization algorithm for low-intensity x-ray CT

Abstract: Purpose: The authors developed an iterative image-reconstruction algorithm for application to low-intensity computed tomography projection data, which is based on constrained, total-variation (TV) minimization. The algorithm design focuses on recovering structure on length scales comparable to a detector bin width. Methods: Recovering the resolution on the scale of a detector bin requires that pixel size be much smaller than the bin width. The resulting image array contains many more pixels than data, and this undersampling is overcome with a combination of Fourier upsampling of each projection and the use of constrained, TV minimization, as suggested by compressive sensing. The presented pseudocode for solving constrained, TV minimization is designed to yield an accurate solution to this optimization problem within 100 iterations. Results: The proposed image-reconstruction algorithm is applied to a low-intensity scan of a rabbit with a thin wire to test the resolution. The proposed algorithm is compared to filtered backprojection (FBP). Conclusions: The algorithm may have some advantage over FBP in that the resulting noise level is lowered at equivalent contrast levels of the wire.

Interpolation-Dependent Image Downsampling

Abstract: Traditional methods for image downsampling commit to remove the aliasing artifacts. However, the influences on the quality of the image interpolated from the downsampled one are usually neglected. To tackle this problem, in this paper, we propose an interpolation-dependent image downsampling (IDID), where interpolation is hinged to downsampling. Given an interpolation method, the goal of IDID is to obtain a downsampled image that minimizes the sum of square errors between the input image and the one interpolated from the corresponding downsampled image. Utilizing a least squares algorithm, the solution of IDID is derived as the inverse operator of upsampling. We also devise a content-dependent IDID for the interpolation methods with varying interpolation coefficients. Numerous experimental results demonstrate the viability and efficiency of the proposed IDID.

Semiblind Sparse Channel Estimation for MIMO-OFDM Systems

Abstract: In this paper, a semiblind algorithm is presented for the estimation of sparse multiple-input-multiple-output orthogonal frequency-division multiplexing (MIMO-OFDM) channels. An analysis of the second-order statistics of the signal that was received through a sparse MIMO channel is first conducted, showing that the correlation matrices of the received signal can be expressed in terms of the most significant taps (MSTs) of the sparse channel. This relationship is used to derive a blind constraint for the effective channel vector that corresponds to the MST position. The blind constraint is then combined with the training-based least square criterion to develop a semiblind approach for the estimation of MSTs of the sparse channel. A signal perturbation analysis of the proposed approach is conducted, showing that the new semiblind solution is not subject to the signal perturbation error when the sparse channel is a decimated version of a full finite impulse response channel. Furthermore, the proposed sparse semiblind algorithm has been extended for the estimation of channels in the upsampling domain for MIMO-OFDM systems with pulse shaping. A number of computer-simulation-based experiments for various sparse channels are carried out to confirm the effectiveness of the proposed semiblind approach.

Multispectral demosaicking using adaptive kernel upsampling

Abstract: Multispectral demosaicking, which estimates full multispectral images from raw data observed using a single image sensor with a color filter array (CFA), is a challenging task because each spectral component is severely undersampled. In this paper, we propose a novel multispectral demosaicking algorithm. We extend existing upsampling algorithms to adaptive kernel upsampling algorithms using an adaptive kernel as a spatial weight and apply them to multispectral demosaicking. We also propose a new CFA and a direct adaptive kernel estimation from the raw data of the proposed CFA. Experimental results with real multispectral images demonstrate the effectiveness of the proposed algorithm.

A novel algorithm combining oversampling and digital lock-in amplifier of high speed and precision.

Abstract: Because of a large amount of arithmetic in the standard digital lock-in detection, a high performance processor is needed to implement the algorithm in real time. This paper presents a novel algorithm that integrates oversampling and high-speed lock-in detection. The algorithm sets the sampling frequency as a whole-number multiple of four of the input signal frequency, and then uses the common downsampling technology to lower the sampling frequency to four times of the input signal frequency. It could effectively remove the noise interference and improve the detection accuracy. After that the phase sensitive detector is implemented. It simply does the addition and subtraction on four points in the period of same phase and replaces almost all the multiplication operations to speed up digital lock-in detection calculation substantially. Furthermore, the correction factor is introduced to improve the calculation accuracy of the amplitude, and an error caused by the algorithm in theory can be eliminated completely. The results of the simulation and actual experiments show that the novel algorithm combining digital lock-in detection and oversampling not only has the high precision, but also has the unprecedented speed. In our work, the new algorithm is suitable for the real-time weak signal detection in the general microprocessor not just digital signal processor.

Image-based bidirectional scene reprojection

Abstract: We introduce a method for increasing the framerate of real-time rendering applications. Whereas many existing temporal upsampling strategies only reuse information from previous frames, our bidirectional technique reconstructs intermediate frames from a pair of consecutive rendered frames. This significantly improves the accuracy and efficiency of data reuse since very few pixels are simultaneously occluded in both frames. We present two versions of this basic algorithm. The first is appropriate for fill-bound scenes as it limits the number of expensive shading calculations, but involves rasterization of scene geometry at each intermediate frame. The second version, our more significant contribution, reduces both shading and geometry computations by performing reprojection using only image-based buffers. It warps and combines the adjacent rendered frames using an efficient iterative search on their stored scene depth and flow. Bidirectional reprojection introduces a small amount of lag. We perform a user study to investigate this lag, and find that its effect is minor. We demonstrate substantial performance improvements (3--4x) for a variety of applications, including vertex-bound and fill-bound scenes, multi-pass effects, and motion blur.

Efficient Full Aperture Processing of TOPS Mode Data Using the Moving Band Chirp $Z$ -Transform

Abstract: The main operational mode of the European Space Agency's upcoming Sentinel-1 operational satellite will be the Terrain Observation by Progressive Scans (TOPS) imaging mode. This paper presents a very efficient wavenumber domain processor for the processing of TOPS mode data. In particular, a novel signal transform, called a moving band chirp Z-transform, is introduced in order to allow the entire azimuth aperture to be focused simultaneously without any need for temporary unaliasing, which requires upsampling, or subaperture processing.

Infrared camera calibration for dense depth map construction

Abstract: In this paper, we introduce a novel and cost effective approach to calibrate the geometric properties of a far-infrared (IR) sensor. We further demonstrate that fully automatic sensor-to-sensor calibration is feasible in a setup involving a laser range scanner, IR cameras as well as conventional cameras. The calibration result then serves as a basis for upsampling range measurements to the resolution of the IR or visible-light camera images. Since our approach allows to rely on IR information instead of visible-light information for upsampling, bad light conditions or even no visible light at all are no limitation. From a practical point of view, we only require one calibration board of relatively small size which facilitates application in outdoor environments and further allows seamless integration of the IR camera in an existing multi-sensor platform. Our experimental results demonstrate that IR images are particularly useful to obtain reasonable depth information for living objects, when visible-light cameras are either blind or require impractical exposure times. In fact, our approach provides a convenient solution to IR camera calibration and integration, an issue which is particularly important in scenarios where sensors are not permanently mounted on vehicles and consequently require on-site adjustment and calibration.

Stereo Acoustic Echo Cancellation Employing Frequency-Domain Preprocessing and Adaptive Filter

Abstract: This paper proposes a windowing frequency domain adaptive filter and an upsampling block transform preprocessing to solve the stereo acoustic echo cancellation problem. The proposed adaptive filter uses windowing functions with smooth cutoff property to reduce the spectral leakage during filter updating, so the utilization of the independent noise introduced by preprocessing in stereo acoustic echo cancellation can be increased. The proposed preprocessing is operated in short blocks with low processing delay, and it uses frequency-domain upsampling to meet the minimal block length requirement given by the band limit of simultaneous masking. Therefore, the simultaneous masking can be well utilized to improve the audio quality. The acoustic echo cancellation simulations and the audio quality evaluation show that, the proposed windowing frequency domain adaptive filter performs better than the conventional frequency domain adaptive filter in both mono and stereo cases, and the upsampling block transform preprocessing provides better audio quality and stereo acoustic echo cancellation performance than the half-wave preprocessing at the same noise level.

Implementation of Linear-Phase FIR Filters for a Rational Sampling-Rate Conversion Utilizing the Coefficient Symmetry

Abstract: This paper proposes an efficient structure for implementing a linear-phase finite-impulse-response (FIR) filter of an arbitrary order N for the sampling-rate conversion by a rational factor of L/M , where L(M) is the integer upsampling (down-sampling) factor to be performed before (after) the actual filter. In this implementation, the coefficient symmetry of the linear-phase filter is exploited as much as possible and the number of delay elements is kept as low as possible while utilizing the following facts. When increasing (decreasing) the sampling rate by a factor of L(M), only every Lth input sample has a nonzero value (only every M th output sample has to be evaluated). In this way, the number of required multiplications per output sample is reduced approximately by a factor of two compared with the conventional polyphase implementation. The proposed implementation is first illustrated using two examples. Based on these examples, guidelines are then given on how to efficiently realize an Nth-order linear-phase FIR filter for a sampling-rate converter having an arbitrary rational conversion factor L/M. Finally, the implementation complexity of the proposed approach is evaluated and some examples are included, showing the efficiency of the proposed implementation compared with other existing ones.

Depth super resolution using bilateral filter

Abstract: Depth maps are used in many applications, e.g. 3D television, stereo matching, segmentation, etc. Recently, a new generation of active 3D range sensors, such as time-of-flight (TOF) cameras, enables recording of full frame depth maps at video frame rate. Unfortunately, depth maps captured with the TOF cameras have limited resolution and poor image quality, being serverely influenced by the random and systematic noise, which makes them innaposite for generating high quality 3D images. In this paper, we proposed a method to enhance the quality and increase the spatial resolution of range data by upsampling the range information with the data from a high resolution video camera. Our algorithm is an adaptive upsampling filter that takes into account the inherent noisy nature of depth data. Thus, we can improve reconstruction quality, boost the resolution of the data to that of the video sensor, and prevent unwanted artifacts like texture copy into geometry.

MobiUP: An Upsampling-Based System Architecture for High-Quality Video Streaming on Mobile Devices

Abstract: Nowadays, mobile video streaming enables people to access digital content, such as online TV shows, music videos, sports reports, and news programs, anytime, anywhere. However, current streaming services in mobile networks are subject to the available wireless bandwidth shared among many users and can only provide videos with limited resolutions. Moreover, on recently developed high-resolution mobile devices, such as iPhone, Google Nexus One, Nokia N97, and SonyEricsson X10, the resolution of video streaming is much lower than the devices can actually support. As a result, existing video upsampling schemes usually introduce visual artifacts. In response to the above problem, we bridge the resolution gap between streaming videos and client screens, and propose a novel upsampling-based system architecture, called MobiUP, to enable high-quality video streaming onto mobile devices. To avoid modifying existing codecs for video streaming, MobiUP upsamples videos with decoded frames and appends a limited amount of metadata to the streaming videos for facilitating high-quality and real-time conversion from low resolution to high fullscreen resolution on the client side. In other words, the proposed upsampling architecture complements current systems. Therefore, MobiUP is generic and flexible, and it can be implemented easily on mobile devices for practical use. The implementation results demonstrate that, although the appended metadata is less than 8% of the total transmitted data, it improves the quality of the upsampled video significantly. Meanwhile, the computation time of MobiUP Client is close to that of bilinear upsampling algorithms implemented on mobile devices.

Resolution Improvement of Infrared Images Using Visible Image Information

Abstract: This letter presents a new framework for improving the spatial resolution of infrared (IR) images based on the high-resolution visible image information. Edge regions in an IR image, which have a strong correlation with the aligned edges in the visible image, are sharpened by using their high frequency patches, which are locally estimated from the visible image. The estimation is performed on the basis of intensity correlations between two images. In addition, in order to improve the resolution in the uncorrelated edge regions and the texture regions where visible image information is not available, we adopt learning-based and reconstruction-based super resolution algorithms, respectively. Experimental results demonstrate that the proposed algorithm improves the spatial resolution compared with the existing upsampling algorithms.

System and method of image upsampling

Abstract: A method includes receiving an image having a first resolution and generating an upsampled image having a second resolution based on the image. A multi-dimensional data structure corresponding to a multi-dimensional image space is generated from the upsampled image. Each node of the data structure is determined based on a weighted sum of values of one or more pixels in the upsampled image. Each of the one or more pixels corresponds to a pixel in the received image and is located within a region of the image space having a vertex defined by the node. A filter modifies the values of the nodes and a second upsampled image is generated based on the modified values of the nodes. Each pixel of the second upsampled image not corresponding to a pixel in the received image is determined based on a weighted sum of the modified values of one or more nodes.

Simultaneous Reactive-Power and Frequency Estimations Using Simple Recursive WLS Algorithm and Adaptive Filtering

Abstract: A new simple approach to the design of digital algorithm for simultaneous reactive-power and frequency estimations of local system is presented. The algorithm is derived using the weighted-least-square method. During the algorithm derivation, a pure sinusoidal voltage model was assumed. Cascade finite-impulse-response (FIR) comb digital filters are used to minimize the noise effect and to eliminate the presence of harmonics effect. The most important point of this paper is the mathematical model that transforms the problem of estimation into an overdetermined set of linear equations. The investigation was simplified because the total similarity to the state of the problem of the active-power and frequency estimations was noticed. The only difference is the adaptive phase shifter applied to the voltage signal. In addition, coefficient-sensitivity problems of the large-order FIR comb cascade structure are overridden by using a multirate (decimation) digital signal processing technique. Even more, by using antialiasing filters, the parameter estimation accuracy is improved. The effectiveness of the proposed techniques is demonstrated by both simulation and experimental results. The algorithm shows a very high level of robustness, as well as high measurement accuracy over a wide range of frequency changes.

Superfast superresolution

Abstract: We propose a fast algorithm for solving the inverse problem of resolution enhancement (superresolution). Robustness is achieved by a non-linear regularizer and a method based on variable splitting is used to obtain an equivalent linear formulation. Special attention is paid to fast implementation using the Fourier transform. In particular, we show that a degradation operator (downsampling) can be implemented in the frequency domain and that all computations can be performed very efficiently without losing robustness. To our knowledge, this is the first attempt towards a very fast SR algorithm, which retains favorable edge-preserving properties of non-linear regularizers.

Fast randomized multi-scale energy minimization for image processing

Abstract: An image processing module performs efficient image enhancement according to a multi-scale energy minimization process. One or more input images are progressively downsampled to generate a pyramid of downsampled images of varying resolution. Starting with the coarsest downsampled image, a label map is generated that maps output pixel positions to pixel positions in the downsampled input images. The label map is then progressively upsampled. At each upsampling stage, the labels are refined according to an energy function configured to produce the desired enhancements. Using the multi-scale energy minimization, the image processing module enhances image via hole-filling and/or super-resolution.

Dynamic Depth of Field on Live Video Streams: A Stereo Solution

Abstract: The ability to produce dynamic Depth of Field effects in live video streams was until recently a quality unique to movie cameras. In this paper, we present a computational camera solution coupled with real-time GPU processing to produce runtime dynamic Depth of Field effects. We first construct a hybrid-resolution stereo camera with a high-res/low-res camera pair. We recover a low-res disparity map of the scene using GPU-based Belief Propagation and subsequently upsample it via fast Cross/Joint Bilateral Upsampling. With the recovered high-resolution disparity map, we warp the high-resolution video stream to nearby viewpoints to synthesize a light field towards the scene. We exploit parallel processing and atomic operations on the GPU to resolve visibility when multiple pixels warp to the same image location. Finally, we generate dynamic Depth of Field effects from the synthesized light field rendering. All processing stages are mapped onto NVIDIA’s CUDA architecture. Our system can produce Depth of Field effects with arbitrary aperture sizes and focal depths for the resolution of 640×480 at 15 fps.

“Rewiring” Filterbanks for Local Fourier Analysis: Theory and Practice

Abstract: This paper describes a series of new results outlining equivalences between certain “rewirings” of filterbank system block diagrams, and the corresponding actions of convolution, modulation, and downsampling operators. This gives rise to a general framework of reverse-order and convolution subband structures in filterbank transforms, which we show to be well suited to the analysis of filterbank coefficients arising from subsampled or multiplexed signals. These results thus provide a means to understand time-localized aliasing and modulation properties of such signals and their subband representations - notions that are notably absent from the global viewpoint afforded by Fourier analysis - as well as signal recovery from sampled sequences based on their filterbank characterizations. The utility of filterbank rewirings is demonstrated by the closed-form analysis of signals subject to degradations such as missing data, spatially or temporally multiplexed data acquisition, or signal-dependent noise, the likes of which are often encountered in practical signal processing applications.

Fast randomized multi-scale energy minimization for inferring depth from stereo image pairs

Abstract: An image processing module infers depth from a stereo image pair according to a multi-scale energy minimization process. A stereo image pair is progressively downsampled to generate a pyramid of downsampled image pairs of varying resolution. Starting with the coarsest downsampled image pair, a disparity map is generated that reflects displacement between corresponding pixels in the stereo image pair. The disparity map is then progressively upsampled. At each upsampling stage, the disparity labels are refined according to an energy function. The disparity labels provide depth information related to surfaces depicted in the stereo image pair.

Depth data upsampling

Abstract: A device and a method of up-sampling low resolution depth data is described, in particular in the form of a range map or a disparity map of stereo content, comprising the following successive steps: up-scaling the low resolution depth data to the desired resolution using a nearest neighbour interpolation to receive an up-scaled disparity map, detecting the horizontal and vertical bounds of each pixel of a high resolution colour image based on intensity differences, wherein the low resolution depth data correspond to the high resolution colour image, combining the up-scaled disparity map and the detected vertical and horizontal bounds by applying an averaging filter among an arbitrary shaped region by the utilization of horizontal and vertical integral data of the up-scaled disparity map to receive a high resolution disparity map.

Apparatus and method for processing an audio signal and for providing a higher temporal granularity for a combined unified speech and audio codec (usac)

Abstract: An apparatus for processing an audio signal is provided. The apparatus comprises a signal processor (110; 205; 405) and a configurator (120; 208; 408). The signal processor (110; 205; 405) is adapted to receive a first audio signal frame having a first configurable number of samples of the audio signal, Moreover, the signal processor (110; 205; 405) is adapted to upsample the audio signal by a configurable upsampling factor to obtain a processed audio signal. Furthermore, the signal processor (110; 205; 405) is adapted to output a second audio signal frame having a second configurable number of samples of the processed audio signal. The configurator 120; 208; 408) is adapted to configure the signal processor (110; 205; 405) based on configuration information such that the configurable upsampling factor is equal to a first upsampling value when a first ratio of the second configurable number of samples to the first configurable number of samples has a first ratio value. Moreover, the configurator ( 120; 208; 408) is adapted to configure the signal processor (110; 205; 405) such that the configurable upsampling factor is equal to a different second upsampling value, when a different second ratio of the second configurable number of samples to the first configurable number of samples has a different second ratio value. The first or the second ratio value is not an integer value.

Fractional rate resampling filter on FPGA

Abstract: A programmable logic device can be configured as a fractional rate resampling filter capable of performing downsampling prior to upsampling without modifying the overall filter response. Input data may be received at a first sample rate and may be downsampled to generate downsampled data. Portions of the downsampled data may be respectively output to different filtering paths. Each filtering path may include a cluster of filter components that corresponds to different subfilters of the overall filter response and may be operable to receive and process the different portions of the downsampled data. Outputs of each cluster may be combined to generate output data at a second sample rate. The resampling filter structure can reduce the number of multiplier circuits used by allowing time-division multiplexing among different filter components.

Edge self-adaptive image amplification method based on non-downsampling Contourlet conversion

Abstract: The invention provides an edge self-adaptive image amplification method based on non-downsampling Contourlet conversion, comprising the following steps: (1) inputting original images, setting resolution ratio of target images, and determining amplification proportional coefficient of the images; (2) carrying out non-downsampling Contourlet conversion on the original images to obtain directional subband coefficient image of transform domain; (3) amplifying the directional subband coefficient image to be the object resolution by adopting the directional self-adaptive interpolation method; (4) estimating the interpolation direction of each point to be interpolated of the target images according to the amplified directional subband coefficient image; (5) adopting the directional self-adaptiveinterpolation method to obtain the pixel value of the point to be interpolated according to the interpolation direction of the target images point to be interpolated; and (6) outputting ultimate amplified image. The invention realizes interpolation at any direction; the edges of the amplified images have high smooth level; and entire visual effect of the images is favorable, thus being applicableto proportional amplification of grey or colorful images.

Real-time spatial and depth upsampling for range data

Abstract: Current active 3D range sensors, such as time-of-flight cameras, enable acquiring of range maps at video frame rate. Unfortunately, the resolution of the range maps is quite limited and the captured data are typically contaminated by noise. We therefore present a simple pipeline to enhance the quality as well as improve the spatial and depth resolution of range data in real time. To improve the spatial resolution of range data, we first upsample the depth information with the data from high resolution video camera. And then, a new strategy is utilized to increase the sub-pixel accuracy. We show that these techniques can greatly improve the reconstruction quality, boost the resolution of the range data to that of video sensor while achieving high computational efficiency for a real-time application.

Full spectrum modulator for digital television signals

Abstract: A full spectrum modulator processes a plurality of CATV channels from separate paths. Each path has (i) a first filter for pulse shaping an input channel signal and upsampling a channel frequency thereof, (ii) an interpolator for interpolating the output of the first filter to a common sample rate, and (iii) a decimator for centering the output of the interpolator on a predetermined channel bandwidth. An IDFT processor receives channel signal outputs from the decimators. A polyphase filter bank receives IDFT processed parallel channel signals from the IDFT processor. A commutator converts the processed parallel channel signals from the polyphase filter bank to a single stream of data. A second filter upsamples the single stream of data to a fixed output sampling rate and low pass filters alias signals therefrom. Both standard and harmonically related carrier CATV channel frequency plans are accommodated.

Image autoregressive interpolation method based on edge detection

Abstract: The invention discloses an image autoregressive interpolation method based on edge detection, and the method is mainly used for solving the problem of high complexity of the existing autoregressive interpolation technique. The method comprises the following steps: carrying out one quarter of downsampling on original images, carrying out edge detection on the downsampling images, approximately obtaining the edge detection images of the to-be interpolated recovery images according to the edge detection images of the downsampling images; classifying the to-be-interpolated pixels of the to-be-interpolated recovery images into to-be-interpolated pixels in smooth area and edge area, adopting bicubic to carry out interpolation on the to-be-interpolated pixels in the smooth area, adopting the autoregressive interpolation method to interpolate the to-be-interpolated pixels in the edge area; and obtaining the image after interpolation. On the premise of ensuring the visual effects of the images, the method in the invention can be utilized to carry out amplification, denoising, mending, deinterlacing and compression on the images.

Fast video interpolation/upsampling using linear motion model

Abstract: Recently, the probabilistic motion field was proposed for super-resolution reconstruction (SRR). In the interpolation step of SRR, a missing pixel can be estimated by the weighted average of neighboring pixels, which are weighted by the errors with the missing pixel. However, the errors are far from true values due to the approximated missing pixel in calculating the errors. Hence, in this paper, we propose a linear motion model to better approximate the errors, which results in a better interpolation quality. Experimental results show that a gain of 0.6 dB in PSNR is achievable using this linear motion model, and only a small number of neighboring pixels have to be used for fast interpolation/upsampling.