Jianping Zhou

Bio: Jianping Zhou is an academic researcher from Apple Inc.. The author has contributed to research in topics: Image stabilization & Frame (networking). The author has an hindex of 18, co-authored 64 publications receiving 2839 citations. Previous affiliations of Jianping Zhou include University of Illinois at Urbana–Champaign & Intel.

The Nonsubsampled Contourlet Transform: Theory, Design, and Applications

Abstract: In this paper, we develop the nonsubsampled contourlet transform (NSCT) and study its applications. The construction proposed in this paper is based on a nonsubsampled pyramid structure and nonsubsampled directional filter banks. The result is a flexible multiscale, multidirection, and shift-invariant image decomposition that can be efficiently implemented via the a trous algorithm. At the core of the proposed scheme is the nonseparable two-channel nonsubsampled filter bank (NSFB). We exploit the less stringent design condition of the NSFB to design filters that lead to a NSCT with better frequency selectivity and regularity when compared to the contourlet transform. We propose a design framework based on the mapping approach, that allows for a fast implementation based on a lifting or ladder structure, and only uses one-dimensional filtering in some cases. In addition, our design ensures that the corresponding frame elements are regular, symmetric, and the frame is close to a tight one. We assess the performance of the NSCT in image denoising and enhancement applications. In both applications the NSCT compares favorably to other existing methods in the literature

Nonsubsampled contourlet transform: construction and application in enhancement

Abstract: We present the nonsubsampled contourlet transform and its application in image enhancement. The nonsubsampled contourlet transform is built upon nonsubsampled pyramids and nonsubsampled directional filter banks and provides a shift-invariant directional multiresolution image representation. Existing methods for image enhancement cannot capture the geometric information of images and tend to amplify noises when they are applied to noisy images since they cannot distinguish noises from weak edges. In contrast, the nonsubsampled contourlet transform extracts the geometric information of images, which can be used to distinguish noises from weak edges. Experimental results show the proposed method achieves better enhancement results than a wavelet-based image enhancement method.

System and method for delivery of dynamically scalable audio/video content over a network

Abstract: The present invention involves a new system and process for streaming delivery of dynamically scalable media content over a network, such as, for example, the Internet or a wireless network, while automatically accounting for both fluctuating network bandwidth and packet loss. A system of rate-distortion based packet selection and organization is used to maximize the quality of streamed media files that have been encoded using any conventional scalable encoder. Media file quality is maximized for available bandwidth by scoring packets comprising encoded media files based on their contribution to the quality of a reconstructed media file, then preferentially transmitting those packets having the highest scores. In addition, where packets are lost during transmission, those packets that will provide the maximum rate-distortion decrease, are preferentially retransmitted prior to other lost packets which, if transmitted in the same time slot, would provide a lesser rate-distortion decease.

Nonsubsampled contourilet transform: filter design and applications in denoising

Abstract: In this paper we study the nonsubsampled contourlet transform. We address the corresponding filter design problem using the Mc-Clellan transformation. We show how zeroes can be imposed in the filters so that the iterated structure produces regular basis functions. The proposed design framework yields filters that can be implemented efficiently through a lifting factorization. We apply the constructed transform in image noise removal where the results obtained are comparable to the state-of-the art, being superior in some cases.

Rolling shutter reduction based on motion sensors

Abstract: This disclosure pertains to devices, methods, and computer readable media for reducing rolling shutter distortion effects in captured video frames based on timestamped positional information obtained from positional sensors in communication with an image capture device In general, rolling shutter reduction techniques are described for generating and applying image segment-specific perspective transforms to already-captured segments (ie, portions) of images in a video sequence, so as to counter or compensate for unwanted distortions that occurred during the read out of the image sensor Such distortions may be due to, for example, the use of CMOS sensors combined with the rapid movement of the image capture device In contrast to the prior art, rolling shutter reduction techniques described herein may be applied to captured images in real-time or near real-time using positional sensor information and without intensive image processing that would require an analysis of the content of the underlying image data

Deep Convolutional Neural Network for Inverse Problems in Imaging

Abstract: In this paper, we propose a novel deep convolutional neural network (CNN)-based algorithm for solving ill-posed inverse problems. Regularized iterative algorithms have emerged as the standard approach to ill-posed inverse problems in the past few decades. These methods produce excellent results, but can be challenging to deploy in practice due to factors including the high computational cost of the forward and adjoint operators and the difficulty of hyperparameter selection. The starting point of this paper is the observation that unrolled iterative methods have the form of a CNN (filtering followed by pointwise non-linearity) when the normal operator ( $H^{*}H$ , where $H^{*}$ is the adjoint of the forward imaging operator, $H$ ) of the forward model is a convolution. Based on this observation, we propose using direct inversion followed by a CNN to solve normal-convolutional inverse problems. The direct inversion encapsulates the physical model of the system, but leads to artifacts when the problem is ill posed; the CNN combines multiresolution decomposition and residual learning in order to learn to remove these artifacts while preserving image structure. We demonstrate the performance of the proposed network in sparse-view reconstruction (down to 50 views) on parallel beam X-ray computed tomography in synthetic phantoms as well as in real experimental sinograms. The proposed network outperforms total variation-regularized iterative reconstruction for the more realistic phantoms and requires less than a second to reconstruct a $512\times 512$ image on the GPU.

Dictionaries for Sparse Representation Modeling

Abstract: Sparse and redundant representation modeling of data assumes an ability to describe signals as linear combinations of a few atoms from a pre-specified dictionary. As such, the choice of the dictionary that sparsifies the signals is crucial for the success of this model. In general, the choice of a proper dictionary can be done using one of two ways: i) building a sparsifying dictionary based on a mathematical model of the data, or ii) learning a dictionary to perform best on a training set. In this paper we describe the evolution of these two paradigms. As manifestations of the first approach, we cover topics such as wavelets, wavelet packets, contourlets, and curvelets, all aiming to exploit 1-D and 2-D mathematical models for constructing effective dictionaries for signals and images. Dictionary learning takes a different route, attaching the dictionary to a set of examples it is supposed to serve. From the seminal work of Field and Olshausen, through the MOD, the K-SVD, the Generalized PCA and others, this paper surveys the various options such training has to offer, up to the most recent contributions and structures.

Rate-distortion optimized streaming of packetized media

Abstract: This paper addresses the problem of streaming packetized media over a lossy packet network in a rate-distortion optimized way. We show that although the data units in a media presentation generally depend on each other according to a directed acyclic graph, the problem of rate-distortion optimized streaming of an entire presentation can be reduced to the problem of error-cost optimized transmission of an isolated data unit. We show how to solve the latter problem in a variety of scenarios, including the important common scenario of sender-driven streaming with feedback over a best-effort network, which we couch in the framework of Markov decision processes. We derive a fast practical algorithm for nearly optimal streaming in this scenario, and we derive a general purpose iterative descent algorithm for locally optimal streaming in arbitrary scenarios. Experimental results show that systems based on our algorithms have steady-state gains of 2-6 dB or more over systems that are not rate-distortion optimized. Furthermore, our systems essentially achieve the best possible performance: the operational distortion-rate function of the source at the capacity of the packet erasure channel.