Statistics for Spatial Data.

In this paper, we propose a novel explicit image filter called guided filter. Derived from a local linear model, the guided filter computes the filtering output by considering the content of a guidance image, which can be the input image itself or another different image. The guided filter can be used as an edge-preserving smoothing operator like the popular bilateral filter [1], but it has better behaviors near edges. The guided filter is also a more generic concept beyond smoothing: It can transfer the structures of the guidance image to the filtering output, enabling new filtering applications like dehazing and guided feathering. Moreover, the guided filter naturally has a fast and nonapproximate linear time algorithm, regardless of the kernel size and the intensity range. Currently, it is one of the fastest edge-preserving filters. Experiments show that the guided filter is both effective and efficient in a great variety of computer vision and computer graphics applications, including edge-aware smoothing, detail enhancement, HDR compression, image matting/feathering, dehazing, joint upsampling, etc.

/pdf/guided-image-filtering-5atmpsksoa.pdf

Guided Image Filtering

Humans perceive the three-dimensional structure of the world with apparent ease. However, despite all of the recent advances in computer vision research, the dream of having a computer interpret an image at the same level as a two-year old remains elusive. Why is computer vision such a challenging problem and what is the current state of the art? Computer Vision: Algorithms and Applications explores the variety of techniques commonly used to analyze and interpret images. It also describes challenging real-world applications where vision is being successfully used, both for specialized applications such as medical imaging, and for fun, consumer-level tasks such as image editing and stitching, which students can apply to their own personal photos and videos. More than just a source of recipes, this exceptionally authoritative and comprehensive textbook/reference also takes a scientific approach to basic vision problems, formulating physical models of the imaging process before inverting them to produce descriptions of a scene. These problems are also analyzed using statistical models and solved using rigorous engineering techniques Topics and features: structured to support active curricula and project-oriented courses, with tips in the Introduction for using the book in a variety of customized courses; presents exercises at the end of each chapter with a heavy emphasis on testing algorithms and containing numerous suggestions for small mid-term projects; provides additional material and more detailed mathematical topics in the Appendices, which cover linear algebra, numerical techniques, and Bayesian estimation theory; suggests additional reading at the end of each chapter, including the latest research in each sub-field, in addition to a full Bibliography at the end of the book; supplies supplementary course material for students at the associated website, http://szeliski.org/Book/. Suitable for an upper-level undergraduate or graduate-level course in computer science or engineering, this textbook focuses on basic techniques that work under real-world conditions and encourages students to push their creative boundaries. Its design and exposition also make it eminently suitable as a unique reference to the fundamental techniques and current research literature in computer vision.

/pdf/computer-vision-algorithms-and-applications-25dn6wu83j.pdf

Computer Vision: Algorithms and Applications

“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

Object tracking is one of the most important components in numerous applications of computer vision. While much progress has been made in recent years with efforts on sharing code and datasets, it is of great importance to develop a library and benchmark to gauge the state of the art. After briefly reviewing recent advances of online object tracking, we carry out large scale experiments with various evaluation criteria to understand how these algorithms perform. The test image sequences are annotated with different attributes for performance evaluation and analysis. By analyzing quantitative results, we identify effective approaches for robust tracking and provide potential future research directions in this field.

/pdf/online-object-tracking-a-benchmark-10et4l3bf2.pdf

Online Object Tracking: A Benchmark

Time-of-Flight (ToF) depth sensing camera is able to obtain depth maps at a high frame rate. However, its low resolution and sensitivity to the noise are always a concern. A popular solution is upsampling the obtained noisy low resolution depth map with the guidance of the companion high resolution color image. However, due to the constrains in the existing upsampling models, the high resolution depth map obtained in such way may suffer from either texture copy artifacts or blur of depth discontinuity. In this paper, a novel optimization framework is proposed with the brand new data term and smoothness term. The comprehensive experiments using both synthetic data and real data show that the proposed method well tackles the problem of texture copy artifacts and blur of depth discontinuity. It also demonstrates sufficient robustness to the noise. Moreover, a data driven scheme is proposed to adaptively estimate the parameter in the upsampling optimization framework. The encouraging performance is maintained even in the case of large upsampling e.g. $8\times$ and $16\times$.

Robust High Quality Image Guided Depth Upsampling

Reliable human parts segmentation on 2D images plays an important role in many human-centric computer vision tasks. While significant achievements have been made on human pose estimation, the performance on human parts segmentation remains low. In this paper, we present a novel technique that we call Pose2Body that robustly conducts human parts segmentation based on the pose estimation results. We partition an image into superpixels and set out to assign a segment label to each superpixel most consistent with the pose. We design special feature vectors for every superpixel-label assignment as well as superpixel-superpixel pairs and model optimal labeling as to solve for a conditional random field (CRF). Comprehensive experiments show that our technique achieves substantial improvements over the state-of-the-art solutions.

Pose2Body: Pose-Guided Human Parts Segmentation

We present a new framework for correcting multiperspective distortions using collineations A collineation describes the transformation between the images of a camera due to changes in sampling and image plane selection We show that image distortions in many previous models of cameras can be effectively reduced via proper collineations To correct distortions in a specific multiperspective camera, we develop an interactive system that allows users to select feature rays from the camera and position them at the desirable pixels Our system then computes the optimal collineation to match the projections of these rays with the corresponding pixels Experiments demonstrate that our system robustly corrects complex distortions without acquiring the scene geometry, and the resulting images appear nearly undistorted

Multiperspective distortion correction using collineations

Motion blurs are pervasive in real captured video data, especially for hand-held cameras and smartphone cameras because of their low frame rate and material quality. This paper presents a novel Kernel-based motion-Blurred target Tracking (KBT) approach to accurately locate objects in motion blurred video sequence, without explicitly performing deblurring. To model the underlying motion blurs, we first augment the target model by synthesizing a set of blurred templates from the target with different blur directions and strengths. These templates are then represented by color histograms regularized by an isotropic kernel. To locate the optimal position for each template, we choose to use the mean shift method for iterative optimization. Finally, the optimal region with maximum similarity to its corresponding template is considered as the target. To demonstrate the effectiveness and efficiency of our method, we collect several video sequences with severe motion blurs and compare KBT with other traditional trackers. Experimental results show that our KBT method can robustly and reliably track strong motion blurred targets.

Kernel-based motion-blurred target tracking

We present a General Linear Camera (GLC) model that unifies many previous camera models into a single representation. The GLC model is capable of describing all perspective (pinhole), orthographic, and many multiperspective (including pushbroom and two-slit) cameras, as well as epipolar plane images. It also includes three new and previously unexplored multiperspective linear cameras. The GLC model is both general and linear in the sense that, given any vector space where rays are represented as points, it describes all 2D affine subspaces (planes) that can be formed by affine combinations of 3 rays. The incident radiance seen along the rays found on subregions of these 2D linear subspaces are a precise definition of a projected image of a 3D scene. We model the GLC imaging process in terms of two separate stages: the mapping of 3D geometry to rays and the sampling of these rays over an image plane. We derive a closed-form solution to projecting 3D points in a scene to rays in a GLC and a notion of GLC collineation analogous to pinhole cameras. Finally, we develop a GLC ray-tracer for the direct rendering of GLC images. The GLC ray-tracer is able to create a broad class of multiperspective effects and it provides flexible collineation controls to reduce multiperspective distortions.

http://www.ims.cuhk.edu.hk/~cis/2007.4/cis_7_4_04.pdf

Jingyi Yu

Papers

Robust High Quality Image Guided Depth Upsampling

Pose2Body: Pose-Guided Human Parts Segmentation

Multiperspective distortion correction using collineations

Kernel-based motion-blurred target tracking

Multiperspective Modeling and Rendering Using General Linear Cameras