From the Publisher: 
The accessible presentation of this book gives both a general view of the entire computer vision enterprise and also offers sufficient detail to be able to build useful applications. Users learn techniques that have proven to be useful by first-hand experience and a wide range of mathematical methods. A CD-ROM with every copy of the text contains source code for programming practice, color images, and illustrative movies. Comprehensive and up-to-date, this book includes essential topics that either reflect practical significance or are of theoretical importance. Topics are discussed in substantial and increasing depth. Application surveys describe numerous important application areas such as image based rendering and digital libraries. Many important algorithms broken down and illustrated in pseudo code. Appropriate for use by engineers as a comprehensive reference to the computer vision enterprise.

Computer vision : a modern approach = 计算机视觉 : 一种现代的方法

Video indexing and retrieval have a wide spectrum of promising applications, motivating the interest of researchers worldwide. This paper offers a tutorial and an overview of the landscape of general strategies in visual content-based video indexing and retrieval, focusing on methods for video structure analysis, including shot boundary detection, key frame extraction and scene segmentation, extraction of features including static key frame features, object features and motion features, video data mining, video annotation, video retrieval including query interfaces, similarity measure and relevance feedback, and video browsing. Finally, we analyze future research directions.

A Survey on Visual Content-Based Video Indexing and Retrieval

High dynamic range (HDR) imaging from a set of sequential exposures is an easy way to capture high-quality images of static scenes, but suffers from artifacts for scenes with significant motion. In this paper, we propose a new approach to HDR reconstruction that draws information from all the exposures but is more robust to camera/scene motion than previous techniques. Our algorithm is based on a novel patch-based energy-minimization formulation that integrates alignment and reconstruction in a joint optimization through an equation we call the HDR image synthesis equation. This allows us to produce an HDR result that is aligned to one of the exposures yet contains information from all of them. We present results that show considerable improvement over previous approaches.

/pdf/robust-patch-based-hdr-reconstruction-of-dynamic-scenes-3nwlmiivng.pdf

Robust patch-based hdr reconstruction of dynamic scenes

Recovering a high dynamic range (HDR) image from a single low dynamic range (LDR) input image is challenging due to missing details in under-/over-exposed regions caused by quantization and saturation of camera sensors. In contrast to existing learning-based methods, our core idea is to incorporate the domain knowledge of the LDR image formation pipeline into our model. We model the HDR-to-LDR image formation pipeline as the (1) dynamic range clipping, (2) non-linear mapping from a camera response function, and (3) quantization. We then propose to learn three specialized CNNs to reverse these steps. By decomposing the problem into specific sub-tasks, we impose effective physical constraints to facilitate the training of individual sub-networks. Finally, we jointly fine-tune the entire model end-to-end to reduce error accumulation. With extensive quantitative and qualitative experiments on diverse image datasets, we demonstrate that the proposed method performs favorably against state-of-the-art single-image HDR reconstruction algorithms.

/pdf/single-image-hdr-reconstruction-by-learning-to-reverse-the-4gk5rm2nhi.pdf

Single-Image HDR Reconstruction by Learning to Reverse the Camera Pipeline

Obtaining a high quality high dynamic range HDR image in the presence of camera and object movement has been a long-standing challenge. Many methods, known as HDR deghosting algorithms, have been developed over the past ten years to undertake this challenge. Each of these algorithms approaches the deghosting problem from a different perspective, providing solutions with different degrees of complexity, solutions that range from rudimentary heuristics to advanced computer vision techniques. The proposed solutions generally differ in two ways: 1 how to detect ghost regions and 2 what to do to eliminate ghosts. Some algorithms choose to completely discard moving objects giving rise to HDR images which only contain the static regions. Some other algorithms try to find the best image to use for each dynamic region. Yet others try to register moving objects from different images in the spirit of maximizing dynamic range in dynamic regions. Furthermore, each algorithm may introduce different types of artifacts as they aim to eliminate ghosts. These artifacts may come in the form of noise, broken objects, under- and over-exposed regions, and residual ghosting. Given the high volume of studies conducted in this field over the recent years, a comprehensive survey of the state of the art is required. Thus, the first goal of this paper is to provide this survey. Secondly, the large number of algorithms brings about the need to classify them. Thus the second goal of this paper is to propose a taxonomy of deghosting algorithms which can be used to group existing and future algorithms into meaningful classes. Thirdly, the existence of a large number of algorithms brings about the need to evaluate their effectiveness, as each new algorithm claims to outperform its precedents. Therefore, the last goal of this paper is to share the results of a subjective experiment which aims to evaluate various state-of-the-art deghosting algorithms.

The State of the Art in HDR Deghosting: A Survey and Evaluation

We propose a new method for ghost-free high dynamic range (HDR) video taken with a camera that captures
alternating short and long exposures. These exposures may be combined using traditional HDR techniques,
however motion in a dynamic scene will lead to ghosting artifacts. Due to occlusions and fast moving objects, a
gradient-based optical flow motion compensation method will fail to eliminate all ghosting. As such, we perform
simpler block-based motion estimation and refine the motion vectors in saturated regions using color similarity in
the adjacent frames. The block-based search allows motion to be calculated directly between adjacent frames over
a larger search range, yet at the cost of decreased motion fidelity. To address this, we investigate a new method
to fix registration errors and block artifacts using a cross-bilateral filter to preserve the edges and structure of
the original frame while retaining the HDR color information. Results show promising dynamic range expansion
for videos with fast local motion.

/pdf/high-dynamic-range-video-with-ghost-removal-5bp9kgxan7.pdf

High dynamic range video with ghost removal

One method to extend the dynamic range of video captured with inexpensive cameras is to alternate the exposure time between frames and combine the information in adjacent frames using post-processing. This method requires no hardware modification, yet traditionally there is a quality tradeoff. Dynamic range expansion corresponds to an increased number of saturated pixels in individual frames, which along with occlusions contributes to registration artifacts. Therefore, we describe a “High Dynamic Range (HDR) Filter” that can mitigate these artifacts to produce a pleasing HDR video without exact frame registration. This filter builds upon the bilateral filter to smooth frames while maintaining important edges. Additionally, the filter strength locally adapts to corresponding motion vectors. Since regions with poor registration generally correspond to higher motion, smoothing here can reduce artifacts without degrading perceptual quality. Results show a significant improvement for HDR videos with fast local motion within saturated regions.

Spatially adaptive filtering for registration artifact removal in HDR video

This paper presents a video summarization technique for rushes that employs high-level feature fusion to identify segments for inclusion. It aims to capture distinct video events using a variety of features: k-means based weighting, speech, camera motion, significant differences in HSV color space, and a dynamic time warping (DTW) based feature that suppresses repeated scenes. The feature functions are used to drive a weighted k-means based clustering to identify visually distinct, important segments that constitute the final summary. The optimal weights corresponding to the individual features are obtained using a gradient descent algorithm that maximizes the recall of ground truth events from representative training videos. Analysis reveals a lengthy computation time but high quality results (60% average recall over 42 test videos) as based on manually-judged inclusion ofdistinct shots. The summaries were judged relatively easy to view and had an average amount of redundancy.

/pdf/feature-fusion-and-redundancy-pruning-for-rush-video-n4u0njrbjh.pdf

Feature fusion and redundancy pruning for rush video summarization

With the advent of glasses-free autostereoscopic handheld displays, a number of applications may be enhanced by 3D perception, particularly mobile video communications. On handheld devices, front-facing stereo cameras can capture the two views necessary for 3D display. However, the short distance between the user and cameras introduces large disparities and other stereoscopic challenges that have traditionally plagued close-up stereo photography. Maintaining both viewer comfort and 3D fusion under varying conditions is therefore a technological priority. In this paper, we discuss the main stereoscopic concerns of handheld 3D video communications and outline a new post-processing technique to remap on-screen disparities for viewer comfort.

Disparity remapping for handheld 3D video communications

We describe a new method for High Dynamic Range (HDR) Video using alternating exposures that adds no additional cost or bandwidth requirements to individual IP cameras, making it suitable for large scale security and surveillance systems. Sufficient dynamic range is crucial to the efficacy of a surveillance system, as saturated pixels mean a camera can no longer “see” its surrounding environment. High costs associated with hardware for improved dynamic range make them unsuitable for very large networks with hundreds or even thousands of cameras. We outline a scalable software method that uses post-processing to combine the information in adjacent frames of a video sequence captured with alternating short and long exposures. In particular, we introduce a novel bi-directional motion estimation module that utilizes block-based motion vectors to register frames with large differences in global brightness and fast local motion within saturated regions. An HDR post-processing solution can be deployed at a central location to process individual camera streams on an “as needed” basis, removing additional costs at the device-end. Furthermore, cameras continue to transmit low dynamic range frames, so there is no additional bandwidth requirement. Results show significant gains in video quality for inexpensive cameras when exposed to brightness variations common in security and surveillance.

http://vivonets.ece.ucsb.edu/mangiat_milcom2011.pdf

Stephen Mangiat

Papers

High dynamic range video with ghost removal

Spatially adaptive filtering for registration artifact removal in HDR video

Feature fusion and redundancy pruning for rush video summarization

Disparity remapping for handheld 3D video communications

Inexpensive High Dynamic Range Video for large scale security and surveillance