Planning algorithms are impacting technical disciplines and industries around the world, including robotics, computer-aided design, manufacturing, computer graphics, aerospace applications, drug design, and protein folding. This coherent and comprehensive book unifies material from several sources, including robotics, control theory, artificial intelligence, and algorithms. The treatment is centered on robot motion planning but integrates material on planning in discrete spaces. A major part of the book is devoted to planning under uncertainty, including decision theory, Markov decision processes, and information spaces, which are the “configuration spaces” of all sensor-based planning problems. The last part of the book delves into planning under differential constraints that arise when automating the motions of virtually any mechanical system. Developed from courses taught by the author, the book is intended for students, engineers, and researchers in robotics, artificial intelligence, and control theory as well as computer graphics, algorithms, and computational biology.

Planning Algorithms: Introductory Material

Rendering the semantic content of an image in different styles is a difficult image processing task. Arguably, a major limiting factor for previous approaches has been the lack of image representations that explicitly represent semantic information and, thus, allow to separate image content from style. Here we use image representations derived from Convolutional Neural Networks optimised for object recognition, which make high level image information explicit. We introduce A Neural Algorithm of Artistic Style that can separate and recombine the image content and style of natural images. The algorithm allows us to produce new images of high perceptual quality that combine the content of an arbitrary photograph with the appearance of numerous wellknown artworks. Our results provide new insights into the deep image representations learned by Convolutional Neural Networks and demonstrate their potential for high level image synthesis and manipulation.

/pdf/image-style-transfer-using-convolutional-neural-networks-xq8m0q3iz5.pdf

Image Style Transfer Using Convolutional Neural Networks

In this paper, we propose a novel explicit image filter called guided filter. Derived from a local linear model, the guided filter computes the filtering output by considering the content of a guidance image, which can be the input image itself or another different image. The guided filter can be used as an edge-preserving smoothing operator like the popular bilateral filter [1], but it has better behaviors near edges. The guided filter is also a more generic concept beyond smoothing: It can transfer the structures of the guidance image to the filtering output, enabling new filtering applications like dehazing and guided feathering. Moreover, the guided filter naturally has a fast and nonapproximate linear time algorithm, regardless of the kernel size and the intensity range. Currently, it is one of the fastest edge-preserving filters. Experiments show that the guided filter is both effective and efficient in a great variety of computer vision and computer graphics applications, including edge-aware smoothing, detail enhancement, HDR compression, image matting/feathering, dehazing, joint upsampling, etc.

/pdf/guided-image-filtering-5atmpsksoa.pdf

Guided Image Filtering

We present a system for accurate real-time mapping of complex and arbitrary indoor scenes in variable lighting conditions, using only a moving low-cost depth camera and commodity graphics hardware. We fuse all of the depth data streamed from a Kinect sensor into a single global implicit surface model of the observed scene in real-time. The current sensor pose is simultaneously obtained by tracking the live depth frame relative to the global model using a coarse-to-fine iterative closest point (ICP) algorithm, which uses all of the observed depth data available. We demonstrate the advantages of tracking against the growing full surface model compared with frame-to-frame tracking, obtaining tracking and mapping results in constant time within room sized scenes with limited drift and high accuracy. We also show both qualitative and quantitative results relating to various aspects of our tracking and mapping system. Modelling of natural scenes, in real-time with only commodity sensor and GPU hardware, promises an exciting step forward in augmented reality (AR), in particular, it allows dense surfaces to be reconstructed in real-time, with a level of detail and robustness beyond any solution yet presented using passive computer vision.

/pdf/kinectfusion-real-time-dense-surface-mapping-and-tracking-pl8e8pbmre.pdf

KinectFusion: Real-time dense surface mapping and tracking

Humans perceive the three-dimensional structure of the world with apparent ease. However, despite all of the recent advances in computer vision research, the dream of having a computer interpret an image at the same level as a two-year old remains elusive. Why is computer vision such a challenging problem and what is the current state of the art? Computer Vision: Algorithms and Applications explores the variety of techniques commonly used to analyze and interpret images. It also describes challenging real-world applications where vision is being successfully used, both for specialized applications such as medical imaging, and for fun, consumer-level tasks such as image editing and stitching, which students can apply to their own personal photos and videos. More than just a source of recipes, this exceptionally authoritative and comprehensive textbook/reference also takes a scientific approach to basic vision problems, formulating physical models of the imaging process before inverting them to produce descriptions of a scene. These problems are also analyzed using statistical models and solved using rigorous engineering techniques Topics and features: structured to support active curricula and project-oriented courses, with tips in the Introduction for using the book in a variety of customized courses; presents exercises at the end of each chapter with a heavy emphasis on testing algorithms and containing numerous suggestions for small mid-term projects; provides additional material and more detailed mathematical topics in the Appendices, which cover linear algebra, numerical techniques, and Bayesian estimation theory; suggests additional reading at the end of each chapter, including the latest research in each sub-field, in addition to a full Bibliography at the end of the book; supplies supplementary course material for students at the associated website, http://szeliski.org/Book/. Suitable for an upper-level undergraduate or graduate-level course in computer science or engineering, this textbook focuses on basic techniques that work under real-world conditions and encourages students to push their creative boundaries. Its design and exposition also make it eminently suitable as a unique reference to the fundamental techniques and current research literature in computer vision.

/pdf/computer-vision-algorithms-and-applications-25dn6wu83j.pdf

Computer Vision: Algorithms and Applications

Volume rendering not only produces high quality 3D images of single organs directly from the intensity data in CT, MR, SPECT, PET, and ultrasound images, but it also can be used to show the relationship among multiple anatomic, treatment, and image objects. In this paper we will explain methods, show results, and discuss the effectiveness of 1. simultaneously volume rendering multiple organs from a single image data set, using transparency and color; 2. rendering polygonally defined treatment objects such as prostheses and radiation treatment beams with volume rendered anatomy. We will also discuss ongoing work in 1. rendering multiple image sets, e.g., radiation dose distributions and medical image data, into a single image via volume rendering, e.g., to show the relation between isodose surfaces and anatomic objects; 2. texture mapping grey scale slices onto clipping planes on volume rendered anatomy, to show the relationship between physiological and anatomic data, or to show the subtle anatomic intensity variations in its 3D context.

Volume Rendering For Display Of Multiple Organs, Treatment Objects, And Image Intensities

Images are detected in a manner that addresses various challenges as discussed herein. As may be consistent with one or more embodiments, aspects are directed to an apparatus having sets of photosensors that detect light rays received at different angles from a specimen via a microlens array, with the light rays detected by each set of photosensors representing an aliased view of the specimen. An output indicative of aliased views of the specimen is provided. Certain embodiments further include a logic circuit that processes the output and generates a deconvolved volume image by combining aliased views of the specimen as detected by the photosensors.

Optics apparatus with detection of light rays received at different angles for output indicative of aliased views

Many computational photography techniques take the form, ”Capture a burst of images varying camera setting X (exposure, gain, focus, lighting), then align and combine them to produce a single photograph exhibiting better Y (dynamic range, signal-to-noise, depth of field). Unfortunately, these techniques may fail on moving scenes because the images are captured sequentially, so objects are in different positions in each image, and robust local alignment is difficult to achieve. To overcome this limitation, we propose using multi-bucket sensors, which allow the images to be captured in time-slice-interleaved fashion. This interleaving produces images with nearly identical positions for moving objects, making alignment unnecessary. To test our proposal, we have designed and fabricated a 4-bucket, VGA-resolution CMOS image sensor, and we have applied it to high dynamic range (HDR) photography. Our sensor permits 4 different exposures to be captured at once with no motion difference between the exposures. Also, since our protocol employs non-destructive analog addition of time slices, it requires

/pdf/applications-of-multi-bucket-sensors-to-computational-50omydyt3o.pdf

Applications of Multi-Bucket Sensors to Computational Photography

The light field is a four-dimensional function representing radiance along rays. By inserting a microlens array into the optical train of an ordinary microscope, we can record these light fields. From these we can generate perspective views, refocused images, focal stacks, and volume data. Inserting a similar array into a microscope’s illumination path, we can control the light falling on a specimen in space and angle. This control can be used to simulate exotic microscope illumination modalities, to increase the contrast of ordinary fluorescence microscopy, and to correct digitally for optical aberrations in a microscope imaging system.

Light field photography, microscopy, and illumination

The present disclosure relates to methods and systems that may reduce pixel noise due to defective sensor elements in optical imaging systems. Namely, a camera may capture a burst of images with an image sensor while adjusting a focus distance setting of an optical element. For example, the image burst may be captured during an autofocus process. The plurality of images may be averaged or otherwise merged to provide a single, aggregate image frame. Such an aggregate image frame may appear blurry. In such a scenario, “hot” pixels, “dead” pixels, or otherwise defective pixels may be more easily recognized and/or corrected. As an example, a defective pixel may be removed from a target image or otherwise corrected by replacing a value of the defective pixel with an average value of neighboring pixels.

Marc Levoy

Papers

Volume Rendering For Display Of Multiple Organs, Treatment Objects, And Image Intensities

Optics apparatus with detection of light rays received at different angles for output indicative of aliased views

Applications of Multi-Bucket Sensors to Computational Photography

Light field photography, microscopy, and illumination

Calibration of defective image sensor elements