We investigate conditional adversarial networks as a general-purpose solution to image-to-image translation problems. These networks not only learn the mapping from input image to output image, but also learn a loss function to train this mapping. This makes it possible to apply the same generic approach to problems that traditionally would require very different loss formulations. We demonstrate that this approach is effective at synthesizing photos from label maps, reconstructing objects from edge maps, and colorizing images, among other tasks. Moreover, since the release of the pix2pix software associated with this paper, hundreds of twitter users have posted their own artistic experiments using our system. As a community, we no longer hand-engineer our mapping functions, and this work suggests we can achieve reasonable results without handengineering our loss functions either.

/pdf/image-to-image-translation-with-conditional-adversarial-5058t24et2.pdf

Image-to-Image Translation with Conditional Adversarial Networks

We investigate conditional adversarial networks as a general-purpose solution to image-to-image translation problems. These networks not only learn the mapping from input image to output image, but also learn a loss function to train this mapping. This makes it possible to apply the same generic approach to problems that traditionally would require very different loss formulations. We demonstrate that this approach is effective at synthesizing photos from label maps, reconstructing objects from edge maps, and colorizing images, among other tasks. Indeed, since the release of the pix2pix software associated with this paper, a large number of internet users (many of them artists) have posted their own experiments with our system, further demonstrating its wide applicability and ease of adoption without the need for parameter tweaking. As a community, we no longer hand-engineer our mapping functions, and this work suggests we can achieve reasonable results without hand-engineering our loss functions either.

Computer vision applications have come to rely increasingly on superpixels in recent years, but it is not always clear what constitutes a good superpixel algorithm. In an effort to understand the benefits and drawbacks of existing methods, we empirically compare five state-of-the-art superpixel algorithms for their ability to adhere to image boundaries, speed, memory efficiency, and their impact on segmentation performance. We then introduce a new superpixel algorithm, simple linear iterative clustering (SLIC), which adapts a k-means clustering approach to efficiently generate superpixels. Despite its simplicity, SLIC adheres to boundaries as well as or better than previous methods. At the same time, it is faster and more memory efficient, improves segmentation performance, and is straightforward to extend to supervoxel generation.

/pdf/slic-superpixels-compared-to-state-of-the-art-superpixel-5d8sq04am9.pdf

SLIC Superpixels Compared to State-of-the-Art Superpixel Methods

Preparing the books to read every day is enjoyable for many people. However, there are still many people who also don't like reading. This is a problem. But, when you can support others to start reading, it will be better. One of the books that can be recommended for new readers is experience and education. This book is not kind of difficult book to read. It can be read and understand by the new readers.

Experience and Education.

3D shape is a crucial but heavily underutilized cue in today's computer vision systems, mostly due to the lack of a good generic shape representation. With the recent availability of inexpensive 2.5D depth sensors (e.g. Microsoft Kinect), it is becoming increasingly important to have a powerful 3D shape representation in the loop. Apart from category recognition, recovering full 3D shapes from view-based 2.5D depth maps is also a critical part of visual understanding. To this end, we propose to represent a geometric 3D shape as a probability distribution of binary variables on a 3D voxel grid, using a Convolutional Deep Belief Network. Our model, 3D ShapeNets, learns the distribution of complex 3D shapes across different object categories and arbitrary poses from raw CAD data, and discovers hierarchical compositional part representation automatically. It naturally supports joint object recognition and shape completion from 2.5D depth maps, and it enables active object recognition through view planning. To train our 3D deep learning model, we construct ModelNet - a large-scale 3D CAD model dataset. Extensive experiments show that our 3D deep representation enables significant performance improvement over the-state-of-the-arts in a variety of tasks.

/pdf/3d-shapenets-a-deep-representation-for-volumetric-shapes-1ii3phcu91.pdf

3D ShapeNets: A deep representation for volumetric shapes

ThreadTone is an NPR representation of an input image by half-toning using threads on a circle. Current approaches to create ThreadTone paintings greedily draw the chords on the circle. We introduce the concept of chord space, and design a new algorithm to improve the quality of the thread painting. We use an optimization process that estimates the fitness of every chord in the chord space, and an error-diffusion based sampling process that selects a moderate number of chords to produce the output painting. We used an image similarity measure to evaluate the quality of our thread painting and also conducted a user study. Our approach can produce high quality results on portraits, sketches as well as cartoon pictures.

Automatic thread painting generation

Image augmentation techniques apply transformation functions such as rotation, shearing, or color distortion on an input image These augmentations were proven useful in improving neural networks' generalization ability In this paper, we present a novel augmentation operation, InAugment, that exploits image internal statistics The key idea is to copy patches from the image itself, apply augmentation operations on them, and paste them back at random positions on the same image This method is simple and easy to implement and can be incorporated with existing augmentation techniques We test InAugment on two popular datasets -- CIFAR and ImageNet We show improvement over state-of-the-art augmentation techniques Incorporating InAugment with Auto Augment yields a significant improvement over other augmentation techniques (eg, +1% improvement over multiple architectures trained on the CIFAR dataset) We also demonstrate an increase for ResNet50 and EfficientNet-B3 top-1's accuracy on the ImageNet dataset compared to prior augmentation methods Finally, our experiments suggest that training convolutional neural network using InAugment not only improves the model's accuracy and confidence but its performance on out-of-distribution images

InAugment: Improving Classifiers via Internal Augmentation

An image processing method comprising receiving a stream of high-resolution digital images at a computing device, downsampling each high-resolution digital image to create a low-resolution image, making a low-resolution hint map by segmenting the low- resolution image into a low-resolution foreground and a low-resolution background, and defining a low-resolution buffer zone, where each pixel in the low-resolution buffer zone is associated with a foreground parent pixel and a background parent pixel, creating a high- resolution hint map from the low-resolution hint map, determining an α value for each pixel in the high-resolution buffer zone, and recoloring the pixel as a function of α and a recoloring criterion.

Real-time alpha compositing for high-resolution image stream

Consider the geo-localization task of finding the pose of a camera in a large 3D scene from a single image. Most existing CNN-based methods use as input textured images. We aim to experimentally explore whether texture and correlation between nearby images are necessary in a CNN-based solution for the geo-localization task. To do so, we consider lean images, textureless projections of a simple 3D model of a city. They only contain information related to the geometry of the scene viewed (edges, faces, and relative depth). The main contributions of this paper are: (i) to demonstrate the ability of CNNs to recover camera pose using lean images; and (ii) to provide insight into the role of geometry in the CNN learning process.

/pdf/on-the-role-of-geometry-in-geo-localization-3avei3xdjr.pdf

Ariel Shamir

Papers

Automatic thread painting generation

InAugment: Improving Classifiers via Internal Augmentation

Real-time alpha compositing for high-resolution image stream

On the role of geometry in geo-localization

Verfahren zur Bild-Retargierung unter Einbeziehung des Inhalts