Humans perceive the three-dimensional structure of the world with apparent ease. However, despite all of the recent advances in computer vision research, the dream of having a computer interpret an image at the same level as a two-year old remains elusive. Why is computer vision such a challenging problem and what is the current state of the art? Computer Vision: Algorithms and Applications explores the variety of techniques commonly used to analyze and interpret images. It also describes challenging real-world applications where vision is being successfully used, both for specialized applications such as medical imaging, and for fun, consumer-level tasks such as image editing and stitching, which students can apply to their own personal photos and videos. More than just a source of recipes, this exceptionally authoritative and comprehensive textbook/reference also takes a scientific approach to basic vision problems, formulating physical models of the imaging process before inverting them to produce descriptions of a scene. These problems are also analyzed using statistical models and solved using rigorous engineering techniques Topics and features: structured to support active curricula and project-oriented courses, with tips in the Introduction for using the book in a variety of customized courses; presents exercises at the end of each chapter with a heavy emphasis on testing algorithms and containing numerous suggestions for small mid-term projects; provides additional material and more detailed mathematical topics in the Appendices, which cover linear algebra, numerical techniques, and Bayesian estimation theory; suggests additional reading at the end of each chapter, including the latest research in each sub-field, in addition to a full Bibliography at the end of the book; supplies supplementary course material for students at the associated website, http://szeliski.org/Book/. Suitable for an upper-level undergraduate or graduate-level course in computer science or engineering, this textbook focuses on basic techniques that work under real-world conditions and encourages students to push their creative boundaries. Its design and exposition also make it eminently suitable as a unique reference to the fundamental techniques and current research literature in computer vision.

/pdf/computer-vision-algorithms-and-applications-25dn6wu83j.pdf

Computer Vision: Algorithms and Applications

“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

Point clouds provide a flexible geometric representation suitable for countless applications in computer graphics; they also comprise the raw output of most 3D data acquisition devices. While hand-designed features on point clouds have long been proposed in graphics and vision, however, the recent overwhelming success of convolutional neural networks (CNNs) for image analysis suggests the value of adapting insight from CNN to the point cloud world. Point clouds inherently lack topological information, so designing a model to recover topology can enrich the representation power of point clouds. To this end, we propose a new neural network module dubbed EdgeConv suitable for CNN-based high-level tasks on point clouds, including classification and segmentation. EdgeConv acts on graphs dynamically computed in each layer of the network. It is differentiable and can be plugged into existing architectures. Compared to existing modules operating in extrinsic space or treating each point independently, EdgeConv has several appealing properties: It incorporates local neighborhood information; it can be stacked applied to learn global shape properties; and in multi-layer systems affinity in feature space captures semantic characteristics over potentially long distances in the original embedding. We show the performance of our model on standard benchmarks, including ModelNet40, ShapeNetPart, and S3DIS.

/pdf/dynamic-graph-cnn-for-learning-on-point-clouds-20wlcru9tg.pdf

Dynamic Graph CNN for Learning on Point Clouds

The PASCAL Visual Object Classes Challenge

We introduce Bootstrap Your Own Latent (BYOL), a new approach to self-supervised image representation learning. BYOL relies on two neural networks, referred to as online and target networks, that interact and learn from each other. From an augmented view of an image, we train the online network to predict the target network representation of the same image under a different augmented view. At the same time, we update the target network with a slow-moving average of the online network. While state-of-the art methods rely on negative pairs, BYOL achieves a new state of the art without them. BYOL reaches $74.3\%$ top-1 classification accuracy on ImageNet using a linear evaluation with a ResNet-50 architecture and $79.6\%$ with a larger ResNet. We show that BYOL performs on par or better than the current state of the art on both transfer and semi-supervised benchmarks. Our implementation and pretrained models are given on GitHub.

Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning

Learning without prejudice: Avoiding bias in webly-supervised action recognition

This paper presents recent and ongoing hardware and software upgrades to our humanoid robot LOLA. The purpose of these modifications is to achieve dynamic multi-contact locomotion, i. e., fast bipedal walking with additional hand-environment support for increased stability and robustness against unforeseen disturbances. The upper body of LOLA has been completely redesigned with an enhanced lightweight torso frame and more robust arms with additional degrees of freedom, which extend the reachable workspace. The mechanical structure of the torso is optimized for stiffness with the help of an experimental modal analysis performed on the real hardware, while the new arm topology is the result of kinematic optimization for typical use-cases in a multi-contact setting. We also propose extensive changes to our software framework, which include a complete redesign of the onboard, real-time perception and navigation module. Although the hardware upgrade is finished and the overall software design is complete, the implementation of various modules is still work in progress.

LOLA v1.1 – An Upgrade in Hardware and Software Design for Dynamic Multi-Contact Locomotion

This paper presents a novel statistical method for background subtraction aimed at robustness with regards to common disturbance factors such as sudden illumination changes, variations of the camera parameters, noise The proposed approach relies on a novel non-linear parametric model for the local effect of disturbance factors on a neighbourhood of pixel intensities Assuming additive gaussian noise, we also propose Bayesian estimation of model parameters by means of a maximum-a-posteriori regression and a statistical change detection test Experimental results demonstrate that the proposed approach is state-of-the-art in sequences where disturbance factors yield linear as well as non-linear intensity transformations

/pdf/non-linear-parametric-bayesian-regression-for-robust-4ywc43urad.pdf

Non-linear parametric Bayesian regression for robust background subtraction

Lightweight and semantically meaningful environment maps are crucial for many applications in robotics and autonomous driving to facilitate higher-level tasks such as navigation and planning. In this paper we present a novel approach to incrementally build a meaningful and lightweight semantic map directly as a 3D mesh from a monocular or stereo sequence. Our system leverages existing feature-based visual odometry paired with learned depth prediction and semantic image segmentation to identify and reconstruct semantically relevant environment structure. We introduce a probabilistic fusion scheme to incrementally refine and extend a 3D mesh with semantic labels for each face without intermediate voxel-based fusion. To demonstrate its effectiveness, we evaluate our system in outdoor driving scenarios with monocular depth prediction and stereo and present quantitative and qualitative reconstruction results with comparison to ground truth. Our results show that the proposed approach achieves reconstruction quality comparable to current state-of-the-art voxel-based methods while being much more lightweight both in storage and computation.

Lightweight Semantic Mesh Mapping for Autonomous Vehicles

Novel view synthesis from a single image aims at generating novel views from a single input image of an object Several works recently achieved remarkable results, though require some form of multi-view supervision at training time, therefore limiting their deployment in real scenarios This work aims at relaxing this assumption enabling training of conditional generative model for novel view synthesis in a completely unsupervised manner We first pre-train a purely generative decoder model using a GAN formulation while at the same time training an encoder network to invert the mapping from latent code to images Then we swap encoder and decoder and train the network as a conditioned GAN with a mixture of auto-encoder-like objective and self-distillation At test time, given a view of an object, our model first embeds the image content in a latent code and regresses its pose wrt a canonical reference system, then generates novel views of it by keeping the code and varying the pose We show that our framework achieves results comparable to the state of the art on ShapeNet and that it can be employed on unconstrained collections of natural images, where no competing method can be trained

Federico Tombari

Papers

Learning without prejudice: Avoiding bias in webly-supervised action recognition

LOLA v1.1 – An Upgrade in Hardware and Software Design for Dynamic Multi-Contact Locomotion

Non-linear parametric Bayesian regression for robust background subtraction

Lightweight Semantic Mesh Mapping for Autonomous Vehicles

Unsupervised Novel View Synthesis from a Single Image