Showing papers by "Takeo Kanade published in 2019"
TL;DR: The Panoptic Studio system and method are the first in reconstructing full body motion of more than five people engaged in social interactions without using markers, and empirically demonstrate the impact of the number of views in achieving this goal.
Abstract: We present an approach to capture the 3D motion of a group of people engaged in a social interaction. The core challenges in capturing social interactions are: (1) occlusion is functional and frequent; (2) subtle motion needs to be measured over a space large enough to host a social group; (3) human appearance and configuration variation is immense; and (4) attaching markers to the body may prime the nature of interactions. The Panoptic Studio is a system organized around the thesis that social interactions should be measured through the integration of perceptual analyses over a large variety of view points. We present a modularized system designed around this principle, consisting of integrated structural, hardware, and software innovations. The system takes, as input, 480 synchronized video streams of multiple people engaged in social activities, and produces, as output, the labeled time-varying 3D structure of anatomical landmarks on individuals in the space. Our algorithm is designed to fuse the “weak” perceptual processes in the large number of views by progressively generating skeletal proposals from low-level appearance cues, and a framework for temporal refinement is also presented by associating body parts to reconstructed dense 3D trajectory stream. Our system and method are the first in reconstructing full body motion of more than five people engaged in social interactions without using markers. We also empirically demonstrate the impact of the number of views in achieving this goal.
279 citations
Patent•
11 Jun 2019
TL;DR: In this article, a system for analyzing images of objects such as vehicles is proposed, which includes a user interface device configured to capture a set of images depicting a target vehicle, and transfer the set of image to a server that stores a base image models.
Abstract: A system for analyzing images of objects such as vehicles. According to certain aspects, the system includes a user interface device configured to capture a set of images depicting a target vehicle, and transfer the set of images to a server that stores a set of base image models. The server analyzes the set of images using a base image model corresponding to the target vehicle, a set of correlational filters, and a set of convolutional neural networks (CNNs) to determine a set of changes to the target vehicle as depicted in the set of images. The server further transmits, to the user interface device, information indicative of the set of changes for a user to view or otherwise access.
10 citations
Posted Content•
TL;DR: This work is the first to be able to make one GAN framework work on all such object reshaping tasks, especially the cross-domain tasks on handling multiple diverse datasets and comparisons with the state-of-the-art models when they are made comparable.
Abstract: The aim of this work is learning to reshape the object in an input image to an arbitrary new shape, by just simply providing a single reference image with an object instance in the desired shape. We propose a new Generative Adversarial Network (GAN) architecture for such an object reshaping problem, named ReshapeGAN. The network can be tailored for handling all kinds of problem settings, including both within-domain (or single-dataset) reshaping and cross-domain (typically across mutiple datasets) reshaping, with paired or unpaired training data. The appearance of the input object is preserved in all cases, and thus it is still identifiable after reshaping, which has never been achieved as far as we are aware. We present the tailored models of the proposed ReshapeGAN for all the problem settings, and have them tested on 8 kinds of reshaping tasks with 13 different datasets, demonstrating the ability of ReshapeGAN on generating convincing and superior results for object reshaping. To the best of our knowledge, we are the first to be able to make one GAN framework work on all such object reshaping tasks, especially the cross-domain tasks on handling multiple diverse datasets. We present here both ablation studies on our proposed ReshapeGAN models and comparisons with the state-of-the-art models when they are made comparable, using all kinds of applicable metrics that we are aware of.