Recent advances in image-based 3D human shape estimation have been driven by the significant improvement in representation power afforded by deep neural networks. Although current approaches have demonstrated the potential in real world settings, they still fail to produce reconstructions with the level of detail often present in the input images. We argue that this limitation stems primarily form two conflicting requirements; accurate predictions require large context, but precise predictions require high resolution. Due to memory limitations in current hardware, previous approaches tend to take low resolution images as input to cover large spatial context, and produce less precise (or low resolution) 3D estimates as a result. We address this limitation by formulating a multi-level architecture that is end-to-end trainable. A coarse level observes the whole image at lower resolution and focuses on holistic reasoning. This provides context to an fine level which estimates highly detailed geometry by observing higher-resolution images. We demonstrate that our approach significantly outperforms existing state-of-the-art techniques on single image human shape reconstruction by fully leveraging 1k-resolution input images.

/pdf/pifuhd-multi-level-pixel-aligned-implicit-function-for-high-2sf1hg8i1x.pdf

PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization

Biometric Recognition - Security and Privacy Concerns.

A home entertainment system for video games and other applications includes a main unit and handheld controllers. The handheld controllers sense their own motion by detecting illumination emitted by emitters positioned at either side of a display. The controllers can be plugged into expansion units that customize the overall control interface for particular applications including but not limited to legacy video games.

Video game system with wireless modular handheld controller

A game operating device (controller) includes a longitudinal housing, and a holding portion held by hand to be wrapped by its palm it is formed in the housing. A direction switch is provided on an upper surface at a position where it can be operated by thumb of the hand holding the holding portion, and a start switch and a select switch are provided backward thereof. An X button 46 and a Y button are further arranged in line on the upper surface of the housing. An imaging information arithmetic unit is provided at a front end of the housing in a longitudinal direction in such a manner that an imaging device thereof is exposed from a front-end surface. A concave portion is formed on a lower surface at a position corresponding to the direction switch. The concave portion includes a valley and two inclined surfaces. An A button capable of being operated by index finger of the hand holding the holding portion is provided on the backward inclined surface. By processing an image signal obtained by imaging an infrared ray from LED modules by the imaging device, it is possible to obtain an operation signal varying according to a position and/or attitude of the controller.

Game operating device

Acceleration data which is output from an acceleration sensor is obtained. A rotation motion of an input device around a predetermined direction as a rotation axis is determined by comparing a start point in a two-dimensional coordinate system which is represented by the first acceleration data obtained in a predetermined period, and an end point in the two-dimensional coordinate system which is represented by the last acceleration data obtained in the predetermined period. Coordinate axes of the two-dimensional coordinate system are defined based on components of the two axial directions of the acceleration data, and an origin of the two-dimensional coordinate system represents a value of the acceleration data in the state where no acceleration including the acceleration of gravity acts upon the acceleration sensor. Motion data including at least the determined rotation motion is output.

Accelerometer-based controller

Existing large scale display systems generally adopt an indirect approach to user interaction This is due to the use of standard desktop-oriented devices, such as a mouse on a desk, to control the large wall-sized display By using an infrared laser pointer and an infrared tracking device, a more direct interaction with the large display can be achieved, thereby reducing the cognitive load of the user and improving their mobility The challenge in designing such systems is to allow users to interact with objects on the display naturally and easily Our system addresses this with hotspots, regions surrounding objects of interest, and gestures, movements made with the laser pointer which triggers an action, similar to those found in modern web browsers (eg Mozilla and Opera) Finally, these concepts are demonstrated by an add-in module for Microsoft® PowerPoint® using the NaturalPointTM Smart-NavTM tracking device

/pdf/direct-interaction-with-large-scale-display-systems-using-26r1xa8hcf.pdf

Direct interaction with large-scale display systems using infrared laser tracking devices

This paper focuses on combining front and back device interaction on grasped devices, using touch-based gestures. We designed generic interactions for discrete, continuous, and combined gesture commands that are executed without hand-eye control because the performing fingers are hidden behind a grasped device. We designed the interactions in such a way that the thumb can always be used as a proprioceptive reference for guiding finger movements, applying embodied knowledge about body structure. In a user study, we tested these touch-based interactions for their performance and users' task-load perception. We combined two iPads together back-to-back to form a double-sided touch screen device: the PinchPad. We discuss the main errors that led to a decrease in accuracy, identify stable features that reduce the error rate, and discuss the role of 'body schema' in designing gesture-based interactions where the user cannot see their hands properly.

PinchPad: performance of touch-based gestures while grasping devices

This paper presents a neural network to estimate a detailed depth map of the foreground human in a single RGB image. The result captures geometry details such as cloth wrinkles, which are important in visualization applications. To achieve this goal, we separate the depth map into a smooth base shape and a residual detail shape and design a network with two branches to regress them respectively. We design a training strategy to ensure both base and detail shapes can be faithfully learned by the corresponding network branches. Furthermore, we introduce a novel network layer to fuse a rough depth map and surface normals to further improve the final result. Quantitative comparison with fused `ground truth' captured by real depth cameras and qualitative examples on unconstrained Internet images demonstrate the strength of the proposed method.

/pdf/a-neural-network-for-detailed-human-depth-estimation-from-a-2xrcnv3kt5.pdf

A Neural Network for Detailed Human Depth Estimation From a Single Image

In this paper, we present an interaction technique that supports the use of tablet devices for interaction and collaboration with large displays. Users can interact with a subset of the large workspace on their tablet, while the same area is visualized on the large display as a rectangular frame. We propose an efficient management and navigation interface through the use of an interactive world-in-miniature view and multitouch gestures. Users can intuitively manage their views on their tablets, navigate between different areas of the workspace, or enlarge them for a closer look. Additionally, they can quickly jump to the same view that their collaborators are looking at.

Supporting interaction and collaboration on large displays using tablet devices

We present research that investigates the amount of guidance required by users for precise back-of-device interaction. We explore how pointing effectiveness is influenced by the presence or absence of visual guidance feedback. Participants were asked to select targets displayed on an iPad device, by touching and releasing them from underneath the device. Another iPad was used to detect finger positions from the rear. Results showed that participants were able to select targets as accurately without visual feedback of finger position as they were with it. Additionally, no significant increase in workload was identified when visual feedback was removed. Our results show that users do not require complex techniques to visualize finger position on the rear of device. Visual feedback does not affect any performance parameters, such as effectiveness, perceived performance, and the number of trials needed to select a target. We also outline the implications of our findings and our future work to fully investigate the effect of visual guidance feedback.

/pdf/does-proprioception-guide-back-of-device-pointing-as-well-as-2acjc7qpfk.pdf

Kelvin Cheng

Papers

Direct interaction with large-scale display systems using infrared laser tracking devices

PinchPad: performance of touch-based gestures while grasping devices

A Neural Network for Detailed Human Depth Estimation From a Single Image

Supporting interaction and collaboration on large displays using tablet devices

Does proprioception guide back-of-device pointing as well as vision?