Course Description This is an advanced course in which we explore the field of Statistical Optics. Topics covered include such subjects as the statistical properties of natural (thermal) and laser light, spatial and temporal coherence, effects of partial coherence on optical imaging instruments, effects on imaging due to randomly inhomogeneous media, and a statistical treatment of the detection of light. Development of this more comprehensive model of the behavior of light draws upon the use of tools traditionally available to the electrical engineer, such as linear system theory and the theory of stochastic processes.

http://ieeexplore.ieee.org/iel5/3/23094/01073023.pdf

Statistical optics

A self-scanned 1024 element photodiode array and minicomputer are used to measure the phase (wavefront) in the interference pattern of an interferometer to lambda/100. The photodiode array samples intensities over a 32 x 32 matrix in the interference pattern as the length of the reference arm is varied piezoelectrically. Using these data the minicomputer synchronously detects the phase at each of the 1024 points by a Fourier series method and displays the wavefront in contour and perspective plot on a storage oscilloscope in less than 1 min (Bruning et al. Paper WE16, OSA Annual Meeting, Oct. 1972). The array of intensities is sampled and averaged many times in a random fashion so that the effects of air turbulence, vibrations, and thermal drifts are minimized. Very significant is the fact that wavefront errors in the interferometer are easily determined and may be automatically subtracted from current or subsequent wavefrots. Various programs supporting the measurement system include software for determining the aperture boundary, sum and difference of wavefronts, removal or insertion of tilt and focus errors, and routines for spatial manipulation of wavefronts. FFT programs transform wavefront data into point spread function and modulus and phase of the optical transfer function of lenses. Display programs plot these functions in contour and perspective. The system has been designed to optimize the collection of data to give higher than usual accuracy in measuring the individual elements and final performance of assembled diffraction limited optical systems, and furthermore, the short loop time of a few minutes makes the system an attractive alternative to constraints imposed by test glasses in the optical shop.

Digital wavefront measuring interferometer for testing optical surfaces and lenses

Humans move their hands and bodies together to communicate and solve tasks. Capturing and replicating such coordinated activity is critical for virtual characters that behave realistically. Surprisingly, most methods treat the 3D modeling and tracking of bodies and hands separately. Here we formulate a model of hands and bodies interacting together and fit it to full-body 4D sequences. When scanning or capturing the full body in 3D, hands are small and often partially occluded, making their shape and pose hard to recover. To cope with low-resolution, occlusion, and noise, we develop a new model called MANO (hand Model with Articulated and Non-rigid defOrmations). MANO is learned from around 1000 high-resolution 3D scans of hands of 31 subjects in a wide variety of hand poses. The model is realistic, low-dimensional, captures non-rigid shape changes with pose, is compatible with standard graphics packages, and can fit any human hand. MANO provides a compact mapping from hand poses to pose blend shape corrections and a linear manifold of pose synergies. We attach MANO to a standard parameterized 3D body shape model (SMPL), resulting in a fully articulated body and hand model (SMPL+H). We illustrate SMPL+H by fitting complex, natural, activities of subjects captured with a 4D scanner. The fitting is fully automatic and results in full body models that move naturally with detailed hand motions and a realism not seen before in full body performance capture. The models and data are freely available for research purposes at http://mano.is.tue.mpg.de.

Embodied hands: modeling and capturing hands and bodies together

Eye tracking has a long history in medical and psychological research as a tool for recording and studying human visual behavior. Real-time gaze-based text entry can also be a powerful means of communication and control for people with physical disabilities. Following recent technological advances and the advent of affordable eye trackers, there is a growing interest in pervasive attention-aware systems and interfaces that have the potential to revolutionize mainstream human-technology interaction. In this chapter, we provide an introduction to the state-of-the art in eye tracking technology and gaze estimation. We discuss challenges involved in using a perceptual organ, the eye, as an input modality. Examples of real life applications are reviewed, together with design solutions derived from research results. We also discuss how to match the user requirements and key features of different eye tracking systems to find the best system for each task and application.

Eye Tracking and Eye-Based Human–Computer Interaction

Large datasets are the cornerstone of recent advances in computer vision using deep learning. In contrast, existing human motion capture (mocap) datasets are small and the motions limited, hampering progress on learning models of human motion. While there are many different datasets available, they each use a different parameterization of the body, making it difficult to integrate them into a single meta dataset. To address this, we introduce AMASS, a large and varied database of human motion that unifies 15 different optical marker-based mocap datasets by representing them within a common framework and parameterization. We achieve this using a new method, MoSh++, that converts mocap data into realistic 3D human meshes represented by a rigged body model. Here we use SMPL [Loper et al., 2015], which is widely used and provides a standard skeletal representation as well as a fully rigged surface mesh. The method works for arbitrary marker sets, while recovering soft-tissue dynamics and realistic hand motion. We evaluate MoSh++ and tune its hyperparameters using a new dataset of 4D body scans that are jointly recorded with markerbased mocap. The consistent representation of AMASS makes it readily useful for animation, visualization, and generating training data for deep learning. Our dataset is significantly richer than previous human motion collections, having more than 40 hours of motion data, spanning over 300 subjects, more than 11000 motions, and will be publicly available to the research community.

/pdf/amass-archive-of-motion-capture-as-surface-shapes-4301lyij8c.pdf

AMASS: Archive of Motion Capture As Surface Shapes

We present a digital signal processing technique that reduces the speckle content in reconstructed digital holograms. The method is based on sequential sampling of the discrete Fourier transform of the reconstructed image field. Speckle reduction is achieved at the expense of a reduced intensity and resolution, but this trade-off is shown to be greatly superior to that imposed by the traditional mean and median filtering techniques. In particular, we show that the speckle can be reduced by half with no loss of resolution (according to standard definitions of both metrics).

http://www.cs.may.ie/~tnaughton/pubs/varasto/MHMFCJN2007p.pdf

Reduction of speckle in digital holography by discrete Fourier filtering.

We propose a task-specific digital holographic capture system for three-dimensional scenes, which can reduce the amount of data sent from the camera system to the receiver and can effectively reconstruct partially occluded objects. The system requires knowledge of the object of interest, but it does not require a priori knowledge of either the occlusion or the distance the object is from the camera. Subwindows of the camera-plane Fresnel field are digitally propagated to reveal different perspectives of the scene, and these are combined to overcome the unknown foreground occlusions. The nature of the occlusions and the effect of subwindows are analyzed thoroughly by using the Wigner distribution function. We demonstrate that a careful combination of reconstructions from subwindows can reveal features that are not apparent in a reconstruction from the whole hologram. We provide results by using optically captured digital holograms of real-world objects and simulated occlusions.

/pdf/reconstruction-of-partially-occluded-objects-encoded-in-3rqwyn5sna.pdf

Reconstruction of partially occluded objects encoded in three-dimensional scenes by using digital holograms

We present a method for real-time bare hand tracking that utilizes natural hand synergies to reduce the complexity and improve the plausibility of the hand posture estimation. The hand pose and posture are estimated by fitting a virtual hand model to the 3D point cloud obtained from a Kinect camera using an inverse kinematics approach. We use real human hand movements captured with a Vicon motion tracking system as the ground truth for deriving natural hand synergies based on principal component analysis. These synergies are integrated in the tracking scheme by optimizing the posture in a reduced parameter space. Tracking in this reduced space combined with joint limit avoidance constrains the posture estimation to natural hand articulations. The information loss associated with dimension reduction can be dealt with by employing a hierarchical optimization scheme. We show that our synergistic hand tracking approach improves runtime performance and increases the quality of the posture estimation.

Real-time hand tracking using synergistic inverse kinematics

Grasping and manual interaction for robots so far has largely been approached with an emphasis on physics and control aspects. Given the richness of human manual interaction, we argue for the consideration of the wider field of “manual intelligence” as a perspective for manual action research that brings the cognitive nature of human manual skills to the foreground. We briefly sketch part of a research agenda along these lines, argue for the creation of a manual interaction database as an important cornerstone of such an agenda, and describe the manual interaction lab recently set up at CITEC to realize this goal and to connect the efforts of robotics and cognitive science researchers towards making progress for a more integrated understanding of manual intelligence.

/pdf/approaching-manual-intelligence-37nyg6warz.pdf

Approaching Manual Intelligence

We extend a recent low cost real-time method of hand tracking and pose estimation in order to control an anthropomorphic robot hand. The approach is data-driven and based on matching the current image of a color-gloved hand with the best fitting image in a database to retrieve the posture. Then, using depth information from a Kinect camera and a color-sensitive iterative closest point-to-triangle algorithm we can very accurately estimate the absolute position and orientation of the hand. The effectiveness of the approach is demonstrated in an application in which we actively control a 20 DOF anthropomorphic robot hand in a manual interaction grasping task.

Jonathan Maycock

Papers

Reduction of speckle in digital holography by discrete Fourier filtering.

Reconstruction of partially occluded objects encoded in three-dimensional scenes by using digital holograms

Real-time hand tracking using synergistic inverse kinematics

Approaching Manual Intelligence

Real-time hand tracking with a color glove for the actuation of anthropomorphic robot hands