In 2008, Zhou et al. presented a survey paper summarizing the previous ten years of ISMAR publications, which provided invaluable insights into the research challenges and trends associated with that time period. Ten years later, we review the research that has been presented at ISMAR conferences since the survey of Zhou et al., at a time when both academia and the AR industry are enjoying dramatic technological changes. Here we consider the research results and trends of the last decade of ISMAR by carefully reviewing the ISMAR publications from the period of 2008–2017, in the context of the first ten years. The numbers of papers for different research topics and their impacts by citations were analyzed while reviewing them—which reveals that there is a sharp increase in AR evaluation and rendering research. Based on this review we offer some observations related to potential future research areas or trends, which could be helpful to AR researchers and industry members looking ahead.

https://par.nsf.gov/servlets/purl/10105851

Revisiting Trends in Augmented Reality Research: A Review of the 2nd Decade of ISMAR (2008–2017)

Physical buttons have the unique ability to provide low-attention and vision-free interactions through their intuitive tactile clues. Unfortunately, the physicality of these interfaces makes them static, limiting the number and types of user interfaces they can support. On the other hand, touch screen technologies provide the ultimate interface flexibility, but offer no inherent tactile qualities. In this paper, we describe a technique that seeks to occupy the space between these two extremes - offering some of the flexibility of touch screens, while retaining the beneficial tactile properties of physical interfaces. The outcome of our investigations is a visual display that contains deformable areas, able to produce physical buttons and other interface elements. These tactile features can be dynamically brought into and out of the interface, and otherwise manipulated under program control. The surfaces we describe provide the full dynamics of a visual display (through rear projection) as well as allowing for multitouch input (though an infrared lighting and camera setup behind the display). To illustrate the tactile capabilities of the surfaces, we describe a number of variations we uncovered in our exploration and prototyping. These go beyond simple on/off actuation and can be combined to provide a range of different possible tactile expressions. A preliminary user study indicates that our dynamic buttons perform much like physical buttons in tactile search tasks.

/pdf/providing-dynamically-changeable-physical-buttons-on-a-4co1i96w6d.pdf

Providing dynamically changeable physical buttons on a visual display

In this chapter, we first present a summary of findings from two previous studies on the limitations of using flat displays with embodied conversational agents (ECAs) in the contexts of face-to-face human-agent interaction. We then motivate the need for a three dimensional display of faces to guarantee accurate delivery of gaze and directional movements and present Furhat, a novel, simple, highly effective, and human-like back-projected robot head that utilizes computer animation to deliver facial movements, and is equipped with a pan-tilt neck. After presenting a detailed summary on why and how Furhat was built, we discuss the advantages of using optically projected animated agents for interaction. We discuss using such agents in terms of situatedness, environment, context awareness, and social, human-like face-to-face interaction with robots where subtle nonverbal and social facial signals can be communicated. At the end of the chapter, we present a recent application of Furhat as a multimodal multiparty interaction system that was presented at the London Science Museum as part of a robot festival,. We conclude the paper by discussing future developments, applications and opportunities of this technology.

/pdf/furhat-a-back-projected-human-like-robot-head-for-multiparty-2tnwk4twds.pdf

Furhat: a back-projected human-like robot head for multiparty human-machine interaction

A system that produces a dynamic haptic effect and generates a drive signal that includes a gesture signal and a real or virtual device sensor signal. The haptic effect is modified dynamically based on both the gesture signal and the real or virtual device sensor signal such as from an accelerometer or gyroscope, or by a signal created from processing data such as still images, video or sound. The haptic effect may optionally be modified dynamically by using the gesture signal and the real or virtual device sensor signal and a physical model, or may optionally be applied concurrently to multiple devices which are connected via a communication link. The haptic effect may optionally be encoded into a data file on a first device. The data file is then communicated to a second device and the haptic effect is read from the data file and applied to the second device.

Interactivity model for shared feedback on mobile devices

We introduce CrossY, a simple drawing application developed as a benchmark to demonstrate the feasibility of goal crossing as the basis for a graphical user interface. We show that crossing is not only as expressive as the current point-and-click interface, but also offers more flexibility in interaction design. In particular, crossing encourages the fluid composition of commands which supports the development of more fluid interfaces. While crossing was previously identified as a potential substitute for the classic point-and-click interaction, this work is the first to report on the practical aspects of implementing an interface based on goal crossing as the fundamental building block.

CrossY: a crossing-based drawing application

Applications such as telepresence and training involve the display of real or synthetic humans to multiple viewers. When attempting to render the humans with conventional displays, non-verbal cues such as head pose, gaze direction, body posture, and facial expression are difficult to convey correctly to all viewers. In addition, a framed image of a human conveys only a limited physical sense of presence—primarily through the display's location. While progress continues on articulated robots that mimic humans, the focus has been on the motion and behavior of the robots. We introduce a new approach for robotic avatars of real people: the use of cameras and projectors to capture and map the dynamic motion and appearance of a real person onto a humanoid animatronic model. We call these devices animatronic Shader Lamps Avatars (SLA).We present a proof-of-concept prototype comprised of a camera, a tracking system, a digital projector, and a life-sized styrofoam head mounted on a pan-tilt unit. The system captures imagery of a moving, talking user and maps the appearance and motion onto the animatronic SLA, delivering a dynamic, real-time representation of the user to multiple viewers.

/pdf/animatronic-shader-lamps-avatars-1p4l4irszj.pdf

Animatronic Shader Lamps Avatars

This paper presents a technique to add the tactile cues of real buttons to virtual buttons displayed on mobile devices with touch screens. When the user's finger is on the display, tactile feedback coveys a feeling of button location and activation. We describe two implementations of the technique, using a personal digital assistant (PDA) and a pressure sensitive tablet.

Tactile virtual buttons for mobile devices

Methods, systems, and computer readable media for generating autostereo three-dimensional views of a scene for a plurality of viewpoints are disclosed. According to one system, a display is configured to display images from plural different viewpoints using a barrier located in front of the display, where the barrier has a pseudo-random arrangement of light ports through which images on the display are viewable. A renderer coupled to the display simultaneously renders images from the different viewpoints such that pixels that should appear differently from the different viewpoints are displayed in a predetermined manner. The pseudo-random arrangement of the light ports in the barrier smoothes interference between the different viewpoints as perceived by viewers located at the different viewpoints.

Methods, systems, and computer readable media for generating autostereo three-dimensional views of a scene for a plurality of viewpoints using a pseudo-random hole barrier

Methods, systems, and computer readable media for shader lamps-based avatars of real and virtual people are disclosed. According to one method, shader lamps-based avatars of real and virtual objects are displayed on physical target objects. The method includes obtaining visual information of a source object and generating at least a first data set of pixels representing a texture image of the source object. At least one of a size, shape, position, and orientation of a 3D physical target object are determined. A set of coordinate data associated with various locations on the surface of the target object are also determined. The visual information is mapped to the physical target object. Mapping includes defining a relationship between the first and second sets of data, wherein each element of the first set is related to each element of the second set. The mapped visual information is displayed on the physical target object using a display module, such as one or more projectors located at various positions around the target object.

Methods, systems, and computer readable media for shader-lamps based physical avatars of real and virtual people

Applications such as telepresence and training involve the display of real or synthetic humans to multiple viewers. When attempting to render the humans with conventional displays, non-verbal cues such as head pose, gaze direction, body posture, and facial expression are difficult to convey correctly to all viewers. In addition, a framed image of a human conveys only a limited physical sense of presence—primarily through the display’s location. While progress continues on articulated robots that mimic humans, the focus has been on the motion and behavior of the robots rather than on their appearance. We introduce a new approach for robotic avatars of real people: the use of cameras and projectors to capture and map both the dynamic motion and the appearance of a real person onto a humanoid animatronic model. We call these devices animatronic Shader Lamps Avatars (SLA). We present a proof-of-concept prototype comprised of a camera, a tracking system, a digital projector, and a life-sized styrofoam head mounted on a pan-tilt unit. The system captures imagery of a moving, talking user and maps the appearance and motion onto the animatronic SLA, delivering a dynamic, real-time representation of the user to multiple viewers.

/pdf/animatronic-shader-lamps-avatars-57h7w15l0j.pdf

Andrew Nashel

Papers

Animatronic Shader Lamps Avatars

Tactile virtual buttons for mobile devices

Methods, systems, and computer readable media for generating autostereo three-dimensional views of a scene for a plurality of viewpoints using a pseudo-random hole barrier

Methods, systems, and computer readable media for shader-lamps based physical avatars of real and virtual people

Animatronic shader lamps avatars