Showing papers on "Augmented reality published in 2016"

Holoportation: Virtual 3D Teleportation in Real-time

Abstract: We present an end-to-end system for augmented and virtual reality telepresence, called Holoportation. Our system demonstrates high-quality, real-time 3D reconstructions of an entire space, including people, furniture and objects, using a set of new depth cameras. These 3D models can also be transmitted in real-time to remote users. This allows users wearing virtual or augmented reality displays to see, hear and interact with remote participants in 3D, almost as if they were present in the same physical space. From an audio-visual perspective, communicating and interacting with remote users edges closer to face-to-face communication. This paper describes the Holoportation technical system in full, its key interactive capabilities, the application scenarios it enables, and an initial qualitative study of using this new communication medium.

Pose Estimation for Augmented Reality: A Hands-On Survey

Abstract: Augmented reality (AR) allows to seamlessly insert virtual objects in an image sequence. In order to accomplish this goal, it is important that synthetic elements are rendered and aligned in the scene in an accurate and visually acceptable way. The solution of this problem can be related to a pose estimation or, equivalently, a camera localization process. This paper aims at presenting a brief but almost self-contented introduction to the most important approaches dedicated to vision-based camera localization along with a survey of several extension proposed in the recent years. For most of the presented approaches, we also provide links to code of short examples. This should allow readers to easily bridge the gap between theoretical aspects and practical implementations.

Virtual laboratories for education in science, technology, and engineering

Abstract: Within education, concepts such as distance learning, and open universities, are now becoming more widely used for teaching and learning. However, due to the nature of the subject domain, the teaching of Science, Technology, and Engineering are still relatively behind when using new technological approaches (particularly for online distance learning). The reason for this discrepancy lies in the fact that these fields often require laboratory exercises to provide effective skill acquisition and hands-on experience. Often it is difficult to make these laboratories accessible for online access. Either the real lab needs to be enabled for remote access or it needs to be replicated as a fully software-based virtual lab. We argue for the latter concept since it offers some advantages over remotely controlled real labs, which will be elaborated further in this paper.We are now seeing new emerging technologies that can overcome some of the potential difficulties in this area. These include: computer graphics, augmented reality, computational dynamics, and virtual worlds. This paper summarizes the state of the art in virtual laboratories and virtual worlds in the fields of science, technology, and engineering. The main research activity in these fields is discussed but special emphasis is put on the field of robotics due to the maturity of this area within the virtual-education community. This is not a coincidence; starting from its widely multidisciplinary character, robotics is a perfect example where all the other fields of engineering and physics can contribute. Thus, the use of virtual labs for other scientific and non-robotic engineering uses can be seen to share many of the same learning processes. This can include supporting the introduction of new concepts as part of learning about science and technology, and introducing more general engineering knowledge, through to supporting more constructive (and collaborative) education and training activities in a more complex engineering topic such as robotics. The objective of this paper is to outline this problem space in more detail and to create a valuable source of information that can help to define the starting position for future research. State of the art in dynamics-based virtual laboratories.Defining the criteria for critical evaluation of existing technologies.State of the art in virtual worlds.Future advances in the field of virtual-world based laboratories.

Systematic review on the effectiveness of augmented reality applications in medical training

Abstract: Background Computer-based applications are increasingly used to support the training of medical professionals. Augmented reality applications (ARAs) render an interactive virtual layer on top of reality. The use of ARAs is of real interest to medical education because they blend digital elements with the physical learning environment. This will result in new educational opportunities. The aim of this systematic review is to investigate to which extent augmented reality applications are currently used to validly support medical professionals training.

A comprehensive survey of augmented reality assembly research

Abstract: In the past two decades, augmented reality (AR) has received a growing amount of attention by researchers in the manufacturing technology community, because AR can be applied to address a wide range of problems throughout the assembly phase in the lifecycle of a product, e.g., planning, design, ergonomics assessment, operation guidance and training. However, to the best of authors’ knowledge, there has not been any comprehensive review of AR-based assembly systems. This paper aims to provide a concise overview of the technical features, characteristics and broad range of applications of AR-based assembly systems published between 1990 and 2015. Among these selected articles, two thirds of them were published between 2005 and 2015, and they are considered as recent pertinent works which will be discussed in detail. In addition, the current limitation factors and future trends in the development will also be discussed.

Resolving the Vergence-Accommodation Conflict in Head-Mounted Displays

Abstract: The vergence-accommodation conflict (VAC) remains a major problem in head-mounted displays for virtual and augmented reality (VR and AR). In this review, I discuss why this problem is pivotal for nearby tasks in VR and AR, present a comprehensive taxonomy of potential solutions, address advantages and shortfalls of each design, and cover various ways to better evaluate the solutions. The review describes how VAC is addressed in monocular, stereoscopic, and multiscopic HMDs, including retinal scanning and accommodation-free displays. Eye-tracking-based approaches that do not provide natural focal cues—gaze-guided blur and dynamic stereoscopy—are also covered. Promising future research directions in this area are identified.

Effects of Virtual Reality and Augmented Reality on Visitor Experiences in Museum

Abstract: This study aims to investigate the impact of Virtual Reality (VR) and Augmented Reality (AR) on the overall visitor experience in the context of museum. In tourism research, few attempts have been made to examine factors which enhance visitor experience using new and emerging technologies such as VR & AR respectively, however research on visitor experience in the mixed environment by combining both VR & AR is scarce. In particular, this study examined the impact of social presence on visitor experience in the mixed (VR & AR) environment by applying social presence theory and experience economy theory. Questionnaires administered to 163 museum visitors revealed that social presence in mixed (VR & AR) environments is a strong predictor of four realms of experience economy. Furthermore, all aspects of experience economy, except for esthetic experience, have a significant influence on visitor experience, which consequently induce the tourists’ intention to revisit Geevor museum. The results of this study provide theoretical and managerial implications for adoption of VR & AR technologies in museum.

Learning anatomy via mobile augmented reality: Effects on achievement and cognitive load.

Abstract: Augmented reality (AR), a new generation of technology, has attracted the attention of educators in recent years. In this study, a MagicBook was developed for a neuroanatomy topic by using mobile augmented reality (mAR) technology. This technology integrates virtual learning objects into the real world and allow users to interact with the environment using mobile devices. The purpose of this study was to determine the effects of learning anatomy via mAR on medical students' academic achievement and cognitive load. The mixed method was applied in the study. The random sample consisted of 70 second-year undergraduate medical students: 34 in an experimental group and 36 in a control group. Academic achievement test and cognitive load scale were used as data collection tool. A one-way MANOVA test was used for analysis. The experimental group, which used mAR applications, reported higher achievement and lower cognitive load. The use of mAR applications in anatomy education contributed to the formation of an effective and productive learning environment. Student cognitive load decreased as abstract information became concrete in printed books via multimedia materials in mAR applications. Additionally, students were able to access the materials in the MagicBook anytime and anywhere they wanted. The mobile learning approach helped students learn better by exerting less cognitive effort. Moreover, the sensory experience and real time interaction with environment may provide learning satisfaction and enable students to structure their knowledge to complete the learning tasks. Anat Sci Educ 9: 411-421. © 2016 American Association of Anatomists.

Transforming Experience: The Potential of Augmented Reality and Virtual Reality for Enhancing Personal and Clinical Change

Abstract: During our life we undergo many personal changes: we change our house, our school, our work and even our friends and partners. However, our daily experience shows clearly that in some situations subjects are unable to change even if they want to. The recent advances in psychology and neuroscience are now providing a better view of personal change, the change affecting our assumptive world: a) the focus of personal change is reducing the distance between self and reality (conflict); b) this reduction is achieved through (1) an intense focus on the particular experience creating the conflict or (2) an internal or external reorganization of this experience; c) personal change requires a progression through a series of different stages; d) clinical psychology is often used to facilitate personal change when subjects are unable to move forward. Starting from these premises, the aim of this paper is to review the potential of virtuality for enhancing the processes of personal and clinical change. First, the paper will focus on the two leading virtual technologies – Augmented Reality (AR) and Virtual Reality (VR) – exploring their current uses in behavioral health and the outcomes of the 28 available systematic reviews and meta-analyses. Then the paper discusses the added value provided by VR and AR in transforming our external experience, by focusing on the high level of self-reflectiveness and personal efficacy induced by their emotional engagement and sense of presence. Finally, it outlines the potential future use of virtuality for transforming our inner experience by structuring, altering and/or replacing our bodily self-consciousness. The final outcome may be a new generation of transformative experiences that provide knowledge that is epistemically inaccessible to the individual until he or she has that experience, while at the same time transforming the individual’s worldview.

Augmented and virtual reality in surgery—the digital surgical environment: applications, limitations and legal pitfalls

Abstract: The continuing enhancement of the surgical environment in the digital age has led to a number of innovations being highlighted as potential disruptive technologies in the surgical workplace. Augmented reality (AR) and virtual reality (VR) are rapidly becoming increasingly available, accessible and importantly affordable, hence their application into healthcare to enhance the medical use of data is certain. Whether it relates to anatomy, intraoperative surgery, or post-operative rehabilitation, applications are already being investigated for their role in the surgeons armamentarium. Here we provide an introduction to the technology and the potential areas of development in the surgical arena.

CAT: high-precision acoustic motion tracking

Abstract: Video games, Virtual Reality (VR), Augmented Reality (AR), and Smart appliances (e.g., smart TVs) all call for a new way for users to interact and control them. This paper develops high-preCision Acoustic Tracker (CAT), which aims to replace a traditional mouse and let a user play games, interact with VR/AR headsets, and control smart appliances by moving a smartphone in the air. Achieving high tracking accuracy is essential to provide enjoyable user experience. To this end, we develop a novel system that uses audio signals to achieve mm-level tracking accuracy. It lets multiple speakers transmit inaudible sounds at different frequencies. Based on the received sound, our system continuously estimates the distance and velocity of the mobile with respect to the speakers to continuously track it. At its heart lies a distributed Frequency Modulated Continuous Waveform (FMCW) that can accurately estimate the absolute distance between a transmitter and a receiver that are separate and unsynchronized. We further develop an optimization framework to combine FMCW estimation with Doppler shifts and Inertial Measurement Unit (IMU) measurements to enhance the accuracy, and efficiently solve the optimization problem. We implement two systems: one on a desktop and another on a mobile phone. Our evaluation and user study show that our system achieves high tracking accuracy and ease of use using existing hardware.

Augmented reality smart glasses: an investigation of technology acceptance drivers

Abstract: Microsoft Hololens and Google Glass (Project Aura) are two examples of a new stream of wearable technology devices called Augmented Reality Smart Glasses that might substantially influence media usage in the near future. In this study, the authors draw upon prior technology acceptance research and propose an exploratory model of antecedents to smart glasses adoption. An empirical study reveals the importance of various drivers such as functional benefits, ease of use, individual difference variables, brand attitudes, and social norms. Although smart glasses are worn in a similar manner to fashion accessories and capture various personal information, self-presentation benefits and potential privacy concerns seem less likely to influence smart glasses adoption. The findings provide pre-market knowledge about smart glasses that can help scholars and managers understand this new technology.

"It's an illusion, but it looks real!" Consumer affective, cognitive and behavioural responses to augmented reality applications

Abstract: The paper investigates two augmented reality (AR) applications and corresponding consumer responses to their media characteristics. Firstly, it discusses the role of interactivity with AR technology. Secondly, it introduces augmentation as a salient media characteristic of AR applications and tests measurement items of perceived augmentation. Two experimental studies replicate the research design of Van Noort et al. [Van Noort, G., Voorveld, H. A. M., & van Reijmersdal, E. A. (2012). Interactivity in brand web sites: cognitive, affective, and behavioral responses explained by consumers’ online flow experience. Journal of Interactive Marketing, 26, 223–234], applying it in the context of AR. The results show that perceived augmentation represents a fitting concept for understanding consumer responses to AR features and, furthermore, that flow mediates effects of perceived augmentation on consumers’ affective responses and behavioural intentions. AR features on the other hand do not increase percept...

Effects of an augmented reality-based educational game on students' learning achievements and attitudes in real-world observations

Abstract: Augmented reality (AR) has been recognized as a potential technology to help students link what they are observing in the real world to their prior knowledge. One of the most challenging issues of AR-based learning is the provision of effective strategy to help students focus on what they need to observe in the field. In this study, a competitive gaming approach is proposed to support AR-based learning activities conducted in real-world contexts. An experiment has been conducted on an elementary school ecology course to explore the effectiveness of the proposed approach in comparison with the conventional AR-based mobile learning approach in field trips. The experimental results show that the AR-based gaming approach can improve not only students' learning attitudes, but also their learning performance on the field trip. Accordingly, discussions and some suggestions for future work are provided.

Real-time Joint Tracking of a Hand Manipulating an Object from RGB-D Input

Abstract: Real-time simultaneous tracking of hands manipulating and interacting with external objects has many potential applications in augmented reality, tangible computing, and wearable computing. However, due to difficult occlusions, fast motions, and uniform hand appearance, jointly tracking hand and object pose is more challenging than tracking either of the two separately. Many previous approaches resort to complex multi-camera setups to remedy the occlusion problem and often employ expensive segmentation and optimization steps which makes real-time tracking impossible. In this paper, we propose a real-time solution that uses a single commodity RGB-D camera. The core of our approach is a 3D articulated Gaussian mixture alignment strategy tailored to hand-object tracking that allows fast pose optimization. The alignment energy uses novel regularizers to address occlusions and hand-object contacts. For added robustness, we guide the optimization with discriminative part classification of the hand and segmentation of the object. We conducted extensive experiments on several existing datasets and introduce a new annotated hand-object dataset. Quantitative and qualitative results show the key advantages of our method: speed, accuracy, and robustness.

Room2Room: Enabling Life-Size Telepresence in a Projected Augmented Reality Environment

Abstract: Room2Room is a telepresence system that leverages projected augmented reality to enable life-size, co-present interaction between two remote participants. Our solution recreates the experience of a face-to-face conversation by performing 3D capture of the local user with color + depth cameras and projecting their life-size virtual copy into the remote space. This creates an illusion of the remote person's physical presence in the local space, as well as a shared understanding of verbal and non-verbal cues (e.g., gaze, pointing.) In addition to the technical details of two prototype implementations, we contribute strategies for projecting remote participants onto physically plausible locations, such that they form a natural and consistent conversational formation with the local participant. We also present observations and feedback from an evaluation with 7 pairs of participants on the usability of our solution for solving a collaborative, physical task.

Real-time Joint Tracking of a Hand Manipulating an Object from RGB-D Input

Abstract: Real-time simultaneous tracking of hands manipulating and interacting with external objects has many potential applications in augmented reality, tangible computing, and wearable computing. However, due to difficult occlusions, fast motions, and uniform hand appearance, jointly tracking hand and object pose is more challenging than tracking either of the two separately. Many previous approaches resort to complex multi-camera setups to remedy the occlusion problem and often employ expensive segmentation and optimization steps which makes real-time tracking impossible. In this paper, we propose a real-time solution that uses a single commodity RGB-D camera. The core of our approach is a 3D articulated Gaussian mixture alignment strategy tailored to hand-object tracking that allows fast pose optimization. The alignment energy uses novel regularizers to address occlusions and hand-object contacts. For added robustness, we guide the optimization with discriminative part classification of the hand and segmentation of the object. We conducted extensive experiments on several existing datasets and introduce a new annotated hand-object dataset. Quantitative and qualitative results show the key advantages of our method: speed, accuracy, and robustness.

Annexing Reality: Enabling Opportunistic Use of Everyday Objects as Tangible Proxies in Augmented Reality

Abstract: Advances in display and tracking technologies hold the promise of increasingly immersive augmented-reality experiences. Unfortunately, the on-demand generation of haptic experiences is lagging behind these advances in other feedback channels. We present Annexing Reality; a system that opportunistically annexes physical objects from a user's current physical environment to provide the best-available haptic sensation for virtual objects. It allows content creators to a priori specify haptic experiences that adapt to the user's current setting. The system continuously scans user's surrounding, selects physical objects that are similar to given virtual objects, and overlays the virtual models on to selected physical ones reducing the visual-haptic mismatch. We describe the developer's experience with the Annexing Reality system and the techniques utilized in realizing it. We also present results of a developer study that validates the usability and utility of our method of defining haptic experiences.

Augmented reality as multimedia: the case for situated vocabulary learning.

Abstract: Augmented reality (AR) has the potential to create compelling learning experiences. However, there are few research works exploring the design and evaluation of AR for educational settings. In our research, we treat AR as a type of multimedia that is situated in authentic environments and apply multimedia learning theory as a framework for developing our educational applications. We share our experiences in developing a handheld AR system and one specific use case, namely, situated vocabulary learning. Results of our evaluations show that we are able to create AR applications with good system usability. More importantly, our preliminary evaluations show that AR may lead to better retention of words and improve student attention and satisfaction.

Touch fee interface for augmented reality systems

Abstract: A method and system for augmented reality. Images of a real world scene are obtained from one or more from one or more image sensors. An orientation and/or location of the image sensors is obtained from one or more state sensors. A real world object at which a predefined pointing object is performing a predefined gesture in the images of the real world scene is identified and data associated with the identified object is displayed on a viewing device. The invention also provides a computer program comprising computer program code means for performing all the steps of the method of the invention when said program is run on a computer.

Systems and methods using virtual reality or augmented reality environments for the measurement and/or improvement of human vestibulo-ocular performance

Abstract: A system and method for using a virtual reality or an augmented reality environment for the measurement and/or improvement of human vestibulo-ocular performance can be implemented by combining a video camera based eye orientation sensor, a head orientation sensor, a display, and an electronic circuit that connects the eye sensor, head sensor, and display. The system and method can be operated in the range of frequencies between 0.01 Hertz and 15 Hertz. The system and method can use a Fourier transform to compute a gain and a phase. The system and method can be used for measuring vestibulo-ocular reflex, dynamic visual acuity, dynamic visual stability, kinetic visual acuity, retinal image stability, or foveal fixation stability in a non-clinical setting. The system and method can be completely portable, head-worn, and self-contained.

Augmented reality virtual content platform apparatuses, methods and systems

Abstract: The AUGMENTED REALITY VIRTUAL CONTENT PLATFORM APPARATUSES, METHODS AND SYSTEMS (“ARV”) provides a photo driven ad-platform that transforms digital media placements into immersive and immediately shareable brand-consumer engagements via GPS-linked virtual photo components instantiated on a user mobile device. Within embodiments, users may create and share photographs augmented with brands or other images and accompanying messages on various social networks using their Smartphones or tablets to earn rewards. In one implementation, merchants and/or advertisers may populate the mobile augmented reality space as fans and consumers may share their photos on social networks and spread the word virally.

Quality metric for spherical panoramic video

Abstract: Virtual reality (VR)/ augmented reality (AR) applications allow users to view artificial content of a surrounding space simulating presence effect with a help of special applications or devices. Synthetic contents production is well known process form computer graphics domain and pipeline has been already fixed in the industry. However emerging multimedia formats for immersive entertainment applications such as free-viewpoint television (FTV) or spherical panoramic video require different approaches in content management and quality assessment. The international standardization on FTV has been promoted by MPEG. This paper is dedicated to discussion of immersive media distribution format and quality estimation process. Accuracy and reliability of the proposed objective quality estimation method had been verified with spherical panoramic images demonstrating good correlation results with subjective quality estimation held by a group of experts.

Discrete Event Simulation and Virtual Reality Use in Industry: New Opportunities and Future Trends

Abstract: This paper reviews the area of combined discrete event simulation (DES) and virtual reality (VR) use within industry. While establishing a state of the art for progress in this area, this paper makes the case for VR DES as the vehicle of choice for complex data analysis through interactive simulation models, highlighting both its advantages and current limitations. This paper reviews active research topics such as VR and DES real-time integration, communication protocols, system design considerations, model validation, and applications of VR and DES. While summarizing future research directions for this technology combination, the case is made for smart factory adoption of VR DES as a new platform for scenario testing and decision making. It is put that in order for VR DES to fully meet the visualization requirements of both Industry 4.0 and Industrial Internet visions of digital manufacturing, further research is required in the areas of lower latency image processing, DES delivery as a service, gesture recognition for VR DES interaction, and linkage of DES to real-time data streams and Big Data sets.

Personalized augmented reality for anatomy education.

Abstract: Anatomy education is a challenging but vital element in forming future medical professionals. In this work, a personalized and interactive augmented reality system is developed to facilitate education. This system behaves as a "magic mirror" which allows personalized in-situ visualization of anatomy on the user's body. Real-time volume visualization of a CT dataset creates the illusion that the user can look inside their body. The system comprises a RGB-D sensor as a real-time tracking device to detect the user moving in front of a display. In addition, the magic mirror system shows text information, medical images, and 3D models of organs that the user can interact with. Through the participation of 7 clinicians and 72 students, two user studies were designed to respectively assess the precision and acceptability of the magic mirror system for education. The results of the first study demonstrated that the average precision of the augmented reality overlay on the user body was 0.96 cm, while the results of the second study indicate 86.1% approval for the educational value of the magic mirror, and 91.7% approval for the augmented reality capability of displaying organs in three dimensions. The usefulness of this unique type of personalized augmented reality technology has been demonstrated in this paper.

The Effect of the Internet of Things (IoT) on Education Business Model

Abstract: The introduction of the Internet of Things (IoT) in education, which allows Internet based communications to happen between physical objects, sensors and controllers, has changed educational institutions massively. By embedding sensors in objects and integrating cloud computing, augmented reality, wearable technologies and big data in this platform, different parameters of the educational environment can be measured and analysed to provide useful information. It also has created a new interaction between people and the environment in educational organisation. In this research based on the recent IoT projects in education, we will categorize the application of IoT in education into four groups: energy management and real time ecosystem monitoring, monitoring student's healthcare, classroom access control and improving teaching and learning. We will investigate and analyse how this platform has changed the Education Business Model and added new value propositions in such organizations based on the Canvas Business Model.