Challenges and opportunities on AR/VR technologies for manufacturing systems in the context of industry 4.0: A state of the art review

Appearance is not everything - Preferred feature combinations for care robots

Major natural disasters have occurred frequently in the last few years, resulting in increased loss of life and economic damage. Most emergency responders do not have first-hand experience with major natural disasters, and thus, there is an urgent need for pre-disaster training. Due to the scenes unreality of traditional emergency drills, the failure to appeal to the target audience and the novel coronavirus pandemic, people are forced to maintain safe social distancing. Therefore, it is difficult to carry out transregional or transnational emergency drills in many countries under the lockdown. There is an increasing demand for simulation training systems that use virtual reality, augmented reality, and mixed reality visualization technologies to simulate major natural disasters. The simulation training system related to natural disasters provides a new way for popular emergency avoidance science education and emergency rescue personnel to master work responsibilities and improve emergency response capabilities. However, to our knowledge, there is no overview of the simulation training system for major natural disasters. Hence, this paper uncovers the visualization techniques commonly used in simulation training systems, and compares, analyses and summarizes the architecture and functions of the existing simulation training systems for different emergency phases of common natural disasters. In addition, the limitations of the existing simulation training system in practical applications and future development directions are discussed to provide reference for relevant researchers to better understand the modern simulation training system. 

/pdf/review-on-visualization-technology-in-simulation-training-1pagprwr.pdf

Review on visualization technology in simulation training system for major natural disasters

• Role of Augmented Reality in assembly and maintenance/repair is analysed. • Software and hardware elements of AR to aid manufacturing systems are discussed. • Challenges in AR tracking and registration techniques are discussed. • Future trends of AR to aid manufacturing systems are discussed. Manufacturing industries are currently experiencing the fourth revolution with the rapid advancements in immersive technologies for human-machine interaction (HMI) and flexible manufacturing systems (FMS). Product variance is limited due to barriers in the knowledge transfer between the stakeholders that existed in the pre and post-manufacturing phases. Augmented reality (AR) is a promising technology that can offer a high degree of adaptability and independence to support knowledge transfer at the most crucial manufacturing stages such as assembly, repair & maintenance. This article is focused on presenting the state of the art literature on AR technologies in product assembly and disassembly from the Maintenance/Repair perspective. The working of various modules in AR technology on facilitating a user-friendly guiding platform and its applications are discussed with suitable illustrations. The critical difficulties, such as tracking and rendering techniques for estimating human movement and environment experiences are observed to extend the adaptability of the technology. Future research potential, such as enhancing the virtual interface for reality, identifying worker behaviours, and enabling sharing and collaboration between multiple streams in an industrial context are analyzed. 

Challenges and opportunities on Augmented reality-based guidance in product assembly and maintenance/repair perspective: A state of the art review

The use of extended reality (XR) technologies, namely Augmented Reality (AR), Virtual Reality (VR) and Mixed Reality (MR) in education, has attracted much attention in recent years. Many educators have described how XR benefits learners and how useful AR and VR technologies are in the classroom. However, creating AR and VR educational tools, apps or learning environments is a complex process, hence providing an immersive learning experience using these technologies is not a straightforward journey. As a result, the adoption of these emerging technologies in education might be delayed or halted despite their reported benefits to today’s learners. In this paper, websites, technical articles, academic journals, reports and mobile app stores, relating to the use of XR technologies in education, have been examined. A number of themes have emerged and been reported in this paper, which provides a roadmap for those who would like to create XR experiences for learning and training purposes. The paper also describes the factors that should be considered when selecting an option to follow to introduce such immersive learning experiences.

https://www.mdpi.com/2079-9292/11/21/3547/pdf?version=1667890190

Creating an Immersive XR Learning Experience: A Roadmap for Educators

In recent years, with the current advancements in Robotics and Artificial Intelligence (AI), robots have the potential to support the field of healthcare. Robotic systems are often introduced in the care of the elderly, children, and persons with disabilities, in hospitals, in rehabilitation and walking assistance, and other healthcare situations. In this survey paper, the recent advances in robotic technology applied in the healthcare domain are discussed. The paper provides detailed information about state-of-the-art research in care, hospital, assistive, rehabilitation, and walking assisting robots. The paper also discusses the open challenges healthcare robots face to be integrated into our society.

A Survey of Robots in Healthcare

Recently, the use of extended reality (XR) systems has been on the rise, to tackle various domains such as training, education, safety, etc. With the recent advances in augmented reality (AR), virtual reality (VR) and mixed reality (MR) technologies and ease of availability of high-end, commercially available hardware, the manufacturing industry has seen a rise in the use of advanced XR technologies to train its workforce. While several research publications exist on applications of XR in manufacturing training, a comprehensive review of recent works and applications is lacking to present a clear progress in using such advance technologies. To this end, we present a review of the current state-of-the-art of use of XR technologies in training personnel in the field of manufacturing. First, we put forth the need of XR in manufacturing. We then present several key application domains where XR is being currently applied, notably in maintenance training and in performing assembly task. We also reviewed the applications of XR in other vocational domains and how they can be leveraged in the manufacturing industry. We finally present some current barriers to XR adoption in manufacturing training and highlight the current limitations that should be considered when looking to develop and apply practical applications of XR.

A Review of Extended Reality (XR) Technologies for Manufacturing Training

The ability to have unmanned ground vehicles navigate unmapped off-road terrain has high impact potential in application areas ranging from supply and logistics, to search and rescue, to planetary exploration. To achieve this, robots must be able to estimate the traversability of the terrain they are facing, in order to be able to plan a safe path through rugged terrain. In the work described here, we pursue the idea of fine-tuning a generic visual recognition network to our task and to new environments, but without requiring any manually labelled data. Instead, we present an autonomous data collection method that allows the robot to derive ground truth labels by attempting to traverse a scene and using localization to decide if the traversal was successful. We then present and experimentally evaluate two deep learning architectures that can be used to adapt a pre-trained network to a new environment. We prove that the networks successfully adapt to their new task and environment from a relatively small dataset.

Improving Traversability Estimation Through Autonomous Robot Experimentation.

Training on simulation data has proven invaluable in applying machine learning in robotics. However, when looking at robot vision in particular, simulated images cannot be directly used no matter how realistic the image rendering is, as many physical parameters (temperature, humidity, wear-and-tear in time) vary and affect texture and lighting in ways that cannot be encoded in the simulation. In this article we propose a different approach for extracting value from simulated environments: although neither of the trained models can be used nor are any evaluation scores expected to be the same on simulated and physical data, the conclusions drawn from simulated experiments might be valid. If this is the case, then simulated environments can be used in early-stage experimentation with different network architectures and features. This will expedite the early development phase before moving to (harder to conduct) physical experiments in order to evaluate the most promising approaches. In order to test this idea we created two simulated environments for the Unity engine, acquired simulated visual datasets, and used them to reproduce experiments originally carried out in a physical environment. The comparison of the conclusions drawn in the physical and the simulated experiments is promising regarding the validity of our approach. 

https://www.mdpi.com/2227-7080/10/1/7/pdf?version=1642765163

A Simulated Environment for Robot Vision Experiments

We present an environment for simulated experiments in field robotics, and especially in experiments on estimating the traversability of foliage and other objects that appear as obstacles but that can be overcome by the robot without circumventing them. The simulated environment is developed in the Unity real-time development platform, integrated with the ROS middleware. In the preliminary experiments presented here, we demonstrate that our environment is able to simulate the sensory input needed in order to train supervised traversability estimation models.

https://dl.acm.org/doi/pdf/10.1145/3453892.3462214

Christos Sevastopoulos

Papers

A Survey of Robots in Healthcare

A Review of Extended Reality (XR) Technologies for Manufacturing Training

Improving Traversability Estimation Through Autonomous Robot Experimentation.

A Simulated Environment for Robot Vision Experiments

A Simulated Environment for Traversability Estimation Experiments in Field Robotics Applications