Digital twin can be defined as a virtual representation of a physical asset enabled through data and simulators for real-time prediction, optimization, monitoring, controlling, and improved decision making. Recent advances in computational pipelines, multiphysics solvers, artificial intelligence, big data cybernetics, data processing and management tools bring the promise of digital twins and their impact on society closer to reality. Digital twinning is now an important and emerging trend in many applications. Also referred to as a computational megamodel, device shadow, mirrored system, avatar or a synchronized virtual prototype, there can be no doubt that a digital twin plays a transformative role not only in how we design and operate cyber-physical intelligent systems, but also in how we advance the modularity of multi-disciplinary systems to tackle fundamental barriers not addressed by the current, evolutionary modeling practices. In this work, we review the recent status of methodologies and techniques related to the construction of digital twins mostly from a modeling perspective. Our aim is to provide a detailed coverage of the current challenges and enabling technologies along with recommendations and reflections for various stakeholders.

/pdf/digital-twin-values-challenges-and-enablers-from-a-modeling-3hpoelaogn.pdf

Digital Twin: Values, Challenges and Enablers From a Modeling Perspective

Ambient Intelligence (AmI) is a new paradigm in information technology aimed at empowering people's capabilities by means of digital environments that are sensitive, adaptive, and responsive to human needs, habits, gestures, and emotions. This futuristic vision of daily environment will enable innovative human-machine interactions characterized by pervasive, unobtrusive, and anticipatory communications. Such innovative interaction paradigms make AmI technology a suitable candidate for developing various real life solutions, including in the healthcare domain. This survey will discuss the emergence of AmI techniques in the healthcare domain, in order to provide the research community with the necessary background. We will examine the infrastructure and technology required for achieving the vision of AmI, such as smart environments and wearable medical devices. We will summarize the state-of-the-art artificial intelligence (AI) methodologies used for developing AmI system in the healthcare domain, including various learning techniques (for learning from user interaction), reasoning techniques (for reasoning about users' goals and intensions), and planning techniques (for planning activities and interactions). We will also discuss how AmI technology might support people affected by various physical or mental disabilities or chronic disease. Finally, we will point to some of the successful case studies in the area and we will look at the current and future challenges to draw upon the possible future research paths.

/pdf/a-survey-on-ambient-intelligence-in-healthcare-wkvwfodws9.pdf

A Survey on Ambient Intelligence in Healthcare

Tactile sensors, most commonly referred as strain and pressure sensors, can collect mechanical property data of the human body and local environment, to provide valuable insights into the human health status or artificial intelligence systems. The introduction of a high level of wearability (bendability and stretchability) to tactile sensors can dramatically enhance their interfaces with the contact objects, providing chronically reliable functions. Therefore, the developed wearable tactile sensors are capable of conformably covering arbitrary curved surface over their stiff counterparts without incurring damage, emerging as a promising development direction toward the Internet of Things (IoT) applications. Fundamental parameters of the wearable tactile sensors such as sensitivity and stretchability have experienced unprecedented advancement, owing to the progress of device fabrication techniques and material structural engineering. Moreover, novel smart materials and mechanically durable sensor design concepts endow these sensors with multi-functionality integration (e.g., simultaneous force, temperature and humidity detection, simultaneous pressure and strain discrimination) and stirring properties (e.g., biocompatibility, biodegradability, self-healing, self-powering and visualization), further broadening the application scope of current wearable tactile sensors. Besides, it is desirable that a tactile sensor is compatible with a printing process that presents a new era of feasible wearable technology due to its large-area and high-throughput production capability. In addition to the development of sensors, packaging, and integration of the rest of the tactile device system (data memory, signal conversion, power supply, wireless transmission, feedback actuator, etc.) to build a wearable platform also emerge as major research frontiers in recent years. This review attempts to summarize the current state-of-the-art wearable tactile sensors concerning basic concepts, functional materials, sensing mechanism, promising applications, performance optimization strategies, multifunctional sensing, and system integration. Finally, the discussion will be presented regarding potential challenges, pathways, and opportunities.

Recent advances in wearable tactile sensors: Materials, sensing mechanisms, and device performance

It is now more than 50 years since the time when clinical monitoring of individuals in the home and community settings was first envisioned. Until recently, technologies to enable such vision were lacking. However, wearable sensors and systems developed over the past decade have provided the tools to finally implement and deploy technology with the capabilities required by researchers in the field of patients' home monitoring. As discussed, potential applications of these technologies include the early diagnosis of diseases such as congestive heart failure, the prevention of chronic conditions such as diabetes, improved clinical management of neurodegenerative conditions such as Parkinson's disease, and the ability to promptly respond to emergency situations such as seizures in patients with epilepsy and cardiac arrest in subjects undergoing cardiovascular monitoring. Current research efforts are now focused on the development of more complex systems for home monitoring of individuals with a variety of preclinical and clinical conditions. Recent research on the clinical assessment of wearable technology promises to deliver methodologies that are expected to lead to clinical adoption within the next five to ten years. In particular, combining home robots and wearable technology is likely to be a key step toward achieving the goal of effectively monitoring patients in the home. These efforts to merge home robots and wearable technology are expected to enable a new generation of complex systems with the ability to monitor subjects' status, facilitate the administration of interventions, and provide an invaluable tool to respond to emergency situations.

/pdf/wearable-sensors-and-systems-pq3jmo58iv.pdf

Wearable Sensors and Systems

The Augmented Reality (AR) technology enhances the human perception of the world by combining the real environment with the virtual space. With the explosive growth of powerful, less expensive mobile devices, and the emergence of sophisticated communication infrastructure, Mobile Augmented Reality (MAR) applications are gaining increased popularity. MAR allows users to run AR applications on mobile devices with greater mobility and at a lower cost. The emerging 5G communication technologies act as critical enablers for future MAR applications to achieve ultra-low latency and extremely high data rates while Multi-access Edge Computing (MEC) brings enhanced computational power closer to the users to complement MAR. This paper extensively discusses the landscape of MAR through the past and its future prospects with respect to the 5G systems and complementary technology MEC. The paper especially provides an informative analysis of the network formation of current and future MAR systems in terms of cloud, edge, localized, and hybrid architectural options. The paper discusses key application areas for MAR and their future with the advent of 5G technologies. The paper also discusses the requirements and limitations of MAR technical aspects such as communication, mobility management, energy management, service offloading and migration, security, and privacy and analyzes the role of 5G technologies.

/pdf/a-survey-on-mobile-augmented-reality-with-5g-mobile-edge-11f7i8j1c6.pdf

A Survey on Mobile Augmented Reality With 5G Mobile Edge Computing: Architectures, Applications, and Technical Aspects

Neuronavigation has become an essential neurosurgical tool in pursuing minimal invasiveness and maximal safety, even though it has several technical limitations. Augmented reality (AR) neuronavigation is a significant advance, providing a real-time updated 3D virtual model of anatomical details, overlaid on the real surgical field. Currently, only a few AR systems have been tested in a clinical setting. The aim is to review such devices. We performed a PubMed search of reports restricted to human studies of in vivo applications of AR in any neurosurgical procedure using the search terms "Augmented reality" and "Neurosurgery." Eligibility assessment was performed independently by two reviewers in an unblinded standardized manner. The systems were qualitatively evaluated on the basis of the following: neurosurgical subspecialty of application, pathology of treated lesions and lesion locations, real data source, virtual data source, tracking modality, registration technique, visualization processing, display type, and perception location. Eighteen studies were included during the period 1996 to September 30, 2015. The AR systems were grouped by the real data source: microscope (8), hand- or head-held cameras (4), direct patient view (2), endoscope (1), and X-ray fluoroscopy (1) head-mounted display (1). A total of 195 lesions were treated: 75 (38.46 %) were neoplastic, 77 (39.48 %) neurovascular, and 1 (0.51 %) hydrocephalus, and 42 (21.53 %) were undetermined. Current literature confirms that AR is a reliable and versatile tool when performing minimally invasive approaches in a wide range of neurosurgical diseases, although prospective randomized studies are not yet available and technical improvements are needed.

Augmented reality in neurosurgery: a systematic review

Aim We present a newly designed, localiser-free, head-mounted system featuring augmented reality as an aid to maxillofacial bone surgery, and assess the potential utility of the device by conducting a feasibility study and validation. Methods Our head-mounted wearable system facilitating augmented surgery was developed as a stand-alone, video-based, see-through device in which the visual features were adapted to facilitate maxillofacial bone surgery. We implement a strategy designed to present augmented reality information to the operating surgeon. LeFort1 osteotomy was chosen as the test procedure. The system is designed to exhibit virtual planning overlaying the details of a real patient. We implemented a method allowing performance of waferless, augmented-reality assisted bone repositioning. In vitro testing was conducted on a physical replica of a human skull, and the augmented reality system was used to perform LeFort1 maxillary repositioning. Surgical accuracy was measured with the aid of an optical navigation system that recorded the coordinates of three reference points (located in anterior, posterior right, and posterior left positions) on the repositioned maxilla. The outcomes were compared with those expected to be achievable in a three-dimensional environment. Data were derived using three levels of surgical planning, of increasing complexity, and for nine different operators with varying levels of surgical skill. Results The mean error was 1.70 ± 0.51 mm. The axial errors were 0.89 ± 0.54 mm on the sagittal axis, 0.60 ± 0.20 mm on the frontal axis, and 1.06 ± 0.40 mm on the craniocaudal axis. The simplest plan was associated with a slightly lower mean error (1.58 ± 0.37 mm) compared with the more complex plans (medium: 1.82 ± 0.71 mm; difficult: 1.70 ± 0.45 mm). The mean error for the anterior reference point was lower (1.33 ± 0.58 mm) than those for both the posterior right (1.72 ± 0.24 mm) and posterior left points (2.05 ± 0.47 mm). No significant difference in terms of error was noticed among operators, despite variations in surgical experience. Feedback from surgeons was acceptable; all tests were completed within 15 min and the tool was considered to be both comfortable and usable in practice. Conclusion We used a new localiser-free, head-mounted, wearable, stereoscopic, video see-through display to develop a useful strategy affording surgeons access to augmented reality information. Our device appears to be accurate when used to assist in waferless maxillary repositioning. Our results suggest that the method can potentially be extended for use with many surgical procedures on the facial skeleton. Further, our positive results suggest that it would be appropriate to proceed to in vivo testing to assess surgical accuracy under real clinical conditions.

Augmented reality as an aid in maxillofacial surgery: validation of a wearable system allowing maxillary repositioning.

Augmented reality (AR) has been successfully providing surgeons an extensive visual information of surgical anatomy to assist them throughout the procedure. AR allows surgeons to view surgical field through the superimposed 3D virtual model of anatomical details. However, open surgery presents new challenges. This study provides a comprehensive overview of the available literature regarding the use of AR in open surgery, both in clinical and simulated settings. In this way, we aim to analyze the current trends and solutions to help developers and end/users discuss and understand benefits and shortcomings of these systems in open surgery. We performed a PubMed search of the available literature updated to January 2018 using the terms (1) "augmented reality" AND "open surgery", (2) "augmented reality" AND "surgery" NOT "laparoscopic" NOT "laparoscope" NOT "robotic", (3) "mixed reality" AND "open surgery", (4) "mixed reality" AND "surgery" NOT "laparoscopic" NOT "laparoscope" NOT "robotic". The aspects evaluated were the following: real data source, virtual data source, visualization processing modality, tracking modality, registration technique, and AR display type. The initial search yielded 502 studies. After removing the duplicates and by reading abstracts, a total of 13 relevant studies were chosen. In 1 out of 13 studies, in vitro experiments were performed, while the rest of the studies were carried out in a clinical setting including pancreatic, hepatobiliary, and urogenital surgeries. AR system in open surgery appears as a versatile and reliable tool in the operating room. However, some technological limitations need to be addressed before implementing it into the routine practice.

Augmented reality in open surgery.

A novel algorithm for human fall detection by means of a tri-axial accelerometer, is described. A module constituted by the accelerometer and an on board processing unit was designed and realized. The system is conceived to be used in a multi-sensor network context for the remote monitoring of personnel working in very severe conditions (firefighters and civil protection operators). In the real application the module is thought to be integrated in the operator uniform collar. The algorithm is based on the detection of a critical trunk inclination in correspondence of an high rotational velocity. A Kalman filter was designed in order to separate the signal component due to gravity (i.e useful to extract the subject orientation) from the one due to the system acceleration. In comparison with the existing solutions the realized algorithm presents many advantages: no training is needed, low computational costs, fast time response and good performances also during critical activities (e.g jumping, running).

Development of a novel algorithm for human fall detection using wearable sensors

Purpose: Benefits of minimally invasive neurosurgery mandate the development of ergonomic paradigms for neuronavigation. Augmented Reality (AR) systems can overcome the shortcomings of commercial neuronavigators. The aim of this work is to apply a novel AR system, based on a head-mounted stereoscopic video see-through display, as an aid in complex neurological lesion targeting. Effectiveness was investigated on a newly designed patient-specific head mannequin featuring an anatomically realistic brain phantom with embedded synthetically created tumors and eloquent areas.Materials and methods: A two-phase evaluation process was adopted in a simulated small tumor resection adjacent to Broca’s area. Phase I involved nine subjects without neurosurgical training in performing spatial judgment tasks. In Phase II, three surgeons were involved in assessing the effectiveness of the AR-neuronavigator in performing brain tumor targeting on a patient-specific head phantom.Results: Phase I revealed the ability ...

Fabrizio Cutolo

Papers

Augmented reality in neurosurgery: a systematic review

Augmented reality as an aid in maxillofacial surgery: validation of a wearable system allowing maxillary repositioning.

Augmented reality in open surgery.

Development of a novel algorithm for human fall detection using wearable sensors

A new head-mounted display-based augmented reality system in neurosurgical oncology: a study on phantom.