Electrification turned over a new leaf in aviation by introducing new types of aerial vehicles along with new means of transportation. Addressing a plethora of use cases, drones are gaining attention in the industry and increasingly appear in the sky. Emerging concepts of flying taxi enable passengers to be transported over several tens of kilometers. Therefore, unmanned traffic management systems are under development to cope with the complexity of future airspace, thereby resulting in unprecedented communication needs. Moreover, the long-term increase in the number of commercial airplanes pushes the limits of voice-oriented communications, and future options such as single-pilot operations demand robust connectivity. In this survey, we provide a comprehensive review and vision for enabling the connectivity applications of aerial vehicles utilizing current and future communication technologies. We begin by categorizing the connectivity use cases per aerial vehicle and analyzing their connectivity requirements. By reviewing more than 500 related studies, we aim for a comprehensive approach to cover wireless communication technologies, and provide an overview of recent findings from the literature toward the possibilities and challenges of employing the wireless communication standards. After analyzing the proposed network architectures, we list the open-source testbed platforms to facilitate future investigations by researchers. This study helped us observe that while numerous works focused on cellular technologies to enable connectivity for aerial platforms, a single wireless technology is not sufficient to meet the stringent connectivity demands of the aerial use cases, especially for the piloting operations. We identified the need of further investigations on multi-technology heterogeneous network architectures to enable robust and real-time connectivity in the sky. Future works should consider suitable technology combinations to develop unified aerial networks that can meet the diverse quality of service demands of the aerial use cases.

A Survey of Wireless Networks for Future Aerial Communications (FACOM)

The aim of this study is to investigate the augmented reality (AR) immersion experiences of university students studying in science education. The relationship between interest, usability, emotional investment, focus of attention, presence and flow was examined for university students studying in the science education department who used AR technology. The research design adopted is a correlational method, an established experimental research method. The sample consisted of 180-university students studying in the science education department (32 males, 148 females) of the Faculty of Education. The data obtained were analyzed according to the structural equation model. A model was developed which is capable of explaining 71% of the variance in university students’ flow experiences. According to our model, focus of attention and presence of university students have an influence on their flow experiences. The study also showed that emotional investment and presence of university students influences their focus of attention. In addition, usability and emotional investment of the students influences their interest.

A Model for Augmented Reality Immersion Experiences of University Students Studying in Science Education.

Objective: The present article undertakes a systematic review of the current state of science for Single-Pilot Operations (SPO) and Reduced-Crew Operations (RCO) in commercial aviation.Background: ...

Progressing Toward Airliners’ Reduced-Crew Operations: A Systematic Literature Review

There is a current trend in commercial aviation that points toward a possible transition from two-crew to single-pilot operations (SPO). The workload on the single pilot is expected to be a major issue for SPO. In order to find the best support solutions for the pilot in SPO, a thorough understanding of pilot workload is required. The present study aims at evaluating pilot workload by means of eye-tracking metrics. A flight simulator study was conducted with commercial pilots. Their task was to fly short approach and landing scenarios with or without the support of a second pilot. The results showed that fixation frequencies were higher during SPO, average dwell durations decreased, and participants transitioned more frequently between different areas of interest. These results suggest that particularly the temporal demand requires adequate support for a possible transition to SPO. The eye-tracking metrics support results obtained from subjective workload ratings.

Eye-Tracking Metrics as an Indicator of Workload in Commercial Single-Pilot Operations

To increase flight safety and operational availability for helicopters, the potential benefits of helmet-mounted displays (HMD) are investigated, with a focus on maritime flight operations. Helicopters have long downtimes, due to harsh weather conditions or other visual impairments, especially in maritime scenarios. Flying in these poor conditions can drastically reduce flight safety. It is often difficult to recognize the horizon due to sea fog, and the absence of reference objects can complicate the maritime flight. These conditions and especially the downtimes cost money or, at worst, life’s. Therefore, DLR integrated the augmented reality glasses Microsoft HoloLens into DLR’s simulator AVES to use it as HMD for pilots. Subsequently, displays and symbology were developed and evaluated. To carry out a piloted simulator study, a maritime scenario was created to measure changes in the pilots’ performance with the HMD, like workload or situational awareness. The paper focuses (a) on the integration of the HoloLens into the simulator with its challenges, solutions and findings, (b) on the symbology and (c) on the piloted simulator study. Both the quality of the HoloLens as HMD and the study results are very positive. The pilots rated high usability, reduced workload, increased situational awareness and increased safety.

Increasing helicopter flight safety in maritime operations with a head-mounted display

This paper presents a conceptual design of an additional assistant system with AR-Glasses for single-pilot operations of commercial transport aircraft. In particular, an analysis regarding the use of the Microsoft HoloLens as a guidance device and co-pilot compensation, to perform useful and timely alerts based on airline emergency procedures. The augmented reality (AR) device was introduced to raise the ability to handle complex work flows under time pressure by adding artificial visual guidance to the crew member. The goal was to enhance situation awareness and confidence taking carefully time critical decisions without the support of a second crew member. The test scenario included an engine fire in a single cockpit operation environment during cruise flight. For the survey, 24 experienced and type rated active Airbus A320 pilots have been compared in two groups to each other using an Airbus A320 fix based simulator. Within the test scenario, the reaction time initiating the standardized action items (based on Airbus A320 quick reference handbook) have been recorded. The first group had only support by the airplane-side systems (e.g. ECAM display), while the second group used the AR-glasses. Compared to the conventional airplane-side systems support to deal with engine fire situations, direct visual instructions have been given by the AR-glasses and have been always linked to a displayed three-dimensional visual marker. This should immediately inform the pilot where the focus of attention must be on and thus ensure that the actions are carried out more quickly. By guiding the pilot through visual instructions, manual scanning of the cockpit parameters have been unnecessary. The direct instructions in the glasses and the additional information expected to shorten the reaction time. At the same time this should relieved the pilot and reduce the workload in order to get a more efficient human machine system and enhanced situation awareness.

Single Pilot Operations with AR-Glasses using Microsoft HoloLens

Building further on previous PBN to precision final approach transition work (PBN to xLS), EUROCONTROL in collaboration with the Technical University of Berlin (TUB) and under the SESAR1 work programme, conducted an experiment investigating the combined lateral and vertical performance of 5 different aircraft types when transitioning from a curved PBN procedure to a precision final approach procedure (using an ILS, GLS or more generically an xLS landing system). While earlier work has concentrated primarily on either lateral or vertical performance during these operations, the current work focusses on the combined lateral and vertical transition aspects. Arinc 424 procedures, consisting of a 180 degree turn using the Radius-To-Fix path terminator, connected to either a 3NM or a 6NM Final Approach Segment were implemented in 5 different aircraft simulators of the following types: A340, B777, B737, E190 and Dash8Q400. To test the procedures under the most realistic conditions, simulated lateral navigation errors ranging between −0.15NM and 0.15NM were introduced in the PBN segments of each procedure, while non-standard temperatures ranging between ISA-37 and ISA+35 were implemented in the simulator. Besides the effect of the lateral navigation errors on the xLS transition already studied in the previous work, the additional effects of deviations in the vertical profile caused by the non-standard temperatures were investigated. Also, a number of scenarios were added to the test cases which contained steeper glide paths with angles up to 4 degrees. Conclusions were formulated regarding the different aircraft capabilities and performances, as well as flight crew considerations. The overall conclusion of this work is that it is possible for all investigated aircraft to transition directly from a curved PBN procedure to an xLS procedure without the requirement of an intermediate segment between final approach course and glide path interception. This is true under the conditions that the glide path is intercepted at a certain distance from the threshold and from a PBN segment containing a defined vertical path which is significantly shallower than the glide path. Additionally, final approach course and glide path capture will require flight crew interventions other than just arming the approach mode in certain situations, i.e. high above ISA temperature deviations and lateral navigation errors. The final objective of this work is to progress to set of procedure design criteria which would enable the design and publication of PBN to xLS procedures.

Simulations investigating combined effect of lateral and vertical navigation errors on PBN to xLS transition

Single Pilot Operations is a topic with the potential to significantly affect the future of commercial aviation. While financially attractive for airlines, Single Pilot Operations bring forth important safety concerns, especially regarding the lack of human redundancy in the flight deck, an increased workload for the single pilot, reduced situational awareness and a higher risk of human error.It is assumed that potential problems affecting Single Pilot Operations could be addressed by implementing an Augmented Reality (AR) device in the flight deck, by presenting additional information and supporting hints within the pilot’s field of view.This paper sets out to demonstrate the conduction of checklists aided by the Microsoft HoloLens. A system, called Pilot Assist, was developed that allows the pilot to conduct checklists interactively with a Microsoft HoloLens. Pilot Assist was developed and demonstrated with a fixed based Airbus A320 simulator at the Technical University of Wildau.By using the HoloLens’ speech recognition and spatial mapping capabilities – scanning and recognizing the environment around the user – it is possible to trigger the execution of checklists. The user is then prompted towards the location of each checklist item in the cockpit, where information regarding necessary actions is projected. Pilot Assist is integrated with the aircraft systems, which allows to obtain aircraft status data in real time and automate the progress through the checklists.Pilots, researchers and engineers had the chance to qualitatively assess the Pilot Assist tool. the experts agreed Pilot Assist provides beneficial support during single pilot operation considering the current prototypical nature of the system. They pointed to limitations of both Pilot Assist and the HoloLens itself, but shared optimism as to how this technology and similar applications could indeed impact the future of flight operations.

A Holographic Checklist Assistant for the Single Pilot

Flight tests performed in an A330 simulator have shown that the existing standardized Ground Based Augmentation System (GBAS) CAT I could be suitable for CAT II operations if anomalous ionospheric errors induced during solar storms could be mitigated either operationally or technically. Tests in an Airbus A330 Level D (fully certified training simulator) flight simulator were performed by EUROCONTROL in cooperation with Technical University of Berlin, to investigate whether the worst case errors of a GBAS CAT I precision approach system (10m vertical deviation) would still be acceptable for CAT II operations. Additionally similar assessment was performed for the new approach classification termed “Other Than Standard” (OTS) CAT II which allows operation of CAT II approved aircraft down to a decision height (DH) of 100 ft on runways not meeting full Cat II requirements. OTS CAT II relaxes the approach lighting requirements which is compensated by a higher required Runway Visual Range (RVR) and higher guidance performance requirements (i.e. autoland). GBAS CAT I suitability to support OTS CAT II and CAT II operation would provide a new potential benefit for airlines that have invested in a GBAS CAT I system. Those benefits could be increased access to runways served by GBAS in low visibility conditions. It could improve as well traffic flow in low visibility operation by replacing the standard Instrument Landing System (ILS), which requires larger separation on final approach during CAT II/III operations due to technical design. The tests have shown that the existing GBAS CAT I standardized system, with a Vertical Alert Limit (VAL) of 10 m as per ICAO Annex 10, would be operationally suitable for CAT II and OTS CAT II operations. The tests demonstrated that failures up to 10m vertical deviation are operationally acceptable for CAT II operations as the landing point was always well within the landing box criteria used for the demonstration of autoland systems. Lateral deviations were easily detected but pilots expressed difficulty in detecting the vertical errors. Some but not all pilots detected errors larger than 10m (i.e. 15m). The visual cues were sufficient at the decision height (DH) to conduct the OTS CAT II operation. The perception was that due to the increased RVR required for OTS CAT II, the visual cues at DH were even better in OTS CAT II conditions than in standard CAT II conditions.

Ferdinand Behrend

Papers

Single Pilot Operations with AR-Glasses using Microsoft HoloLens

Simulations investigating combined effect of lateral and vertical navigation errors on PBN to xLS transition

A Holographic Checklist Assistant for the Single Pilot

CAT II / OTS CAT II Operations: Using excisting CAT I ground based augmentation system

Simulations investigating curved departure and arrival procedures using GNSS based vertical guidance