Deep learning is recently showing outstanding results for solving a wide variety of robotic tasks in the areas of perception, planning, localization, and control. Its excellent capabilities for learning representations from the complex data acquired in real environments make it extremely suitable for many kinds of autonomous robotic applications. In parallel, Unmanned Aerial Vehicles (UAVs) are currently being extensively applied for several types of civilian tasks in applications going from security, surveillance, and disaster rescue to parcel delivery or warehouse management. In this paper, a thorough review has been performed on recent reported uses and applications of deep learning for UAVs, including the most relevant developments as well as their performances and limitations. In addition, a detailed explanation of the main deep learning techniques is provided. We conclude with a description of the main challenges for the application of deep learning for UAV-based solutions.

/pdf/a-review-of-deep-learning-methods-and-applications-for-1tbhgi7yqs.pdf

A Review of Deep Learning Methods and Applications for Unmanned Aerial Vehicles

Current access infrastructures are characterized by heterogeneity, low latency, high throughput, and high computational capability, enabling massive concurrent connections and various services. Unfortunately, this design does not pay significant attention to mobile services in underserved areas. In this context, the use of aerial radio access networks (ARANs) is a promising strategy to complement existing terrestrial communication systems. Involving airborne components such as unmanned aerial vehicles, drones, and satellites, ARANs can quickly establish a flexible access infrastructure on demand. ARANs are expected to support the development of seamless mobile communication systems toward a comprehensive sixth-generation (6G) global access infrastructure. This paper provides an overview of recent studies regarding ARANs in the literature. First, we investigate related work to identify areas for further exploration in terms of recent knowledge advancements and analyses. Second, we define the scope and methodology of this study. Then, we describe ARAN architecture and its fundamental features for the development of 6G networks. In particular, we analyze the system model from several perspectives, including transmission propagation, energy consumption, communication latency, and network mobility. Furthermore, we introduce technologies that enable the success of ARAN implementations in terms of energy replenishment, operational management, and data delivery. Subsequently, we discuss application scenarios envisioned for these technologies. Finally, we highlight ongoing research efforts and trends toward 6G ARANs.

Survey on Aerial Radio Access Networks: Toward a Comprehensive 6G Access Infrastructure

The use of UAVs in areas ranging from agriculture over urban services to entertainment or simply as a hobby has rapidly grown over the last years. Regarding serious/commercial applications, UAVs have been considered in the literature, especially as mobile sensing/actuation platforms (i.e., as a delivery platform for an increasingly wide range of sensors and actuators). With regard to timely, cost-effective and very rich data acquisition, both, NEC Research as well as TNO are pursuing investigations into the use of UAVs and swarms of UAVs for scenarios where high-resolution requirements, prohibiting environments or tight time constraints render traditional approaches ineffective. In this review article, we provide a brief overview of safety and security-focused application areas that we identified as main targets for industrial and commercial projects, especially in the context of intelligent autonomous systems and autonomous/semi-autonomously operating swarms. We discuss a number of challenges related to the deployment of UAVs in general and to their deployment within the identified application areas in particular. As such, this article is meant to serve as a review and overview of the literature and the state-of-the-art, but also to offer an outlook over our possible (near-term) future work and the challenges that we will face there.

Review: Using Unmanned Aerial Vehicles (UAVs) as Mobile Sensing Platforms (MSPs) for Disaster Response, Civil Security and Public Safety

Natural hazards have the potential to cause catastrophic damage and significant socioeconomic loss. The actual damage and loss observed in the recent decades has shown an increasing trend. As a result, disaster managers need to take a growing responsibility to proactively protect their communities by developing efficient management strategies. A number of research studies apply artificial intelligence (AI) techniques to process disaster-related data for supporting informed disaster management. This study provides an overview of current applications of AI in disaster management during its four phases: mitigation, preparedness, response, and recovery. It presents example applications of different AI techniques and their benefits for supporting disaster management at different phases, as well as some practical AI-based decision support tools. We find that the majority of AI applications focus on the disaster response phase. This study also identifies challenges to inspire the professional community to advance AI techniques for addressing them in future research.

Applications of artificial intelligence for disaster management

In search and rescue activities, unmanned aerial vehicles (UAV) should exploit sound information to compensate for poor visual information. This paper describes the design and implementation of a UAV-embedded microphone array system for sound source localization in outdoor environments. Four critical development problems included water-resistance of the microphone array, efficiency in assembling, reliability of wireless communication, and sufficiency of visualization tools for operators. To solve these problems, we developed a spherical microphone array system (SMAS) consisting of a microphone array, a stable wireless network communication system, and intuitive visualization tools. The performance of SMAS was evaluated with simulated data and a demonstration in the field. Results confirmed that the SMAS provides highly accurate localization, water resistance, prompt assembly, stable wireless communication, and intuitive information for observers and operators.

/pdf/design-of-uav-embedded-microphone-array-system-for-sound-2hdcn4uld6.pdf

Design of UAV-Embedded Microphone Array System for Sound Source Localization in Outdoor Environments

This paper addresses online outdoor sound source localization using a microphone array embedded in an unmanned aerial vehicle (UAV). In addition to sound source localization, sound source enhancement and robust communication method are also described. This system is one instance of deployment of our continuously developing open source software for robot audition called HARK (Honda Research Institute Japan Audition for Robots with Kyoto University). To improve the robustness against outdoor acoustic noise, we propose to combine two sound source localization methods based on MUSIC (multiple signal classification) to cope with trade-off between latency and noise robustness. The standard Eigenvalue decomposition based MUSIC (SEVD-MUSIC) has smaller latency but less noise robustness, whereas the incremental generalized singular value decomposition based MUSIC (iGSVD-MUSIC) has higher noise robustness but larger latency. A UAV operator can use an appropriate method according to the situation. A sound enhancement method called online robust principal component analysis (ORPCA) enables the operator to detect a target sound source more easily. To improve the stability of wireless communication, and robustness of the UAV system against weather changes, we developed data compression based on free lossless audio codec (FLAC) extended to support a 16 ch audio data stream via UDP, and developed a water-resistant microphone array. The resulting system successfully worked in an outdoor search and rescue task in ImPACT Tough Robotics Challenge in November 2016.

Development of microphone-array-embedded UAV for search and rescue task

This paper addresses sound source separation and identification for noise-contaminated acoustic signals recorded with a microphone array embedded in an Unmanned Aerial Vehicle (UAV), aiming at people's voice detection quickly and widely in a disaster situation. The key approach to achieve this is Deep Neural Network (DNN), but it is well known that training a DNN needs a huge dataset to improve its performance. In a practical application, building such a dataset is not often realistic owing to the cost of manual data annotation. Therefore, we propose a Partially-Shared Deep Neural Network (PS-DNN) which can learn multiple tasks at the same time with a small amount of annotated data. Preliminary results show that the PS-DNN outperforms conventional DNN-based approaches which require fully-annotated data in training in terms of identification accuracy. In addition, it maintains performance even when noise-suppressed signals are used for sound source separation training, and partially annotated data is used for sound source identification training.

Partially Shared Deep Neural Network in sound source separation and identification using a UAV-embedded microphone array

This paper addresses reduction of computational cost in training of a Deep Neural Network (DNN), in particular, for sound identification using highly noise-contaminated sound recorded with a microphone array embedded in an Unmanned Aerial Vehicle (UAV), aiming at people’s voice detection quickly and widely in a disastrous situation. It is known that a DNN training method called end-to-end training shows high performance, since it uses a huge neural network with high non-linearity which is trained with a large amount of raw input signals without preprocessing. Its computational cost is, however, expensive due to the high complexity of the neural network. Therefore, we propose two-stage DNN training using two separately-trained networks; denoising of sound sources and sound source identification. Since the huge network is divided into two smaller networks, the complexity of the networks is expected to decrease and each of them can consider a specific model of denoising and identification. This results in faster convergence and computational cost reduction in DNN training. Preliminary results showed that only 71 % of training time was necessary with the proposed two staged network, while maintaining the accuracy of sound source identification, compared to end-to-end training using noisy acoustic signals recorded with an 8 ch circular microphone array embedded in a UAV.

Takayuki Morito

Papers

Development of microphone-array-embedded UAV for search and rescue task

Partially Shared Deep Neural Network in sound source separation and identification using a UAV-embedded microphone array

Reduction of Computational Cost Using Two-Stage Deep Neural Network for Training for Denoising and Sound Source Identification