Localization of sound sources in robotics: A review

Current access infrastructures are characterized by heterogeneity, low latency, high throughput, and high computational capability, enabling massive concurrent connections and various services. Unfortunately, this design does not pay significant attention to mobile services in underserved areas. In this context, the use of aerial radio access networks (ARANs) is a promising strategy to complement existing terrestrial communication systems. Involving airborne components such as unmanned aerial vehicles, drones, and satellites, ARANs can quickly establish a flexible access infrastructure on demand. ARANs are expected to support the development of seamless mobile communication systems toward a comprehensive sixth-generation (6G) global access infrastructure. This paper provides an overview of recent studies regarding ARANs in the literature. First, we investigate related work to identify areas for further exploration in terms of recent knowledge advancements and analyses. Second, we define the scope and methodology of this study. Then, we describe ARAN architecture and its fundamental features for the development of 6G networks. In particular, we analyze the system model from several perspectives, including transmission propagation, energy consumption, communication latency, and network mobility. Furthermore, we introduce technologies that enable the success of ARAN implementations in terms of energy replenishment, operational management, and data delivery. Subsequently, we discuss application scenarios envisioned for these technologies. Finally, we highlight ongoing research efforts and trends toward 6G ARANs.

Survey on Aerial Radio Access Networks: Toward a Comprehensive 6G Access Infrastructure

Lightweight and optimized sound source localization and tracking methods for open and closed microphone array configurations

This paper introduces DREGON, a novel publicly-available dataset that aims at pushing research in sound source localization using a microphone array embedded in an unmanned aerial vehicle (UAV). The dataset contains both clean and noisy in-flight audio recordings continuously annotated with the 3D position of the target sound source using an accurate motion capture system. In addition, various signals of interests are available such as the rotational speed of individual rotors and inertial measurements at all time. Besides introducing the dataset, this paper sheds light on the specific properties, challenges and opportunities brought by the emerging task of UAV-embedded sound source localization. Several baseline methods are evaluated and compared on the dataset, with real-time applicability in mind. Very promising results are obtained for the localization of a broad-band source in loud noise conditions, while speech localization remains a challenge under extreme noise levels.

/pdf/dregon-dataset-and-methods-for-uav-embedded-sound-source-4eyjbkcann.pdf

DREGON: Dataset and Methods for UAV-Embedded Sound Source Localization

In recent years, research and development in aerial robotics (i.e., unmanned aerial vehicles, UAVs) has been growing at an unprecedented speed, and there is a need to summarize the background, latest developments, and trends of UAV research. Along with a general overview on the definition, types, categories, and topics of UAV, this work describes a systematic way to identify 1,318 high-quality UAV papers from more than thirty thousand that have been appeared in the top journals and conferences. On top of that, we provide a bird's-eye view of UAV research since 2001 by summarizing various statistical information, such as the year, type, and topic distribution of the UAV papers. We make our survey list public and believe that the list can not only help researchers identify, study, and compare their work, but is also useful for understanding research trends in the field. From our survey results, we find there are many types of UAV, and to the best of our knowledge, no literature has attempted to summarize all types in one place. With our survey list, we explain the types within our survey and outline the recent progress of each. We believe this summary can enhance readers' understanding on the UAVs and inspire researchers to propose new methods and new applications.

Recent Developments in Aerial Robotics: A Survey and Prototypes Overview

In search and rescue activities, unmanned aerial vehicles (UAV) should exploit sound information to compensate for poor visual information. This paper describes the design and implementation of a UAV-embedded microphone array system for sound source localization in outdoor environments. Four critical development problems included water-resistance of the microphone array, efficiency in assembling, reliability of wireless communication, and sufficiency of visualization tools for operators. To solve these problems, we developed a spherical microphone array system (SMAS) consisting of a microphone array, a stable wireless network communication system, and intuitive visualization tools. The performance of SMAS was evaluated with simulated data and a demonstration in the field. Results confirmed that the SMAS provides highly accurate localization, water resistance, prompt assembly, stable wireless communication, and intuitive information for observers and operators.

/pdf/design-of-uav-embedded-microphone-array-system-for-sound-2hdcn4uld6.pdf

Design of UAV-Embedded Microphone Array System for Sound Source Localization in Outdoor Environments

This paper addresses online outdoor sound source localization using a microphone array embedded in an unmanned aerial vehicle (UAV). In addition to sound source localization, sound source enhancement and robust communication method are also described. This system is one instance of deployment of our continuously developing open source software for robot audition called HARK (Honda Research Institute Japan Audition for Robots with Kyoto University). To improve the robustness against outdoor acoustic noise, we propose to combine two sound source localization methods based on MUSIC (multiple signal classification) to cope with trade-off between latency and noise robustness. The standard Eigenvalue decomposition based MUSIC (SEVD-MUSIC) has smaller latency but less noise robustness, whereas the incremental generalized singular value decomposition based MUSIC (iGSVD-MUSIC) has higher noise robustness but larger latency. A UAV operator can use an appropriate method according to the situation. A sound enhancement method called online robust principal component analysis (ORPCA) enables the operator to detect a target sound source more easily. To improve the stability of wireless communication, and robustness of the UAV system against weather changes, we developed data compression based on free lossless audio codec (FLAC) extended to support a 16 ch audio data stream via UDP, and developed a water-resistant microphone array. The resulting system successfully worked in an outdoor search and rescue task in ImPACT Tough Robotics Challenge in November 2016.

Development of microphone-array-embedded UAV for search and rescue task

Unmanned multirotor helicopters have been expected for various critical tasks such as rescue missions, and it is important to sense the environment to achieve those tasks. In order to realize such sensor systems, this paper considers acoustic information recognition from a hovering helicopter which a microphone array system. As the noise of rotors distorts the acoustic information, noise reduction is necessary. Since the rotor noise varies even at the hovering flight, an acoustic model of the rotor noise is derived taking the dynamics of the helicopter into account, and the model is utilized to evaluate the performance of the microphone array. The proposed approach was verified by evaluating the optimality of a real device that was empirically tuned in authors previous work, and the computed configuration of the microphone array and the developed one coincide well, which ensures the validity of the approach. As the validation through a practical application, the signal was processed by Delay-and-Sum Beam Former to localize the sound source. The system was able to find peaks of the power that corresponded to the sound source.

Design model of microphone arrays for multirotor helicopters

In order to achieve acoustic surveillance by using a multirotor helicopter, a microphone array specially tailored for the helicopter is considered in this paper. Several preliminary experiments were conducted to evaluate how the noise generated by rotors distorted the target sound signal from the view point of sound source localization, and the microphone array was developed based on results of the experiments. The microphone array was also installed on the helicopter, and it was tested to record the human voice during an autonomous waypoint flight in an outdoor environment. Investigating the measured auditory signal, it was shown that the proposed mechanism was capable to be utilized for the acoustic search in the outdoor environment.

A microphone array configuration for an auditory quadrotor helicopter system

High expectations are placed on the use of unmanned aerial vehicles (UAVs) in such tasks as rescue operations, which require a system that makes use of visual or auditory information to recognize the surrounding environment. As an example of such a system, this study examines the recognition of the environment using a helicopter mounted with a microphone array. Because the rotors of a helicopter generate high noise during operation, it is necessary to reduce the effects of this noise and those from other sources to record the audio signals coming from the ground with onboard microphones. In particular, because of helicopter body control, the rotor speed changes continuously and causes an unsteady rotor noise, which implies that it would be effective to arrange the microphones at a sufficient distance from the rotors. When a large microphone array is employed, however, the array weight may alter the helicopter’s flight characteristics and increase the noise, presenting a dilemma. This paper presents a model of rotor noise that takes into account the effect of the microphone array on the helicopter’s dynamic characteristics and proposes a method of evaluating the optimality of the array configuration, which is necessary for design. The validity of the proposed method is investigated using a multirotor helicopter mounted with a microphone array previously developed by the authors. In addition, an application example for locating sound sources on the ground using this helicopter is presented.

Takahiro Ishiki

Papers

Design of UAV-Embedded Microphone Array System for Sound Source Localization in Outdoor Environments

Development of microphone-array-embedded UAV for search and rescue task

Design model of microphone arrays for multirotor helicopters

A microphone array configuration for an auditory quadrotor helicopter system

Evaluation of Microphone Array for Multirotor Helicopters