Unmanned Aerial Vehicles (UAVs) have been employed for multiple tasks in the last decade, including planetary exploration. A common task for UAVs is mapping, with maps most commonly represented in variable-resolution form, such as Octrees or their 2D-counterparts, Quadtrees. Convolutional Neural Networks (CNNs) have revolutionized sensor data processing, but are heavily influenced by the training data and the resolution of the images. In addition, Octree and Quadtree-based map representations typically insert values into the deepest level of the tree, regardless of the position of the UAV and the resulting image resolution. To alleviate both these issues, we propose a novel Quadtree implementation that considers the resolution of the observations when calculating insertion indices. The probabilistic mapping approach considers the tree structure when looking up probabilistic priors to insert generated observations. The system is implemented in ROS and tested in simulation and experiments with vertical, horizontal and mixed motions at different frequencies of observations. The map reproduction is evaluated with the Kullback-Leibler divergence and qualitatively. Increasing the frequency and approaching a target for closer inspection is shown to improve the quality of the map reproduction. Results also indicate that low-frequency, noisy observations can be overcome by increasing the frequency, opening up avenues for further research on mapping CNN observations. For a video demonstration visit: https://youtu.be/FdKu8BzjKOI 

/pdf/resolution-adaptive-quadtrees-for-semantic-segmentation-2go88hbj.pdf

Resolution-adaptive Quadtrees for Semantic Segmentation Mapping in UAV Applications

Radio altimeters are an important component of modern helicopter on-board systems. These devices currently involve the use of narrowband deterministic signals, that limits their potential technical characteristics. Given the significant breakthrough in the development of wideband and ultra-wideband radio electronics, it is promising to create on-board radio complexes capable of obtaining the necessary information using wideband stochastic signals. At the same time, when developing such complexes, it is important to use optimal synthesis methods for radio systems, which will allow optimal signal processing algorithms and potential accuracy parameters to be obtained. In this work, the algorithm to measure flight altitude for a helicopter or an unmanned aerial vehicle based on the processing of wideband and ultra-wideband pulsed stochastic signals is synthesized for the first time by the maximum-likelihood method. When formulating the problem, the mathematical model of the signal and observation is specified, and their statistical characteristics are investigated. The peculiarity of the synthesis task is the use of a noise pulse transmitter, which implements the function of an underlying surface illuminator, as well as considering the signal structure destruction during its radiation, propagation, and reflection. This signal shape destruction makes it impossible to synthesize a radar with internally coherent processing when working on one receiving antenna. In accordance with the synthesized algorithm, a simulation model of a pulsed radar with a stochastic probing signal has been developed and the results of its modeling are presented.

/pdf/signal-processing-algorithm-for-monopulse-noise-noncoherent-2kecvy2e.pdf

Signal Processing Algorithm for Monopulse Noise Noncoherent Wideband Helicopter Altitude Radar

Studies of past life forms on other planets are possible through the identification and localisation of desiccation cracks in ancient water bodies such as lakes, rivers and seas. Unmanned aerial vehicles (UAVs) are increasingly being used as a viable remote sensing solution for planetary exploration, as desiccation cracks are difficult to identify with the naked eye and are normally located in complex and unreachable environments. However, most UAVs have a strong reliance on human operators through their communication systems, as UAVs have limited onboard decision-making capabilities for autonomous navigation in such environments. UAV navigation in real-world scenarios is also challenging as data captured from their sensors is imperfect, and outputs from computer vision systems are, sometimes, inaccurate. These sensory and onboard vision limitations cause partial observability of the state of surveyed environments, inducing uncertainty in optimal path planning. This paper proposes a UAV system for autonomous onboard navigation, identification, and mapping of desiccation cracks for planetary exploration. The navigation problem is mathematically formulated as a partially observable Markov decision process (POMDP), where a motion strategy can be obtained by solving the POMDP in real time using the augmented belief tree (ABT) solver. The framework discussed in this work is validated with real flight tests using two desiccation crack patterns distributed across the surveyed area. Real-time segmentation from streamed camera frames of desiccation cracks is achieved through inference onboard the aircraft using a ResNet18 Convolutional Neural Network (CNN) model, and an OpenCV AI Kit (OAK)-D camera. Results from real flight tests indicate that the system can reduce levels of object detection uncertainty to locate and map desiccation cracks in environments under partial observability. The system design allows further adaptation for similar time-critical applications requiring increased levels of UAV autonomy in unstructured environments under uncertainty and partial observability, such as humanitarian relief, wildlife monitoring, and surveillance. 

https://eprints.qut.edu.au/226787/8/101887533.pdf

Autonomous Mapping of Desiccation Cracks via a Probabilistic-based Motion Planner Onboard UAVs

Future Mars Rotorcrafts will require the ability to precisely navigate to previously visited locations in order to return to a safe landing site or execute precise scientific measurements, such as sample acquisition or targeted sensing. To enable a future Mars Science Helicopter to perform in-flight loop closures, we present an on-board visual loop closure detection system based on a Bag-of-Words (BoW) approach that is efficient enough to run in real-time on the anticipated computationally constrained flight avionics. Our system establishes image-to-image associations between incoming images of a downward-looking navigation camera and previously observed geo-tagged keyframes stored in a database. The system extracts ORB features which are quantized into a BoW histogram using a custom one million words hierarchical vocabulary, trained on synthetic images from a Mars simulation. An efficient database query using an inverted index produces a set of candidate frames which we check for geometrical consistency. For efficient feature matching, we leverage the vocabulary to perform fast approximate nearest neighbor search. The geometrical check accepts loop closure pairs whose essential matrix is supported by a minimum number of feature matches, which are pre-selected based on a rotational consistency check. The vocabulary and the methods used for the geometrical consistency checks were chosen to maximize the performance while allowing single-thread real-time execution on a computationally constrained embedded processor. We demonstrate and evaluate the proposed system both on simulated and real-world data, including flight data from the Mars Helicopter <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Ingenuity</i> . 

Visual Loop Closure Detection for a Future Mars Science Helicopter

The next generation of Mars rotorcrafts requires on-board autonomous hazard avoidance landing. To this end, this work proposes a system that performs continuous multi-resolution height map reconstruction and safe landing spot detection. Structure-from-Motion measurements are aggregated in a pyramid structure using a novel Optimal Mixture of Gaus-sians formulation that provides a comprehensive uncertainty model. Our multiresolution pyramid is built more efficiently and accurately than past work by decoupling pyramid filling from the measurement updates of different resolutions. To detect the safest landing location, after an optimized hazard segmentation, we use a mean shift algorithm on multiple distance transform peaks to account for terrain roughness and uncertainty. The benefits of our contributions are evaluated on real and synthetic flight data.

Optimizing Terrain Mapping and Landing Site Detection for Autonomous UAVs

Future Mars Rotorcrafts require advanced navigation capabilities to enable all terrain access for science investigations with long distance flights that are executed fully autonomously. A critical component is the ability to safely land in hazardous terrain as part of a mission, or triggered by an emergency situation. In this paper, we present an advanced navigation system for continuous on-board terrain reconstruction for the purpose of hazard-free landing site detection for the autonomous navigation of a Mars Science Helicopter - a JPL research concept that investigates the feasibility of flying a multi-kilogram science payload at various Mars science locations, with flight ranges of multiple kilometers per flight. Our approach builds on a vision-based perception system that incorporates an on-board visual-inertial state estimator augmented by a laser altimeter (range-VIO), and a structure-from-motion 3D reconstruction approach that uses a single, downward-looking camera to provide dense depth measurements while the vehicle is in motion. Depth measurements are accumulated in a local, robot-centric, multi- resolution elevation map that is analyzed by a landing site detector to extract safe landing areas below the rotorcraft, based on a heuristic that includes slope, roughness and the presence of landing hazards. Detected landing sites are prioritized by an on-board autonomy engine that either selects suitable landing sites for immediate landing maneuvers, or can explore a terrain location as part of a mission in order to find a best landing site in a pre-planned area. We demonstrate and evaluate our approach on simulated data and data acquired with a surrogate unmanned aerial system (UAS) executing flights over relevant terrain.

Autonomous Safe Landing Site Detection for a Future Mars Science Helicopter

In this paper, we propose a resource-efficient approach to provide an autonomous UAV with an on-board perception method to detect safe, hazard-free landing sites during flights over complex 3D terrain. We aggregate 3D measurements acquired from a sequence of monocular images by a Structure-from-Motion approach into a local, robot-centric, multi-resolution elevation map of the overflown terrain, which fuses depth measurements according to their lateral surface resolution (pixel-footprint) in a probabilistic framework based on the concept of dynamic Level of Detail. Map aggregation only requires depth maps and the associated poses, which are obtained from an onboard Visual Odometry algorithm. An efficient landing site detection method then exploits the features of the underlying multi-resolution map to detect safe landing sites based on slope, roughness, and quality of the reconstructed terrain surface. The evaluation of the performance of the mapping and landing site detection modules are analyzed independently and jointly in simulated and real-world experiments in order to establish the efficacy of the proposed approach.

Pascal Schoppmann

Papers

Autonomous Safe Landing Site Detection for a Future Mars Science Helicopter

Multi-Resolution Elevation Mapping and Safe Landing Site Detection with Applications to Planetary Rotorcraft.