Showing papers by "Giuseppe Loianno published in 2021"

Cooperative Transportation of Cable Suspended Payloads With MAVs Using Monocular Vision and Inertial Sensing

Abstract: Micro Aerial Vehicles (MAVs) have the great potential to be deployed in commercial or health care services such as e-commerce package delivery, transportation of medicines, same-day food delivery, and other time-sensitive transportation tasks. A team of MAVs can cooperatively transport objects to overcome the physical limitations of a single vehicle, while concurrently increasing the system's resilience to vehicles’ failures. In this letter, we address the state estimation, control and trajectory tracking problems of cooperative transportation of cable suspended rigid body payloads with MAVs using monocular vision and inertial sensing. The key contributions are (a) a distributed vision-based coordinated control of the cable-suspended rigid body payload on SE(3), (b) a distributed estimation approach that allows each agent to estimate its cable direction and velocity independently, and (c) a new cooperative estimation scheme that can infer the payload's full 6-DoF states. This is obtained by sharing the robots’ local position estimates and their relative position with respect to their corresponding attachment points on the payload. It allows to infer the payload's 6-DoF state when it is not directly measurable by each vehicle. The proposed solution runs in real-time on board and is validated through experimental results with multiple quadrotors transporting a rigid body payload via cables.

Mobile Manipulator for Autonomous Localization, Grasping and Precise Placement of Construction Material in a Semi-Structured Environment

Abstract: Mobile manipulators have the potential to revolutionize modern agriculture, logistics and manufacturing. In this work, we present the design of a ground-based mobile manipulator for automated structure assembly. The proposed system is capable of autonomous localization, grasping, transportation and deployment of construction material in a semi-structured environment. Special effort was put into making the system invariant to lighting changes, and not reliant on external positioning systems. Therefore, the presented system is self-contained and capable of operating in outdoor and indoor conditions alike. Finally, we present means to extend the perceptive radius of the vehicle by using it in cooperation with an autonomous drone, which provides aerial reconnaissance. Performance of the proposed system has been evaluated in a series of experiments conducted in real-world conditions.

Tracking and Relative Localization of Drone Swarms With a Vision-Based Headset

Abstract: We address the detection, tracking, and relative localization of the agents of a drone swarm from a human perspective using a headset equipped with a single camera and an Inertial Measurement Unit (IMU). We train and deploy a deep neural network detector on image data to detect the drones. A joint probabilistic data association filter resolves the detection problems and couples this information with the headset IMU data to track the agents. In order to estimate the drones’ relative poses in 3D space with respect to the human, we use an additional deep neural network that processes image regions of the drones provided by the tracker. Finally, to speed up the deep neural networks’ training, we introduce an automated labeling process relying on a motion capture system. Several experimental results validate the effectiveness of the proposed approach. The approach is real-time, does not rely on any communication between the human and the drones, and can scale to a large number of agents, often called swarms. It can be used to spatially task a swarm of drones and also employed without a headset for formation control and coordination of terrestrial vehicles.

Lightweight UAV-based Measurement System for Air-to-Ground Channels at 28 GHz

Abstract: Wireless communication at millimeter wave frequencies is an attractive option for high-bit-rate connectivity to unmanned aerial vehicles (UAVs). However, conducting the channel measurements necessary to assess the communication performance at these frequencies has been challenging due to the severe payload and power restrictions in commercial UAVs. This work presents a novel lightweight (approximately 1.3kg) channel measurement system at 28GHz installed on a commercially available UAV. A ground transmitter equipped with a horn antenna conveys sounding signals to a UAV equipped with a lightweight spectrum analyzer. We demonstrate that the measurements can be highly influenced by the onboard antenna pattern as shaped by the UAV’s frame. A calibration procedure is presented to correct for the resulting angular variations in antenna gain. The measurement setup is then validated on real flights from an airstrip at distances in excess of 300m.

PCMPC: Perception-Constrained Model Predictive Control for Quadrotors with Suspended Loads using a Single Camera and IMU

Abstract: In this paper, we address the Perception– Constrained Model Predictive Control (PCMPC) and state estimation problems for quadrotors with cable suspended payloads using a single camera and Inertial Measurement Unit (IMU). We design a receding–horizon control strategy for cable suspended payloads directly formulated on the system manifold configuration space SE (3) ×S2. The approach considers the system dynamics, actuator limits and the camera’s Field Of View (FOV) constraint to guarantee the payload’s visibility during motion. The monocular camera, IMU, and vehicle’s motor speeds are combined to provide estimation of the vehicle’s states in 3D space, the payload’s states, the cable’s direction and velocity. The proposed control and state estimation solution runs in real-time at 500 Hz on a small quadrotor equipped with a limited computational unit. The approach is validated through experimental results considering a cable suspended payload trajectory tracking problem at different speeds.

Defending a Perimeter from a Ground Intruder Using an Aerial Defender: Theory and Practice

Abstract: The perimeter defense game has received interest in recent years as a variant of the pursuit-evasion game. A number of previous works have solved this game to obtain the optimal strategies for defender and intruder, but the derived theory considers the players as point particles with first-order assumptions. In this work, we aim to apply the theory derived from the perimeter defense problem to robots with realistic models of actuation and sensing and observe performance discrepancy in relaxing the first-order assumptions. In particular, we focus on the hemisphere perimeter defense problem where a ground intruder tries to reach the base of a hemisphere while an aerial defender constrained to move on the hemisphere aims to capture the intruder. The transition from theory to practice is detailed, and the designed system is simulated in Gazebo. Two metrics for parametric analysis and comparative study are proposed to evaluate the performance discrepancy.

Lightweight UAV-based Measurement System for Air-to-Ground Channels at 28 GHz

Abstract: Wireless communication at millimeter wave frequencies has attracted considerable attention for the delivery of high-bit-rate connectivity to unmanned aerial vehicles (UAVs). However, conducting the channel measurements necessary to assess communication at these frequencies has been challenging due to the severe payload and power restrictions in commercial UAVs. This work presents a novel lightweight (approximately 1.3 kg) channel measurement system at 28 GHz installed on a commercially available UAV. A ground transmitter equipped with a horn antenna conveys sounding signals to a UAV equipped with a lightweight spectrum analyzer. We demonstrate that the measurements can be highly influenced by the antenna pattern as shaped by the UAV's frame. A calibration procedure is presented to correct for the resulting angular variations in antenna gain. The measurement setup is then validated on real flights from an airstrip at distances in excess of 300 m.

Aggressive Visual Perching with Quadrotors on Inclined Surfaces.

Abstract: Autonomous Micro Aerial Vehicles (MAVs) have the potential to be employed for surveillance and monitoring tasks. By perching and staring on one or multiple locations aerial robots can save energy while concurrently increasing their overall mission time without actively flying. In this paper, we address the estimation, planning, and control problems for autonomous perching on inclined surfaces with small quadrotors using visual and inertial sensing. We focus on planning and executing of dynamically feasible trajectories to navigate and perch to a desired target location with on board sensing and computation. Our planner also supports certain classes of nonlinear global constraints by leveraging an efficient algorithm that we have mathematically verified. The on board cameras and IMU are concurrently used for state estimation and to infer the relative robot/target localization. The proposed solution runs in real-time on board a limited computational unit. Experimental results validate the proposed approach by tackling aggressive perching maneuvers with flight envelopes that include large excursions from the hover position on inclined surfaces up to 90$^\circ$, angular rates up to 600~deg/s, and accelerations up to 10m/s^2.

Semi-dense visual-inertial odometry and mapping for computationally constrained platforms

Abstract: In this paper we present a direct semi-dense stereo Visual-Inertial Odometry (VIO) algorithm enabling autonomous flight for quadrotor systems with Size, Weight, and Power (SWaP) constraints. The proposed approach is validated through experiments on a 250 g, 22 cm diameter quadrotor equipped with a stereo camera and an IMU. Semi-dense methods have superior performance in low texture areas, which are often encountered in robotic tasks such as infrastructure inspection. However, due to the measurement size and iterative nonlinear optimization, these methods are computationally more expensive. As the scale of the platform shrinks down, the available computation of the on-board CPU becomes limited, making autonomous navigation using optimization-based semi-dense tracking a hard problem. We show that our direct semi-dense VIO performs comparably to other state-of-the-art methods, while taking less CPU than other optimization-based approaches, making it suitable for computationally-constrained small platforms. Our method takes less amount of CPU than the state-of-the-art semi-dense method, VI-Stereo-DSO, due to a simpler framework in the algorithm and a multi-threaded code structure allowing us to run real-time state estimation on an ARM board. With a low texture dataset obtained with our quadrotor platform, we show that this method performs significantly better than sparse methods in low texture conditions encountered in indoor navigation. Finally, we demonstrate autonomous flight on a small platform using our direct semi-dense Visual-Inertial Odometry. Supplementary code, low texture datasets and videos can be found on our github repo: https://github.com/KumarRobotics/sdd_vio .

Millimeter-Wave UAV Coverage in Urban Environments

Abstract: With growing interest in mmWave connectivity for UAVs, a basic question is whether networks intended for terrestrial users can provide sufficient aerial coverage as well. To assess this possibility, the paper proposes a novel evaluation methodology using generative models trained on detailed ray tracing data. These models capture complex propagation characteristics and can be readily combined with antenna and beamforming assumptions. Extensive simulation using these models indicate that standard (street-level and downtilted) base stations at typical microcellular densities can indeed provide satisfactory UAV coverage. Interestingly, the coverage is possible via a conjunction of antenna sidelobes and strong reflections. With sparser deployments, the coverage is only guaranteed at progressively higher altitudes. Additional dedicated (rooftop-mounted and uptilted) base stations strengthen the coverage provided that their density is comparable to that of the standard deployment, and would be instrumental for sparse deployments of the latter.

PCMPC: Perception-Constrained Model Predictive Control for Quadrotors with Suspended Loads using a Single Camera and IMU

Abstract: In this paper, we address the Perception--Constrained Model Predictive Control (PCMPC) and state estimation problems for quadrotors with cable suspended payloads using a single camera and Inertial Measurement Unit (IMU). We design a receding--horizon control strategy for cable suspended payloads directly formulated on the system manifold configuration space SE(3)xS^2. The approach considers the system dynamics, actuator limits and the camera's Field Of View (FOV) constraint to guarantee the payload's visibility during motion. The monocular camera, IMU, and vehicle's motor speeds are combined to provide estimation of the vehicle's states in 3D space, the payload's states, the cable's direction and velocity. The proposed control and state estimation solution runs in real-time at 500 Hz on a small quadrotor equipped with a limited computational unit. The approach is validated through experimental results considering a cable suspended payload trajectory tracking problem at different speeds.

Design and Deployment of an Autonomous Unmanned Ground Vehicle for Urban Firefighting Scenarios.

Abstract: Autonomous mobile robots have the potential to solve missions that are either too complex or dangerous to be accomplished by humans. In this paper, we address the design and autonomous deployment of a ground vehicle equipped with a robotic arm for urban firefighting scenarios. We describe the hardware design and algorithm approaches for autonomous navigation, planning, fire source identification and abatement in unstructured urban scenarios. The approach employs on-board sensors for autonomous navigation and thermal camera information for source identification. A custom electro{mechanical pump is responsible to eject water for fire abatement. The proposed approach is validated through several experiments, where we show the ability to identify and abate a sample heat source in a building. The whole system was developed and deployed during the Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2020, for Challenge No. 3 Fire Fighting Inside a High-Rise Building and during the Grand Challenge where our approach scored the highest number of points among all UGV solutions and was instrumental to win the first place.

Comparative Analysis of Agent-Oriented Task Assignment and Path Planning Algorithms Applied to Drone Swarms.

Abstract: Autonomous drone swarms are a burgeoning technology with significant applications in the field of mapping, inspection, transportation and monitoring. To complete a task, each drone has to accomplish a sub-goal within the context of the overall task at hand and navigate through the environment by avoiding collision with obstacles and with other agents in the environment. In this work, we choose the task of optimal coverage of an environment with drone swarms where the global knowledge of the goal states and its positions are known but not of the obstacles. The drones have to choose the Points of Interest (PoI) present in the environment to visit, along with the order to be visited to ensure fast coverage. We model this task in a simulation and use an agent-oriented approach to solve the problem. We evaluate different policy networks trained with reinforcement learning algorithms based on their effectiveness, i.e. time taken to map the area and efficiency, i.e. computational requirements. We couple the task assignment with path planning in an unique way for performing collision avoidance during navigation and compare a grid-based global planning algorithm, i.e. Wavefront and a gradient-based local planning algorithm, i.e. Potential Field. We also evaluate the Potential Field planning algorithm with different cost functions, propose a method to adaptively modify the velocity of the drone when using the Huber loss function to perform collision avoidance and observe its effect on the trajectory of the drones. We demonstrate our experiments in 2D and 3D simulations.

Defending a Perimeter from a Ground Intruder Using an Aerial Defender: Theory and Practice

Abstract: The perimeter defense game has received interest in recent years as a variant of the pursuit-evasion game. A number of previous works have solved this game to obtain the optimal strategies for defender and intruder, but the derived theory considers the players as point particles with first-order assumptions. In this work, we aim to apply the theory derived from the perimeter defense problem to robots with realistic models of actuation and sensing and observe performance discrepancy in relaxing the first-order assumptions. In particular, we focus on the hemisphere perimeter defense problem where a ground intruder tries to reach the base of a hemisphere while an aerial defender constrained to move on the hemisphere aims to capture the intruder. The transition from theory to practice is detailed, and the designed system is simulated in Gazebo. Two metrics for parametric analysis and comparative study are proposed to evaluate the performance discrepancy.

Selective and Hierarchical Allocation of Sensing Resources for Anomalous Target Identification in Exploratory Missions

Abstract: We present an approach for selective, hierarchical allocation of sensing resources that aims to maximize information gain in exploratory missions such as search and rescue (SAR) or surveillance in an efficient manner. Specifically, we propose a methodology for perception-enabled SAR or crowd surveillance driven by anomaly detection based on low-level statistical assessment of a region. The characterizations of previously-observed regions are used to populate a window of observations that serves as “short-term memory,” providing a contextually-appropriate characterization of proximate regions in the scene. Currently-observed regions are compared with this short-term memory window, and if sufficiently dissimilar, can be considered as candidates for the presence of a SAR target or unexpected event. We adaptively allocate additional sensing resources for subsequent exploration of anomalous regions through a novel utility function that balances varied mission objectives and constraints including exploratory sensing actions, maintaining situational awareness, or ensuring some degree of confidence in self-localization. Simulation results validate the proposed approach and demonstrate its benefits with regards to efficiency in exploration while maximizing potential information gain and balancing other mission requirements and objectives.

VIPose: Real-time Visual-Inertial 6D Object Pose Tracking.

Abstract: Estimating the 6D pose of objects is beneficial for robotics tasks such as transportation, autonomous navigation, manipulation as well as in scenarios beyond robotics like virtual and augmented reality. With respect to single image pose estimation, pose tracking takes into account the temporal information across multiple frames to overcome possible detection inconsistencies and to improve the pose estimation efficiency. In this work, we introduce a novel Deep Neural Network (DNN) called VIPose, that combines inertial and camera data to address the object pose tracking problem in real-time. The key contribution is the design of a novel DNN architecture which fuses visual and inertial features to predict the objects' relative 6D pose between consecutive image frames. The overall 6D pose is then estimated by consecutively combining relative poses. Our approach shows remarkable pose estimation results for heavily occluded objects that are well known to be very challenging to handle by existing state-of-the-art solutions. The effectiveness of the proposed approach is validated on a new dataset called VIYCB with RGB image, IMU data, and accurate 6D pose annotations created by employing an automated labeling technique. The approach presents accuracy performances comparable to state-of-the-art techniques, but with the additional benefit of being real-time.