Introduction - The aims of this book the features of this book organization and background the analytical mind: strengh and weakness. Fundamentals of linear algebra - Vector and matrix calculus Eigenvalue problem linear systems and optimization matrix and tensor algebra. Probabilities and statistical estimation - probability distributions manifolds and local distributions gaussian distributions and X2 distributions statistical estimation for gaussian models general statistical estimation maximum likelihood estimation Akaike information criterion. Representation of geometric objects - image points and image lines space points and space lines space planes conics space conics and quadrics coordinate transformation and projection. Geometric correction - general theory correction of image points and image lines correction of space points and space lines correction of space planes orthogonality correction conic incidence correction. 3-D computation by stereo vision - epipolar constraint optimal correction of correspondence 3-D reconstruction of points 3-D reconstruction of lines optimal back projection onto a space plane scenes infinitely far away camera calibration errors. Parametric fitting - general theory optimal fitting for image points optimal fitting for image lines optimal fitting for space points optimal fitting for space lines optimal fitting for space planes. Optimal filter - general theory iterative estimation scheme effective gradient approximation reduction from the klaman filter estimation from linear hypotheses. Renormalization - eigenvector fit unbiased eigenvector generalized eigenvalue fit renormalization lincarization second order renormalization. Applications of geometric estimation - image line fitting conic fitting space plane fitting by range sensing space plane fitting by stereo vision. 3-D motion analysis - general theory lincarization and renormalization optimal correction and decomposition reliability of 3-D reconstruction critical surfaces 3-D reconstruction from planar surface motion camera rotation and information. 3-D interpretation of optical flow - optical flow detection theoretical basis of 3-D interpretation optimal estimation of motion parameters. (Part contents).

Statistical optimization for geometric computation : theory and practice

Automatic obstacle detection and avoidance is a key component for the success of micro-aerial vehicles (MAVs) in the future. As the payload of MAVs is highly constrained, cameras are attractive sensors because they are both lightweight and provide rich information about the environment. In this paper, we present an approach that allows a quadrotor with a single monocular camera to locally generate collision-free waypoints. We acquire a small set of images while the quadrotor is hovering from which we compute a dense depth map. Based on this depth map, we render a 2D scan and generate a suitable waypoint for navigation. In our experiments, we found that the pose variation during hovering is already sufficient to obtain suitable depth maps. The computation takes less than one second which renders our approach applicable for obstacle avoidance in real-time. We demonstrate the validity of our approach in challenging environments where we navigate a Parrot Ardrone quadrotor successfully through narrow passages including doors, boxes, and people.

/pdf/collision-avoidance-for-quadrotors-with-a-monocular-camera-392zl3cixz.pdf

Collision Avoidance for Quadrotors with a Monocular Camera

Micro aerial vehicles (MAV) pose a challenge in designing sensory systems and algorithms due to their size and weight constraints and limited computing power. We present an efficient 3D multi-resolution map that we use to aggregate measurements from a lightweight continuously rotating laser scanner. We estimate the robot's motion by means of visual odometry and scan registration, aligning consecutive 3D scans with an incrementally built map. By using local multi-resolution, we gain computational ef- ficiency by having a high resolution in the near vicinity of the robot and a lower resolution with increasing distance from the robot, which correlates with the sensor's characteristics in relative distance accuracy and measurement density. Compared to uniform grids, local multi-resolution leads to the use of fewer grid cells without loosing information and consequently results in lower computational costs. We efficiently and accurately register new 3D scans with the map in order to estimate the motion of the MAV and update the map in-flight. In experiments, we demonstrate superior accuracy and efficiency of our registration approach compared to state-of-the- art methods such as GICP. Our approach builds an accurate 3D obstacle map and estimates the vehicle's trajectory in real-time. I. INTRODUCTION Micro aerial vehicles (MAV) such as quadrotors have attracted attention in the field of aerial robotics. Their size and weight limitations pose a challenge in designing sensory systems. Most of today's MAVs are equipped with ultra sound sensors and camera systems due to their minimal size and weight. While these small and lightweight sensors provide valuable information, they suffer from a limited field- of-view and are sensitive to illumination conditions. Only few systems (1), (2), (3), (4) are equipped with 2D laser range finders (LRF) that are used for navigation. In contrast, we build a continuously rotating laser scanner that is minimalistic in terms of size and weight and thus particularly well suited for obstacle perception and localiza- tion on MAVs, allowing for environment perception in all directions. We use a hybrid multi-resolution map that stores occu- pancy information and the respective distance measurements. Measurements are stored in grid cells with increasing cell size from the robot's center. Thus, we gain computational efficiency by having a high resolution in the close proximity to the sensor and a lower resolution with increasing distance, which correlates with the sensor's characteristics in relative distance accuracy and measurement density. Compared to

/pdf/local-multi-resolution-representation-for-6d-motion-2lbh8oiy0y.pdf

Local multi-resolution representation for 6D motion estimation and mapping with a continuously rotating 3D laser scanner

. The HD(CP)2 Observational Prototype Experiment (HOPE) was performed as a major 2-month field experiment in Julich, Germany, in April and May 2013, followed by a smaller campaign in Melpitz, Germany, in September 2013. HOPE has been designed to provide an observational dataset for a critical evaluation of the new German community atmospheric icosahedral non-hydrostatic (ICON) model at the scale of the model simulations and further to provide information on land-surface–atmospheric boundary layer exchange, cloud and precipitation processes, as well as sub-grid variability and microphysical properties that are subject to parameterizations. HOPE focuses on the onset of clouds and precipitation in the convective atmospheric boundary layer. This paper summarizes the instrument set-ups, the intensive observation periods, and example results from both campaigns. HOPE-Julich instrumentation included a radio sounding station, 4 Doppler lidars, 4 Raman lidars (3 of them provide temperature, 3 of them water vapour, and all of them particle backscatter data), 1 water vapour differential absorption lidar, 3 cloud radars, 5 microwave radiometers, 3 rain radars, 6 sky imagers, 99 pyranometers, and 5 sun photometers operated at different sites, some of them in synergy. The HOPE-Melpitz campaign combined ground-based remote sensing of aerosols and clouds with helicopter- and balloon-based in situ observations in the atmospheric column and at the surface. HOPE provided an unprecedented collection of atmospheric dynamical, thermodynamical, and micro- and macrophysical properties of aerosols, clouds, and precipitation with high spatial and temporal resolution within a cube of approximately 10  ×  10  ×  10 km3. HOPE data will significantly contribute to our understanding of boundary layer dynamics and the formation of clouds and precipitation. The datasets have been made available through a dedicated data portal. First applications of HOPE data for model evaluation have shown a general agreement between observed and modelled boundary layer height, turbulence characteristics, and cloud coverage, but they also point to significant differences that deserve further investigations from both the observational and the modelling perspective.

/pdf/the-hd-cp-2-observational-prototype-experiment-hope-an-2u19n3o746.pdf

The HD(CP)2 Observational Prototype Experiment (HOPE) - An overview

Micro aerial vehicles, such as multirotors, are particularly well suited for the autonomous monitoring, inspection, and surveillance of buildings, e.g., for maintenance or disaster management. Key prerequisites for the fully autonomous operation of micro aerial vehicles are real-time obstacle detection and planning of collision-free trajectories. In this article, we propose a complete system with a multimodal sensor setup for omnidirectional obstacle perception consisting of a three-dimensional 3D laser scanner, two stereo camera pairs, and ultrasonic distance sensors. Detected obstacles are aggregated in egocentric local multiresolution grid maps. Local maps are efficiently merged in order to simultaneously build global maps of the environment and localize in these. For autonomous navigation, we generate trajectories in a multilayered approach: from mission planning over global and local trajectory planning to reactive obstacle avoidance. We evaluate our approach and the involved components in simulation and with the real autonomous micro aerial vehicle. Finally, we present the results of a complete mission for autonomously mapping a building and its surroundings.

Multilayered Mapping and Navigation for Autonomous Micro Aerial Vehicles

Reliably perceiving obstacles and avoiding collisions is key for the fully autonomous application of micro aerial vehicles (MAVs), Limiting factors for increasing autonomy and complexity of MAVs are limited onboard sensing and limited onboard processing power. In this paper, we propose a complete system with a multimodal sensor setup for omnidirectional obstacle perception. We developed a lightweight 3D laser scanner and visual obstacle detection using wide-angle stereo cameras. Detected obstacles are aggregated in egocentric grid maps. We implemented a fast reactive collision avoidance approach for safe operation in the vicinity of structures like buildings or vegetation.

http://www.ais.uni-bonn.de/papers/ECMR_2013_Nieuwenhuisen_Multimodal_Obstacle_Avoidance.pdf

Multimodal obstacle detection and collision avoidance for micro aerial vehicles

Online pose estimation and mapping in unknown environments is essential for most mobile robots. Especially autonomous unmanned aerial vehicles require good pose estimates at comparably high frequencies. In this paper, we propose an effective system for online pose and simultaneous map estimation designed for light-weight UAVs. Our system consists of two components: (1) real-time pose estimation combining RTK-GPS and IMU at 100 Hz and (2) an effective SLAM solution running at 10 Hz using image data from an omnidirectional multi-fisheye-camera system. The SLAM procedure combines spatial resection computed based on the map that is incrementally refined through bundle adjustment and combines the image data with raw GPS observations and IMU data on keyframes. The overall system yields a real-time, georeferenced pose at 100 Hz in GPS-friendly situations. Additionally, we obtain a precise pose and feature map at 10 Hz even in cases where the GPS is not observable or underconstrained. Our system has been implemented and thoroughly tested on a 5 kg copter and yields accurate and reliable pose estimation at high frequencies. We compare the point cloud obtained by our method with a model generated from georeferenced terrestrial laser scanner.

Fast and effective online pose estimation and mapping for UAVs

This paper presents a concept and first experiments on a keyframe-based incremental bundle adjustment for real-time structure and motion estimation in an unknown scene. In order to avoid periodic batch steps, we use the software iSAM2 for sparse nonlinear incremental optimization, which is highly efficient through incremental variable reordering and fluid relinearization. We adapted the software to allow for (1) multi-view cameras by taking the rigid transformation between the cameras into account, (2) omnidirectional cameras as it can handle arbitrary bundles of rays and (3) scene points at infinity, which improve the estimation of the camera orientation as points at the horizon can be observed over long periods of time. The real-time bundle adjustment refers to sets of keyframes, consisting of frames, one per camera, taken in a synchronized way, that are initiated if a minimal geometric distance to the last keyframe set is exceeded. It uses interest points in the keyframes as observations, which are tracked in the synchronized video streams of the individual cameras and matched across the cameras, if possible. First experiments show the potential of the incremental bundle adjustment w.r.t. time requirements. Our experiments are based on a multi-camera system with four fisheye cameras, which are mounted on a UAV as two stereo pairs, one looking ahead and one looking backwards, providing a large field of view.

/pdf/incremental-real-time-bundle-adjustment-for-multi-camera-1v4rakzykb.pdf

Incremental real-time bundle adjustment for multi-camera systems with points at infinity

. We present a novel approach for a rigorous bundle adjustment for omnidirectional and multi-view cameras, which enables an efficient maximum-likelihood estimation with image and scene points at infinity. Multi-camera systems are used to increase the resolution, to combine cameras with different spectral sensitivities (Z/I DMC, Vexcel Ultracam) or – like omnidirectional cameras – to augment the effective aperture angle (Blom Pictometry, Rollei Panoscan Mark III). Additionally multi-camera systems gain in importance for the acquisition of complex 3D structures. For stabilizing camera orientations – especially rotations – one should generally use points at the horizon over long periods of time within the bundle adjustment that classical bundle adjustment programs are not capable of. We use a minimal representation of homogeneous coordinates for image and scene points. Instead of eliminating the scale factor of the homogeneous vectors by Euclidean normalization, we normalize the homogeneous coordinates spherically. This way we can use images of omnidirectional cameras with single-view point like fisheye cameras and scene points, which are far away or at infinity. We demonstrate the feasibility and the potential of our approach on real data taken with a single camera, the stereo camera FinePix Real 3D W3 from Fujifilm and the multi-camera system Ladybug 3 from Point Grey.

/pdf/bundle-adjustment-for-multi-camera-systems-with-points-at-18mjtqzowl.pdf

Bundle Adjustment for Multi-Camera Systems with Points at Infinity

. We present a novel approach for dense 3-D cloud reconstruction above an area of 10 × 10 km2 using two hemispheric sky imagers with fisheye lenses in a stereo setup. We examine an epipolar rectification model designed for fisheye cameras, which allows the use of efficient out-of-the-box dense matching algorithms designed for classical pinhole-type cameras to search for correspondence information at every pixel. The resulting dense point cloud allows to recover a detailed and more complete cloud morphology compared to previous approaches that employed sparse feature-based stereo or assumed geometric constraints on the cloud field. Our approach is very efficient and can be fully automated. From the obtained 3-D shapes, cloud dynamics, size, motion, type and spacing can be derived, and used for radiation closure under cloudy conditions, for example. Fisheye lenses follow a different projection function than classical pinhole-type cameras and provide a large field of view with a single image. However, the computation of dense 3-D information is more complicated and standard implementations for dense 3-D stereo reconstruction cannot be easily applied. Together with an appropriate camera calibration, which includes internal camera geometry, global position and orientation of the stereo camera pair, we use the correspondence information from the stereo matching for dense 3-D stereo reconstruction of clouds located around the cameras. We implement and evaluate the proposed approach using real world data and present two case studies. In the first case, we validate the quality and accuracy of the method by comparing the stereo reconstruction of a stratocumulus layer with reflectivity observations measured by a cloud radar and the cloud-base height estimated from a Lidar-ceilometer. The second case analyzes a rapid cumulus evolution in the presence of strong wind shear.

/pdf/cloud-photogrammetry-with-dense-stereo-for-fisheye-cameras-3n1jdiqjue.pdf

Johannes Schneider

Papers

Multimodal obstacle detection and collision avoidance for micro aerial vehicles

Fast and effective online pose estimation and mapping for UAVs

Incremental real-time bundle adjustment for multi-camera systems with points at infinity

Bundle Adjustment for Multi-Camera Systems with Points at Infinity

Cloud photogrammetry with dense stereo for fisheye cameras