Building an efficient and reliable collision perception visual system is a challenging problem for future robots and autonomous vehicles. The biological visual neural networks, which have evolved over millions of years in nature and are working perfectly in the real world, could be ideal models for designing artificial vision systems. In the locust’s visual pathways, a lobula giant movement detector (LGMD), that is, the LGMD2, has been identified as a looming perception neuron that responds most strongly to darker approaching objects relative to their backgrounds; similar situations which many ground vehicles and robots are often faced with. However, little has been done on modeling the LGMD2 and investigating its potential in robotics and vehicles. In this article, we build an LGMD2 visual neural network which possesses the similar collision selectivity of an LGMD2 neuron in locust via the modeling of biased-ON and -OFF pathways splitting visual signals into parallel ON/OFF channels. With stronger inhibition (bias) in the ON pathway, this model responds selectively to darker looming objects. The proposed model has been tested systematically with a range of stimuli including real-world scenarios. It has also been implemented in a micro-mobile robot and tested with real-time experiments. The experimental results have verified the effectiveness and robustness of the proposed model for detecting darker looming objects against various dynamic and cluttered backgrounds.

/pdf/a-robust-collision-perception-visual-neural-network-with-1k99pu7ibk.pdf

A Robust Collision Perception Visual Neural Network With Specific Selectivity to Darker Objects

A review of GNSS-independent UAV navigation techniques

Most current unmanned aerial vehicles (UAVs) primarily use a global positioning system (GPS) and an inertial measurement unit (IMU) for position estimation. However, compared to birds and insects, the abilities of current UAVs to recognize the environment are not sufficient. To achieve autonomous flight of UAVs, like birds, the UAVs should be able to process and respond to information from their surrounding environment immediately. Therefore, in this paper, we propose a direct visual servoing system for UAVs, using an onboard high-speed monocular camera. There are two advantages of this system. First, the high image sampling rates help to improve the ability to recognize the environment. Second, the issue of control latency can be effectively solved because the position control signals are transmitted to the flight controller directly. In the experiment, the UAV could recognize a target at update rates of about 350 Hz, and a target tracking task was successfully realized.

Autonomous Target Tracking of UAV Using High-Speed Visual Feedback

Accurate and timely perception of collision in highly variable environments is still a challenging problem for artiﬁcial visual systems. As a source of inspiration, the lobula giant movement detectors (LGMDs) in locust’s visual pathways have been studied intensively, and modelled as quick collision detectors against challenges from various scenarios including vehicles and robots. However, the state-of-the-art LGMD models have not achieved acceptable robustness to deal with more challenging scenarios like the various vehicle driving scenes, due to the lack of adaptive signal processing mechanisms. To address this problem, we propose an improved neuronal system model, called LGMD+, that is featured by novel modelling of spatiotemporal inhibition dynamics with biological plausibilities including 1) lateral inhibitionswithglobalbiasesdeﬁnedbyavariantofGaussiandistribution,spatially,and2)anadaptivefeedforward inhibition mediation pathway, temporally. Accordingly, the LGMD+ performs more effectively to detect merely approaching objects threatening head-on collision risks by appropriately suppressing motion distractors caused by vibrations, near-miss or approaching stimuli with deviations from the centre view. Through evolutionary learning with a systematic dataset of various crash and non-collision driving scenarios, the LGMD+ shows improved robustness outperforming the previous related methods. After evolution, its computational simplicity, ﬂexibility and robustness have also been well demonstrated by real-time experiments of autonomous micro-mobile robots.

/pdf/improved-collision-perception-neuronal-system-model-with-4yue59x5ra.pdf

Improved Collision Perception Neuronal System Model with Adaptive Inhibition Mechanism and Evolutionary Learning

Similar to most visual animals, the crab Neohelice granulata relies predominantly on visual information to escape from predators, to track prey and for selecting mates. It, therefore, needs specialized neurons to process visual information and determine the spatial location of looming objects. In the crab Neohelice granulata, the Monostratified Lobula Giant type1 (MLG1) neurons have been found to manifest looming sensitivity with finely tuned capabilities of encoding spatial location information. MLG1s neuronal ensemble can not only perceive the location of a looming stimulus, but are also thought to be able to influence the direction of movement continuously, for example, escaping from a threatening, looming target in relation to its position. Such specific characteristics make the MLG1s unique compared to normal looming detection neurons in invertebrates which can not localize spatial looming. Modeling the MLG1s ensemble is not only critical for elucidating the mechanisms underlying the functionality of such neural circuits, but also important for developing new autonomous, efficient, directionally reactive collision avoidance systems for robots and vehicles. However, little computational modeling has been done for implementing looming spatial localization analogous to the specific functionality of MLG1s ensemble. To bridge this gap, we propose a model of MLG1s and their pre-synaptic visual neural network to detect the spatial location of looming objects. The model consists of 16 homogeneous sectors arranged in a circular field inspired by the natural arrangement of 16 MLG1s' receptive fields to encode and convey spatial information concerning looming objects with dynamic expanding edges in different locations of the visual field. Responses of the proposed model to systematic real-world visual stimuli match many of the biological characteristics of MLG1 neurons. The systematic experiments demonstrate that our proposed MLG1s model works effectively and robustly to perceive and localize looming information, which could be a promising candidate for intelligent machines interacting within dynamic environments free of collision. This study also sheds light upon a new type of neuromorphic visual sensor strategy that can extract looming objects with locational information in a quick and reliable manner.

/pdf/a-looming-spatial-localization-neural-network-inspired-by-1e5c870r.pdf

A Looming Spatial Localization Neural Network Inspired by MLG1 Neurons in the Crab Neohelice

Building a reliable and efficient collision avoidance system for unmanned aerial vehicles (UAVs) is still a challenging problem. This research takes inspiration from locusts, which can fly in dense swarms for hundreds of miles without collision. In the locust’s brain, a visual pathway of LGMD-DCMD (lobula giant movement detector and descending contra-lateral motion detector) has been identified as collision perception system guiding fast collision avoidance for locusts, which is ideal for designing artificial vision systems. However, there is very few works investigating its potential in real-world UAV applications. In this paper, we present an LGMD based competitive collision avoidance method for UAV indoor navigation. Compared to previous works, we divided the UAV’s field of view into four subfields each handled by an LGMD neuron. Therefore, four individual competitive LGMDs (C-LGMD) compete for guiding the directional collision avoidance of UAV. With more degrees of freedom compared to ground robots and vehicles, the UAV can escape from collision along four cardinal directions (e.g. the object approaching from the left-side triggers a rightward shifting of the UAV). Our proposed method has been validated by both simulations and real-time quadcopter arena experiments.

An LGMD Based Competitive Collision Avoidance Strategy for UAV

Building a reliable and efficient collision avoidance system for unmanned aerial vehicles (UAVs) is still a challenging problem. This research takes inspiration from locusts, which can fly in dense swarms for hundreds of miles without collision. In the locust's brain, a visual pathway of LGMD-DCMD (lobula giant movement detector and descending contra-lateral motion detector) has been identified as collision perception system guiding fast collision avoidance for locusts, which is ideal for designing artificial vision systems. However, there is very few works investigating its potential in real-world UAV applications. In this paper, we present an LGMD based competitive collision avoidance method for UAV indoor navigation. Compared to previous works, we divided the UAV's field of view into four subfields each handled by an LGMD neuron. Therefore, four individual competitive LGMDs (C-LGMD) compete for guiding the directional collision avoidance of UAV. With more degrees of freedom compared to ground robots and vehicles, the UAV can escape from collision along four cardinal directions (e.g. the object approaching from the left-side triggers a rightward shifting of the UAV). Our proposed method has been validated by both simulations and real-time quadcopter arena experiments.

The invention provides an unmanned aerial vehicle biomimetic obstacle avoidance control system based on LGMD. The system comprises a flight control subsystem, an optical flow sensor, a drive motor, anembedded LGMD detector, a camera, a wireless communication module and a ground station PC. The flow sensor and the embedded LGMD detector are electrically connected with the flight control subsystem.The wireless communication module and the drive motor are electrically connected with the output end of the flight control subsystem. The input end of the embedded LGMD detector is connected with theoutput end of the camera through signals. The wireless communication module is wirelessly connected with the ground station PC. The invention further provides an unmanned aerial vehicle biomimetic obstacle avoidance control method based on LGMD. By constructing an LGMD neural network and segmenting an image of field of view, spatial direction selection and scene prediction are realized during theflight of an unmanned aerial vehicle. The unmanned aerial vehicle can carry out real-time, efficient and obstacle avoidance flying in an unknown environment.

Unmanned aerial vehicle biomimetic obstacle avoidance control system and method based on LGMD

Nowadays, great uncertainty still exists on the urban- and regional-scale anthropogenic CO2 emission estimation based on emission inventories. In order to achieve the carbon peaking and neutrality targets for China, it is urgent to accurately estimate anthropogenic CO2 emissions at regional scales, especially in large urban agglomerations. Using two inventories (EDGAR v6.0 inventory and a modified inventory combining EDGAR v6.0 with GCG v1.0) as prior anthropogenic CO2 emission datasets andtaking themas input data respectively, this study utilized the WRF-STILT atmospheric transport model to simulate atmospheric CO2 concentration in the Yangtze River Delta region from December 2017 to February 2018. The simulated atmospheric CO2 concentrations were further improved by referencing atmospheric CO2 concentration observation at a tall tower in Quanjiao County of Anhui Province and using the scaling factors obtained from the Bayesian inversion method. An estimation of anthropogenic CO2 emission flux in the Yangtze River Delta regionwas finally accomplished. The results indicated that:①in winter, in comparison to the atmospheric CO2 concentration simulated based on EDGAR v6.0, the atmospheric CO2 concentration simulated based on the modified inventory was more consistent with observed values. ②The simulated atmospheric CO2 concentration was higher than observation at night and lower than observation during the daytime. The CO2 emission data of emission inventories could not fully reflect the diurnal variation in anthropogenic emissions, andtheoverestimation, caused by the simulated low-atmospheric boundary layer height at night, of the contribution from point sources with higher emission height near the observation station were the main reasons. ③The simulation performance on atmospheric CO2 concentration was greatly affected by the emission bias of the EDGAR grid points that significantly contributed to concentrations of the observation station, and this indicated that the uncertainty in the spatial distribution in EDGAR emission was the main factor influencing the simulation accuracy. ④The posterior anthropogenic CO2 emission flux in the Yangtze River Delta from December 2017 to February 2018 was around (0.184±0.006) mg·(m2·s)-1and (0.183±0.007) mg·(m2·s)-1 based on EDGAR and the modified inventory, respectively. It is suggested that the inventories with higher temporal and spatial resolutions and more accurate spatial emission distribution should be selected as the prior emissions to obtain a more accurate estimation of the regional anthropogenic CO2 emissions.

Xingzao Ma

Papers

An LGMD Based Competitive Collision Avoidance Strategy for UAV

An LGMD Based Competitive Collision Avoidance Strategy for UAV

Unmanned aerial vehicle biomimetic obstacle avoidance control system and method based on LGMD

[Simulation of Anthropogenic CO2 Emissions in the Yangtze River Delta Based on Different Emission Inventories].