The use of unmanned aerial vehicles (UAVs) is growing rapidly across many civil application domains, including real-time monitoring, providing wireless coverage, remote sensing, search and rescue, delivery of goods, security and surveillance, precision agriculture, and civil infrastructure inspection. Smart UAVs are the next big revolution in the UAV technology promising to provide new opportunities in different applications, especially in civil infrastructure in terms of reduced risks and lower cost. Civil infrastructure is expected to dominate more than $45 Billion market value of UAV usage. In this paper, we present UAV civil applications and their challenges. We also discuss the current research trends and provide future insights for potential UAV uses. Furthermore, we present the key challenges for UAV civil applications, including charging challenges, collision avoidance and swarming challenges, and networking and security-related challenges. Based on our review of the recent literature, we discuss open research challenges and draw high-level insights on how these challenges might be approached.

Unmanned Aerial Vehicles: A Survey on Civil Applications and Key Research Challenges

In recent years, deep learning (DL), a rebranding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, and natural language processing. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV, e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should not only be aware of advancements such as DL, but also be leading researchers in this area. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools, and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as they relate to (i) inadequate data sets, (ii) human-understandable solutions for modeling physical phenomena, (iii) big data, (iv) nontraditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial, and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.

Comprehensive survey of deep learning in remote sensing: theories, tools, and challenges for the community

Deep learning is recently showing outstanding results for solving a wide variety of robotic tasks in the areas of perception, planning, localization, and control. Its excellent capabilities for learning representations from the complex data acquired in real environments make it extremely suitable for many kinds of autonomous robotic applications. In parallel, Unmanned Aerial Vehicles (UAVs) are currently being extensively applied for several types of civilian tasks in applications going from security, surveillance, and disaster rescue to parcel delivery or warehouse management. In this paper, a thorough review has been performed on recent reported uses and applications of deep learning for UAVs, including the most relevant developments as well as their performances and limitations. In addition, a detailed explanation of the main deep learning techniques is provided. We conclude with a description of the main challenges for the application of deep learning for UAV-based solutions.

/pdf/a-review-of-deep-learning-methods-and-applications-for-1tbhgi7yqs.pdf

A Review of Deep Learning Methods and Applications for Unmanned Aerial Vehicles

Smart cities contain intelligent things which can intelligently automatically and collaboratively enhance life quality, save people's lives, and act a sustainable resource ecosystem. To achieve these advanced collaborative technologies such as drones, robotics, artificial intelligence, and Internet of Things (IoT) are required to increase the smartness of smart cities by improving the connectivity, energy efficiency, and quality of services (QoS). Therefore, collaborative drones and IoT play a vital role in supporting a lot of smart-city applications such as those involved in communication, transportation, agriculture,safety and security, disaster mitigation, environmental protection, service delivery, energy saving, e-waste reduction, weather monitoring, healthcare, etc. This paper presents a survey of the potential techniques and applications of collaborative drones and IoT which have recently been proposed in order to increase the smartness of smart cities. It provides a comprehensive overview highlighting the recent and ongoing research on collaborative drone and IoT in improving the real-time application of smart cities. This survey is different from previous ones in term of breadth, scope, and focus. In particular, we focus on the new concept of collaborative drones and IoT for improving smart-city applications. This survey attempts to show how collaborative drones and IoT improve the smartness of smart cities based on data collection, privacy and security, public safety, disaster management, energy consumption and quality of life in smart cities. It mainly focuses on the measurement of the smartness of smart cities, i.e., environmental aspects, life quality, public safety, and disaster management.

/pdf/survey-on-collaborative-smart-drones-and-internet-of-things-3xjk9xubzg.pdf

Survey on Collaborative Smart Drones and Internet of Things for Improving Smartness of Smart Cities

Next-generation wireless networks must support ultra-reliable, low-latency communication and intelligently manage a massive number of Internet of Things (IoT) devices in real-time, within a highly dynamic environment. This need for stringent communication quality-of-service (QoS) requirements as well as mobile edge and core intelligence can only be realized by integrating fundamental notions of artificial intelligence (AI) and machine learning across the wireless infrastructure and end-user devices. In this context, this paper provides a comprehensive tutorial that introduces the main concepts of machine learning, in general, and artificial neural networks (ANNs), in particular, and their potential applications in wireless communications. For this purpose, we present a comprehensive overview on a number of key types of neural networks that include feed-forward, recurrent, spiking, and deep neural networks. For each type of neural network, we present the basic architecture and training procedure, as well as the associated challenges and opportunities. Then, we provide an in-depth overview on the variety of wireless communication problems that can be addressed using ANNs, ranging from communication using unmanned aerial vehicles to virtual reality and edge caching.For each individual application, we present the main motivation for using ANNs along with the associated challenges while also providing a detailed example for a use case scenario and outlining future works that can be addressed using ANNs. In a nutshell, this article constitutes one of the first holistic tutorials on the development of machine learning techniques tailored to the needs of future wireless networks.

Machine Learning for Wireless Networks with Artificial Intelligence: A Tutorial on Neural Networks

Following an avalanche, one of the factors that affect victims’ chance of survival is the speed with which they are located and dug out. Rescue teams use techniques like trained rescue dogs and electronic transceivers to locate victims. However, the resources and time required to deploy rescue teams are major bottlenecks that decrease a victim’s chance of survival. Advances in the field of Unmanned Aerial Vehicles (UAVs) have enabled the use of flying robots equipped with sensors like optical cameras to assess the damage caused by natural or manmade disasters and locate victims in the debris. In this paper, we propose assisting avalanche search and rescue (SAR) operations with UAVs fitted with vision cameras. The sequence of images of the avalanche debris captured by the UAV is processed with a pre-trained Convolutional Neural Network (CNN) to extract discriminative features. A trained linear Support Vector Machine (SVM) is integrated at the top of the CNN to detect objects of interest. Moreover, we introduce a pre-processing method to increase the detection rate and a post-processing method based on a Hidden Markov Model to improve the prediction performance of the classifier. Experimental results conducted on two different datasets at different levels of resolution show that the detection performance increases with an increase in resolution, while the computation time increases. Additionally, they also suggest that a significant decrease in processing time can be achieved thanks to the pre-processing step.

A Convolutional Neural Network Approach for Assisting Avalanche Search and Rescue Operations with UAV Imagery

In this letter, we face the problem of multilabeling unmanned aerial vehicle (UAV) imagery, typically characterized by a high level of information content, by proposing a novel method based on convolutional neural networks. These are exploited as a means to yield a powerful description of the query image, which is analyzed after subdividing it into a grid of tiles. The multilabel classification task of each tile is performed by the combination of a radial basis function neural network and a multilabeling layer (ML) composed of customized thresholding operations. Experiments conducted on two different UAV image data sets demonstrate the promising capability of the proposed method compared to the state of the art, at the expense of a higher but still contained computation time.

A Deep Learning Approach to UAV Image Multilabeling

This paper presents a prototype to assist blind people in indoor environments.The prototype incorporates recognition and guidance units.It comprises also a voice-user interface.Tests in a public indoor space demonstrate promising capabilities. Assistive technologies for blind people are showing a fast growth, providing useful tools to support daily activities and to improve social inclusion. Most of these technologies are mainly focused on helping blind people to navigate and avoid obstacles. Other works emphasize on providing them assistance to recognize their surrounding objects. Very few of them however couple both aspects (i.e., navigation and recognition). With the aim to address the aforesaid needs, we describe in this paper an innovative prototype, which offers the capabilities to (i) move autonomously and to (ii) recognize multiple objects in public indoor environments. It incorporates lightweight hardware components (camera, IMU, and laser sensors), all mounted on a reasonably-sized integrated device to be placed on the chest. It requires the indoor environment to be 'blind-friendly', i.e., prior information about it should be prepared and loaded in the system beforehand. Its algorithms are mainly based on advanced computer vision and machine learning approaches. The interaction between the user and the system is performed through speech recognition and synthesis modules. The prototype offers to the user the possibility to (i) walk across the site to reach the desired destination, avoiding static and mobile obstacles, and (ii) ask the system through vocal interaction to list the prominent objects in the user's field of view. We illustrate the performances of the proposed prototype through experiments conducted in a blind-friendly indoor space equipped at our Department premises.

Recovering the sight to blind people in indoor environments with smart technologies

We describe a novel multilabel classification approach based on a support vector machine (SVM) for the extremely high-resolution remote sensing images. Its underlying ideas consist to: 1) exploit inter-label relationships by means of a structured SVM and 2) incorporate spatial contextual information by adding to the cost function a term that encourages spatial smoothness into the structural SVM optimization process. The resulting formulation appears as an extension of the traditional SVM learning, in which our proposed model integrates the output structure and spatial information simultaneously during the training. Numerical experiments conducted on two different UAV- and airborne-acquired sets of images show the interesting properties of the proposed model, in particular, in terms of classification accuracy.

Spatial and Structured SVM for Multilabel Image Classification

In recent years, unmanned aerial vehicles (UAVs) have been widely used for civilian remote sensing applications. One of them is to assess damages due to man-made or natural disasters and search for bodies in the debris. In this work, we propose to support avalanche search and rescue (SAR) operation with UAVs. The image acquired by the UAV is processed through a pre-trained convolutional neural network (CNN) to extract discriminative features. A linear support vector machine (SVM) is integrated at the top of the CNN to detect objects of interest. Experimental results show encouraging detection performance at a reasonable processing time.

Abdallah Zeggada

Papers

A Convolutional Neural Network Approach for Assisting Avalanche Search and Rescue Operations with UAV Imagery

A Deep Learning Approach to UAV Image Multilabeling

Recovering the sight to blind people in indoor environments with smart technologies

Spatial and Structured SVM for Multilabel Image Classification

Convolutional neural networks for near real-time object detection from UAV imagery in avalanche search and rescue operations