The PASCAL Visual Object Classes Challenge

This paper presents a new meteorological photo classification system based on the Multi-channel Convolutional Neural Network (CNN) and improved Frame Difference Method (FDM). This system can work in an embedded system with limited computational resources and categorize cloud observation photos captured by ground cameras. We propose the improved FDM extractor to detect and extract cloud-like objects from large photos into small images. Then, these small images are sent to a Multi-channel CNN image classifier. We construct the classifier and train it on the photo-set that we established. After combining the extractor and classifier to form the classification system, the images can be classified into three different types of clouds, namely, cumulus, cirrus and stratus, based on their meteorological features. The testing phase uses 200 actual photos of real scenes as the experimental data. The results show that the classification accuracy can reach 94%, which indicates that the system has a competitive classification ability. Moreover, the time cost and computational resource consumption for image recognition are greatly reduced. By using this system, meteorologists can lighten their workload of processing meteorological data.

/pdf/cloud-shape-classification-system-based-on-multi-channel-cnn-1qknfdf1w8.pdf

Cloud Shape Classification System Based on Multi-Channel CNN and Improved FDM

The emerging field of autonomous UAV cinematography is examined through a tutorial for non-experts, which also presents the required underlying technologies and connections with different UAV application domains. Current industry practices are formalized by presenting a UAV shot-type taxonomy composed of framing shot types, single-UAV camera motion types, and multiple-UAV camera motion types. Visually pleasing combinations of framing shot types and camera motion types are identified, while the presented camera motion types are modeled geometrically and graded into distinct energy consumption classes and required technology complexity levels for autonomous capture. Two specific strategies are prescribed, namely focal length compensation and multidrone compensation, for partially overcoming a number of issues arising in UAV live outdoor event coverage, deemed as the most complex UAV cinematography scenario. Finally, the shot types compatible with each compensation strategy are explicitly identified. Overall, this tutorial both familiarizes readers coming from different backgrounds with the topic in a structured manner and lays necessary groundwork for future advancements.

Autonomous UAV Cinematography: A Tutorial and a Formalized Shot-Type Taxonomy

Shot type constraints in UAV cinematography for autonomous target tracking

The use of camera-equipped Unmanned Aerial Vehicles (UAVs, or “drones”) for a wide range of aerial video capturing applications, including media production, surveillance, search and rescue operations, etc., has exploded in recent years. Technological progress has led to commercially available UAVs with a degree of cognitive autonomy and perceptual capabilities, such as automated, on-line detection and tracking of target objects upon the captured footage. However, the limited computational hardware, the possibly high camera-to-target distance and the fact that both the UAV/camera and the target(s) are moving, makes it challenging to achieve both high accuracy and stable real-time performance. In this paper, the current state-of-the-art on real-time object detection/tracking is overviewed. Additionally, a relevant, modular implementation suitable for on-drone execution (running on top of the popular Robot Operating System) is presented and empirically evaluated on a number of relevant datasets. The results indicate that a sophisticated, neural network-based detection and tracking system can be deployed at real-time even on embedded devices.

Embedded UAV Real-Time Visual Object Detection and Tracking

Over the past decade, Deep Convolutional Neural Networks with heavy architectures and large numbers of parameters have achieved state-of-the-art results and eclipsed other methods in multiple visual analysis tasks, including object detection. However, the real-time requirements of such tasks directly conflict with the restricted computational capabilities of embedded systems, prohibiting the immediate deployment of bulky models, and necessitating their optimization for inference. Parameter pruning techniques reduce the number of parameters while reducing the input size leads to smaller internal representations, leading by extension to fewer computational operations. Furthermore, inference optimization schemes provided by Deep Learning frameworks can yield significant speed ups, for example by allowing half-precision floating point operations. We investigate the behavior of various model configurations in object detection tasks and perform a comparative study on inference optimization methods which aim to reduce the computational cost of Convolutional Neural Networks, while examining the effect of such methods on their performance, and propose architecture modifications for this purpose.

Convolutional Neural Networks for Visual Information Analysis with Limited Computing Resources

The goal of few-shot image learning is to utilize a very small amount of training examples in order to train a machine learning model to recognize a given number of image classes. While humans can perform such a task pretty much effortlessly, applying the same mechanism to deep learning visual recognition systems is a much more difficult task, having a wide range of real-world visual recognition applications. In this paper, we investigate the behavior of such few-shot methods in the context of drone vision cinematography for sports event filming, in order to recognize new image classes by taking into consideration the fact that this new class we wish to identify is a subclass of an already known class. More specifically we use UAV footage to recognize certain types of athletes, belonging to a subset of an original athlete class, utilizing only a handful of recorded images of this athlete subclass. We examine the effects of such methods on image recognition accuracy while proposing a novel approach for accuracy optimizations. The overall task is evaluated on actual cycling race UAV footage.

/pdf/few-shot-image-recognition-for-uav-sports-cinematography-d4gvs3y8ex.pdf

Few-Shot Image Recognition for UAV Sports Cinematography

Powerline inspection operations involve capturing and inspecting visual footage of powerline elements in electric transmission infrastructures. Current technological advantages in the areas of robotics and machine learning are towards enabling the utilization of completely autonomous Unmanned Aerial Vehicles (UAVs) to carry out such tasks. One of the tasks to be addressed is the robust, precise, and fast powerline object detection problem. To this end, UAVs are required to perform visual object detection autonomously, with high accuracy and fast algorithm execution speed, for providing image regions of interest to be inspected by humans or even be used as input for autonomously controlling the UAV/camera. However, the limited computational resources of the on-board devices of such systems heavily affect the type of neural network architectures that can potentially be deployed. In this work, we study state-of-the-art object detectors in an attempt to find an acceptable trade-off between detection accuracy and inference speed that will allow the exploitation of UAVs for autonomous powerline inspection purposes. To this end, we publicly release a powerline inspection dataset and state a benchmark evaluation with recently proposed object detectors based on deep learning.

A UAV Object Detection Benchmark for Vision-assisted Powerline Element Inspection

Powerline inspection operations involve capturing and inspecting visual footage of powerline elements from elevated positions above and around the powerline and are currently performed with the help of helicopters and/or Unmanned Aerial Vehicles (UAVs). Current technological advances in the areas of robotics and machine learning are towards enabling fully autonomous operations. To this end, one of the tasks to be addressed is the robust, precise and fast powerline object detection problem. Recently introduced Transformer-based object detection methods demonstrate time and accuracy advances with respect to previous works. In this work, we present an enhanced Transformer-based architecture that further improves the state-of-the-art by incorporating a content-specific object query generator and by substituting the original attention operation with a whitening-inspired transformation at certain stages of the architecture. We evaluate our method in a recently captured powerline detection dataset and we show that our novel contributions offer a significant boost regarding detection accuracy.

Whitening Transformation inspired Self-Attention for Powerline Element Detection

Powerline inspection operations involve capturing and inspecting visual (RBG/therma/LiDAR) footage of powerline elements from elevated positions above and around the powerline, which are currently performed with the help of helicopters and/or Unmanned Aerial Vehicles (UAV). Current technological advances in the areas of robotics and machine learning are towards enabling fully autonomous operations. To this end, one of the tasks to be addressed is the robust, precise and fast powerline object detection problem. End-to-end Object Detection with Transformers (DETR) was a recently introduced method that demonstrates time and accuracy advances with respect to other detectors. However, this architecture comes with some computational complexity issues, which can mainly be attributed to Transformer encoder/decoder components, limiting its applicability in fast processing high-resolution feature maps and long sequences in general. To address these issues, we propose incorporating low-complexity Transformer implementations and evaluate them in a recently captured powerline detection dataset.

Emmanouil Patsiouras

Papers

Convolutional Neural Networks for Visual Information Analysis with Limited Computing Resources

Few-Shot Image Recognition for UAV Sports Cinematography

A UAV Object Detection Benchmark for Vision-assisted Powerline Element Inspection

Whitening Transformation inspired Self-Attention for Powerline Element Detection

On the Detection of Powerline Elements with Efficient Transformers