2017 25th European Signal Processing Conference (EUSIPCO)

UAV assistance paradigm: State-of-the-art in applications and challenges

Human activity recognition (HAR) systems attempt to automatically identify and analyze human activities using acquired information from various types of sensors. Although several extensive review papers have already been published in the general HAR topics, the growing technologies in the field as well as the multi-disciplinary nature of HAR prompt the need for constant updates in the field. In this respect, this paper attempts to review and summarize the progress of HAR systems from the computer vision perspective. Indeed, most computer vision applications such as human computer interaction, virtual reality, security, video surveillance and home monitoring are highly correlated to HAR tasks. This establishes new trend and milestone in the development cycle of HAR systems. Therefore, the current survey aims to provide the reader with an up to date analysis of vision-based HAR related literature and recent progress in the field. At the same time, it will highlight the main challenges and future directions.

/pdf/vision-based-human-activity-recognition-a-survey-9vr3o8g5p7.pdf

Vision-based human activity recognition: a survey

This paper represents the result of the IAEG C35 Commission “Monitoring methods and approaches in engineering geology applications” workgroup aimed to describe a general overview of unmanned aerial vehicles (UAVs) and their potentiality in several engineering geology applications. The use of UAV has progressively increased in the last decade and nowadays started to be considered a standard research instrument for the acquisition of images and other information on demand over an area of interest. UAV represents a cheap and fast solution for the on-demand acquisition of detailed images of an area of interest and the creation of detailed 3D models and orthophoto. The use of these systems required a good background of data processing and a good drone pilot ability for the management of the flight mission in particular in a complex environment.

The use of unmanned aerial vehicles (UAVs) for engineering geology applications

Multimodal summarization has drawn much attention due to the rapid growth of multimedia data. The output of the current multimodal summarization systems is usually represented in texts. However, we have found through experiments that multimodal output can significantly improve user satisfaction for informativeness of summaries. In this paper, we propose a novel task, multimodal summarization with multimodal output (MSMO). To handle this task, we first collect a large-scale dataset for MSMO research. We then propose a multimodal attention model to jointly generate text and select the most relevant image from the multimodal input. Finally, to evaluate multimodal outputs, we construct a novel multimodal automatic evaluation (MMAE) method which considers both intra-modality salience and inter-modality relevance. The experimental results show the effectiveness of MMAE.

/pdf/msmo-multimodal-summarization-with-multimodal-output-2zwljgole6.pdf

MSMO: Multimodal Summarization with Multimodal Output

Camera-equipped unmanned aerial vehicles (UAVs), or “drones,” are a recent addition to standard audiovisual shooting technologies. As drone cinematography is expected to further revolutionize media production, this paper presents an overview of the state-of-the-art in this area, along with a brief review of current commercial UAV technologies and legal restrictions on their deployment. A novel taxonomy of UAV cinematography visual building blocks, in the context of filming outdoor events where targets (e.g., athletes) must be actively followed, is additionally proposed. Such a taxonomy is necessary for progress in intelligent/autonomous UAV shooting, which has the potential of addressing current technology challenges. Subsequently, the concepts and advantages inherent in multiple-UAV cinematography are introduced. The core of multiple-UAV cinematography consists in identifying different combinations of multiple single-UAV camera motion types, assembled in meaningful sequences. Finally, based on the defined UAV/camera motion types, tools for managing a partially autonomous, multiple-UAV fleet from the director’s point of view are presented. Although the overall focus is on cinematic coverage of sports events, the majority of our contributions also apply in different scenarios, such as movies/TV production, newsgathering, or advertising.

High-Level Multiple-UAV Cinematography Tools for Covering Outdoor Events

Autonomous UAV cinematography is an active research field with exciting potential for the media industry. It bears the promise of greatly facilitating UAV shooting for various applications, while significantly reducing the costs compared to manual shooting. However, the general problem has not been clearly defined and the challenges arising from current legislation and technology restrictions have not been fully charted. A complete overview of issues related to autonomous UAV cinematography is needed, pertaining to the current situation in the field, so as to guide immediate-future research. The purpose of this paper is to lay exactly this groundwork, with the expectation of providing a global perspective to multiple domain-specific research communities. The outlined issues are partitioned into challenges deriving from ethical/legal/safety considerations and from operational/production requirements. A brief survey of current technological solutions, including their limitations, is also provided for each issue.

Challenges in Autonomous UAV Cinematography: An Overview

A salient dictionary learning framework for activity video summarization via key-frame extraction

The emerging field of autonomous UAV cinematography is examined through a tutorial for non-experts, which also presents the required underlying technologies and connections with different UAV application domains. Current industry practices are formalized by presenting a UAV shot-type taxonomy composed of framing shot types, single-UAV camera motion types, and multiple-UAV camera motion types. Visually pleasing combinations of framing shot types and camera motion types are identified, while the presented camera motion types are modeled geometrically and graded into distinct energy consumption classes and required technology complexity levels for autonomous capture. Two specific strategies are prescribed, namely focal length compensation and multidrone compensation, for partially overcoming a number of issues arising in UAV live outdoor event coverage, deemed as the most complex UAV cinematography scenario. Finally, the shot types compatible with each compensation strategy are explicitly identified. Overall, this tutorial both familiarizes readers coming from different backgrounds with the topic in a structured manner and lays necessary groundwork for future advancements.

Ioannis Mademlis

Papers

High-Level Multiple-UAV Cinematography Tools for Covering Outdoor Events

Challenges in Autonomous UAV Cinematography: An Overview

A salient dictionary learning framework for activity video summarization via key-frame extraction

Autonomous UAV Cinematography: A Tutorial and a Formalized Shot-Type Taxonomy

Shot type constraints in UAV cinematography for autonomous target tracking