The demand for various multimedia applications is rapidly increasing due to the recent advance in the computing and network infrastructure, together with the widespread use of digital video technology. Among the key elements for the success of these applications is how to effectively and efficiently manage and store a huge amount of audio visual information, while at the same time providing user-friendly access to the stored data. This has fueled a quickly evolving research area known as video abstraction. As the name implies, video abstraction is a mechanism for generating a short summary of a video, which can either be a sequence of stationary images (keyframes) or moving images (video skims). In terms of browsing and navigation, a good video abstract will enable the user to gain maximum information about the target video sequence in a specified time constraint or sufficient information in the minimum time. Over past years, various ideas and techniques have been proposed towards the effective abstraction of video contents. The purpose of this article is to provide a systematic classification of these works. We identify and detail, for each approach, the underlying components and how they are addressed in specific works.

Video abstraction: A systematic review and classification

Industrial automation based on cyber-physical systems technologies

The emergence of wearable devices such as portable cameras and smart glasses makes it possible to record life logging first-person videos. Browsing such long unstructured videos is time-consuming and tedious. This paper studies the discovery of moments of user's major or special interest (i.e., highlights) in a video, for generating the summarization of first-person videos. Specifically, we propose a novel pairwise deep ranking model that employs deep learning techniques to learn the relationship between high-light and non-highlight video segments. A two-stream network structure by representing video segments from complementary information on appearance of video frames and temporal dynamics across frames is developed for video highlight detection. Given a long personal video, equipped with the highlight detection model, a highlight score is assigned to each segment. The obtained highlight segments are applied for summarization in two ways: video time-lapse and video skimming. The former plays the highlight (non-highlight) segments at low (high) speed rates, while the latter assembles the sequence of segments with the highest scores. On 100 hours of first-person videos for 15 unique sports categories, our highlight detection achieves the improvement over the state-of-the-art RankSVM method by 10.5% in terms of accuracy. Moreover, our approaches produce video summary with better quality by a user study from 35 human subjects.

/pdf/highlight-detection-with-pairwise-deep-ranking-for-first-1q859coe4m.pdf

Highlight Detection with Pairwise Deep Ranking for First-Person Video Summarization

http://www.micc.unifi.it/bertini/download/papers/cviu2003-pp.pdf

Semantic annotation of soccer videos: automatic highlights identification

In this paper, we propose a novel audio-visual feature-based framework for event detection in broadcast video of multiple different field sports. Features indicating significant events are selected and robust detectors built. These features are rooted in characteristics common to all genres of field sports. The evidence gathered by the feature detectors is combined by means of a support vector machine, which infers the occurrence of an event based on a model generated during a training phase. The system is tested generically across multiple genres of field sports including soccer, rugby, hockey, and Gaelic football and the results suggest that high event retrieval and content rejection statistics are achievable.

/pdf/event-detection-in-field-sports-video-using-audio-visual-2dxayoz64h.pdf

Event detection in field sports video using audio-visual features and a support vector Machine

Advances in the media and entertainment industries, for example streaming audio and digital TV, present new challenges for managing large audio-visual collections. Efficient and effective retrieval from large content collections forms an important component of the business models for content holders and this is driving a need for research in audio-visual search and retrieval. Current content management systems support retrieval using low-level features, such as motion, colour, texture, beat and loudness. However, low-level features often have little meaning for the human users of these systems, who much prefer to identify content using high-level semantic descriptions or concepts. This creates a gap between the system and the user that must be bridged for these systems to be used effectively. The research presented in this paper describes our approach to bridging this gap in a specific content domain, sports video. Our approach is based on a number of automatic techniques for feature detection used in combination with heuristic rules determined through manual observations of sports footage. This has led to a set of models for interesting sporting events-goal segments-that have been implemented as part of an information retrieval system. The paper also presents results comparing output of the system against manually identified goals.

Automatic detection of 'Goal' segments in basketball videos

This paper describes an event-based synchronization mechanism, which is at the core of the inter-media synchronization in the upcoming standard for multimedia presentation, PREMO. The synchronization mechanism of PREMO is a powerful tool, based on a small number of concepts, and on cooperation among active objects, and represents a synthesis of various synchronization models described in the literature. This model can serve as a basis for the implementation of complex synchronization patterns in multimedia presentations, both purely event-based, as well as time-based.

A standard model for multimedia synchronization: PREMO synchronization objects

MADE is the acronym for an E SPRIT III project aimed at developing a programming environment for multimedia applications. The resulting software library is based on C++ and will operate on both UNIX workstations and PCbased platforms. This paper gives a technical overview of the project and describes a number of application scenarios where the MADE environment will provide significant help for multimedia programming.

http://ieeexplore.ieee.org/iel2/952/7230/00292452.pdf

MADE: A Multimedia Application Development Environment.

An Approach to the Formal Specification of Configurable Models of Graphics Systems

PREMO is a major new ISO/IEC standard for graphics and multimedia, which addresses many of the concerns that have been expressed about existing graphics standards. In particular, it addresses the issues of configuration, extension, and interoperation of and between PREMO implementations. This paper gives an overview of PREMO and highlights its most significant features.

https://www.inf.ufg.br/~eduardo/mm/PREMO-p111-herman.pdf

Graham J. Reynolds

Papers

Automatic detection of 'Goal' segments in basketball videos

A standard model for multimedia synchronization: PREMO synchronization objects

MADE: A Multimedia Application Development Environment.

An Approach to the Formal Specification of Configurable Models of Graphics Systems

PREMO: an ISO standard for presentation environment for multimedia objects