The past decade has witnessed a rapid proliferation of video cameras in all walks of life and has resulted in a tremendous explosion of video content. Several applications such as content-based video annotation and retrieval, highlight extraction and video summarization require recognition of the activities occurring in the video. The analysis of human activities in videos is an area with increasingly important consequences from security and surveillance to entertainment and personal archiving. Several challenges at various levels of processing-robustness against errors in low-level processing, view and rate-invariant representations at midlevel processing and semantic representation of human activities at higher level processing-make this problem hard to solve. In this review paper, we present a comprehensive survey of efforts in the past couple of decades to address the problems of representation, recognition, and learning of human activities from video and related applications. We discuss the problem at two major levels of complexity: 1) "actions" and 2) "activities." "Actions" are characterized by simple motion patterns typically executed by a single human. "Activities" are more complex and involve coordinated actions among a small number of humans. We will discuss several approaches and classify them according to their ability to handle varying degrees of complexity as interpreted above. We begin with a discussion of approaches to model the simplest of action classes known as atomic or primitive actions that do not require sophisticated dynamical modeling. Then, methods to model actions with more complex dynamics are discussed. The discussion then leads naturally to methods for higher level representation of complex activities.

/pdf/machine-recognition-of-human-activities-a-survey-4jrl67n0jj.pdf

Machine Recognition of Human Activities: A Survey

Research on sensor-based activity recognition has, recently, made significant progress and is attracting growing attention in a number of disciplines and application domains. However, there is a lack of high-level overview on this topic that can inform related communities of the research state of the art. In this paper, we present a comprehensive survey to examine the development and current status of various aspects of sensor-based activity recognition. We first discuss the general rationale and distinctions of vision-based and sensor-based activity recognition. Then, we review the major approaches and methods associated with sensor-based activity monitoring, modeling, and recognition from which strengths and weaknesses of those approaches are highlighted. We make a primary distinction in this paper between data-driven and knowledge-driven approaches, and use this distinction to structure our survey. We also discuss some promising directions for future research.

Sensor-Based Activity Recognition

Purpose – This paper aims to serve two main purposes. In the first instance it aims to it provide an overview addressing the state‐of‐the‐art in the area of activity recognition, in particular, in the area of object‐based activity recognition. This will provide the necessary material to inform relevant research communities of the latest developments in this area in addition to providing a reference for researchers and system developers who ware working towards the design and development of activity‐based context aware applications. In the second instance this paper introduces a novel approach to activity recognition based on the use of ontological modeling, representation and reasoning, aiming to consolidate and improve existing approaches in terms of scalability, applicability and easy‐of‐use.Design/methodology/approach – The paper initially reviews the existing approaches and algorithms, which have been used for activity recognition in a number of related areas. From each of these, their strengths and w...

Ontology‐based activity recognition in intelligent pervasive environments

Although events are ubiquitous in multimedia, no common notion of events has emerged. Events appear in multimedia presentation formats, programming frameworks, and databases, as well as in next-generation multimedia applications such as eChronicles, life logs, or the Event Web. A common event model for multimedia could serve as a unifying foundation for all of these applications

Toward a Common Event Model for Multimedia Applications

The lack of a formal model of events hinders interoperability in distributed event-based systems. In this paper, we present a formal model of events, called Event-Model-F. The model is based on the foundational ontology DOLCE+DnS Ultralight (DUL) and provides comprehensive support to represent time and space, objects and persons, as well as mereological, causal, and correlative relationships between events. In addition, the Event-Model-F provides a flexible means for event composition, modeling event causality and event correlation, and representing different interpretations of the same event. The Event-Model-F is developed following the pattern-oriented approach of DUL, is modularized in different ontologies, and can be easily extended by domain specific ontologies.

F--a model of events based on the foundational ontology dolce+DnS ultralight

The notion of "events" is extremely important in characterizing the contents of video. An event is typically triggered by some kind of change of state captured in the video, such as when an object starts moving. The ability to reason with events is a critical step toward video understanding. This article describes the findings of a recent workshop series that has produced an ontology framework for representing video events-called Video Event Representation Language (VERL) -and a companion annotation framework, called Video Event Markup Language (VEML). One of the key concepts in this work is the modeling of events as composable, whereby complex events are constructed from simpler events by operations such as sequencing, iteration, and alternation. The article presents an extensible event and object ontology expressed in VERL and discusses a detailed example of applying VERL and VEML to the description of a "tailgating" event in surveillance video.

/pdf/verl-an-ontology-framework-for-representing-and-annotating-3dnqsacpvn.pdf

J.R. Smith

Papers

VERL: an ontology framework for representing and annotating video events