A system and method is provided for editing and parsing compressed digital information. The compressed digital information may include visual information which is edited and parsed in the compressed domain. In a preferred embodiment, the present invention provides a method for detecting moving objects in a compressed digital bitstream which represents a sequence of fields or frames of video information for one or more captured scenes of video.

Methods and architecture for indexing and editing compressed video over the world wide web

Caption boxes which are embedded in video content can be located and the text within the caption boxes decoded. Real time processing is enhanced by locating caption box regions in the compressed video domain and performing pixel based processing operations within the region of the video frame in which a caption box is located. The captions boxes are further refined by identifying word regions within the caption boxes and then applying character and word recognition processing to the identified word regions. Domain based models are used to improve text recognition results. The extracted caption box text can be used to detect events of interest in the video content and a semantic model applied to extract a segment of video of the event of interest.

System and method for extracting text captions from video and generating video summaries

Systems and methods for describing video content establish video description records which include an object set (24), an object hierarchy (26) and entity relation graphs (28). Video objects can include global ojects, segment objects and local objects. The video objects are further defined by a number of features organized in classes, which in turn are further defined by a number of feature descriptors (36, 38 and 40). The relationships (44) between and among the objects in the object set (24) are defined by the object hierarchy (26) and entity relation graphs (28). The video description records provide a standard vehicle for describing the content and context of video information for subsequent access and processing by computer applications such as search engines, filters, and archive systems.

Video description system and method

A framework for retrieving images by spatial similarity (FRISS) in ima ge databases is presented. In this framework, a robust retrieval by spatial similarity (RSS) algorithm is defined as one that incorporates both directional and topological spatial constraints, retrieves similar images, and recognized images even after they undergo translation, scaling, rotation (both perfect and multiple), or any arbitrary combination of transformatioins. The FRISS framework is discussed and used as a base for comparing various existing RSS algorithms. Analysis shows that none of them satisfies all the FRISS specifications. An algorithm, SIMdtc, is then presented. SIMdtc introduces the concept of a rotation correction angle(RCA) to align objects in one image spatially closer to matching objects in another image for more accurate similarity assessment. Similarity between two images is a function of the number of common objects between them and the closeness of directional and topological spatial relationships between object pairs in both images. The SIMdtc retrieval is invariant under translation, scaling, and perfect rotation, and the algorithm is able to rank multiple rotation variants. The algorithm was tested using synthetic images and the TESSA image database. Analysis shows the robustness of the SIMdtc algorithm over current algorithms.

A robust framework for content-based retrieval by spatial similarity in image databases

We present an approach for indexing animated objects and efficiently answering queries about their position in time and space. In particular, we consider an animated movie as a spatiotemporal evolution. A movie is viewed as an ordered sequence of frames, where each frame is a 2D space occupied by the objects that appear in that frame. The queries of interest are range queries of the form, "find the objects that appear in area S between frames f/sub i/ and f/sub j//sup "/ as well as nearest neighbor queries such as, "find the q nearest objects to a given position A between frames f/sub i/ and f/sub j//sup "/. The straightforward approach to index such objects considers the frame sequence as another dimension and uses a 3D access method (such as an R-Tree or its variants). This, however, assigns long "lifetime" intervals to objects that appear through many consecutive frames. Long intervals are difficult to cluster efficiently in a 3D index. Instead, we propose to reduce the problem to a partial-persistence problem. Namely, we use a 2D access method that is made partially persistent. We show that this approach leads to faster query performance while still using storage proportional to the total number of changes in the frame evolution, What differentiates this problem from traditional temporal indexing approaches is that objects are allowed to move and/or change their extent continuously between frames. We present novel methods to approximate such object evolutions, We formulate an optimization problem for which we provide an optimal solution for the case where objects move linearly. Finally, we present an extensive experimental study of the proposed methods. While we concentrate on animated movies, our approach is general and can be applied to other spatiotemporal applications as well.

/pdf/indexing-animated-objects-using-spatiotemporal-access-3wftwfryz4.pdf

Indexing animated objects using spatiotemporal access methods

We describe a general multimedia query language, called MOQL, based on ODMG's Object Query Language (OQL). In contrast to previous multimedia query languages that are either designed for one particular medium (e.g. images) or specialized for a particular application (e.g., medical imaging), MOQL is general in its treatment of multiple media and di erent applications. The language includes constructs to capture the temporal and spatial relationships in multimedia data as well as functions for query presentation. We illustrate the language features by query examples. The language is implemented for a multimedia database built on top of ObjectStore.

https://webdocs.cs.ualberta.ca/~duane/publications/pdf/1997wmmis.pdf

MOQL: A Multimedia Object Query Language

One of the key aspects of videos is the temporal relationship between video frames. In this paper we propose a tree-based model for specifying the temporal semantics of video data. We present a unique way of integrating our video model into an object database management system which has rich multimedia temporal operations. We further show how temporal histories are used to model video data, explore the video object base using object-oriented techniques. Such a seamless integration gives a uniform interface to end users. The integrated video object base management systems supports a broad range of temporal queries.© (1997) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Modeling video temporal relationships in an object database management system

In this paper we consider various spatial relationships that are of general interest for retrieving data from multimedia databases. We present a uni ed representation of spatial objects for both topological and directional relations. Such a representation is based on Allen's temporal interval algebra. We also present a set of spatial inference rules, which allow us to make heterogeneous spatial relation deductions from existing directional and topological relations. For example, if we know A north of B, B overlap with C, and C north of D, then we deriveA above D. Since all the rules are propositional Horn clauses, they can be easily integrated into any multimedia database by using either a simple inference engine or a lookup table.

/pdf/spatial-reasoning-rules-in-multimedia-management-systems-3lfsfpzs3s.pdf

Spatial Reasoning Rules in Multimedia Management Systems

Amidst the dramatic developments of computer technology in the last decades, we have seen a growing interest in supporting multimedia data. Compared with traditional data, the most noticeable features of multimedia data are their spatiality (the spatial layout of objects and their relationships), temporality (time-dependence in media such as video and audio), and presentation requirements (imposed in response to user queries, such as synchronization of different media). Developing database management support for multimedia data poses interesting challenges.

Many multimedia database management systems have been developed to deal with these multimedia features. However, they suffer some common problems: lack of generality in media modeling, lack of precise knowledge of spatial information, limited spatio-temporal functionality, ineffective query languages, and inefficient query processing. The main purpose of this work is to investigate these problems and provide solutions. The first step of the research is to design a multimedia data model which is built upon and extends existing models. The proposed model is based on a temporal interval algebra. The establishment of a multimedia data model provides a theoretical foundation that is used to investigate multimedia query languages. The result is a general-purpose, object-oriented multimedia query language which supports content-based information retrieval. A prototype has been built to validate the proposed concepts.

/pdf/modeling-and-querying-multimedia-data-3zivq3az7i.pdf

Modeling and querying multimedia data

Declarative query languages are an important feature of database management systems and have played an important role in their success. As database management technology enters the multimedia information system area, the availability of special-purpose query languages for multimedia applications will be equally important. In this report 1 , we survey multimedia query languages and query models. Particularly, we look at those systems from the point of view of well-deened queries, fuzzy queries, visual queries, and query presentations. Several research issues, such as generic multimedia query languages, incremental queries, fuzzy queries, spatio-temporal queries, feature storage and organization, are also identiied. In our opinion these are vital issues for the success and development of a multimedia query language.

/pdf/query-languages-in-multimedia-database-systems-5dwnvwvrtj.pdf

John Z. Li

Papers

MOQL: A Multimedia Object Query Language

Modeling video temporal relationships in an object database management system

Spatial Reasoning Rules in Multimedia Management Systems

Modeling and querying multimedia data

Query Languages in Multimedia Database Systems