Multimedia database

About: Multimedia database is a research topic. Over the lifetime, 1404 publications have been published within this topic receiving 19856 citations. The topic is also known as: Multimedia database & MMDB.

Retrieval of multimedia stimuli with semantic and emotional cues: Suggestions from a controlled study

Abstract: The ability to efficiently search pictures with annotated semantics and emotion is an important problem for Human-Computer Interaction with considerable interdisciplinary significance. Accuracy and speed of the multimedia retrieval process depends on the chosen metadata annotation model. The quality of such multifaceted retrieval is opposed to the potential complexity of data setup procedures and development of multimedia annotations. Additionally, a recent study has shown that databases of emotionally annotated multimedia are still being predominately searched manually which highlights the need to study this retrieval modality. To this regard we present a study with N = 75 participants aimed to evaluate the influence of keywords and dimensional emotions in manual retrieval of pictures. The study showed that if the multimedia database is comparatively small emotional annotations are sufficient to achieve a fast retrieval despite comparatively lesser overall accuracy. In a larger dataset semantic annotations became necessary for efficient retrieval although they contributed to a slower beginning of the search process. The experiment was performed in a controlled environment with a team of psychology experts. The results were statistically consistent with validates measures of the participants' perceptual speed.

Conceptual modeling of multimedia databases

Abstract: The gap between the semantic content of multimedia data and its underlying physical representation is one of the main problems in the modern multimedia research in general, and, in particular, in the field of multimedia database modeling. We believe that one of the principal reasons of this problem is the attempt to conceptually represent multimedia data in a way, which is similar to its low-level representation by applications dealing with encoding standards, feature-based multimedia analysis, etc. In our opinion, such conceptual representation of multimedia contributes to the semantic gap by separating the representation of multimedia information from the representation of the universe of discourse of an application, to which the multimedia information pertains. In this research work we address the problem of conceptual modeling of multimedia data in a way to deal with the above-mentioned limitations. First, we introduce two different paradigms of conceptual understanding of the essence of multimedia data, namely: multimedia as data and multimedia as metadata. The multimedia as data paradigm, which views multimedia data as the subject of modeling in its own right, is inherent to so-called multimedia-centric applications, where multimedia information itself represents the main part of the universe of discourse. The examples of such kind of applications are digital photo collections or digital movie archives. On the other hand, the multimedia as metadata paradigm, which is inherent to so-called multimedia-enhanced applications, views multimedia data as just another (optional) source of information about whatever universe of discourse that the application pertains to. An example of a multimedia-enhanced application is a human-resource database augmented with employee photos. Here the universe of discourse is the totality of company employees, while their photos simply represent an additional (possibly optional) kind of information describing the universe of discourse. The multimedia conceptual modeling approach that we present in this work allows addressing multimedia-centric applications, as well as, in particular, multimedia-enhanced applications. The model that we propose builds upon MADS (Modeling Application Data with Spatio-temporal features), which is a rich conceptual model defined in our laboratory, and which is, in particular, characterized by structural completeness, spatio-temporal modeling capabilities, and multirepresentation support. The proposed multimedia model is provided in the form of a new modeling dimension of MADS, whose orthogonality principle allows to integrate the new multimedia modeling dimension with already existing modeling features of MADS. The following multimedia modeling constructs are provided: multimedia datatypes, simple and complex representational constraints (relationships), a multimedia partitioning mechanism, and multimedia multirepresentation features. Following the description of our conceptual multimedia modeling approach based on MADS, we present the peculiarities of logical multimedia modeling and of conceptual-to-logical inter-layer transformations. We provide a set of mapping guidelines intended to help the schema designer in coming up with rich logical multimedia document representations of the application domain, which conform with the conceptual multimedia schema. The practical interest of our research is illustrated by a mock-up application, which has been developed to support the theoretical ideas described in this work. In particular, we show how the abstract conceptual set-based representations of multimedia data elements, as well as simple and complex multimedia representational relationships can be implemented using Oracle DBMS.

Construction of interactive video information system by applying results of object recognition

Abstract: Although numerous attempts have been made to determine algorithms and approaches for building up a video information system, not many practical applications have been proposed. In this paper, a novel interactive video information system called the Drama Characters' Popularity Voting System (DCPVS) is constructed by applying the results of off-line object recognition. The system's purpose is to provide description annotation, retrieval, and statistics in the video associated with an object, such as a character as a basic unit, over the Internet. By using the proposed system, multiple users in a network can enjoy the same video and can vote for the characters they like in it. The voting information is collected and stored in the server, which then provides the statistics regarding the popularity of different characters or the voting rates within different periods of the video.

A Multimedia Database System VHM and Its Application

Abstract: Video Hyper Media (VHM) is an environment for creating and executing interactive multimedia applications The database of the VHM comprises multimedia contents such as still pictures, moving pictures, and sounds and scenarios that describe the behavior and dynamic relationships of compound objects (combinations of the multimedia contents) With the interactive edit function of the VHM, applications such as the on-screen subject search and a video walk-through application can be developed without any programming As an example of an application created with the VHM, we developed a railway line information system and confirmed the effectiveness of the VHM

SPY-TEC+ : AN INTEGRATED INDEX STRUCTURE FOR k-NEAREST NEIGHBOR QUERIES WITH SEMANTIC PREDICATES IN MULTIMEDIA DATABASE

Abstract: Recently, advanced multimedia applications, such as geographic information system, and content-based multimedia retrieval system, require the efficient processing of k-nearest neighbor queries over large collection of multimedia objects. These queries usually include the semantic information that is represented by text, as well as the visual information that is represented by a high-dimensional feature vector. Among the available techniques for processing such queries, the incremental nearest neighbor algorithm proposed by Hjaltason and Samet is known as the best choice. However, the R-tree used in their algorithm has no facility capable of partially pruning the candidate tuples that will turn out not to satisfy the semantic predicate. Also, the R-tree does not perform sufficiently well on high-dimensional data even though it provides good results on low or middle-dimensional data. These drawbacks may lead to a poor performance when processing the query. In this paper, we propose an integrated index structure, so-called SPY-TEC+, that provides an efficient method for indexing the visual and semantic feature at the same time using the SPY-TEC that was proposed for indexing high-dimensional data, and the signature file. We also propose an efficient incremental nearest neighbor algorithm for processing k-nearest neighbor queries with visual and semantic predicates on the SPY-TEC+. Finally, we show that the SPY-TEC+ enhances the performance of the SPY-TEC for processing k-nearest neighbor queries with visual and semantic predicates through various experiments.