Showing papers on "Multimedia database published in 2004"

A rule-based video annotation system

Abstract: A generic system for automatic annotation of videos is introduced. The proposed approach is based on the premise that the rules needed to infer a set of high-level concepts from low-level descriptors cannot be defined a priori. Rather, knowledge embedded in the database and interaction with an expert user is exploited to enable system learning. Underpinning the system at the implementation level is preannotated data that dynamically creates signification links between a set of low-level features extracted directly from the video dataset and high-level semantic concepts defined in the lexicon. The lexicon may consist of words, icons, or any set of symbols that convey the meaning to the user. Thus, the lexicon is contingent on the user, application, time, and the entire context of the annotation process. The main system modules use fuzzy logic and rule mining techniques to approximate human-like reasoning. A rule-knowledge base is created on a small sample selected by the expert user during the learning phase. Using this rule-knowledge base, the system automatically assigns keywords from the lexicon to nonannotated video clips in the database. Using common low-level video representations, the system performance was assessed on a database containing hundreds of broadcasting videos. The experimental evaluation showed robust and high annotation accuracy. The system architecture offers straightforward expansion to relevance feedback and autonomous learning capabilities.

Similarity search in multimedia databases

Abstract: The research on multimedia databases involves different areas in Computer Science, such as computer graphics, databases, and information retrieval. There are many practical applications that benefit from this research, e.g., molecular biology, medicine, CAD/CAM, and geography. An important characteristic of these applications is the variety of data that should be supported, e.g., text, images (both still and moving), and audio. This implies that the development of a multimedia information system is considerably more complex than a traditional information system. An important research issue in the field of multimedia databases is the content-based retrieval of similar objects. Given a multimedia query object, the search for an exact match in a database is not meaningful in most applications, because the probability that two multimedia objects are identical is negligible (unless they are digital copies from the same source). For this reason, the development of efficient and effective similarity search techniques has become an important topic in the multimedia database research community. The goal of this advanced technology seminar is to provide an overview of the similarity search problem and to present the state-of-art techniques for performing efficient and effective similarity queries in multimedia databases. The seminar begins with an introduction and a motivation of multimedia databases. The two main approaches for describing multimedia objects (as elements in a metric space or in a vector space) are introduced, as well as a description of the ”Multimedia Content Description Interface” (MPEG)-7 standard. The efficiency issue is addressed for both metric and vector space approaches, describing the data structures and algorithms used to answer similarity queries. For the effectiveness issue, the seminar introduces some widely used retrieval performance measures. Several examples of techniques for particular multimedia applications (text, image, CAD, 3D objects, audio and video) are presented. The seminar outline is as follows:

Modeling and Management of Fuzzy Information in Multimedia Database Applications

Abstract: In this paper, we firstly present a conceptual data model for multimedia database applications based on ExIFO2 model. The ExIFO2 data model is chosen as the conceptual model since it handles complex objects along with their uncertain and imprecise properties. We enhanced this conceptual model in order to meet the multimedia data requirements. In addition to uncertain and imprecise information, we present a way of handling relationships among objects of multimedia database applications. Events that might be extracted from video or audio are also considered in this study. Secondly, the conceptual model is mapped to a logical model, which the fuzzy object-oriented data (FOOD) model is chosen, for storing and manipulating the multimedia objects. This mapping is done in a way that it preserves most of the information represented at the conceptual level. Finally, in this study videos of football (soccer) games is selected as the multimedia database application to show how we handle crisp and fuzzy querying and retrieval of fuzzy and crisp data from the database. A program has been developed to draw ExIFO2 schemas and to map the schema to FOOD code automatically.

Querying distributed multimedia databases and data sources for sensor data fusion

Abstract: Sensor data fusion imposes a number of novel requirements on query languages and query processing techniques. A spatial/temporal query language called /spl Sigma/QL has been proposed to support the retrieval and fusion of multimedia information from multiple sources and databases. In this paper we investigate fusion techniques, multimedia data transformations and /spl Sigma/QL query processing techniques for sensor data fusion. Fusion techniques including fusion by the merge operation, the detection of moving objects, and the incorporation of belief values, have been developed. An experimental prototype has been implemented and tested to demonstrate the feasibility of these techniques.

A Query by Humming System using MPEG-7 Descriptors

Abstract: Query by Humming (QBH) is a method for searching in a multimedia database system containing meta data descriptions of songs. The database can be searched by hummed queries, this means that a user can hum a melody into a microphone which is connected to the computer hosting the system. The QBH system searches the database for songs which are similar to the input query and presents the result to the user as a list of matching songs. This paper presents a modular QBH system using MPEG-7 descriptors in all processing stages. Due to the modular design all components can easily be substituted. The system is evaluated by changing parameters defined by the MPEG-7 descriptors.

A multimedia database workbench for content and context retrieval

Abstract: Multimedia databases are extending the scope of traditional databases to handle the complex structure of multimedia objects. Models for multimedia information must include representations for the structure and content of several media in a form that allows flexibility in retrieval. Content-based retrieval is the main motivation behind recent research in multimedia databases. The task of searching in video and audio content is made hard by the nature of audiovisual data where, unlike text, there is no direct syntactic channel between the object and its meaning [(R. Zhao and W. I. Grosy, 2002), (J. Chen et al., 2004), (M. Bertini et al., 2003)]. We propose a model for multimedia content storage and retrieval accounting for both context and content information and taking advantage of their dependencies for effective retrieval. We then describe a prototype multimedia database with a retrieval interface. It has been used as a workbench for testing the representation model and integrating tools for feature extraction, information interchange and retrieval. The workbench allows an easy inclusion of new tools for content analysis and new methods for context- and content-based retrieval while offering storage and access for both the actual digital content and its metadata.

P-Karaoke: personalized karaoke system

Abstract: In this demonstration, a personalized Karaoke system, P-Karaoke, is proposed. In the P-Karaoke system, personal home videos and photographs, which are automatically selected from users' multimedia database according to their content, users' preferences or music, are utilized as the background videos of the Karaoke. The selected video clips, photographs, music and lyrics are well aligned to compose a Karaoke video, connecting by specific content-based transitions.

ICBR - Multimedia Management System for Intelligent Content Based Retrieval

Abstract: This paper presents a system designed for the management of multimedia databases that embarks upon the problem of efficient media processing and representation for automatic semantic classification and modelling. Its objectives are founded on the integration of a large-scale wildlife digital media archive with a manually annotated semantic metadata organised in a structured taxonomy and media classification system. Novel techniques will be applied to temporal analysis, intelligent key-frame extraction, animal gait analysis, semantic modelling and audio classification. The system demonstrator will be developed as a part of an ICBR project within the 3C Research programme of convergent technology research for digital media processing and communications.

R*-Histograms: efficient representation of spatial relations between objects of arbitrary topology

Abstract: Representation of relative spatial relations between objects is often required in many multimedia database applications because spatial relations between objects in an image convey important information about the image. Quantitative representation of spatial relations taking into account shape, size, orientation and distance is often required. The R-Histogram is such a quantitative representation of spatial relations between two objects. However, this method only considers pixels on the object boundary, assuming that the objects are homeomorphic to a 2-ball. For objects with more complicated topology, we propose in this paper the R*-Histogram, a new extension to the R-Histogram. The R*-Histogram generalizes the R-Histogram by taking into account all the pixels in the objects. We also introduce an efficient O(kN log N) time algorithm to compute the R*-Histogram, which is asymptotically faster than the original O(N2) time algorithm for the R-Histogram even when k=O(n). Here, N=n2 denotes the number of pixels in the processed n x n image and k is the number of different directions considered. The effectiveness of the R*-Histogram is evaluated empirically with a Query By Example (QBE) system on a database of 2000 synthetic images containing objects with complicated shape and topology. Experiments have shown that the similarly search results match human intuition very well.

An implementation method of automatic metadata extraction method for music data and its application to semantic associative search

Abstract: In this paper, we present a method for implementing a Kansei (sensitivity) operator for an automatic metadata extraction method used for media data. The Kansei operator, which operates on metadata extracted from media data, performs mapping based on human sensitivity by applying a logarithmic function that reflects human sensitivity based on Fechner's law. The proposed method enables metadata that reflects human sensitivity to be extracted according to the Kansei operator for metadata that was obtained according to weighted word categories output by a conventional automatic metadata extraction method. In addition, using a semantic associative search method enables searching to be performed, which matches human intuition for the media data categories. In this paper, we combine the Kansei operator with an automatic metadata extraction method for music data to implement an automatic metadata extraction method having a Kansei operator for music data. In addition, they show the effectiveness of the proposed method experimentally. © 2004 Wiley Periodicals, Inc. Syst Comp Jpn, 35(6): 59–78, 2004; Published online in Wiley InterScience (). DOI 10.1002sscj.10380

A metadata approach to multimedia database federations

Abstract: Recent research in federated database systems has advanced in the direction of federations of multimedia databases. However, in each project studied, there has been little emphasis placed on the subject of metamodels. Conversely, standard database systems (relational, object-relational and object-oriented) define metadata models that lack multimedia and federation features. In this paper, a specification of a metadata architecture for a multimedia database federation is presented. The architecture is based on a new object-oriented metamodel that has support for multimedia types and federated metadata. By using this metamodel, administrators and users of a multimedia federation can determine precisely how data is structured and the types of operations that are possible.

An efficient bitmap indexing method for similarity search in high dimensional multimedia databases

Abstract: The paper proposes a new indexing mechanism for similarity search in high-dimensional multimedia database; it quickly filters out irrelevant objects using a bitmap index, in which the characteristic of each object is approximated as a bit-string. The bits in a bitstring that are set to '1' denote the representative dimensions of an object that their attribute values are a relatively larger value than others. Since two objects are dissimilar if their representative dimensions are so much different, the degree of dissimilarity can be computed easily by XORing the bit-strings of two objects and counting the number of '1' s in the resulting bit-string. Experimental results with more than 100,000 images show that a remarkable speed-up can be obtained with the proposed indexing method compared to the VA-file and linear scan method because of the simple XORing operation in the filtering process, although there is some loss in search accuracy

A multimodal database framework for multimedia meeting annotations

Abstract: The main objective of this paper is to present a flexible annotation management framework for a multimedia database system, applied to meeting recordings. Presented research and development activities are carried out within the scope of the IM2 project in which annotations play an important role in describing raw data from various points of view and in enhancing the query process. We focus on a database system capable of managing annotations (e.g. text transcriptions, dialog acts, speaker space position, etc.) and keeping links with raw data (audio, video, digital documents). This database provides a schema evolution mechanism and a meta-description layer ensuring flexible and incremental annotation definitions. To enhance this database system, some research works are currently in progress: a predictive methodology for schema evolution and a query technique that deals with fuzzy concepts and ontological commitments. We describe our on-going prototype development, in which we focus on data storage and interactive data access.

Managing large-scale multimedia repositories

Abstract: Capturing and organizing vast volumes of images, such as scientific and medical data, requires new information processing techniques for context of pattern recognition and data mining. In content based retrieval, the main task is the seeking of entries in an image database that are most similar, in some sense, to a given query object. The volume of the data is large, and the feature vectors are, typically, of high dimensionality. In high dimensions, the curse of dimensionality is an issue as the search space grows exponentially with the dimensions. In addition, it is impractical to store all the extracted feature vectors from millions of images in main memory. The time spent accessing the feature vectors on hard storage devices overwhelmingly dominates the time complexity of the search. The time complexity problem is further emphasized when the search is to be performed multiple times in an interactive scenario. One of the main contributions of this dissertation is to enable efficient, effective, and interactive data access. We introduce a modified texture descriptor that has comparable performance but nearly half the dimensionality and less computational expense. Moreover, based on the statistical properties of the texture descriptors, we propose an adaptive for approximate nearest neighbor search indexing approach. In content-based retrieval systems, exact search and retrieval in the feature space is often wasteful. We present an approximate similarity search method for large feature datasets. It improves similarity retrieval efficiency without compromising on the retrieval quality. We also address the computation bottleneck of a real-life system interface. We propose a similarity search scheme that exploits correlations between two consecutive nearest neighbor sets and considerably accelerates interactive search, particularly in the context of relevance feedback mechanisms that support distance metric update approach. In multimedia query processing, the main task is the seeking of entries in a multimedia database that are most similar to a given query object. Since feature descriptors approximately capture information contained in images, they often do not capture visual concepts contained in those images. Semantic analysis of multimedia content is needed. We introduce a framework for learning and summarizing basic semantic concepts in scientific datasets. Moreover, we present a method to detect coarse spatial patterns and visual concepts in image and video datasets. Experiments on a large set of aerial images and video data are presented.

METIS: a flexible database foundation for unified media management

Abstract: Current multimedia database systems largely focus on specific media types and/or application domains. This paper gives a sketch of METIS, a database solution for the unified management of any kind of digital media, characterized by profound customizability for domain or application needs. The system core is based on an expressive data model that can be adapted to any scheme for media management, description, and classification desired, and can accommodate arbitrary query operators, similarity measures, and/or feature extraction algorithms. METIS is based on a persistence abstraction layer allowing the exchange of storage back-ends and includes a customizable web front-end for media management. Thereby, METIS establishes a common database foundation for a wide spectrum of multimedia applications.

An architecture for streamlining the implementation of biomedical text/image databases on the Web

Abstract: To date the implementation of biomedical text/images databases on the Web has been hindered by the lack of software flexibility to enable new datasets to be "rolled in " to existing database systems without extensive programming modifications. An R&D division of the National Library of Medicine (NLM) is creating a new database system, the multimedia database tool (MDT), built on an architecture that moves the required customization for a particular dataset from the programmer to the database administrator level, thereby reducing potentially high labor costs. The first database application to be supported will be uterine cervix data from the National Cancer Institute, including 100,000 digitized images, and NHANES II and III databases now hosted by the NLM Web-based medical information retrieval system (WebMIRS).

Fast and robust video clip search using index structure

Abstract: Content based retrieval of similar multimedia objects (e.g. images, text, and videos) is an important research issue in the field of multimedia database. In this demo, we present a fast and robust video clip searching system. This system consists of two major modules, namely, robust video representation and fast searching. Different from traditional key frame-based histogram methods, we employ the cumulative histogram to represent the ordinal features and color range features for a video segment. This representation provides a spatio-temporal description of the whole segment. Our experiment has shown it is effective for capturing the patterns of short video clips such as commercial, program lead in/out, flying logo in sports video, etc. In order to improve the performance in searching large video database, we introduce the index structure to deal with video search from the viewpoint of query processing (e.g. K-NN query, Range query, etc.) in high-dimensional spaces. Different query processing support different search tasks. In this demo, we employ the mrkd-tree index structure and the proposed video representation to fulfill fast and robust search of short video clips (i.e. news video lead-in/out, replay logo, commercial) in large video collections with the total length of 15 hours.

Electronic multimedia data management for remote population in Ecuador

Abstract: Purpose : A large part of medicine is the management of patient related information inclusive of demographic data, historical data, data-rich multimedia diagnostic studies and longitudinal disease management. Managing the volume of information generated from patient examination and disease management into an accurate and seamless record challenges traditional record systems. This array of patient relevant data is currently recorded with handwritten notes and reviewed by the physician only after accessing the patient's medical record in the office. By integrating an electronic medical record (EMR) into patient management, the physician can access and manage patient data efficiently and accurately. Many successful EMR applications are available for advanced health systems. This study applied a Spanish language EMR developed with the clinical practitioners at remote clinics in Ecuador to support patient management. The data files were shared electronically with their regional collaborating clinics and internationally for clinical consultations. Methods : A relational database format was established with user-friendly interface to contain the necessary fields for data acquisition inclusive of manual data field entry as well as drop down fields for rapid data entry. Still and video format images were integrated to accommodate results from radiographs, sonograms, microscopic and colposcopy examinations. Application of the software at the general medicine clinics in a remote area of Ecuador provided an interface on a desktop computer to capture and share data for general medicine cases, general surgery, pediatrics, as well as obstetrics. The patient files were archived in a secure format within the computer at the clinic with restricted entry by password. Results : Staff and clinicians in the clinics easily recorded demographic, and specific patient care information. Clinicians were able to query electronic files for easy update and for follow-up in patient care practices. The software design allowed file compression in order to include various data file formats from multiple freestanding diagnostic devices to a consolidated EMR. The EMR was utilized locally at its source and shared with consultants at Virginia Commonwealth University (VCU) in a teleconsultation format. Conclusion : The EMR described here can support integration of diverse data formats from multiple diagnostic devices and provides a uniform interface in compliance with federal regulations. Additionally, the EMR data can be extracted in XML format for secure export independent of operating systems. Such a comprehensive database serves the needs with minimal lines of code, minimal need for technical support, simplicity of usage and responsive to local medical and linguistic realities.

Web Access to Large Audiovisual Assets Based on User Preferences

Abstract: Current multimedia databases contain a wealth of information in the form of audiovisual as well as text data. Even though efficient search algorithms have been developed for either media, there still exists the need for abstract presentation and summarization of the results of database users' queries. Moreover, multimedia retrieval systems should be capable of providing the user with additional information related to the specific subject of the query, as well as suggest other topics which could be identified to attract the interest of users with a similar profile. In this paper, we present solutions to these issues, giving as an example an integrated architecture we have developed, along with notions that support efficient and secure Internet access to audiovisual/video databases. Segmentation of each video in shots is followed by shot classification in a number of predetermined categories. Generation of users' profiles according to the categories, enhanced by relevance feedback, permits an efficient presentation of retrieved video shots or characteristic frames in terms of the user interest in them. Moreover, this clustering scheme assists the notion of ‘lateral’ links that enable the user to continue retrieval with data of similar nature or content to those already returned. Furthermore, user groups are formed and modeled by registering actual preferences and practices. This enables the system to ‘predict’ information that is possibly relevant to the user's interest and present it along with the returned results. The concepts utilized in this system can be smoothly integrated in MPEG-7 compatible multimedia database systems.

Video cataloging system for real-time scene change detection of news video

Abstract: It is necessary for various multimedia database applications to develop efficient and fast storage, indexing, browsing, and retrieval of video. We make a cataloging system, which has important information of video data. In this paper, we proposed a method to process huge amount of broadcasting data. It can detect scene-change quickly and effectively for MPEG stream. To extract the DC-image, minimal decoding is employed form MPEG-1 compressed domain. Scene-change is detected by the modified histogram comparative method that combines luminance and color component. A neural network is used to judge whether it is scene-change with an anchorperson and to raise precision of an anchorperson scene extraction.

Acquisition and Annotation of Slovenian Broadcast News Database.

Abstract: This paper presents the Slovenian Broadcast News Database project that was started in year 2002 as cooperation between University of Maribor and Slovenian national broadcaster RTV Slovenia. The resulting database will be used for large vocabulary continuous speech recognition and multimedia database retrieval or archive indexation. First some organizational aspects that were needed in initial phase of the project are described. The raw audio and video material was acquired from the original Analog Beta SP Master tapes that are preserved in the RTV Slovenia’s archive. Raw material was copied to DAT and DVD media. Also additional teletext material was collected. The manual annotation of speech material is performed with the Transcriber tool. The annotation rules were defined on the basis of general rules for Broadcast News databases, with some special language dependent sections. Also some statistics on a part of current material are given.

QuaSAQ: An approach to enabling end-to-end QoS for multimedia databases

Abstract: The paper discusses the design and prototype implementation of a QoS-aware multimedia database system. Recent research in multimedia databases has devoted little attention to the aspect of the integration of QoS support at the user level. Our proposed architecture to enable end-to-end QoS control, the QoS-Aware Query Processor (QuaSAQ), satisfies user specified quality requirements. The users need not be aware of detailed low-level QoS parameters, but rather specifies high-level, qualitative attributes. In addition to an overview of key research issues in the design of QoS-aware databases, this paper presents our proposed solutions, and system implementation details. An important issue relates to the enumeration and evaluation of alternative plans for servicing QoS-enhanced queries. This step follows the conventional query execution which results in the identification of objects of interest to the user. We propose a novel cost model for media delivery that explicitly takes the resource utilization of the plan and the current system contention level into account. Experiments run on the QuaSAQ prototype show significantly improved QoS and system throughput.

Aspect-ARM: An aspect-oriented active rule system for heterogeneous multimedia information

Abstract: An active database mechanism is important to realize new information provision in current mobile and ubiquitous computing environments. Applying an active database mechanism to multidatabase environments realizes the automatic and meaningful information provision from various and heterogeneous databases. In this paper, we propose an aspect-oriented active rule system, called Aspect-ARM, for heterogeneous multimedia information resources. The Aspect-ARM provides an ECA rule language, called an aspect-ECA rule language, in adapting the rule system to the multimedia information in active and dynamic multidatabase environments. The aspect-ECA rule language is designed on a framework of conceptual data modeling for structuring complex multimedia data from multiple viewpoints. This model focuses on two concepts: aspects of multimedia data and relationships among multimedia data. We call this data model an aspect-oriented data model. The feature of this model is to deal with meanings included implicitly in media data in active rule evaluation. By introducing aspect-ECA rules with the aspect-oriented data model, it is possible to provide the active database functionality in cross-over multimedia database environments without affecting application structures and local information system structures.

QuaSAQ: An Approach to Enabling End-to-End QoS for Multimedia Databases

Abstract: The paper discusses the design and prototype implementation of a QoS-aware multimedia database system Recent research in multimedia databases has devoted little attention to the aspect of the integration of QoS support at the user level Our proposed architecture to enable end-to-end QoS control, the QoS-Aware Query Processor (QuaSAQ), satisfies user specified quality requirements The users need not be aware of detailed low-level QoS parameters, but rather specifies high-level, qualitative attributes In addition to an overview of key research issues in the design of QoS-aware databases, this paper presents our proposed solutions, and system implementation details An important issue relates to the enumeration and evaluation of alternative plans for servicing QoS-enhanced queries This step follows the conventional query execution which results in the identification of objects of interest to the user We propose a novel cost model for media delivery that explicitly takes the resource utilization of the plan and the current system contention level into account Experiments run on the QuaSAQ prototype show significantly improved QoS and system throughput

On the design of Web-based interactive multimedia contents for English learning

Abstract: We present a Web-based authoring tool to create interactive multimedia contents for English learning. Target users of this authoring tool are English teachers and content designer, therefore, ease of use and simplicity are the fundamental issues. Furthermore, the authoring tool and multimedia database are integrated in the context of intelligent Web-based interactive language learning (IWiLL) system. Special language tools to access corpus, manipulate multimedia elements, and create collaborative learning sessions are designed in the system. These resources and tools on hand yield the potential to create rich and deep interactive multimedia contents.