Extending beyond the boundaries of science, art, and culture, content-based multimedia information retrieval provides new paradigms and methods for searching through the myriad variety of media all over the world. This survey reviews 100p recent articles on content-based multimedia information retrieval and discusses their role in current research directions which include browsing and search paradigms, user studies, affective computing, learning, semantic queries, new features and media types, high performance indexing, and evaluation techniques. Based on the current state of the art, we discuss the major challenges for the future.

/pdf/content-based-multimedia-information-retrieval-state-of-the-41z647ll8w.pdf

Content-based multimedia information retrieval: State of the art and challenges

An adaptive interface for a programmable system, for predicting a desired user function, based on user history, as well as machine internal status and context. The apparatus receives an input from the user and other data. A predicted input is presented for confirmation by the user, and the predictive mechanism is updated based on this feedback. Also provided is a pattern recognition system for a multimedia device, wherein a user input is matched to a video stream on a conceptual basis, allowing inexact programming of a multimedia device. The system analyzes a data stream for correspondence with a data pattern for processing and storage. The data stream is subjected to adaptive pattern recognition to extract features of interest to provide a highly compressed representation that may be efficiently processed to determine correspondence. Applications of the interface and system include a video cassette recorder (VCR), medical device, vehicle control system, audio device, environmental control system, securities trading terminal, and smart house. The system optionally includes an actuator for effecting the environment of operation, allowing closed-loop feedback operation and automated learning.

Adaptive pattern recognition based control system and method

Systems and methods for data classification to facilitate and improve data management within an enterprise are described. The disclosed systems and methods evaluate and define data management operations based on data characteristics rather than data location, among other things. Also provided are methods for generating a data structure of metadata that describes system data and storage operations. This data structure may be consulted to determine changes in system data rather than scanning the data files themselves.

Systems and methods for classifying and transferring information in a storage network

A system and method of the present invention allow users to access, manage, and edit information about content data, often referred to as metadata. Metadata is collected from various sources, added, and maintained in a metadata database. In addition, metadata is dynamically read from the metadata database and dynamically displayed in a graphical user interface in an organized manner, such as a hierarchical tree. In the graphical user interface, a user may add, delete, and/or modify the metadata. As the user changes the metadata, the metadata database is updated and the user's changes are propagated throughout the graphical user interface such that the hierarchical tree is displays the changed metadata.

System and method of managing metadata data

We present a computer audition system that can both annotate novel audio tracks with semantically meaningful words and retrieve relevant tracks from a database of unlabeled audio content given a text-based query. We consider the related tasks of content-based audio annotation and retrieval as one supervised multiclass, multilabel problem in which we model the joint probability of acoustic features and words. We collect a data set of 1700 human-generated annotations that describe 500 Western popular music tracks. For each word in a vocabulary, we use this data to train a Gaussian mixture model (GMM) over an audio feature space. We estimate the parameters of the model using the weighted mixture hierarchies expectation maximization algorithm. This algorithm is more scalable to large data sets and produces better density estimates than standard parameter estimation techniques. The quality of the music annotations produced by our system is comparable with the performance of humans on the same task. Our ldquoquery-by-textrdquo system can retrieve appropriate songs for a large number of musically relevant words. We also show that our audition system is general by learning a model that can annotate and retrieve sound effects.

/pdf/semantic-annotation-and-retrieval-of-music-and-sound-effects-14hge5xy7b.pdf

Semantic Annotation and Retrieval of Music and Sound Effects

The recent progress of Electronic Music Distribution creates a natural pressure for fine-grained musical metadata. This metadata is needed to provide music distribution services which are able to cope with the mere size of music catalogues, and the desire of users to access music titles by similarity. In this context, we describe a project of a global music title metadatabase, and focus in the particular "genre" descriptor. We analyze existing taxonomies of musical genre as found in the music industry and on the Internet, and stress on their inconsistencies. We describe a novel music genre taxonomy based on a few guiding principles, and report on the process of building this taxonomy.

A taxonomy of musical genres

Advances in networking and transmission of digital multimedia data will soon bring huge catalogues of music to users. Accessing these catalogues raises a problem for users and content providers, that we define as the music selection problem. We introduce three main goals to be satisfied in music selection: match user preferences, provide users with new music, and exploit the catalogue in an optimal fashion. We propose a novel approach to music selection, based on computing coherent sequences of music titles, and show that this amounts to solving a combinatorial pattern generation problem. We propose constraint satisfaction techniques to solve it. The resulting system is an enabling technology to build better music delivery services.

/pdf/a-combinatorial-approach-to-content-based-music-selection-1q3g7dyjc9.pdf

A combinatorial approach to content-based music selection

A method and a system for generating sequencing information representing a sequence of items selected in a database. Similarity relation techniques are applied between the items.

Method and system for generating sequencing information representing a sequence of items selected in a database

Accessing large digital music catalogues raises a problem for both users and content providers. We propose a novel approach to music selection called RecitalComposer, which is based on computing coherent sequences of music titles. This amounts to solving a combinatorial pattern generation problem by using constraint satisfaction techniques.

/pdf/a-combinatorial-approach-to-content-based-music-selection-5esuim4g4j.pdf

The present invention relates to a method designed to be used to compute music sequences in all possible contexts and situations, including: Internet adaptive radio, Digital Audio Broadcasting (DAB) with intelligent scheduling, music recommendation systems, and other innovative Electronic Music Distribution (EMD) services in general. These sequences are generated “iteratively”, and step by step. To the above end, there is provided a method of generating incrementally a sequence of items from a database containing said items, characterised in that the sequence is generated by implementing in combination a sequence completion system and a user profiling system, thereby taking into account both sequence coherence and user profile.

Daniel Cazaly

Papers

A taxonomy of musical genres

A combinatorial approach to content-based music selection

Method and system for generating sequencing information representing a sequence of items selected in a database

A combinatorial approach to content-based music selection

Incremental sequence completion system and method