Showing papers on "Multimedia database published in 2020"

I/O Efficient Approximate Nearest Neighbour Search based on Learned Functions

Abstract: Approximate nearest neighbour search (ANNS) in high dimensional space is a fundamental problem in many applications, such as multimedia database, computer vision and information retrieval. Among many solutions, data-sensitive hashing-based methods are effective to this problem, yet few of them are designed for external storage scenarios and hence do not optimized for I/O efficiency during the query processing. In this paper, we introduce a novel data-sensitive indexing and query processing framework for ANNS with an emphasis on optimizing the I/O efficiency, especially, the sequential I/Os. The proposed index consists of several lists of point IDs, ordered by values that are obtained by learned hashing (i.e., mapping) functions on each corresponding data point. The functions are learned from the data and approximately preserve the order in the high-dimensional space. We consider two instantiations of the functions (linear and non-linear), both learned from the data with novel objective functions. We also develop an I/O efficient ANNS framework based on the index. Comprehensive experiments on six benchmark datasets show that our proposed methods with learned index structure perform much better than the state-of-the-art external memory-based ANNS methods in terms of I/O efficiency and accuracy.

A Distributed M-Tree for Similarity Search in Large Multimedia Database on Spark

Abstract: In this chapter, Image2vec or Video2vector are used to convert images and video clips to vectors in large multimedia database. The M-tree is an index structure that can be used for the efficient resolution of similarity queries on complex objects. M-tree can be profitably used for content-based retrieval on multimedia databases provided relevant features have been extracted from the objects. In a large multimedia database, to search for similarities such as k-NN queries and Range queries, distances from the query object to all remaining objects (images or video clips) are calculated. The calculation between query and entities in a large multimedia database is not feasible. This chapter proposes a solution to distribute the M-Tree structure on the Apache Spark framework to solve the Range Query and kNN Query problems in large multimedia database with a lot of images and video clips.

Multimedia detection method based on knowledge graph

Abstract: The invention discloses a multimedia detection method based on a knowledge graph. The multimedia detection method comprises the steps that each user uploads a shared multimedia file and a custom labelto a multimedia database; for a picture file and a video file containing portraits in the multimedia file, face detection, face recognition and face comparison are carried out to construct a portraitlibrary; for the picture file and the video file from which the portraits are removed in the multimedia file, processing is carried out through image classification and target detection, and a sceneand a real object are identified; for text files in the multimedia files, the text files are classified through a universal classifier and an educational classifier, and category labels are marked onthe text files according to classification results; a file atlas is formed for each user; a character relationship network is constructed; and retrieval is performed by the user based on the file atlas, the character library and the character relationship network. And multimedia attribute association retrieval is carried out on the basis of the knowledge graph, so that the user can use the methodconveniently.

An Announcer Shot Detection Algorithm for TV News Programs

Abstract: In this paper, an automatic announcer shot detection algorithm is proposed. Based on the statistical law of frequent occurrence of announcer shots, the candidate announcer shots are found out based on the automatic clustering method, and then the candidate announcer shots are confirmed by neural network classifier according to the temporal and spatial characteristics of the announcer shots. The algorithm is one of the important means to automatically analyze the content of TV news programs, and it is essential to establish a video database index. Experimental results show that the method is accurate and fast, and can be effectively applied to video retrieval system.

Automated songwriting generation system and method thereof

Abstract: This invention provides an automated songwriting generation system and a method thereof comprising a tune analysis engine applied for analyzing a popular music tune structure through a neural network to construct a tune combination model based on a ranking order of a multimedia database; a tune analysis engine applied for analyzing a lyrics structure of popular music and the sentence structure from a text database through the neural network to construct a lyric combination model based on the ranking order of the multimedia database and the text database; a style selection unit that provides a predefined frame having various genre attributes or various style attributes; a lyric selection unit that provides a plurality of lyrics filling fields corresponding to a lyrics set based on the lyric combination model for offering selection or modification; and a tune selection unit that provides plurality of tune filling fields corresponding to a tune set based on the tune combination model for offering selection or modification.

Audio and video multimedia database construction and multimedia subjective quality evaluation method

Abstract: The invention provides an audio and video multimedia database construction and multimedia subjective quality evaluation method. The method comprises the following steps: creating a joint quality subjective evaluation audio and video multimedia database specially used for audio and video multimedia, based on the database, establishing an audio and video multimedia quality evaluation environment, selecting data in the audio and video multimedia database for training a test subject, selecting to-be-tested data from audio and video multimedia database for testing, and carrying out later data processing on the subjective quality evaluation result obtained by testing. Standardized and process audio and video multimedia subjective quality evaluation is realized.