With the proliferation of online services and mobile technologies, the world has stepped into a multimedia big data era. A vast amount of research work has been done in the multimedia area, targeting different aspects of big data analytics, such as the capture, storage, indexing, mining, and retrieval of multimedia big data. However, very few research work provides a complete survey of the whole pine-line of the multimedia big data analytics, including the management and analysis of the large amount of data, the challenges and opportunities, and the promising research directions. To serve this purpose, we present this survey, which conducts a comprehensive overview of the state-of-the-art research work on multimedia big data analytics. It also aims to bridge the gap between multimedia challenges and big data solutions by providing the current big data frameworks, their applications in multimedia analyses, the strengths and limitations of the existing methods, and the potential future directions in multimedia big data analytics. To the best of our knowledge, this is the first survey that targets the most recent multimedia management techniques for very large-scale data and also provides the research studies and technologies advancing the multimedia analyses in this big data era.

Multimedia Big Data Analytics: A Survey

Big data has attracted much attention from academia and industry. But the discussion of big data is disparate, fragmented and distributed among different outlets. This paper conducts a systematic a...

Big data analytics and big data science: a survey

Digital image watermarking is the process of embedding and extracting a watermark covertly on a cover-image. To dynamically adapt image watermarking algorithms, deep learning–based image watermarking schemes have attracted increased attention during recent years. However, existing deep learning–based watermarking methods neither fully apply the fitting ability to learn and automate the embedding and extracting algorithms, nor achieve the properties of robustness and blindness simultaneously. In this paper, a robust and blind image watermarking scheme based on deep learning neural networks is proposed. To minimize the requirement of domain knowledge, the fitting ability of deep neural networks is exploited to learn and generalize an automated image watermarking algorithm. A deep learning architecture is specially designed for image watermarking tasks, which will be trained in an unsupervised manner to avoid human intervention and annotation. To facilitate flexible applications, the robustness of the proposed scheme is achieved without requiring any prior knowledge or adversarial examples of possible attacks. A challenging case of watermark extraction from phone camera–captured images demonstrates the robustness and practicality of the proposal. The experiments, evaluation, and application cases confirm the superiority of the proposed scheme.

An Automated and Robust Image Watermarking Scheme Based on Deep Neural Networks

Machine learning based blind color image watermarking scheme for copyright protection

Digital image watermarking is the process of embedding and extracting a watermark covertly on a cover-image. To dynamically adapt image watermarking algorithms, deep learning-based image watermarking schemes have attracted increased attention during recent years. However, existing deep learning-based watermarking methods neither fully apply the fitting ability to learn and automate the embedding and extracting algorithms, nor achieve the properties of robustness and blindness simultaneously. In this paper, a robust and blind image watermarking scheme based on deep learning neural networks is proposed. To minimize the requirement of domain knowledge, the fitting ability of deep neural networks is exploited to learn and generalize an automated image watermarking algorithm. A deep learning architecture is specially designed for image watermarking tasks, which will be trained in an unsupervised manner to avoid human intervention and annotation. To facilitate flexible applications, the robustness of the proposed scheme is achieved without requiring any prior knowledge or adversarial examples of possible attacks. A challenging case of watermark extraction from phone camera-captured images demonstrates the robustness and practicality of the proposal. The experiments, evaluation, and application cases confirm the superiority of the proposed scheme.

Watermarking plays an important role in identifying the copyright of an image and related issues. The state-of-the-art watermark embedding schemes, spread spectrum and quantization, suffer from host signal interference (HSI) and scaling attacks, respectively. Both of them use a fixed embedding parameter, which is difficult to take both robustness and imperceptibility into account for all images. This paper solves the problems by proposing two novel blind watermarking schemes: a spread spectrum scheme with adaptive embedding strength (SSAES) and a differential quantization scheme with adaptive quantization threshold (DQAQT). Their adaptiveness comes from the proposed adaptive embedding strategy (AEP), which maximizes the embedding strength or quantization threshold by guaranteeing the peak signal-to-noise ratio (PSNR) of the host image after embedding the watermark, and strikes the balance between robustness and imperceptibility. SSAES is HSI free by factoring in the priori knowledge about HSI. In DQAQT, an effective quantization mode is proposed to resist scaling attacks by utilizing the difference between two selected DCT coefficients with high stability. Both SSAES and DQAQT can be easily applied to other watermarking frameworks. We introduce a notion called error threshold to theoretically analyze the performance of our proposed methods in details. The experimental results consistently demonstrate that SSAES and DQAQT outperform the state-of-the-art methods in terms of imperceptibility, robustness, computational cost, and adaptability.

Enhancing Image Watermarking With Adaptive Embedding Parameter and PSNR Guarantee

Fast audio retrieval is crucial for many important applications and yet demanding due to the high dimension nature and increasingly larger volume of audios on the Internet. Although audio fingerprinting can greatly reduce its dimension while keeping audio identifiable, the dimension for audio fingerprints is still too high to scale up for big audio data. The tradeoff between accuracy (measured by precision and recall rate) and efficiency (measured by retrieval time) prevents further reduction in the dimension of fingerprints. This paper shows that a multi-stage filtering strategy can achieve both speedup and high accuracy, with the beginning stages focusing on speedup and the end stage emphasizing accuracy. With this strategy, an efficient cascaded filtering retrieval method is proposed that consists of filtering with Fibonacci hashing, the middle fingerprint, thresholds to quickly select candidate audios, and refining with an accurate and robust fingerprint on the candidate audios. Experiments with 500 000 audios show that the proposed method can achieve a speed gain more than 28 K times that of the Fibonacci hashing retrieval. After applying MP3 conversion, resampling, white noise addition, and background noise addition, the recall rates of the method are all above 99.45%, and the precision is the same as the Philips audio fingerprint, which is close to 100%.

An Efficient Cascaded Filtering Retrieval Method for Big Audio Data

It is challenging to retrieve audio clips from large audio datasets not only due to the high dimensionality of audio but also due to the large number of audios. Fingerprinting methods primarily focus on the use of semantic-level techniques to speed up retrieval and neglect low-level support. This paper shows that the performance of audio retrieval can be exploited by properly organizing and manipulating audio fingerprint data. The proposed sampling and counting method markedly improves the retrieval speed while maintaining a high recall rate and high precision for short audio clips. The proposed inverted index structure for fingerprints quickly shrinks the scope of the candidate set while requiring considerably less memory. The experiments show that the proposed method is faster and yields more consistent performance in terms of recall rate and precision than do the state-of-the-art methods. The proposed method is scalable for big audio data and is robust to background noise, resampling, MP3 conversion, white noise addition, bandpass filtering, chorus, echo, flanger, gsm compression, and tremolo. The average precision and recall rates are 99.995% and 99.78%, respectively, for dataset 1 and average 98.97% and 95.15%, respectively, for dataset 2. Specifically, the precision and recall rates for query audio clips of 3 s in length in the targeted dataset 1 are 99.94% and 99.64%, respectively.

Audio Identification by Sampling Sub-fingerprints and Counting Matches

Fast audio retrieval is crucial for many important applications and yet demanding due to the high dimension nature and increasingly larger volume of audios in the internet. Although audio fingerprinting can greatly reduce its dimension while keeping audio identifiable, the dimension of audio fingerprints is still too high to scale up for big audio data. The tradeoff between the accuracy and the efficiency prevents the further reducing of the dimension of fingerprints. This paper proposes a multi-stage filtering strategy for audio retrieval, with the beginning stages focusing on speed up by using a middle fingerprint with much smaller size to quickly filtering the most likely audios, and the ending stages emphasizing on accuracy by applying an accurate and robust fingerprint on the small set of the most likely audios. A notion called middle fingerprint is devised with considerable small dimension for quickly filtering out most irrelevant audios. A matching algorithm is developed to reduce the computational complexity by comparing the samples at fixed interval of two audios with thresholds. By using the middle fingerprint, audio retrieval can get a speed gain of 12 times on average compared with the Fibonacci Hashing retrieval. By combing the Fibonacci hashing algorithm with the middle filtering retrieval and the matching algorithm, we propose an efficient cascaded filtering retrieval methods, which can further improve the retrieval by 250 times on average. After applying MP3 conversion, resampling, and random shearing, the recall rates of the method are all above 99.47%, and the theoretical accuracy is close to 100%.

A step type voice frequency retrieval method comprises the steps that the Fibonacci Hash algorithm is used to establish a Hash index table for a voice frequency original fingerprint database; the voice frequency original fingerprint database is converted through the BoF algorithm to generate a voice frequency intermediate fingerprint database corresponding to the voice frequency original fingerprint database; by screening for three times, a third serial number set of a possible outcome is screened out, and the retrieval range is narrowed once again; in original fingerprints corresponding to the third serial number set, accurate match retrieval is conducted with the original fingerprints of a voice frequency clip to be retrieved, and the final retrieval result is obtained. By means of the step type voice frequency retrieval method, when the quick voice frequency retrieval is conducted, on the premise that the precision is not lowered, the calculated amount can be reduced, it is achieved that the efficiency is improved, and the use of internal storage can be reduced.

Baoning Niu

Papers

Enhancing Image Watermarking With Adaptive Embedding Parameter and PSNR Guarantee

An Efficient Cascaded Filtering Retrieval Method for Big Audio Data

Audio Identification by Sampling Sub-fingerprints and Counting Matches

An Efficient Cascaded Filtering Retrieval Method for Big Audio Data

Step type voice frequency retrieval method