Video Shot Boundary Detection: A Review
01 Jan 2015-pp 119-127
TL;DR: This paper presents different approaches to shot boundary detection problem, and shows how segmentation plays an important role in digital media processing, pattern recognition, and computer vision.
Abstract: Video image processing is a technique to handle the video data in an effective and efficient way. It is one of the most popular aspects in the video and image based technologies such as surveillance. Shot change boundary detection is also one of the major research areas in video signal processing. Previous works have developed various algorithms in this domain. In this paper, a brief literature survey is presented that establishes an overview of the works that has been done previously. In this paper we have discussed few algorithms that were proposed previously which also includes histogram based, DCT based and motion vector based algorithms as well as their advantages and their limitations.
Citations
More filters
TL;DR: New models and algorithms for object-level video advertising that aims to embed content-relevant ads within a video stream is investigated and a heuristic algorithm is developed to solve the proposed optimization problem.
Abstract: In this paper, we present new models and algorithms for object-level video advertising. A framework that aims to embed content-relevant ads within a video stream is investigated in this context. First, a comprehensive optimization model is designed to minimize intrusiveness to viewers when ads are inserted in a video. For human clothing advertising, we design a deep convolutional neural network using face features to recognize human genders in a video stream. Human parts alignment is then implemented to extract human part features that are used for clothing retrieval. Second, we develop a heuristic algorithm to solve the proposed optimization problem. For comparison, we also employ the genetic algorithm to find solutions approaching the global optimum. Our novel framework is examined in various types of videos. Experimental results demonstrate the effectiveness of the proposed method for object-level video advertising.
159 citations
TL;DR: This paper presents a review of an extensive set for SBD approaches and their development, and the advantages and disadvantages of each approach are comprehensively explored.
Abstract: The recent increase in the number of videos available in cyberspace is due to the availability of multimedia devices, highly developed communication technologies, and low-cost storage devices. These videos are simply stored in databases through text annotation. Content-based video browsing and retrieval are inefficient due to the method used to store videos in databases. Video databases are large in size and contain voluminous information, and these characteristics emphasize the need for automated video structure analyses. Shot boundary detection (SBD) is considered a substantial process of video browsing and retrieval. SBD aims to detect transition and their boundaries between consecutive shots; hence, shots with rich information are used in the content-based video indexing and retrieval. This paper presents a review of an extensive set for SBD approaches and their development. The advantages and disadvantages of each approach are comprehensively explored. The developed algorithms are discussed, and challenges and recommendations are presented.
56 citations
Cites methods from "Video Shot Boundary Detection: A Re..."
...Hence, a robust, efficient, automated SBD method is an urgent requirement [11,19]....
[...]
TL;DR: It was found that strategies for cut-based segmentation, color-based indexing, k-means based dimensionality reduction and data clustering have been the most frequent choices in recent papers.
Abstract: Content-based video retrieval and indexing have been associated with intelligent methods in many applications such as education, medicine and agriculture. However, an extensive and replicable review of the recent literature is missing. Moreover, relevant topics that can support video retrieval, such as dimensionality reduction, have not been surveyed. This work designs and conducts a systematic review to find papers able to answer the following research question: “what segmentation, feature extraction, dimensionality reduction and machine learning approaches have been applied for content-based video indexing and retrieval?”. By applying a research protocol proposed by us, 153 papers published from 2011 to 2018 were selected. As a result, it was found that strategies for cut-based segmentation, color-based indexing, k-means based dimensionality reduction and data clustering have been the most frequent choices in recent papers. All the information extracted from these papers can be found in a publicly available spreadsheet. This work also indicates additional findings and future research directions.
47 citations
TL;DR: The proposed shot boundary detection approach using Genetic Algorithm and Fuzzy Logic is compared to latest techniques and yields better result in terms of F1score parameter.
Abstract: This paper proposed a shot boundary detection approach using Genetic Algorithm and Fuzzy Logic. In this, the membership functions of the fuzzy system are calculated using Genetic Algorithm by taking preobserved actual values for shot boundaries. The classification of the types of shot transitions is done by the fuzzy system. Experimental results show that the accuracy of the shot boundary detection increases with the increase in iterations or generations of the GA optimization process. The proposed system is compared to latest techniques and yields better result in terms of F1score parameter.
45 citations
Cites methods from "Video Shot Boundary Detection: A Re..."
...Color histogram is a global feature extraction technique which is one of the simplest and widely used image feature extractions for shot boundary detection [19]....
[...]
TL;DR: A novel video summarization approach called VISCOM is proposed, which is based on color co-occurrence matrices to describe the video frames and generate a synopsis with the most representative frames, to produce competitive outcomes among the evaluated methods.
Abstract: Video summarization techniques have allowed the content analysis of large volumes of digital video sequences of different categories, such as movies, documentaries, lectures, sports, surveillance, and news. This paper proposes and evaluates a novel video summarization approach called VISCOM, which is based on color co-occurrence matrices to describe the video frames and generate a synopsis with the most representative frames. Experiments conducted on two different data sets of various genres demonstrate the effectiveness of the proposed method in terms of quality. The resulting video summaries are compared against several others using a specific quantitative evaluation metric, producing competitive outcomes among the evaluated methods.
37 citations
Cites background from "Video Shot Boundary Detection: A Re..."
...A disadvantage of the SBD techniques is that they are not applicable to videos that do not have any sort of previous edition (e.g., home videos), since they look for specific transitions between video contents....
[...]
...Approaches to SBD include: pixelwise differences [65], color histograms [26], compressed domain techniques [15] and motion vectors [3]....
[...]
...Most of the SBD algorithms rely on comparisons of color information of video frames, which are done from a specific metric that calculates the distance (dissimilarity) between two different frames [5, 9, 28, 35, 40, 47]....
[...]
...Shot boundary detection (SBD) [45, 53] is a fundamental step in video analysis, being applicable not only in video summarization but also in other tasks such as video indexing, browsing and retrieval....
[...]
References
More filters
07 May 2006
TL;DR: A novel scale- and rotation-invariant interest point detector and descriptor, coined SURF (Speeded Up Robust Features), which approximates or even outperforms previously proposed schemes with respect to repeatability, distinctiveness, and robustness, yet can be computed and compared much faster.
Abstract: In this paper, we present a novel scale- and rotation-invariant interest point detector and descriptor, coined SURF (Speeded Up Robust Features). It approximates or even outperforms previously proposed schemes with respect to repeatability, distinctiveness, and robustness, yet can be computed and compared much faster.
This is achieved by relying on integral images for image convolutions; by building on the strengths of the leading existing detectors and descriptors (in casu, using a Hessian matrix-based measure for the detector, and a distribution-based descriptor); and by simplifying these methods to the essential. This leads to a combination of novel detection, description, and matching steps. The paper presents experimental results on a standard evaluation set, as well as on imagery obtained in the context of a real-life object recognition application. Both show SURF's strong performance.
13,011 citations
TL;DR: Several methods for filter design are described for dual-tree CWT that demonstrates with relatively short filters, an effective invertible approximately analytic wavelet transform can indeed be implemented using the dual- tree approach.
Abstract: The paper discusses the theory behind the dual-tree transform, shows how complex wavelets with good properties can be designed, and illustrates a range of applications in signal and image processing The authors use the complex number symbol C in CWT to avoid confusion with the often-used acronym CWT for the (different) continuous wavelet transform The four fundamentals, intertwined shortcomings of wavelet transform and some solutions are also discussed Several methods for filter design are described for dual-tree CWT that demonstrates with relatively short filters, an effective invertible approximately analytic wavelet transform can indeed be implemented using the dual-tree approach
2,407 citations
TL;DR: A texture segmentation algorithm inspired by the multi-channel filtering theory for visual information processing in the early stages of human visual system is presented, which is based on reconstruction of the input image from the filtered images.
Abstract: This paper presents a texture segmentation algorithm inspired by the multi-channel filtering theory for visual information processing in the early stages of human visual system. The channels are characterized by a bank of Gabor filters that nearly uniformly covers the spatial-frequency domain, and a systematic filter selection scheme is proposed, which is based on reconstruction of the input image from the filtered images. Texture features are obtained by subjecting each (selected) filtered image to a nonlinear transformation and computing a measure of “energy” in a window around each pixel. A square-error clustering algorithm is then used to integrate the feature images and produce a segmentation. A simple procedure to incorporate spatial information in the clustering process is proposed. A relative index is used to estimate the “true” number of texture categories.
2,351 citations
Book•
08 Oct 2001
TL;DR: This book takes a nontraditional nonlinear approach and reflects the fact that most practical applications are nonlinear.
Abstract: From the Publisher:
Kalman filtering is a well-established topic in the field of control and signal processing and represents by far the most refined method for the design of neural networks. This book takes a nontraditional nonlinear approach and reflects the fact that most practical applications are nonlinear. The book deals with important applications in such fields as control, financial forecasting, and idle speed control.
1,960 citations
"Video Shot Boundary Detection: A Re..." refers methods in this paper
...Using Kalman Filtering [48] these features are matched with the features of the subsequent frames, accordingly with the changing pattern of pixel intensity shot boundary is detected....
[...]
TL;DR: A twin-comparison approach has been developed to solve the problem of detecting transitions implemented by special effects, and a motion analysis algorithm is applied to determine whether an actual transition has occurred.
Abstract: Partitioning a video source into meaningful segments is an important step for video indexing. We present a comprehensive study of a partitioning system that detects segment boundaries. The system is based on a set of difference metrics and it measures the content changes between video frames. A twin-comparison approach has been developed to solve the problem of detecting transitions implemented by special effects. To eliminate the false interpretation of camera movements as transitions, a motion analysis algorithm is applied to determine whether an actual transition has occurred. A technique for determining the threshold for a difference metric and a multi-pass approach to improve the computation speed and accuracy have also been developed.
1,360 citations