A fuzzy logic approach for detection of video shot boundaries
TL;DR: A fuzzy logic approach to integrate hybrid features for detecting shot boundaries inside general videos by using publicly available test data set from Carleton University and demonstrating that the proposed algorithm outperforms the representative existing algorithms in terms of the precision and recall rates.
Abstract: Video temporal segmentation is normally the first and important step for content-based video applications. Many features including the pixel difference, colour histogram, motion, and edge information etc. have been widely used and reported in the literature to detect shot cuts inside videos. Although existing research on shot cut detection is active and extensive, it still remains a challenge to achieve accurate detection of all types of shot boundaries with one single algorithm. In this paper, we propose a fuzzy logic approach to integrate hybrid features for detecting shot boundaries inside general videos. The fuzzy logic approach contains two processing modes, where one is dedicated to detection of abrupt shot cuts including those short dissolved shots, and the other for detection of gradual shot cuts. These two modes are unified by a mode-selector to decide which mode the scheme should work on in order to achieve the best possible detection performances. By using the publicly available test data set from Carleton University, extensive experiments were carried out and the test results illustrate that the proposed algorithm outperforms the representative existing algorithms in terms of the precision and recall rates.
Citations
More filters
TL;DR: Experimental results show that the proposed fuzzy color histogram-based shot-boundary detection algorithm effectively detects shot boundaries and reduces false alarms as compared to the state-of-the-art shot- boundary detection algorithms.
Abstract: We present a fuzzy color histogram-based shot-boundary detection algorithm specialized for content-based copy detection applications The proposed method aims to detect both cuts and gradual transitions (fade, dissolve) effectively in videos where heavy transformations (such as cam-cording, insertions of patterns, strong re-encoding) occur Along with the color histogram generated with the fuzzy linking method on L*a*b* color space, the system extracts a mask for still regions and the window of picture-in-picture transformation for each detected shot, which will be useful in a content-based copy detection system Experimental results show that our method effectively detects shot boundaries and reduces false alarms as compared to the state-of-the-art shot-boundary detection algorithms
88 citations
Additional excerpts
...Grana et al. [3] propose a two-step iterative algorithm, unique for both cuts and gradual transitions detection, in the presence of fast object motion and camera operations....
[...]
TL;DR: The problem of making a balance between detection accuracy and speed is addressed in this paper, and a novel fast detection framework is presented that can improve both detection speed and precision.
Abstract: Video shot boundary detection is the initial and fundamental step towards video indexing, browsing and retrieval. Great efforts have been paid on developing accurate shot boundary detection algorithms. However, the high computational cost in shot detection becomes a bottleneck for real-time applications. The problem of making a balance between detection accuracy and speed is addressed in this paper, and a novel fast detection framework is presented. The general framework that employs pre-processing techniques can improve both detection speed and precision. In the pre-processing stage, adaptive local thresholding is adopted to classify non-boundary segments and candidate segments that may contain shot boundaries. The candidate segments are refined using bisection-based comparisons to eliminate non-boundary frames. Only refined candidate segments are preserved for further detections; hence, the speed of shot detection is improved by reducing detection scope. Moreover, prior knowledge about each possible shot boundary such as its type and duration can be obtained in the pre-processing stage, which can accelerate the consequent hard cut and gradual transition detections. Experimental results indicate that the proposed framework is effective in accelerating the shot detection process, and it can also achieve excellent detection accuracies.
65 citations
TL;DR: Wang et al. as mentioned in this paper proposed an efficient convolutional neural network architecture for video summarization via Global Diverse Attention (GDA), which adapts attention mechanism in a global perspective to consider pairwise temporal relations of video frames.
Abstract: Video summarization is an effective way to facilitate video searching and browsing. Most of existing systems employ encoder-decoder based recurrent neural networks, which fail to explicitly diversify the system-generated summary frames while requiring intensive computations. In this paper, we propose an efficient convolutional neural network architecture for video SUMmarization via Global Diverse Attention called SUM-GDA, which adapts attention mechanism in a global perspective to consider pairwise temporal relations of video frames. Particularly, the GDA module has two advantages: (1) it models the relations within paired frames as well as the relations among all pairs, thus capturing the global attention across all frames of one video; (2) it reflects the importance of each frame to the whole video, leading to diverse attention on these frames. Thus, SUM-GDA is beneficial for generating diverse frames to form satisfactory video summary. Extensive experiments on three data sets, i.e., SumMe, TVSum, and VTW, have demonstrated that SUM-GDA and its extension outperform other competing state-of-the-art methods with remarkable improvements. In addition, the proposed models can be run in parallel with significantly less computational costs, which helps the deployment in highly demanding applications.
60 citations
TL;DR: A multi-modal visual features-based SBD framework is employed that aims to analyze the behaviors of visual representation in terms of the discontinuity signal and can achieve good accuracy in both types of video data set compared with other proposed SBD methods.
Abstract: One of the essential pre-processing steps of semantic video analysis is the video shot boundary detection (SBD). It is the primary step to segment the sequence of video frames into shots. Many SBD systems using supervised learning have been proposed for years; however, the training process still remains its principal limitation. In this paper, a multi-modal visual features-based SBD framework is employed that aims to analyze the behaviors of visual representation in terms of the discontinuity signal. We adopt a candidate segment selection that performs without the threshold calculation but uses the cumulative moving average of the discontinuity signal to identify the position of shot boundaries and neglect the non-boundary video frames. The transition detection is structurally performed to distinguish candidate segment into a cut transition and a gradual transition, including fade in/out and logo occurrence. Experimental results are evaluated using the golf video clips and the TREC2001 documentary video data set. Results show that the proposed SBD framework can achieve good accuracy in both types of video data set compared with other proposed SBD methods.
57 citations
TL;DR: This paper presents a review of an extensive set for SBD approaches and their development, and the advantages and disadvantages of each approach are comprehensively explored.
Abstract: The recent increase in the number of videos available in cyberspace is due to the availability of multimedia devices, highly developed communication technologies, and low-cost storage devices. These videos are simply stored in databases through text annotation. Content-based video browsing and retrieval are inefficient due to the method used to store videos in databases. Video databases are large in size and contain voluminous information, and these characteristics emphasize the need for automated video structure analyses. Shot boundary detection (SBD) is considered a substantial process of video browsing and retrieval. SBD aims to detect transition and their boundaries between consecutive shots; hence, shots with rich information are used in the content-based video indexing and retrieval. This paper presents a review of an extensive set for SBD approaches and their development. The advantages and disadvantages of each approach are comprehensively explored. The developed algorithms are discussed, and challenges and recommendations are presented.
56 citations
Cites background from "A fuzzy logic approach for detectio..."
...HT refers to a sudden change in temporal visual information, in which two consecutive shots are attached without any VEP [44,45]....
[...]
...As stated previously, VEP is used in VPP by individuals or institutes....
[...]
...Directors or individuals use VEP for stylistic effects....
[...]
...Concatenation between two or more shots is implemented in the video editing process (VEP) to create a video during the video production process (VPP) [37]....
[...]
...During VEP, HT and ST are generated between shots to form a video....
[...]
References
More filters
Book•
15 Oct 1992TL;DR: A complete guide to the C4.5 system as implemented in C for the UNIX environment, which starts from simple core learning methods and shows how they can be elaborated and extended to deal with typical problems such as missing data and over hitting.
Abstract: From the Publisher:
Classifier systems play a major role in machine learning and knowledge-based systems, and Ross Quinlan's work on ID3 and C4.5 is widely acknowledged to have made some of the most significant contributions to their development. This book is a complete guide to the C4.5 system as implemented in C for the UNIX environment. It contains a comprehensive guide to the system's use , the source code (about 8,800 lines), and implementation notes. The source code and sample datasets are also available on a 3.5-inch floppy diskette for a Sun workstation.
C4.5 starts with large sets of cases belonging to known classes. The cases, described by any mixture of nominal and numeric properties, are scrutinized for patterns that allow the classes to be reliably discriminated. These patterns are then expressed as models, in the form of decision trees or sets of if-then rules, that can be used to classify new cases, with emphasis on making the models understandable as well as accurate. The system has been applied successfully to tasks involving tens of thousands of cases described by hundreds of properties. The book starts from simple core learning methods and shows how they can be elaborated and extended to deal with typical problems such as missing data and over hitting. Advantages and disadvantages of the C4.5 approach are discussed and illustrated with several case studies.
This book and software should be of interest to developers of classification-based intelligent systems and to students in machine learning and expert systems courses.
21,674 citations
"A fuzzy logic approach for detectio..." refers background in this paper
...5 decision tree [16] to formulate optimal rules for final decision making....
[...]
Book•
01 Dec 1994
TL;DR: This chapter discusses Fuzzy Systems Simulation, specifically the development of Membership Functions and the Extension Principle, and some of the methods used to derive these functions.
Abstract: About the Author. Preface to the Third Edition. 1 Introduction. The Case for Imprecision. A Historical Perspective. The Utility of Fuzzy Systems. Limitations of Fuzzy Systems. The Illusion: Ignoring Uncertainty and Accuracy. Uncertainty and Information. The Unknown. Fuzzy Sets and Membership. Chance Versus Fuzziness. Sets as Points in Hypercubes. Summary. References. Problems. 2 Classical Sets and Fuzzy Sets. Classical Sets. Operations on Classical Sets. Properties of Classical (Crisp) Sets. Mapping of Classical Sets to Functions. Fuzzy Sets. Fuzzy Set Operations. Properties of Fuzzy Sets. Alternative Fuzzy Set Operations. Summary. References. Problems. 3 Classical Relations and Fuzzy Relations. Cartesian Product. Crisp Relations. Cardinality of Crisp Relations. Operations on Crisp Relations. Properties of Crisp Relations. Composition. Fuzzy Relations. Cardinality of Fuzzy Relations. Operations on Fuzzy Relations. Properties of Fuzzy Relations. Fuzzy Cartesian Product and Composition. Tolerance and Equivalence Relations. Crisp Equivalence Relation. Crisp Tolerance Relation. Fuzzy Tolerance and Equivalence Relations. Value Assignments. Cosine Amplitude. Max Min Method. Other Similarity Methods. Other Forms of the Composition Operation. Summary. References. Problems. 4 Properties of Membership Functions, Fuzzification, and Defuzzification. Features of the Membership Function. Various Forms. Fuzzification. Defuzzification to Crisp Sets. -Cuts for Fuzzy Relations. Defuzzification to Scalars. Summary. References. Problems. 5 Logic and Fuzzy Systems. Part I Logic. Classical Logic. Proof. Fuzzy Logic. Approximate Reasoning. Other Forms of the Implication Operation. Part II Fuzzy Systems. Natural Language. Linguistic Hedges. Fuzzy (Rule-Based) Systems. Graphical Techniques of Inference. Summary. References. Problems. 6 Development of Membership Functions. Membership Value Assignments. Intuition. Inference. Rank Ordering. Neural Networks. Genetic Algorithms. Inductive Reasoning. Summary. References. Problems. 7 Automated Methods for Fuzzy Systems. Definitions. Batch Least Squares Algorithm. Recursive Least Squares Algorithm. Gradient Method. Clustering Method. Learning From Examples. Modified Learning From Examples. Summary. References. Problems. 8 Fuzzy Systems Simulation. Fuzzy Relational Equations. Nonlinear Simulation Using Fuzzy Systems. Fuzzy Associative Memories (FAMS). Summary. References. Problems. 9 Decision Making with Fuzzy Information. Fuzzy Synthetic Evaluation. Fuzzy Ordering. Nontransitive Ranking. Preference and Consensus. Multiobjective Decision Making. Fuzzy Bayesian Decision Method. Decision Making Under Fuzzy States and Fuzzy Actions. Summary. References. Problems. 10 Fuzzy Classification. Classification by Equivalence Relations. Crisp Relations. Fuzzy Relations. Cluster Analysis. Cluster Validity. c-Means Clustering. Hard c-Means (HCM). Fuzzy c-Means (FCM). Fuzzy c-Means Algorithm. Classification Metric. Hardening the Fuzzy c-Partition. Similarity Relations from Clustering. Summary. References. Problems. 11 Fuzzy Pattern Recognition. Feature Analysis. Partitions of the Feature Space. Single-Sample Identification. Multifeature Pattern Recognition. Image Processing. Summary. References. Problems. 12 Fuzzy Arithmetic and the Extension Principle. Extension Principle. Crisp Functions, Mapping, and Relations. Functions of Fuzzy Sets Extension Principle. Fuzzy Transform (Mapping). Practical Considerations. Fuzzy Arithmetic. Interval Analysis in Arithmetic. Approximate Methods of Extension. Vertex Method. DSW Algorithm. Restricted DSW Algorithm. Comparisons. Summary. References. Problems. 13 Fuzzy Control Systems. Control System Design Problem. Control (Decision) Surface. Assumptions in a Fuzzy Control System Design. Simple Fuzzy Logic Controllers. Examples of Fuzzy Control System Design. Aircraft Landing Control Problem. Fuzzy Engineering Process Control. Classical Feedback Control. Fuzzy Control. Fuzzy Statistical Process Control. Measurement Data Traditional SPC. Attribute Data Traditional SPC. Industrial Applications. Summary. References. Problems. 14 Miscellaneous Topics. Fuzzy Optimization. One-Dimensional Optimization. Fuzzy Cognitive Mapping. Concept Variables and Causal Relations. Fuzzy Cognitive Maps. Agent-Based Models. Summary. References. Problems. 15 Monotone Measures: Belief, Plausibility, Probability, and Possibility. Monotone Measures. Belief and Plausibility. Evidence Theory. Probability Measures. Possibility and Necessity Measures. Possibility Distributions as Fuzzy Sets. Possibility Distributions Derived from Empirical Intervals. Deriving Possibility Distributions from Overlapping Intervals. Redistributing Weight from Nonconsonant to Consonant Intervals. Comparison of Possibility Theory and Probability Theory. Summary. References. Problems. Index.
4,958 citations
"A fuzzy logic approach for detectio..." refers background or methods in this paper
...This is supported by many other applications in resolving problems under uncertain circumstances in computer vision [15]....
[...]
...[15], all the membership values are subject to AND and OR operations as defined below:...
[...]
...To select the value of TG, we adopted a minimum entropy algorithm [15] to optimise the value of TG via the training sequences....
[...]
Book•
[...]
TL;DR: This text intentionally omits theories of machine vision that do not have sufficient practical applications at the time, and basic concepts are introduced with only essential mathematical elements.
Abstract: This text is intended to provide a balanced introduction to machine vision. Basic concepts are introduced with only essential mathematical elements. The details to allow implementation and use of vision algorithm in practical application are provided, and engineering aspects of techniques are emphasized. This text intentionally omits theories of machine vision that do not have sufficient practical applications at the time.
2,365 citations
TL;DR: A conceptual solution to the shot-boundary detection problem is presented in the form of a statistical detector that is based on minimization of the average detection-error probability and the performance of the detector is demonstrated regarding two most widely used types of shot boundaries: hard cuts and dissolves.
Abstract: Partitioning a video sequence into shots is the first step toward video-content analysis and content-based video browsing and retrieval. A video shot is defined as a series of interrelated consecutive frames taken contiguously by a single camera and representing a continuous action in time and space. As such, shots are considered to be the primitives for higher level content analysis, indexing, and classification. The objective of this paper is twofold. First, we analyze the shot-boundary detection problem in detail and identify major issues that need to be considered in order to solve this problem successfully. Then, we present a conceptual solution to the shot-boundary detection problem in which all issues identified in the previous step are considered. This solution is provided in the form of a statistical detector that is based on minimization of the average detection-error probability. We model the required statistical functions using a robust metric for visual content discontinuities (based on motion compensation) and take into account all (a priori) knowledge that we found relevant to shot-boundary detection. This knowledge includes the shot-length distribution, visual discontinuity patterns at shot boundaries, and characteristic temporal changes of visual features around a boundary. Major advantages of the proposed detector are its robust and sequence-independent performance, while there is also the possibility to detect different types of shot boundaries simultaneously. We demonstrate the performance of our detector regarding two most widely used types of shot boundaries: hard cuts and dissolves.
513 citations
15 Oct 1994
TL;DR: This work uses the model driven approach to digital video segmentation to classify the edit effects used in video and film production and systematically design the feature detectors for detecting edit effects in digital video.
Abstract: The data driven, bottom up approach to video segmentation has ignored the inherent structure that exists in video. This work uses the model driven approach to digital video segmentation. Mathematical models of video based on video production techniques are formulated. These models are used to classify the edit effects used in video and film production. The classes and models are used to systematically design the feature detectors for detecting edit effects in digital video. Digital video segmentation is formulated as a feature based classification problem. Experimental results from segmenting cable television programming with cuts, fades, dissolves and page translate edits are presented.
279 citations
"A fuzzy logic approach for detectio..." refers background in this paper
...(8) decides to go to the gradual shot-cut-mode, variance changes can be monitored to detect the boundary of gradual shot cuts [2,17]....
[...]