Temporal color correlograms for video retrieval
11 Aug 2002-Vol. 1, pp 10267
TL;DR: The efficiency of the temporal color correlogram and HSV color correlograms are evaluated against other retrieval systems participating the TREC video track evaluation and against color histograms used commonly in content-based retrieval.
Abstract: This paper presents a novel method to retrieve segmented video shots based on their color content. The temporal color correlogram captures the spatiotemporal relationship of colors in a video shot using cooccurrence statistics. The temporal color correlogram extends the HSV color correlogram that has been found to be very effective in content-based image retrieval. Temporal color correlograms compute the autocorrelation of quantized HSV color values from a set of frame samples taken from a video shot. In this paper, the efficiency of the temporal color correlogram and HSV color correlograms are evaluated against other retrieval systems participating the TREC video track evaluation and against color histograms used commonly in content-based retrieval. We used queries and relevance judgments on the I I hours of segmented MPEG-1 video provided to track participants. Tests are executed using our content-based multimedia retrieval system that was specifically developed for multimedia information retrieval applications.
Citations
More filters
01 Jan 2009
TL;DR: In this paper color extraction and comparison were performed using the three color histograms, conventional color histogram (CCH), invariant colorhistogram (ICH) and fuzzy linking color Histogram (FCH) to address the problem of spatial relationship fuzzy linkingColor histograms.
Abstract: Advances in data storage and image acquisition technologies have enabled the creation of large image datasets. In this scenario, it is necessary to develop appropriate information systems to efficiently manage these collections. The most common approaches use Content-Based Image Retrieval (CBIR). The goal of CBIR systems is to support image retrieval based on content e.g., shape, color, texture. In this paper color extraction and comparison were performed using the three color histograms, conventional color histogram (CCH), invariant color histogram (ICH) and fuzzy color histogram (FCH) .The conventional color histogram (CCH) of an image indicates the frequency of occurrence of every color in an image. The appealing aspect of the CCH is its simplicity and ease of computation. There are however, several difficulties associated with the CCH. The first of these is the high dimensionality of the CCH, even after drastic quantization of the color space. Another downside of the CCH is that it does not take into consideration color similarity across different bins and cannot handle rotation and translation. To address the problem of rotation and translation an invariant color histograms(ICH) based on the color gradients is used and to address the problem of spatial relationship fuzzy linking color histogram (FCH) is used.
71 citations
05 Jun 2012
TL;DR: Despite the very simple way to generate the visual dictionary, which has taken photos at random, the results show that the approach presents high accuracy relative to the state-of-the art solutions.
Abstract: This paper presents a novel approach for video representation, called bag-of-scenes. The proposed method is based on dictionaries of scenes, which provide a high-level representation for videos. Scenes are elements with much more semantic information than local features, specially for geotagging videos using visual content. Thus, each component of the representation model has self-contained semantics and, hence, it can be directly related to a specific place of interest. Experiments were conducted in the context of the MediaEval 2011 Placing Task. The reported results show our strategy compared to those from other participants that used only visual content to accomplish this task. Despite our very simple way to generate the visual dictionary, which has taken photos at random, the results show that our approach presents high accuracy relative to the state-of-the art solutions.
42 citations
Cites methods from "Temporal color correlograms for vid..."
...In this method, videos are described by the overall distribution of low-level features, such as color, texture, edge, or other visual properties [24,32,34]....
[...]
TL;DR: The proposed method for video sequence matching based on the invariance of color correlation is demonstrated to be robust against most typical video content-preserving operations, including geometric distortion, blurring, noise contamination, contrast enhancement, and strong re-encoding.
Abstract: Video sequence matching aims to locate a query video clip in a video database. It plays an important role in reducing storage redundancy and detecting video copies for copyright protection. In this paper, we propose an effective method for video sequence matching based on the invariance of color correlation. The proposed method first splits each key-frame into nonoverlapping blocks. For each block, we sort the red, green, and blue color components according to their average intensities, and use the percentage of the color correlation to generate a frame feature with a small size. Finally, the resulting video feature is made up of the consecutive frame features, which is demonstrated to be robust against most typical video content-preserving operations, including geometric distortion, blurring, noise contamination, contrast enhancement, and strong re-encoding. The experimental results show that the proposed method outperforms the existing methods in the literature, as well as the method based on the traditional color histogram. Furthermore, the time and space complexity of our algorithm are both satisfactory, which are very important for many real-time applications.
39 citations
Cites methods from "Temporal color correlograms for vid..."
...Finally, we use the consecutive frame features for the purpose of video sequence matching....
[...]
27 Jun 2004
TL;DR: Results indicate improvements in browsing efficiency when automatic speech recognition transcripts are incorporated into browsing by visual similarity, and performed well in overall comparison with interactive video retrieval systems in TRECVID 2003 evaluation.
Abstract: The paper describes cluster-temporal browsing of news video databases. Cluster-temporal browsing combines content similarities and temporal adjacency into a single representation. Visual, conceptual and lexical features are used to organize and view similar shot content. Interactive experiments with eight test users have been carried out using a database of roughly 60 hours of news video. Results indicate improvements in browsing efficiency when automatic speech recognition transcripts are incorporated into browsing by visual similarity. The cluster-temporal browsing application received positive comments from the test users and performed well in overall comparison with interactive video retrieval systems in TRECVID 2003 evaluation.
37 citations
Cites background from "Temporal color correlograms for vid..."
...In [8][9] two low-level shot features have been introduced that escalate traditional video features by shifting from the static key frame context to the temporal properties of video color and structure....
[...]
15 Oct 2004
TL;DR: Weighted fusion of text, concept and visual features improved the performance over text search baseline, and expanded query term list of text queries gave also notable increase in performance over the baseline text search.
Abstract: This paper describes revised content-based search experiments in the context of TRECVID 2003 benchmark. Experiments focus on measuring content-based video retrieval performance with following search cues: visual features, semantic concepts and text. The fusion of features uses weights and similarity ranks. Visual similarity is computed using Temporal Gradient Correlogram and Temporal Color Correlogram features that are extracted from the dynamic content of a video shot. Automatic speech recognition transcripts and concept detectors enable higher-level semantic searching. 60 hours of news videos from TRECVID 2003 search task were used in the experiments. System performance was evaluated with 25 pre-defined search topics using average precision. In visual search, multiple examples improved the results over single example search. Weighted fusion of text, concept and visual features improved the performance over text search baseline. Expanded query term list of text queries gave also notable increase in performance over the baseline text search
25 citations
Cites background from "Temporal color correlograms for vid..."
...Its efficiency against traditional color descriptors has been reported in [12]....
[...]
References
More filters
Book•
01 Jan 1995
TL;DR: This chapter discusses the development of Hardware and Software for Computer Graphics, and the design methodology of User-Computer Dialogues, which led to the creation of the Simple Raster Graphics Package.
Abstract: 1 Introduction Image Processing as Picture Analysis The Advantages of Interactive Graphics Representative Uses of Computer Graphics Classification of Applications Development of Hardware and Software for Computer Graphics Conceptual Framework for Interactive Graphics 2 Programming in the Simple Raster Graphics Package (SRGP)/ Drawing with SRGP/ Basic Interaction Handling/ Raster Graphics Features/ Limitations of SRGP/ 3 Basic Raster Graphics Algorithms for Drawing 2d Primitives Overview Scan Converting Lines Scan Converting Circles Scan Convertiing Ellipses Filling Rectangles Fillign Polygons Filling Ellipse Arcs Pattern Filling Thick Primiives Line Style and Pen Style Clipping in a Raster World Clipping Lines Clipping Circles and Ellipses Clipping Polygons Generating Characters SRGP_copyPixel Antialiasing 4 Graphics Hardware Hardcopy Technologies Display Technologies Raster-Scan Display Systems The Video Controller Random-Scan Display Processor Input Devices for Operator Interaction Image Scanners 5 Geometrical Transformations 2D Transformations Homogeneous Coordinates and Matrix Representation of 2D Transformations Composition of 2D Transformations The Window-to-Viewport Transformation Efficiency Matrix Representation of 3D Transformations Composition of 3D Transformations Transformations as a Change in Coordinate System 6 Viewing in 3D Projections Specifying an Arbitrary 3D View Examples of 3D Viewing The Mathematics of Planar Geometric Projections Implementing Planar Geometric Projections Coordinate Systems 7 Object Hierarchy and Simple PHIGS (SPHIGS) Geometric Modeling Characteristics of Retained-Mode Graphics Packages Defining and Displaying Structures Modeling Transformations Hierarchical Structure Networks Matrix Composition in Display Traversal Appearance-Attribute Handling in Hierarchy Screen Updating and Rendering Modes Structure Network Editing for Dynamic Effects Interaction Additional Output Features Implementation Issues Optimizing Display of Hierarchical Models Limitations of Hierarchical Modeling in PHIGS Alternative Forms of Hierarchical Modeling 8 Input Devices, Interaction Techniques, and Interaction Tasks Interaction Hardware Basic Interaction Tasks Composite Interaction Tasks 9 Dialogue Design The Form and Content of User-Computer Dialogues User-Interfaces Styles Important Design Considerations Modes and Syntax Visual Design The Design Methodology 10 User Interface Software Basic Interaction-Handling Models Windows-Management Systems Output Handling in Window Systems Input Handling in Window Systems Interaction-Technique Toolkits User-Interface Management Systems 11 Representing Curves and Surfaces Polygon Meshes Parametric Cubic Curves Parametric Bicubic Surfaces Quadric Surfaces 12 Solid Modeling Representing Solids Regularized Boolean Set Operations Primitive Instancing Sweep Representations Boundary Representations Spatial-Partitioning Representations Constructive Solid Geometry Comparison of Representations User Interfaces for Solid Modeling 13 Achromatic and Colored Light Achromatic Light Chromatic Color Color Models for Raster Graphics Reproducing Color Using Color in Computer Graphics 14 The Quest for Visual Realism Why Realism? Fundamental Difficulties Rendering Techniques for Line Drawings Rendering Techniques for Shaded Images Improved Object Models Dynamics Stereopsis Improved Displays Interacting with Our Other Senses Aliasing and Antialiasing 15 Visible-Surface Determination Functions of Two Variables Techniques for Efficient Visible-Surface Determination Algorithms for Visible-Line Determination The z-Buffer Algorithm List-Priority Algorithms Scan-Line Algorithms Area-Subdivision Algorithms Algorithms for Octrees Algorithms for Curved Surfaces Visible-Surface Ray Tracing 16 Illumination And Shading Illumination Modeling Shading Models for Polygons Surface Detail Shadows Transparency Interobject Reflections Physically Based Illumination Models Extended Light Sources Spectral Sampling Improving the Camera Model Global Illumination Algorithms Recursive Ray Tracing Radiosity Methods The Rendering Pipeline 17 Image Manipulation and Storage What Is an Image? Filtering Image Processing Geometric Transformations of Images Multipass Transformations Image Compositing Mechanisms for Image Storage Special Effects with Images Summary 18 Advanced Raster Graphic Architecture Simple Raster-Display System Display-Processor Systems Standard Graphics Pipeline Introduction to Multiprocessing Pipeline Front-End Architecture Parallel Front-End Architectures Multiprocessor Rasterization Architectures Image-Parallel Rasterization Object-Parallel Rasterization Hybrid-Parallel Rasterization Enhanced Display Capabilities 19 Advanced Geometric and Raster Algorithms Clipping Scan-Converting Primitives Antialiasing The Special Problems of Text Filling Algorithms Making copyPixel Fast The Shape Data Structure and Shape Algebra Managing Windows with bitBlt Page Description Languages 20 Advanced Modeling Techniques Extensions of Previous Techniques Procedural Models Fractal Models Grammar-Based Models Particle Systems Volume Rendering Physically Based Modeling Special Models for Natural and Synthetic Objects Automating Object Placement 21 Animation Conventional and Computer-Assisted Animation Animation Languages Methods of Controlling Animation Basic Rules of Animation Problems Peculiar to Animation Appendix: Mathematics for Computer Graphics Vector Spaces and Affine Spaces Some Standard Constructions in Vector Spaces Dot Products and Distances Matrices Linear and Affine Transformations Eigenvalues and Eigenvectors Newton-Raphson Iteration for Root Finding Bibliography Index 0201848406T04062001
5,692 citations
IBM1
TL;DR: The Query by Image Content (QBIC) system as discussed by the authors allows queries on large image and video databases based on example images, user-constructed sketches and drawings, selected color and texture patterns, camera and object motion, and other graphical information.
Abstract: Research on ways to extend and improve query methods for image databases is widespread. We have developed the QBIC (Query by Image Content) system to explore content-based retrieval methods. QBIC allows queries on large image and video databases based on example images, user-constructed sketches and drawings, selected color and texture patterns, camera and object motion, and other graphical information. Two key properties of QBIC are (1) its use of image and video content-computable properties of color, texture, shape and motion of images, videos and their objects-in the queries, and (2) its graphical query language, in which queries are posed by drawing, selecting and other graphical means. This article describes the QBIC system and demonstrates its query capabilities. QBIC technology is part of several IBM products. >
3,957 citations
3,375 citations
01 Feb 1997
TL;DR: The VisualSEEk system is novel in that the user forms the queries by diagramming spatial arrangements of color regions by utilizing color information, region sizes and absolute and relative spatial locations.
Abstract: We describe a highly functional prototype system for searching by visual features in an image database. The VisualSEEk system is novel in that the user forms the queries by diagramming spatial arrangements of color regions. The system nds the images that contain the most similar arrangements of similar regions. Prior to the queries, the system automatically extracts and indexes salient color regions from the images. By utilizing e cient indexing techniques for color information, region sizes and absolute and relative spatial locations, a wide variety of complex joint color/spatial queries may be computed.
2,084 citations
17 Jun 1997
TL;DR: Experimental evidence suggests that this new image feature called the color correlogram outperforms not only the traditional color histogram method but also the recently proposed histogram refinement methods for image indexing/retrieval.
Abstract: We define a new image feature called the color correlogram and use it for image indexing and comparison. This feature distills the spatial correlation of colors, and is both effective and inexpensive for content-based image retrieval. The correlogram robustly tolerates large changes in appearance and shape caused by changes in viewing positions, camera zooms, etc. Experimental evidence suggests that this new feature outperforms not only the traditional color histogram method but also the recently proposed histogram refinement methods for image indexing/retrieval.
1,956 citations