Showing papers on "Content-based image retrieval published in 2017"

Recent Advance in Content-based Image Retrieval: A Literature Survey

Abstract: The explosive increase and ubiquitous accessibility of visual data on the Web have led to the prosperity of research activity in image search or retrieval. With the ignorance of visual content as a ranking clue, methods with text search techniques for visual retrieval may suffer inconsistency between the text words and visual content. Content-based image retrieval (CBIR), which makes use of the representation of visual content to identify relevant images, has attracted sustained attention in recent two decades. Such a problem is challenging due to the intention gap and the semantic gap problems. Numerous techniques have been developed for content-based image retrieval in the last decade. The purpose of this paper is to categorize and evaluate those algorithms proposed during the period of 2003 to 2016. We conclude with several promising directions for future research.

Unsupervised Visual Hashing with Semantic Assistant for Content-Based Image Retrieval

Abstract: As an emerging technology to support scalable content-based image retrieval (CBIR), hashing has recently received great attention and became a very active research domain. In this study, we propose a novel unsupervised visual hashing approach called semantic-assisted visual hashing (SAVH). Distinguished from semi-supervised and supervised visual hashing, its core idea is to effectively extract the rich semantics latently embedded in auxiliary texts of images to boost the effectiveness of visual hashing without any explicit semantic labels. To achieve the target, a unified unsupervised framework is developed to learn hash codes by simultaneously preserving visual similarities of images, integrating the semantic assistance from auxiliary texts on modeling high-order relationships of inter-images, and characterizing the correlations between images and shared topics. Our performance study on three publicly available image collections: Wiki , MIR Flickr , and NUS-WIDE indicates that SAVH can achieve superior performance over several state-of-the-art techniques.

Fusion of Deep Learning and Compressed Domain Features for Content-Based Image Retrieval

Abstract: This paper presents an effective image retrieval method by combining high-level features from convolutional neural network (CNN) model and low-level features from dot-diffused block truncation coding (DDBTC). The low-level features, e.g., texture and color, are constructed by vector quantization -indexed histogram from DDBTC bitmap, maximum, and minimum quantizers. Conversely, high-level features from CNN can effectively capture human perception. With the fusion of the DDBTC and CNN features, the extended deep learning two-layer codebook features is generated using the proposed two-layer codebook, dimension reduction, and similarity reweighting to improve the overall retrieval rate. Two metrics, average precision rate and average recall rate (ARR), are employed to examine various data sets. As documented in the experimental results, the proposed schemes can achieve superior performance compared with the state-of-the-art methods with either low-or high-level features in terms of the retrieval rate. Thus, it can be a strong candidate for various image retrieval related applications.

New content-based image retrieval system based on optimised integration of DCD, wavelet and curvelet features

Abstract: A new content-based image retrieval (CBIR) scheme is proposed based on the optimised combination of the colour and texture features to enhance the image retrieval precision. This work focuses on a uniform partitioning scheme which is applied in the Hue, Saturation and Value (HSV) colour space to extract dominant colour descriptor (DCD) features. In the proposed CBIR scheme, the DCD features are initially extracted as the colour features, and then an appropriate similarity measure is applied. Also, several wavelet and curvelet features are defined as texture features to overcome the noise and the problem of image translation. Finally, the colour and texture features are optimally combined by using the particle swarm optimisation algorithm. The findings show that not only the proposed colour, wavelet and curvelet features outperform the existing ones but also their optimum combination has a better accuracy in comparison with several contemporary CBIR systems. The performance analysis shows that the proposed method improves the average precision metric from 67.85 to 71.05% for DCD, 58.90 to 65.43% for wavelet and 53.18 to 56.00% for curvelet using Corel dataset. In addition, the optimum combination presents the average precision of %76.50 which is significantly higher than the other state-of-the-art methods.

An efficient similarity measure for content based image retrieval using memetic algorithm

Abstract: Content based image retrieval (CBIR) systems work by retrieving images which are related to the query image (QI) from huge databases. The available CBIR systems extract limited feature sets which confine the retrieval efficacy. In this work, extensive robust and important features were extracted from the images database and then stored in the feature repository. This feature set is composed of color signature with the shape and color texture features. Where, features are extracted from the given QI in the similar fashion. Consequently, a novel similarity evaluation using a meta-heuristic algorithm called a memetic algorithm (genetic algorithm with great deluge) is achieved between the features of the QI and the features of the database images. Our proposed CBIR system is assessed by inquiring number of images (from the test dataset) and the efficiency of the system is evaluated by calculating precision-recall value for the results. The results were superior to other state-of-the-art CBIR systems in regard to precision.

GEO matching regions: multiple regions of interests using content based image retrieval based on relative locations

Abstract: Information retrieval systems are getting more attention in the era of multimedia technologies such as an image, video, audio and text files. The large numbers of images are challenges in computer systems field to store, manage data effectively and efficiently. The shape retrieval feature of different objects in the image also remains a difficult problem due to distinct angle view of different objects in a scene only; few studies have reported solution to the problem of finding relative locations of ROIs. In this paper, we proposed three methods such as1. Geolocation-based image retrieval (GLBIR), 2.Unsupervised feature technique Principal component analysis (PCA) and 3.multiple region-based image retrieval. The first proposed (GLBIR) method identifies geo location an image using visual attention based mechanism and its color layout descriptors. These features are extracted from geo-location of query image from Flickr database. Our proposed model does not fully semantic understanding of image content, uses visual metrics for example; the proximity ,color contrast, size and nearness to image's boundaries to locate viewer's attention. We analyzed results and compared with state of art CBIR Systems and GLBIR Technique. Our second method to refine images exploiting and fusing by unsupervised feature technique using principal component analysis (PCA). The visually similar images clustering together with analyses image retrieval process and remove outliers initially retrieved image set by PCA. To evaluation our proposed approach, we used thousands of images downloaded from Flickr and CIFAR-10 databases using Flickr public API. Finally, we determinately proposed a system for image retrieval based on region. It provides a user interface for availing to designate the watershed ROI within an input image. During the retrieval of images, regions' feature vectors having codes of region homogeneous to a region of input image are utilized for comparison. Standard datasets are used for evaluation of proposed approach. The experiment demonstrates and effectiveness of the proposed method to achieve higher annotation performance increases accuracy and reduces image retrieval time. We evaluated our proposed approach on images dataset from Flickr and CIFAR-10.

Analysis of content based image retrieval for plant leaf diseases using color, shape and texture features

Abstract: This research paper is an attempt to present Content Based Image Retrieval (CBIR) system developed for retrieving diseased leaves of soybean. It uses color, shape and texture features of leaf. Color features are extracted using HSV color histogram. Scale Invariant Feature Transform (SIFT) provides shape features in the form of matching key points. Local Binary Pattern (LBP) and Gabor filter are widely used texture features. Novel texture feature named Local Gray Gabor Pattern (LGGP) is proposed by combining LBP and Gabor. Performance of all these features with respect to retrieval precision is tested for three soybean leaf diseases. Further color, shape and texture features are combined to increase performance. It is found that when LGGP is combined with color histogram and SIFT retrieval precision is improved. Retrieval efficiency of about 96%, 68% and 76% is achieved for soybean leaves affected by mosaic virus, septoria brown spot and pod mottle disease respectively. Average retrieval efficiency of 80% (for the top 5 retrieval) and 72% (for the top 10 retrieval) is obtained by combined features. This retrieval precision is database dependent and varies with size of the database and quality of images.

Learning Deep Representations of Medical Images using Siamese CNNs with Application to Content-Based Image Retrieval

Abstract: Deep neural networks have been investigated in learning latent representations of medical images, yet most of the studies limit their approach in a single supervised convolutional neural network (CNN), which usually rely heavily on a large scale annotated dataset for training. To learn image representations with less supervision involved, we propose a deep Siamese CNN (SCNN) architecture that can be trained with only binary image pair information. We evaluated the learned image representations on a task of content-based medical image retrieval using a publicly available multiclass diabetic retinopathy fundus image dataset. The experimental results show that our proposed deep SCNN is comparable to the state-of-the-art single supervised CNN, and requires much less supervision for training.

Selective Deep Convolutional Features for Image Retrieval

Abstract: Convolutional Neural Network (CNN) is a very powerful approach to extract discriminative local descriptors for effective image search. Recent work adopts fine-tuned strategies to further improve the discriminative power of the descriptors. Taking a different approach, in this paper, we propose a novel framework to achieve competitive retrieval performance. Firstly, we propose various masking schemes, namely SIFT-mask, SUM-mask, and MAX-mask, to select a representative subset of local convolutional features and remove a large number of redundant features. We demonstrate that this can effectively address the burstiness issue and improve retrieval accuracy. Secondly, we propose to employ recent embedding and aggregating methods to further enhance feature discriminability. Extensive experiments demonstrate that our proposed framework achieves state-of-the-art retrieval accuracy.

Content-Based Image Retrieval System for Pulmonary Nodules: Assisting Radiologists in Self-Learning and Diagnosis of Lung Cancer

Abstract: Visual information of similar nodules could assist the budding radiologists in self-learning. This paper presents a content-based image retrieval (CBIR) system for pulmonary nodules, observed in lung CT images. The reported CBIR systems of pulmonary nodules cannot be put into practice as radiologists need to draw the boundary of nodules during query formation and feature database creation. In the proposed retrieval system, the pulmonary nodules are segmented using a semi-automated technique, which requires a seed point on the nodule from the end-user. The involvement of radiologists in feature database creation is also reduced, as only a seed point is expected from radiologists instead of manual delineation of the boundary of the nodules. The performance of the retrieval system depends on the accuracy of the segmentation technique. Several 3D features are explored to improve the performance of the proposed retrieval system. A set of relevant shape and texture features are considered for efficient representation of the nodules in the feature space. The proposed CBIR system is evaluated for three configurations such as configuration-1 (composite rank of malignancy "1","2" as benign and "4","5" as malignant), configuration-2 (composite rank of malignancy "1","2", "3" as benign and "4","5" as malignant), and configuration-3 (composite rank of malignancy "1","2" as benign and "3","4","5" as malignant). Considering top 5 retrieved nodules and Euclidean distance metric, the precision achieved by the proposed method for configuration-1, configuration-2, and configuration-3 are 82.14, 75.91, and 74.27 %, respectively. The performance of the proposed CBIR system is close to the most recent technique, which is dependent on radiologists for manual segmentation of nodules. A computer-aided diagnosis (CAD) system is also developed based on CBIR paradigm. Performance of the proposed CBIR-based CAD system is close to performance of the CAD system using support vector machine.

Facial emotion recognition on a dataset using convolutional neural network

Abstract: Nowadays, deep learning is a technique that takes place in many computer vision related applications and studies. While it is put in the practice mostly on content based image retrieval, there is still room for improvement by employing it in diverse computer vision applications. In this study, we aimed to build a Convolutional Neural Network (CNN) based Facial Expression Recognition System (FER), in order to automatically classify expressions presented in Facial Expression Recognition (FER2013) database. Our presented CNN achieved % 57.1 success rate on FER2013 database.

A Self-Balanced Min-Cut Algorithm for Image Clustering

Abstract: Many spectral clustering algorithms have been proposed and successfully applied to image data analysis such as content based image retrieval, image annotation, and image indexing. Conventional spectral clustering algorithms usually involve a two-stage process: eigendecomposition of similarity matrix and clustering assignments from eigenvectors by k-means or spectral rotation. However, the final clustering assignments obtained by the two-stage process may deviate from the assignments by directly optimize the original objective function. Moreover, most of these methods usually have very high computational complexities. In this paper, we propose a new min-cut algorithm for image clustering, which scales linearly to the data size. In the new method, a self-balanced min-cut model is proposed in which the Exclusive Lasso is implicitly introduced as a balance regularizer in order to produce balanced partition. We propose an iterative algorithm to solve the new model, which has a time complexity of O(n) where n is the number of samples. Theoretical analysis reveals that the new method can simultaneously minimize the graph cut and balance the partition across all clusters. A series of experiments were conducted on both synthetic and benchmark data sets and the experimental results show the superior performance of the new method.

Image retrieval based on non-uniform bins of color histogram and dual tree complex wavelet transform

Abstract: Traditional Content-Based Image Retrieval (CBIR) systems were developed for retrieving similar kinds of images from a whole image database based on the given query image. In this paper, the authors have proposed a hierarchical approach for designing a CBIR scheme based on the color and texture features of an image. Initially, a color based approach is adopted and the intermediate results produced by using these color features is appropriate to discard a significant number of non-relevant images from the database. The intermediate database will be the input for the second stage. At this stage, a texture based approach is adopted for retrieving images from the intermediate database. The color features are extracted by computing the statistical parameters of non-uniform quantized histograms of HSV color space while a rotation invariant multi-resolution texture based approach is accomplished on value(V) component of HSV color space for extracting texture features. These texture features are extracted based on the principal texture direction and by taking the energies from various sub-bands of a dual tree complex wavelet transform (DT-CWT). Furthermore, the proposed scheme is suitable to handle mirror images during the retrieval process. The presented scheme has reduced the processing cost due to the consideration of a hierarchical approach. The proposed scheme is tested on the two well-known Corel-1K and GHIM-10K image databases respectively and satisfactory results were achieved in terms of precision, recall and F-score. The proposed scheme is compared with some other existing state of art CBIR schemes and the experimental results validate the improvement over other schemes in most of the instances.

Content-based image retrieval with color and texture features in neutrosophic domain

Abstract: In this paper, a new content-based image retrieval (CBIR) scheme is proposed in neutrosophic (NS) domain. For this task, RGB images are first transformed to three subsets in NS domain and then segmented. For each segment of an image, color features including dominant color discribtor (DCD), histogram and statistic components are extracted. Wavelet features are also extracted as texture features from the whole image. All extracted features from either segmented image or the whole image are combined to create a feature vector. Feature vectors are presented for ant colony optimization (ACO) feature selection which selects the most relevant features. Selected features are used for final retrieval process. Proposed CBIR scheme is evaluated on Corel image dataset. Experimental results show that the proposed method outperforms our prior method (with the same feature vector and feature selection method) by 2% and 1% with respect to precision and recall, respectively. Also, the proposed method achieves the improvement of 13% and 2% in precision and recall, respectively, in comparison with prior methods.

CBIR for classification of cow types using GLCM and color features extraction

Abstract: Cow is one of the animals that have many benefits for humans. There are various types of cows based on benefits such as dairy cows, beef cattle, worker cattle, and others. Cattle breeding should be tailored to the needs of the public. Less knowledge about different types of cattle can reduce the benefits of farmed cattle. Content Based Image Retrieval (CBIR) can be applied to help the problem of distinguishing or knowing the type of cow. The first step of the method proposed in this research is preprocessing by changing the background color, resizing and conversion of color space. Color feature extraction calculates the average and standard deviation of the color intensity of each color component. Next extract the texture feature using Gray Level Cooccurrence Matrix (GLCM) to look for contrast, energy, correlation, homogeneity and entropy at each angle 0°, 45°, 90° and 135° with a mean of 1 averaged. Six color features and five texture features are used as attributes to perform calculations with Euclidean Distance, so it can be known the similarity between images. Cattle types used include Limousin, Simental, Brangus, Peranakan Ongole (PO), and Frisien Holstein (FH). With 100 training images and 20 test images. To measure the accuracy of the proposed CBIR is used Confusion Matrix. Based on the measurement results obtained accuracy of 95% while the precision and recall obtained 100%.

Saliency-weighted graphs for efficient visual content description and their applications in real-time image retrieval systems

Abstract: The exponential growth in the volume of digital image databases is making it increasingly difficult to retrieve relevant information from them. Efficient retrieval systems require distinctive features extracted from visually rich contents, represented semantically in a human perception-oriented manner. This paper presents an efficient framework to model image contents as an undirected attributed relational graph, exploiting color, texture, layout, and saliency information. The proposed method encodes salient features into this rich representative model without requiring any segmentation or clustering procedures, reducing the computational complexity. In addition, an efficient graph-matching procedure implemented on specialized hardware makes it more suitable for real-time retrieval applications. The proposed framework has been tested on three publicly available datasets, and the results prove its superiority in terms of both effectiveness and efficiency in comparison with other state-of-the-art schemes.

Smart surveillance based on video summarization

Abstract: In recent years, video surveillance technology has become ubiquitous in every sphere of our life. But automated video surveillance generates huge quantities of data, which ultimately does rely upon manual inspection at some stage. The present work aims to address this ever increasing gap between the volumes of actual data generated and the volume that can be reasonably inspected manually. It is laborious and time consuming to scrutinize the salient events from the large video databases. We introduce smart surveillance by using video summarization for various applications. Techniques like video summarization epitomizes the vast content of a video in a succinct manner. In this paper, we give an overview how to use an optimal summarization framework for surveillance videos. In addition to reduce the search time we propose to convert content based video retrieval problem into a content based image retrieval problem. We have performed several experiments on different data sets to validate our proposed approach for smart surveillance.

Content-Based Image Retrieval Using Convolutional Neural Networks

Abstract: Content-based image retrieval (CBIR) is a widely used technique for retrieval images from huge and unlabeled image databases. However, users are not satisfied with the traditional information retrieval techniques. Moreover, the emergence of web development and transmission networks and also the amount of images which are available to users continue to grow. Therefore, a permanent and considerable digital image production in many areas takes place. Hence, the rapid access to these huge collections of images and retrieve similar image of a given image (Query) from this large collection of images presents major challenges and requires efficient techniques. The performance of a content-based image retrieval system crucially depends on the feature representation and similarity measurement. For this reason, we present, on this paper, a simple but effective deep learning framework based on Convolutional Neural Networks (CNN) and Support Vector Machine (SVM) for fast image retrieval composed of feature extraction and classification. From several extensive of empirical studies for a variety of CBIR tasks using image database, we obtain some encouraging results which reveals several important insights for improving the CBIR performance.

Retrieval From and Understanding of Large-Scale Multi-modal Medical Datasets: A Review

Abstract: Content-based multimedia retrieval (CBMR) has been an active research domain since the mid 1990s. In medicine visual retrieval started later and has mostly remained a research instrument and less a clinical tool. The limited size of data sets due to privacy constraints is often mentioned as reason for these limitations. Nevertheless, much work has been done in CBMR, including the availability of increasingly large data sets and scientific challenges. Annotated data sets and clinical data for images have now become available and can be combined for multi-modal retrieval. Much has been learned on user behavior and application scenarios. This text is motivated by the advances in medical image analysis and the availability of public large data sets that often include clinical data. It is a systematic review of recent work (concentrating on the period 2011–2017) on multi-modal CBMR and image understanding in the medical domain, where image understanding includes techniques such as detection, localization, and classification for leveraging visual content. With the objective of summarizing the current state of research for multimedia researchers outside the medical field, the text provides ways to get data sets and identifies current limitations and promising research directions. The text highlights advances in the past six years and a trend to use larger scale training data and deep learning approaches that can replace/complement hand-crafted features. Using images alone will likely only work in limited domains but combining multiple sources of data for multi-modal retrieval has the biggest chances of success, particularly for clinical impact.

Histogram refinement for texture descriptor based image retrieval

Abstract: Texture descriptors such as local binary patterns (LBP) have been successfully employed for feature extraction in image retrieval algorithms because of their high discriminating ability and computational efficiency. In this paper, we propose histogram feature refinement methods for enhancing performance of texture descriptor based content-based image retrieval (CBIR) systems. In the proposed approach for histogram refinement, each pixel in the query and database images is classified into one of the two categories based on the analysis of pixel values in its neighborhood. Local patterns corresponding to two sets of pixels are used to generate two histogram features for each image, effectively resulting in splitting of the original global histogram of texture descriptors into two based on the category of each pixel. Resulting histograms are then concatenated to form a single histogram feature. This study also explores three hybrid frameworks for histogram refinement in CBIR systems. Comparison of histogram features corresponding to query and database images are performed using the relative l1 distance metric. Performance evaluation on three publicly available benchmark image databases namely, GHIM 10000, COREL 1000 database, and Brodatz texture database shows that performances of existing texture descriptor based approaches improve considerably when the proposed histogram feature refinement is incorporated. Specifically, the average precision rate is improved by 6.02%, 5.69%, 4.79%, and 4.21% for LBP, local derivative pattern (LDP), local ternary pattern (LTP), and local tetra pattern (LTrP) descriptors, respectively on GHIM 10000 database. The proposed histogram refinement approaches also provide performance improvement for other texture descriptors considered in this study. A general framework for histogram refinement of texture descriptors.Improves retrieval performance of texture descriptor based CBIR systems.Performance improvement in CBIR validated using 9 texture descriptors.Marginal increase in average image retrieval time with 1.2s in the worst case.