Haitham Samih

Bio: Haitham Samih is an academic researcher from Egyptian e-Learning University. The author has contributed to research in topics: Image retrieval & Graph (abstract data type). The author has an hindex of 2, co-authored 3 publications receiving 9 citations.

Enhancing image retrieval for complex queries using external knowledge sources

Abstract: Annotation-based image retrieval associates textual descriptions to images based on human perception. A user query, composed of keywords of choice and for retrieval, are usually matched lexically with the textual descriptions associated for stored images to extract the best matches. This paradigm will not produce appropriate desired results for complex queries if a semantic approach is not considered. This paper proposes an image retrieval framework which integrates external knowledge sources for obtaining a higher-level inference that can both handle complex queries and increase the number of relevant retrievals. The framework includes a parser where a semantic representation graph is initially generated from both image captions and query. The semantic representation of image captions is stored in the form of Resource Description Framework (RDF) triples, while the user query is translated into a SPARQL language query. For better query understanding, the external knowledge sources (ConceptNet, WordNet), are next fused together with the parser’s output in a significant process named query expansion to infer combined and expanded knowledge about the terms used in the query. Also, the expansion process generates a set of expansion rules to semantically expand the user query to adapt the inferred knowledge. The expanded query is matched against the stored RDF triplets to indicate the best matched image retrievals. Retrievals are eventually ranked using a relation similarity metric to obtain a ranked list of relevant images. Experimental studies carried on two Flickr datasets show that the proposed framework outperforms related work with 40% increase in the number of relevant retrievals at almost full accuracy. The framework achieves additionally an average increase for the accuracy at given k in the range of 50–72% for up to the tenth retrieval.

Improving Natural Language Queries Search and Retrieval through Semantic Image Annotation Understanding

Abstract: Retrieving images using detailed natural language queries remains a difficult challenge. Traditional annotation-based image retrieval systems using word matching techniques cannot efficiently support such query types. Significant improvements for this problem can be achieved with a semantic understanding for those query sentences and image annotations. This paper presents a two-stage semantic understanding approach for natural language query sentences. At the first stage, the Stanford parser and a designed rule-based relation extraction tool are used in triple extraction process to efficiently extract the objects attributes, instances and natural language annotations relationships involving these objects. The second stage integrates the extracted relations with external commonsense knowledge source in a mapping process to provide high-level semantic meanings to the extracted triples. Experiments are conducted for evaluating the benefit of the proposed semantic understanding against a testing set of natural language sentences from the Flickr8k dataset. The results show that the proposed approach succeeds to extract relational triples with average accuracy value of 97% for the different types of annotations relationships: attributes and instance relations, multiword dependence relations, and semantic relations.

Semantic Graph Representation and Evaluation for Generated Image Annotations

Abstract: The rapid advancement in generated image annotations, also known as captions, puts a crucial need for efficient and automated methods for those annotations’ evaluation. State-of-art in the automatic evaluation metrics has proven to be inefficient for evaluating the quality of the generated annotations because they rely on certain aspects of word matching, such as the n-gram overlap. This paper presents a semantic-based graph representation and evaluation for generated image annotations. The semantic graph explicitly encodes objects, attributes, and natural language annotations relationships, while consulting ConceptNet as an external knowledge source to provide a rich semantic generated graph. For annotations evaluation, the ConceptNet is extended for use in a proposed semantic evaluation metric, whose input is the semantic graphs. Experimental results show that, over the Flickr8k dataset, the proposed graph-based evaluation metric achieves a higher system-level correlation and rank correlation coefficient value compared to existing related works.

Improving Natural Language Queries Search and Retrieval through Semantic Image Annotation Understanding

Abstract: Retrieving images using detailed natural language queries remains a difficult challenge. Traditional annotation-based image retrieval systems using word matching techniques cannot efficiently support such query types. Significant improvements for this problem can be achieved with a semantic understanding for those query sentences and image annotations. This paper presents a two-stage semantic understanding approach for natural language query sentences. At the first stage, the Stanford parser and a designed rule-based relation extraction tool are used in triple extraction process to efficiently extract the objects attributes, instances and natural language annotations relationships involving these objects. The second stage integrates the extracted relations with external commonsense knowledge source in a mapping process to provide high-level semantic meanings to the extracted triples. Experiments are conducted for evaluating the benefit of the proposed semantic understanding against a testing set of natural language sentences from the Flickr8k dataset. The results show that the proposed approach succeeds to extract relational triples with average accuracy value of 97% for the different types of annotations relationships: attributes and instance relations, multiword dependence relations, and semantic relations.

Semantic approaches for query expansion

Abstract: To increase the Information Retrieval System’s efficiency, there is a requirement to expand the native user query. There are many approaches to enhance the user query in which the primary method consists of semantic-based query expansion (QE). In a semantic-based QE approach, relevant documents are retrieved by considering all the similar terms of a given user query. The semantic-based QE approach helps in dealing with the limitation of low recall and low precision value of the Information retrieval system and deals with ambiguity and vagueness. Query Expansion techniques contain the semantic concepts that are relevant to semantic computing, computational intelligence, and information retrieval area. Computational intelligence technique is required during automatic query expansion for advanced information processing. This paper presents a comprehensive survey of the methods proposed by several researchers considering semantic-based query expansion. It also discusses the merits and demerits of each technique in a detailed manner. This paper presents the view of several semantic query expansion core techniques. This paper also discusses the various keyholes present in today’s era.