Semantic Restructuring of Natural Language Image Captions to Enhance Image Retrieval

Abstract

The rapid growth in the volume of visual information can make the task of finding and accessing visual information of interest, overwhelming for users. Semantic analysis of image captions can be used in conjunction with image retrieval systems (IMR) to retrieve selected images more precisely. To do this, we first exploit a Natural Language Processing (NLP) framework in order to extract concepts from image captions. Next, an ontology-based framework is deployed in order to resolve natural language ambiguities. The novelty of the proposed framework is that the combination of LSI with the Ontology framework enables the combined framework to tolerate ambiguities and variations in the Ontology. A key feature is that the system can find indirectly relevant concepts in image captions and thus leverage these to represent the semantics of images at a higher level. Experimental results show that the use of LSI based NLP combined with an ontological framework significantly enhances image retrieval.