Content-based image retrieval. An introduction to literature and applications

Authors:

  • Jörgen Ahlberg
  • Fredrik Johansson
  • Ronnie Johansson
  • Magnus Jändel
  • Anna Linderhed
  • Pontus Svenson
  • Gustav Tolt

Publish date: 2012-05-28

Report number: FOI-R--3395--SE

Pages: 44

Written in: English

Keywords:

  • CBIR
  • CBIR-system
  • content-based
  • image retrieval
  • information retrieval

Abstract

To search in image and video collections based on visual content is potentially a very powerful technique. Content-Based Image Retrieval, CBIR, has attracted researchers from various research fields: computer vision, artificial intelligence, human factors, and machine learning to name a few. The relatively young age of CBIR as a phenomenon and research area results in an enormous growth of research articles on the topic. The main purpose of this report is to give a brief introduction to the research field of CBIR, the literature and applications. The survey contains an overview of CBIR methods and presents some of the main challenges in CBIR. Most of the suggested CBIR approaches rely on a pre-processing step of feature extraction, aiming at identifying suitable image features to allow for successful retrieval of relevant images from a database containing thousands or millions of images. While low-level features based on colour, texture, and shape are directly related to simple perceptual aspects of image content, there are also higher-level features which are not extracted as easily from pixel data. To automatically derive semantically meaningful features or concepts from images is a hard challenge to which no perfect solutions exist. However, we review attempts and suggestions from the literature. Ontologies are formal descriptions of the concepts and relations in a domain. They are used for bridging the semantic gap between how humans and computers represent the world. Query creation is another vital issue which is critical for the result of the search. Research on multimedia query creation including metadata is thriving but less is done on the harder problem of only using image data in queries. This report includes a limited overview of systems including CBIR capabilities encompassing research prototypes as well as open-source and commercial systems. CBIR also extends into the realms of video data, where, in addition to applying CBIR techniques to individual video frames, the additional time dimension can be explored to detect certain actions, movements or changes. The Swedish Armed Forces today have access to vast amounts of image, video and film material from international missions, but lacks the ability to efficiently search such archives for information. We believe there are several different application areas of interest for the Swedish Armed Forces, particularly for counter-IED analysis, and analysis for air reconnaissance. A conclusion from this study is that while there are indeed large potential benefits of using CBIR, the realization of CBIR systems for military applications requires that potential end-users take active part in the development process in order to work out where CBIR functionality has the greatest impact. The report ends with a brief discussion on the main findings from this study and presents our thoughts on the next steps to be taken in investigating how the Swedish Armed Forces can be provided with relevant CBIR capabilities. An appendix with summaries of studied articles is provided, together with the list of publications resulting from our literature search.