Content-Based Image Retrieval and Filtering (CBIR/CBIF)

The interest in use of multimedia data (digital images and video data) in both scientific and commercial fields has been growing enormously over the last few years. As a result, there is a need to organize, store, search and display multimedia data satisfying user requirements. Consequently, developing Content Based Image retrieval (CBIR) systems for searching multimedia data based on contents have attracted a lot of academic and industrial interest. CBIR comprises techniques for retrieving images/video data (from a collection) based on automatically derived features such as color, texture, shape and motion. CBIR draws many of its methods from the field of image processing, computer vision, statistics, and pattern recognition. Content-Based Image Filtering (CBIF), on the other hand, comprises techniques for filtering images for pornographic contents.

The aim of this project work is to design and implement a system for efficient Content Based Image Retrieval and Filtering that can 1) provide a large variety of image distance measures such as color histograms, texture measures, shape invariants, or semantic scene/object retrieval, which can be used singly or in combination to satisfy a wide range of user needs 2) provide techniques to filter images for pornographic content.

A CBIR system comprises a multimedia database, querying mechanisms for retrieving desired image/video data from the database, a GUI for formulating queries and displaying results, and suitable indexing techniques for storing feature vectors. A CBIF system will comprise techniques to filter images for pornographic content. The joint collaboration will involve work in following areas:

a) Querying mechanisms:
· Color: Retrieval based on matching user specified color attributes with images in multimedia database. For filtering images for pornographic content (CBIF), skin colour would be used.
· Texture: Retrieval based on matching query image textures with images in multimedia database. Texture measures to be considered: degree of contrast, coarseness, directionality, regularity, randomness, Gabor filters, and fractals. For CBIF, skin texture would be used to reduce the number of false positive images by the system (images that are benign).
· Shape: Retrieval based on matching object features in stored images with object features in the query image. Support for both boundary-based and region-based shape representations (using Fourier descriptors and Moment invariants) would be considered. Alternative methods to be explored: elastic deformation of templates, comparison of directional histograms of edges extracted from the image, comparison of skeletal representations of object shapes using graph-matching techniques. For CBIF, shape features would be used to differentiate portrait pictures from the pornographic images.
· Other primitive features: a) Template matching - Accessing data by spatial location, and b) Wavelet transforms - Retrieval by matching wavelet features of query and stored images.
· Semantic retrieval: The project work will concentrate on two problems.
i) Scene recognition: Identify types of scenes depicting natural objects (such as vegetation, trees, sky, and water bodies). Techniques for scene analysis include use of low frequency image components to train a neural network and color neighborhood information extracted from low-resolution images to construct user defined templates.
ii) Object recognition: Identify images containing conspicuous man-made objects (e.g., buildings, towers, bridges, etc.). The work involves developing a model for each class of object to be recognized, identifying image regions that might contain examples of the object, and building up evidence to confirm or rule out the objects presence. Techniques to be used: Model-based object recognition, Perceptual grouping, etc.

b) Graphical user interface (GUI): Developing GUI to formulate queries is crucial in any retrieval system. The GUI for the proposed work will include functionality for query-by-example (user provides a query image and system displays thumbnails of closest matching images), sliders (for specifying color queries), palettes (for specifying texture queries), and facility for sketching desired object on screen for shape queries.

c) Indexing techniques: The retrieval efficiency depends on indexing structures used for storing the image features. The most commonly used are R*-tree, the TV-tree and the SS+-tree, but the overheads of using these are considerable. Recently, use of similarity clustering of images has shown promising results. The proposed work will explore efficient indexing techniques for improving the search efficiency.

d) CBIF: A system to filter out pornographic images using a combinatorial techniques of an icon filter, a graph-photo detector, a color histogram filter, a texture filter, and a wavelet-based shape matching algorithm would be developed (similar to Wavelet Image Pornography Elimination software). For portrait images, the system would find outlines of the skin-tone regions in an image, and use shape-matching algorithms to identify whether the shapes represent a portrait image or pornographic content.
e) Video Retrieval: Algorithms will also be developed to structure a video into a set of scenes, which represents a video table of content (VToC). The VToC can be used for browsing and retrieval of video data.

 

Expected Achievement: An intellegent semantic access to multimedia data in a digital library

Digital libraries access
Content-Based Image Retrieval
Semantic multimedia search
Intelligent access to Digital content

Experts in Image processing, Pattern Recognition, and Computer vision

Experts in soft computing (Neural networks, Genetic algoritms, Fuzzy Logic)

PROFILE 1: Develop algorithms to structure a video into a set of scenes, which represents a video table of content (VToC). The VToC can be used for browsing and retrieval of video data from multimedia database in digital library.

PROFILE 2: Perform an audio search based on an audio clip from digital library archive.

PROFILE 3: Perform Semantic retrieval: a) Scene recognition to identify types of scenes depicting natural objects (such as vegetation, trees, sky, and water bodies).  b) Object recognition: Identify images containing conspicuous man-made objects (e.g., buildings, towers, bridges, etc.).

Institutes having Digital Libraries

Institutes involved in research into Image processing and Pattern Recognition

Yes