Information Society Technologies

Search In Audio Visual Content Using Peer-to-peer IR

IST FP6 Project


Overview

SAPIR extends the power of web searches beyond centralized text and metadata searches to include distributed audio-visual content.

Today, Web searches are dominated by search giants such as Google, Yahoo, or MSN that deploy a centralized approach to indexing and utilize text-only indexes enriched by page rank algorithms. Consequently, while it is possible to search for audio-visual content, the search is limited to associated text and metadata annotations. Supporting real content-based, audio-visual search requires media-specific understanding and extremely high CPU utilization, which would not scale in todayís centralized solutions.

SAPIR aims at breaking this technological barrier by developing a large-scale, distributed P2P architecture that will make it possible to search audio-visual content using the query-by-example paradigm. Considering that a picture is worth a thousand words, an image taken by a cell phone can be used to find information. For example, search for a monument using its photo or search for a full song using its melody. Combining these search hints with optional metadata annotations and user and social networking contexts will provide the next level of search capabilities with precise retrieved results.

Our vision is to conduct innovative research that will lead to a technology where end-users are peers that can produce audio-visual content from their mobile devices. This content will be indexed by super-peers across a scalable P2P network to enable content searches in real-time, while respecting IPR and protecting against spam. To this end, SAPIR brings experts in audio-visual content understanding in the areas of text, audio, image, video, and music analysis.

A common framework for feature extraction from all media contents will be developed for similarity search and ranking along all supported media. To address scalability issues, we will develop a P2P architecture where features can be extracted in one peer and pushed to an indexing peer. The P2P architecture will provide a scalable indexing structure that can be used for multi-feature search. Caching techniques will be developed to increase system performance. To further improve audio-visual retrieval and navigation, SAPIRís consortium combines experts in mobile device technology, along with with experts in social networking and IPR to enable a secure and trusted environment.

This technology can provide a significant advantage to the European community over existing, centralized, text-only search engines and can be applied to various fields such as tourism, government services, healthcare, and more.

Chorus
Sapir is part of the Chorus activities.

CHORUS is a European Coordination Action which aims at creating the conditions of mutual information and cross fertilisation between the European projects dealing with Multimedia Content Search Engines. National and international initiatives are also included in Chorus action.


 
 

Partners

  
IBM Haifa Research Laboratory
IBM Haifa Research Laboratory
 
 
ISTI - CNR
ISTI - CNR
 
 
Max-Planck Institute for Informatics
Max-Planck Institute for Informatics
 
 
University of Padova
University of Padova
 
 
Eurix
Eurix
 
 
Xerox Research Centre Europe
Xerox Research Centre Europe
 
 
Masaryk University
Masaryk University
 
 
Telefonica Investigacion y Desarrollo
Telefonica Investigacion y Desarrollo
 
 
Telenor
Telenor