Information Society Technologies

Search In Audio Visual Content Using Peer-to-peer IR

IST FP6 Project


Work packages and Deliverables

WP 1 - User Requirements, Interfaces, and Scenarios


This WP will develop and test a set of scenarios for multimedia search over mobile terminals. Both analysis of archetypal users (their motivations, expectations and goals) and business analysis will be used to evaluate the scenarios, and to pick out a small number of scenarios which are promising for future exploitation. The selected scenarios will be further detailed with regard to the involved information retrieval functionality and the concepts will be tested in focus groups with small numbers of potential users (journalists, tourists, consumers, or other dependent on the selected scenarios). The results will be an input of user requirements including user interface aspects, and an identification of roles and players, that should inform the activities of the rest of the SAPIR project.

Back

WP 2 - System Architecture, P2P Infrastructure, Routing, Protocols


The purpose of this work package is to define the system architecture of SAPIR in detail. The system will be composed of three parts – content management, P2P indexing and P2P search. In this WP we will define the components, functionality, interfaces, and interaction between those components. The preliminary architecture will be defined in the first year in close collaboration with the work packages on specific functionalities and requirements. Then the second year will be used to carefully revise the architecture and extend, refine, or adjust it to additional requirements based on insights gained during the first year.

Back

WP 3 - Media Analysis and Enrichment for Search


In this WP we will integrate all activities related to media analysis for enabling efficient search and browsing by content. The tools developed in this WP will provide browsing capabilities and content extraction across the following multimedia data - images, videos, audio, music and text. All tools will be developed as UIMA (Unstructured Information Management Architecture) annotators using the uima-apache open source. High performance of media analysis will be achieved by exploiting the UIMA advanced flow control that enables parallelism and remote invocation using Vinci. CAS consumers will output the extracted media features into MPEG-7 format using some specific MPEG-7 extensions that we will contribute to the MPEG-7 committee.

Back

WP 4 – P2P Indexing, Caching and Collaborative Crawling Including Push


The aim of this work package is to provide a methodology and software for building a P2P network for crawling, indexing and searching large volumes of multimedia data. In particular, (i) we will research two different types of indexes: (a) similarity search indexes of metric multimedia features distributed between a dynamic infrastructure of peers forming multiple structured overlays for similarity searching and (b) local multimedia indexes with enough P2P statistics to decide how to route queries between the indexes at query time; (ii) in order to improve the performance of the query elaboration, we will try to develop caching mechanisms that store the answers of the most frequent queries in all peers; (iii) we will investigate a novel distributed push-based crawling model, where content providers publish and “push” information to the P2P indexing nodes instead of being visited by crawling agents.

Back

WP 5 – Complex Search and Ranking in P2P


The aim of this WP is to develop novel and scalable methods for combining text, meta-data and content-search in audio-visual data. Query input will be supplied by the user following the “query by example” paradigm using several audio-visual sources for querying. Scalability is achieved using P2P overlay network combining sophisticated ranking algorithms over several media specific similarity metrics.

Back

WP 6 – Security, Rights, and Trust in P2P


This WP aims for the proper integration of Protected Content in a P2P network, to assure IPR protection, in particular in the different foreseen scenarios. This will improve user (especially those interested in protected content sharing) trust and confidence in the system. This WP will deal with network security and peer trusting, and with content digital right management.

Back

WP 7 – Embedding in Social Networks and Multiple Devices


This WP will focus on the enhancement of user experience with search engines addressing on one hand the research of the most innovative context-based technologies and social network characteristics and its appliance to a multimedia search engine in a P2P environment. On the other hand, the access from multiple devices to the P2P network will be considered to publish and retrieve digital contents, adapting these contents to the particular features of devices. Special emphasis will be put on devices according to the scenario defined in WP1.

Back

WP 8 – Testbed Integration and Trials


This WP will integrate the findings and implementations achieved in other WPs into a testbed, in order to analyze the work done and assess the overall success of the proposed innovations.

The user scenarios detailed in WP1 will be redefined in a more detailed form, keeping in mind that they are "proofs of concept". Accordingly, test plans will be written to examine system key features.

Then, integration will be done on the aforementioned testbed, and reports will be written, measuring the project's accomplishments and proposing future research lines, new improvements and enhancements or better alternatives applicable to those scenarios, if appropriate.

Back

WP 9 - Dissemination & exploitation of the results


In WP9 the dissemination of the results in an exploratory way assumes to simultaneously provide the possibilities to apply and use of the knowledge acquired. The information, products and services which will directly or indirectly derive from the obtained scientific data will be released to the market, by means of website, databases, protocols, presentations and publications.

Back































 
 

Partners

  
IBM Haifa Research Laboratory
IBM Haifa Research Laboratory
 
 
ISTI - CNR
ISTI - CNR
 
 
Max-Planck Institute for Informatics
Max-Planck Institute for Informatics
 
 
University of Padova
University of Padova
 
 
Eurix
Eurix
 
 
Xerox Research Centre Europe
Xerox Research Centre Europe
 
 
Masaryk University
Masaryk University
 
 
Telefonica Investigacion y Desarrollo
Telefonica Investigacion y Desarrollo
 
 
Telenor
Telenor