INEX 2010 Web Service Discovery Track

Overview

An efficient and effective Web services discovery mechanism is important in many computing paradigms including Pervasive Computing, Service-Oriented Computing, and the most recent Cloud Computing, in which Web services constitute the chief building blocks. The Web Service Discovery track aims to investigate techniques for discovery of Web services based on searching service descriptions provided in Web Services Description Language (WSDL) . Participating groups will contribute to topic development and evaluation, which will then allow them to compare the effectiveness of their XML retrieval techniques for the discovery of Web services. This will lead to the development of a test collection that will allow participating groups to undertake future comparative experiments.

Test collection

The Web Service Discovery track will use a collection of WSDL documents. These WSDL documents were directly crawled from real-world public Web services indexed by the Google search engine. The test collection was pre-processed so that only valid WSDL1.1-compliant descriptions are retained for XML-based retrieval.

2010 Topics: 25 Topics
2010 Documents (original): contains 1987 separate documents
2010 Documents version 1: contains 1738 documents (some duplicates removed - however filenames unchanged)
Description of 2010 Dataset

Tasks

Each participating group will be asked to create a set of candidate topics, representative of a range of realistic web service discovery needs. Topics and submissions will be in the same format as the ad hoc track.

Relevance assessments

Each participating group will judge several topics using an assessment system suited to the web service discovery problem; where possible these topics should be amongst those originally created by the group.

Evaluation

The effectiveness of XML web service discovery systems developed by the participants will be evaluated using common scoring techniques based on methods already used in INEX for evaluation of XML retrieval.

Revised Schedule

4 October 2010	Submission deadline for candidate topics
11 October 2010	Release of final set of topics
1 November 2010	Submission deadline for Web Service Discovery search results
5 November 2010	Release of assessment pool
12 November 2010	Deadline for relevance assessments
17 November 2010	Release evaluation results
22 November 2010	Submission deadline for papers for pre-proceedings (all tracks)
13-15 December 2010	INEX Workshop in Netherlands

Organizers

James Thom
RMIT University
james.thom@rmit.edu.au

Chen Wu
Curtin University of Technology
chen.wu@curtin.edu.au