Home

The Information Retrieval area has a strong and long tradition dating back to the 1960s in producing and processing scientific data resulting from the experimental evaluation of search algorithms and search systems. This attitude towards evaluation has led to fast and continuous progress in the evolution of information retrieval systems and search engines.

However, in order to make these data test collections understandable and usable they must be endowed with some auxiliary information, i.e., provenance, quality, context, etc. Therefore, there is a need for metadata models able to describe the main characteristics of evaluation data. In addition, in order to make distributed data collections accessible, sharable, and interoperable, there is a need for advanced data infrastructures.

In contrast, the information retrieval area has barely explored and exploited the possibilities for managing, storing, and effectively accessing the scientific data produced during the evaluation studies by making use of the methods typical of the database and knowledge management areas.
Over the years, the information retrieval area has produced a vast set of large test collections which have become the main benchmark tools of the area and ensure reproducible and comparable experiments. However, these same collections have not been organised into coherent and integrated infrastructures which make them accessible, searchable, citable, exploitable, and re-usable to all possibly interested researchers, developers, and user communities.

It is thus time for these three communities – information retrieval, databases, and knowledge management – to join efforts, meet, and cooperate to address the problem of envisaging and designing useful infrastructures able to coherently manage pertinent data collections and sources of information, and so take concrete steps towards developing them.
Indeed, the information retrieval experts need to recognise this need, while the database and knowledge management experts need to understand the problem and work together to solve it by using the methods and techniques specific to information management.

Therefore, the main objective of the workshop is to gather together experts from these three areas, to encourage them to recognise the urgency of addressing the problem in an integrated and coherent way, and to coordinate efforts towards drawing a roadmap and suggesting best practices for an effective solution of the problem.