CHiC 2011: Cultural Heritage in CLEF

Digital libraries and other information systems that access Cultural Heritage (CH) materials are becoming increasingly complex. Indeed, they must often manage a diverse range of content from different CH institutions – such as libraries, museums, written and audiovisual archives – and have to provide access to them in a unified and coherent way. The content from CH institutions is often multilingual and multimedia (e.g. text, photographs, images, audio recordings, and videos), usually described with metadata in multiple formats and of different levels of complexity. CH institutions have different approaches to managing information and serve diverse user communities, often with specialized needs. This makes the meaning of "search and browse" quite different for users of a library or archive and non-specialist users may not be able to successfully retrieve relevant results or may be disoriented by the kind of results they obtain. Today much effort is being placed on designing and developing effective search systems and tackling issues such as user interfaces, interoperability and metadata enrichment. However, as far as evaluation is concerned, particularly that which is system-oriented, there is less attention. Evaluation approaches are often fragmentary and home-grown.  The CHiC 2011 workshop aims at moving towards a systematic and large-scale evaluation of cultural heritage digital libraries and information access systems and helping to shape a possible roadmap for it.

The objective of this workshop is to review existing use cases in the CH domain and their translation  into potential retrieval and evaluation scenarios that can be used as benchmarks for evaluating CH information access systems. The overall goals are:

1.       To establish what makes searching in the cultural heritage domain distinct from other domains

2.       To gather existing use cases for multilingual information access in the CH domain.

3.       To review existing evaluation resources studies within the CH domain.

4.       To propose appropriate methodologies for evaluating multilingual information access to CH resources.

5.       To define multiple concrete evaluation tasks modeled on IR evaluation initiatives such as CLEF, TREC or INEX