Output details

An uncertainty-aware query selection model for evaluation of IR systems.

<17> This SIGIR (ranked #1 conference in Information Retrieval and rank #16 in Computer Science by Microsoft Academic) paper considers the problem of query selection as a mechanism for reducing the cost of constructing information retrieval test collections. A comprehensive mathematical formulation of the problem is presented. Experiments confirmed the theory on two TREC collections. Microsoft found that the number of judgements needed to evaluate variants of their search engine was approximately halved. Both the National Institute of Standards and Technology (NIST, USA) and the Commonwealth Scientific and Industrial Research Organisation (CSIRO, Australia) have expressed interest in using the framework.