May 23, 2016: Initial 2016 Draft Guidelines available for comment.
TREC 2016 Total Recall Track
The TREC 2016 Total Recall Track will reprise (with minor tweaks) the 2015 task with new datasets and new topics. Several of the 2015 datasets, as well as an on-line test server and baseline model implementation, are available now to TREC 2016 participants.
Please register for TREC 2016, sign up for the Total Recall Discussion Group, and familiarize yourself with the guidelines.
TREC 2015 Total Recall Track (Completed)
Proceedings of TREC 2015 will be available shortly. In the
meantime, please sign up for TREC 2016.
OLD NEWS
June 22, 2015: Supplemental Guidelines finalize requirements for At Home partcipation.
May 3, 2015: Initial Draft Guidelines available for comment.
April 12, 2015: Baseline Model Implementation now available for download.
Overview
The Total Recall Track is part of the Text Retrieval Conference (TREC),
a forum in which academic, commercial, and government researchers conduct experiments and share results
for specific information retrieval tasks.
Participants in the Total Recall Track will investigate tools and strategies to find, given
a topic and a collection of documents, with as little human effort as possible, as nearly all
of the relevant documents as possible.
Prospective participants must register for TREC
as soon as possible.
Participants should adhere to the Initial Draft Guidelines and Supplemental Guidelines.
Participants may conduct pilot experiments through June.
Particpants submitting fully-automated
systems for evaluation ("Sandbox" participation) must submit their systems by September 1.
Particpants
who operate their own systems ("At-home" participation) must conduct the specified retrieval
tasks throught July and August.
Some collections will be available only to Sandbox participation;
other collections may be available only to At-home participation. To ensure access to all
collections, participants should plan for both Sandbox and At-home participation.
Results will be presented in November at TREC 2015, at the National Institute of Standards and Technology, in
Gaithersburg, MD.
Slide Show
This Overview of the Total Recall Track was presented at TREC 2014.
Mailing List
Participants should subscribe to the trec-total-recall discussion group.
Guidelines
Draft guidelines were released May 3. Supplemental guidelines, released June 22, finalize requirements for At Home participation. Future supplemental guidelines will finalize requirements for Sandbox participation.
Sample Tools
Sample tools are provided to accomplish Sandbox and At-home participation. Participants are
free to use or modify these tools, or to use or create their own tools. The results achieved
by the sample tools will be used as a baseline for comparison with particpants' results.
Track Coordinators
Adam Roegiest, University of Waterloo
Gordon V. Cormack, University of Waterloo
Maura R. Grossman, Wachtell Lipton Rosen & Katz
Charles L. A. Clarke, University of Waterloo