TREC 2006 Terabyte Track

Topics Creation Guidelines for the Named Page Finding Task

Overview

The named page finding task simulates the process of using a search engine to locate an interesting page that you found once but can't quite remember where it is. Your goal is to locate interesting, "bookmark-worthy" pages, ones you think you might want to go back and find again. Once you've found such a page, give it a name such as you might assign to a bookmark. This name should approximate what you might type into the web search engine to locate that page again.

You can find pages in one of two ways. One is to ask the search system to take you to a random page. Keep looking at random pages until one strikes your fancy. Another approach is to search for things that are interesting to you, and look through the search results until you find something worth keeping.

You should try to make your page's name specific to that page. To check this, after you've named your page, do a new search with the name as a query, and check to see if other pages come up which could take the same name. The named page itself does not need to appear in these search results, although it's okay if it does. The purpose of this check search is to weed out similar (but not identical) pages that might need to be distinguished in order to obtain a good named page topic. If this isn't possible because too many similar pages exist, start again with a different page. If there are several pages that appear to be exact duplicates of the named page, that's fine. During the evaluation process these pages will be identified and treated as equivalent.

Topic Creation

Each participating group is expected to create 10-15 named page finding topics using the TREC Terabyte track topic creation system. Instructions on its main page explain how to obtain an account. We've attempted to simplify the topic creation process as much as possible, and we hope it will require no more than a few hours of work per group.

After obtaining an account and logging into the system, you can browse the GOV2 collection and create new topics for the named page finding task. Browsing the collection can be done by clicking on "Search document collection" and entering a search query into the search field or by clicking on "View random document". Topics can be created via the "Create/modify/delete topics" link or by filling out the form at the bottom of the page when displaying a document from the collection.

For the purposes of topic creation, links within the collection have been disabled. Links either do nothing or take you outside the collection, possibly displaying newer version of the same document, etc. If you want to navigate to a specific document in the collection, use the "View specific document" form in the navigation frame (on the left-hand side of the browser window).

Please note that, since we are only creating topics for the named page finding task, the description and narrative fields of the topic creation interface should be left empty. Please make sure that, whenever you create a new topic, you include the DOCID of the named page that this topic is referring to.