TREC 2009 synopsis for relevance feedback track

[Ellen Voorhees]

The TREC 2009 relevance feedback track,
along with the web, entity, and million query tracks,
will be using a new document collection this year.
All four tracks are using this new collection so
that the different tracks can inform one another
(and so they can share some resources such as judgments).
This new collection is a very large crawl of the web.
Details about the collection and its current state
are provided at

http://boston.lti.cs.cmu.edu/Data/web09-bst/

It is likely that most of the tracks will not use the full new
collection, but instead a smaller subset will be defined.
Since the full collection is not yet complete, the subset
has not been defined yet either.

The relevance feedback track guidelines cannot be
completed until the document collection is finalized,
but here is the general idea for the track.
Given that there will be a new collection used,
the 2009 relevance feedback track submissions will be in two
phases. The first phase will be straight ad hoc runs
with the goal of finding good documents to feed to feedback
algorithms. These runs will be submitted in June and
small pools will be judged by NIST. The judgments from
these pools will be released by late July. Participants
will then use the judgments (from all groups) to perform
their feedback runs. Feedback runs will be due in late August.

Ellen Voorhees

[李思]

I am wondering whether we could buy only the subset used in fb2009
instead of the full collection. And how to obtain the Dataset, is it
just as the method in http://boston.lti.cs.cmu.edu/Data/web09-bst/ ?

Thank you very much!

[Ellen Voorhees]

I'm afraid those details aren't known yet.
I am hopeful that these issues will be resolved
shortly.

Ellen