qrels file for participants

[Lewis John Mcgibbney]

Hi,
Quick email to check if there is any qrels file which I should have.
In the evaluation subsection of Task 1: Resource Selection [0], it states "We will provide a qrels file for the participants that assigns each search engine a relevance grade for a given query."...
Also how valuable are people finding this qrels file? I have not got it yet and have therefore not used it yet.
Thanks in advance.
Best
Lewis

[0] https://sites.google.com/site/trecfedweb/

[Dolf Trieschnigg]

Hi Lewis,

The qrels are only released after the deadlines - they are used for
evaluating the participating systems. Giving them now would be like
providing the answers up front :-)

Regards, Dolf.

> --
> You received this message because you are subscribed to the Google Groups
> "TREC FedWeb track" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to trec-fedweb...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

[Lewis John Mcgibbney]

Hi Dolf,

No problems at all. I just need clarification on as much of this material as possible.

I found out yesterday that the *main email* was sent out in February... which I missed.
I've been chasing my tail every since in an attempt to ensure that I am in compliance with the FedWeb track.

Thanks for the feedback.
Best
Lewis

--
Lewis