Explicit Description of Run's

[Lewis John Mcgibbney]

Hi All,
I've been working on the resource selection and have a few last minute clarifications.

The FedWeb site [0] states "For a given query, the system should return a ranking such that..." I am working under the assumption that "... a given query" is the queries provided here [1] where the file is FW13-topics.txt. Can someone please confirm?

The reason I ask is that the submission format provided as an example in [0] looks like

100 Q0 FW13-e200 1 29.34 example13a

Whilst the query numbers (I am currently looking at) in FW13-topics.txt begin at 7001 and finish at 7506!

I do notice that in the README.txt which accompanies the dataset I have been working with, it states "The participants are required to submit runs containing all these topics."... I've therefore been working on the assumption that for each topic in FW13-topics.txt, I will submit a top 10 list in the format stated above.

Is this satisfactory?

Kind Regards and thank you in advance.

Best
Lewis

[0] https://sites.google.com/site/trecfedweb/
[1] http://bookstore.ewi.utwente.nl/dolf/fedweb13/

[Dolf Trieschnigg]

Hi Lewis,

The topic file can be found in the active participants section of the
(general) TREC website
http://trec.nist.gov/act_part/tracks13.html
(there is a link called "Federated Web track test topics" just below
Mailing list)

The topic numbers are indeed in the 7000-8000 range, the example on
the FedWeb guidelines did not reflect this. I've updated it to make it
more clear. Thanks for pointing it out.

There are 200 topics in the topic file. For each of these 200 topics
you can submit up to a thousand search engines. Since we have only 157
engines, you should provide a ranking of these 157 engines for each
topic.
(The relevance of an engine is based on the top 10 results from this engine)

There's a perl script to check your run file before submitting it:
http://trec.nist.gov/act_part/tools.html
(the submission form also uses this tool)

I hope this answers your questions.
Regards, Dolf

> --
> You received this message because you are subscribed to the Google Groups
> "TREC FedWeb track" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to trec-fedweb...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Lewis John Mcgibbney]

Hi Dolf,
Thanks  for clarification.
I will write here if there are more queries.
Thank you.
Best
Lewis

>> [0] trecfedweb

>> [1] http://bookstore.ewi.utwente.nl/dolf/fedweb13/
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "TREC FedWeb track" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to trec-fedweb...@googlegroups.com.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>

--
Lewis

[Lewis John Mcgibbney]

Morning,

On Monday, 5 August 2013 21:50:11 UTC-7, Dolf Trieschnigg wrote:

The topic file can be found in the active participants section of the
(general) TREC website
http://trec.nist.gov/act_part/tracks13.html
(there is a link called "Federated Web track test topics" just below
Mailing list)

Any idea how I can obtain a username and password for this site please?
The last username and password I was issue with was on June 28th. This was for the dataset available at http://bookstore.ewi.utwente.nl/dolf/fedweb13/
I've no password to access the site above.
Can you please help?
Thank you
Lewis

[Dong Nguyen]

Hi,

The username and password for the active participants section can be obtained by registering as a participant with TREC. Contact Ellen Voorhees (ellen.v...@nist.gov) for more information.

Kind regards,

Dong

--

[Lewis John Mcgibbney]

Good Morning Dong,

Thank you very much.

Lewis

--
Lewis