Task in Submission

[Zhiqi Huang]

Hi,

This is the first time I have participated in the TREC NeuCLIR track. I have a question about submission.

Do I have to submit runs for all combinations of retrieval settings?

In the task table here https://neuclir.github.io/2023, except for Technical Documents, there are 20 different type of retrieval settings:

3 single language news x 2 query languages x (FF or RR) + multilingual news x 4 query languages x (FF or RR) = 20 run files

Or can I only focus on English query and full retrieval setting?

Also, in the query language column, what does "other" refer to?

Thanks,

Zhiqi

[Dawn J. Lawrie]

Hi Zhiqi,

You do not have to submit runs for all combinations. The main categorization of results is by document set. There is no requirement to submit results for all document sets, although it is encouraged. When you rank your runs, you will be doing that by document set, so you can give a priority of 1 to a Chinese document set run and a Russian document set run, for instance. If you are only interested in the cross language task where you use English queries, all of your runs may use that configuration. 

As for the "other" option for queries, you could use the Human translations of the English queries to Chinese and retrieve Russian documents. This is still a cross language task, and we wanted to be able to support such experiments if researchers were interested in it.

Dawn

--
You received this message because you are subscribed to the Google Groups "neuclir-participants" group.
To unsubscribe from this group and stop receiving emails from it, send an email to neuclir-particip...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/neuclir-participants/b8246206-d97e-442b-b7e7-1f86ef9e66b4n%40googlegroups.com.

--

_________________________________________________
Dawn J. Lawrie Ph.D.
Senior Research Scientist
Human Language Technology Center of Excellence
Johns Hopkins University
810 Wyman Park Drive
Baltimore, MD 21211
law...@jhu.edu
https://hltcoe.jhu.edu/faculty/dawn-lawrie/