Initial Release, TREC-IS 2021-A Guidelines

[Cody Buntain]

Good morning, all!

Attached is a PDF of the submission and task guidelines for 2021-A. 

Big changes include collapsing the three tasks from 2020 back into Task 1 and a move to a JSON-based runfile format. We intend to release the data for 2021-A within the week as well.

We also intend to host a brief workshop toward the end of this month to discuss changes in 2021, questions about 2021-A, and plans for 2021-B, such as spinning up a leaderboard for evaluation for the summer.

The TREC-IS website will be updated in the next day or so as well, where all data through 2020-B will be available. If you need the assessor data sooner, however, feel free to email me.

-All the best,

Cody and the TREC-IS Team