Question about the format of the test data

[Nhat Tran]

Hi,

I want to ask what information is included in the input of the test data. Specifically, for each turn in the dialogue history, is the information about the gold-standard document and span available, or what is provided is just the raw text of the utterances?

 

In the provided dummy test file, the ground truth document spans of the previous turns in the dialogue history is available, but I am not sure if they will be in the real test data.

[Song Feng]

Hi Nhat,

 

Thank you for the questions! 

 

Once you registered with the Shared Task learderboard on eval.ai (https://eval.ai/web/challenges/challenge-page/1437/overview), you will see more information on the “Submit” tab.  It will show you a data folder that includes the data input for the Dev Phase. The test data for final Test Phrase would be in the exact same format as Dev Phase but more examples for evaluation. You will be only provided dialogue history (just utterances) as input along with all documents, which means that there won’t be grounding span or other annotations available. Again, you will see it in the data folder mentioned earlier for a better idea.

 

For test data for the final Test Phase, you won’t see the turns to predict in the dialogue history. We will only select one turn to predict per dialogue for Test Phase.

 

Let me know if you have any other questions. 

 

 

Thanks,

Song