Confusions Regarding the Gold Standards and Baseline Results

52 views

Skip to first unread message

ABDUR REHMAN KHALID

unread,

Jun 5, 2022, 4:23:11 AM6/5/22

to AKBC2022

I am a bit confused in understanding the format of the baseline_results.txt and extractions-groundtruth.oie.

The first sentence in groundtruth.oie is related to the spirits, meanwhile the first sentence in the baseline_results.txt is the Moody's something and in sentences.txt the first sentence is the Annualized Interest rates something, and in assignment we are asked to add the id of sentence if sentences are not same everywhere then how we are supposed to get our results evaluated?

AKBC2022

unread,

Jun 7, 2022, 5:37:05 AM6/7/22

to AKBC2022

Summary: The order of the sentences do not matter.

The sentences in the baseline_result, input sentences and the ground truth extractions are not in order. They are not required to be in order. The results file has id of the sentence in the order in which they are read by the reader. The run.py already takes care of formatting your extractions to the correct format to be written to the results file. Again, here the ids are only generated in the results file and they are not unique ids.