Dear all,
TL;DR
- If you get "Not a Number" (NaN) as a result on a setting that you are NOT participating in, do not worry about it, we will still consider your results on the other subtasks/settings. (See details on why you get NaNs below)
- If you get NaN as a result on a setting that you ARE participating on, then you must fix this error and resubmit. You can email me to delete the submission with errors.
- The practice phase on CodaLab continues to remain open so you can continue to tune your models. Please be sure to submit your results to the correct phase ("eval data" results to the practice phase and the "test data" results to the evaluation phase).
The most likely reasons for getting NaNs is that you have not included the entries associated with a particular setting. For example, if you are participating only on the "zero_shot" setting, then your submission must list the "one_shot" entries with empty (no values) as predictions.
Here is a truncated example:
... Lot of rows not shown here.
41989 GL zero_shot <numeric prediction value (1 or 0)>
87635 GL zero_shot <numeric prediction value (1 or 0)>
93602 GL zero_shot <numeric prediction value (1 or 0)>
11495 EN one_shot <numeric prediction value (1 or 0)>
97180 EN one_shot
71582 EN one_shot
29356 EN one_shot
3378 EN one_shot
... Lot of rows not shown here.
Hope this helps.
Do not hesitate to post here or email if you have any questions.
Best
Harish