Not a Number (NaNs) in Results

[Harish Tayyar Madabushi]

Dear all, 

TL;DR

1. If you get "Not a Number" (NaN) as a result on a setting that you are NOT participating in, do not worry about it, we will still consider your results on the other subtasks/settings. (See details on why you get NaNs below)
2. If you get NaN as a result on a setting that you ARE participating on, then you must fix this error and resubmit. You can email me to delete the submission with errors. 
3. The practice phase on CodaLab continues to remain open so you can continue to tune your models. Please be sure to submit your results to the correct phase ("eval data" results to the practice phase and the "test data" results to the evaluation phase). 
   

The most likely reasons for getting NaNs is that you have not included the entries associated with a particular setting. For example, if you are participating only on the "zero_shot" setting, then your submission must list the "one_shot" entries with empty (no values) as predictions. 

Here is a truncated example: 

... Lot of rows not shown here. 

41989        GL        zero_shot        <numeric prediction value (1 or 0)>
87635        GL        zero_shot        <numeric prediction value (1 or 0)>
93602        GL        zero_shot        <numeric prediction value (1 or 0)>
11495        EN        one_shot         <numeric prediction value (1 or 0)>
97180        EN        one_shot        
71582        EN        one_shot        
29356        EN        one_shot        

3378        EN        one_shot         

... Lot of rows not shown here. 

Hope this helps. 

Do not hesitate to post here or email if you have any questions. 

Best

Harish