SQuAD 2.0 Leaderboard Update

[robi...@stanford.edu]

Hi everyone,

I just pushed an update to the SQuAD 2.0 leaderboard. Behind the scenes, we have patched some vulnerabilities in our testing procedure, which necessitated re-running all previously submitted models. As a result, some models that use randomness (such as ELMo-based models) had slight fluctuations in performance (usually within 0.1 F1, max diff of 0.4 F1). Please let me know if you have any questions!

Robin

[xzhan...@gmail.com]

Hi Robin, we submitted our version 1.1 model on 4/21, but have not got any update or feedback on the leader board while I saw new updates this Monday on the version 2.0 leader board, can you let us know when do you plan to run and update the version 1.1 leader board? Our model name is "Common-sense Governed Bert-123" 

Thanks

Jerry Zhang