Small update to code in repo

20 views
Skip to first unread message

Stephen Mayhew

unread,
Mar 19, 2020, 10:26:06 AM3/19/20
to duolingo-sharedtask-2020
Hello all,

The JHU team pointed out an issue with punctuation that could affect the scoring script (tl;dr we were using string.punctuation, but should have been using unicode punctuation). See here: https://github.com/duolingo/duolingo-sharedtask-2020/pull/8 This has been merged into the master branch. I compared the AWS baseline against gold dev files and only Japanese scores changed, and those only by 0.2%. Nothing changed in the test scores.

For what it's worth, we'd seen this issue when setting up CodaLab and corrected it. So CodaLab scores are correct. We had just forgot to propagate it back to the repo. 

Thanks,
Stephen
Reply all
Reply to author
Forward
0 new messages