You do not have permission to delete messages in this group
Copy link
Report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to duolingo-sharedtask-2020
Hello all,
The JHU team pointed out an issue with punctuation that could affect the scoring script (tl;dr we were using string.punctuation, but should have been using unicode punctuation). See here: https://github.com/duolingo/duolingo-sharedtask-2020/pull/8 This has been merged into the master branch. I compared the AWS baseline against gold dev files and only Japanese scores changed, and those only by 0.2%. Nothing changed in the test scores.
For what it's worth, we'd seen this issue when setting up CodaLab and corrected it. So CodaLab scores are correct. We had just forgot to propagate it back to the repo.