What does a score of -1.0 mean?

[Amr Keleg]

Hi Sasha,

I have just made a submission to subtask 1 and I got a score of -1.0
I think this means that there is something wrong but I am not sure how to fix it.
Could you please help me with this issue?

Thanks,
Amr

[Sasha Spala]

Hi Amr,

Codalab’s leaderboard required us to populate all fields when we report a score. Because we can’t have multiple leaderboards for one competition, that means that we have to provide a score for each subtask even if you’re only submitting to one. The most straightforward way to handle this is to give a “non-score” (in our case, -1) for sub tasks you didn’t submit to. So if you are submitting to subtask 1, you should get a genuine score for subtask 1, and -1 for both subtask 2 and 3.

It’s unclear how Codalab determines the ranking list off to the side, but as long as the correct values are in the leaderboard matrix (where you see the breakdown of scores), you’re all set.

Best,

Sasha

On Feb 22, 2020, at 4:21 PM, Amr Keleg <amrke...@gmail.com> wrote:



--
You received this message because you are subscribed to the Google Groups "DeftEval 2020" group.
To unsubscribe from this group and stop receiving emails from it, send an email to semeval-2020-task...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/semeval-2020-task-6-all/66df8119-8103-42c8-ae0d-1d13e5e4c4c7%40googlegroups.com.