Subtask 1 and 2 Results

[Sasha Spala]

Hello everyone,

Again thank you all for your participation in DeftEval! The official subtask 1 and 2 leaderboards are now available on Github and on the front page of the Codalab site. Because we heard from many of you who did not submit your best score to the leaderboard, or forgot to include your other subtask results in your leaderboard submission, we made the decision to use the best result from each user regardless of whether or not you submitted that result to the leaderboard. If for whatever reason (interesting architecture, alternative method, etc.) you intentionally submitted a different result to the leaderboard, remember that you are still absolutely welcome to submit those ideas to the conference regardless of your leaderboard submission.

 

 

Additionally, if you placed lower than expected in the leaderboard, remember that you are still welcome to make a paper submission to the SemEval conference. We are interested both in high performing models and models that are interesting in architecture, approach, or otherwise significant even though they may score below the top 5, top 10, etc.

 

If you have any questions, please feel free to reach out. We will post the results of subtask 3 as soon as we have the aggregate data together after the submission deadline tomorrow.

Best,

Sasha

[hx...@georgetown.edu]

Hi Sasha,

Thanks for posting the results. I'm just wondering how is this result calculated? Seems like it's not purely F1 score.

Kind regards,

Haotian Xue

[Sasha Spala]

Hi Haotian,

The results are taken directly from the evaluation program output (whatever your scores.txt has in it). Information on the evaluation metrics is available here and the exact code used is here.

 

Best,
Sasha

 

From: <semeval-202...@googlegroups.com> on behalf of "hx...@georgetown.edu" <hx...@georgetown.edu>
Date: Tuesday, March 10, 2020 at 11:37 AM
To: DeftEval 2020 <semeval-202...@googlegroups.com>
Subject: Re: Subtask 1 and 2 Results

 

Hi Sasha,

 

Thanks for posting the results. I'm just wondering how is this result calculated? Seems like it's not purely F1 score.

 

Kind regards,

Haotian Xue

On Tuesday, March 10, 2020 at 11:27:35 AM UTC-4, Sasha Spala wrote:

Hello everyone,

Again thank you all for your participation in DeftEval! The official subtask 1 and 2 leaderboards are now available on Github and on the front page of the Codalab site. Because we heard from many of you who did not submit your best score to the leaderboard, or forgot to include your other subtask results in your leaderboard submission, we made the decision to use the best result from each user regardless of whether or not you submitted that result to the leaderboard. If for whatever reason (interesting architecture, alternative method, etc.) you intentionally submitted a different result to the leaderboard, remember that you are still absolutely welcome to submit those ideas to the conference regardless of your leaderboard submission.

 

 

Additionally, if you placed lower than expected in the leaderboard, remember that you are still welcome to make a paper submission to the SemEval conference. We are interested both in high performing models and models that are interesting in architecture, approach, or otherwise significant even though they may score below the top 5, top 10, etc.

 

If you have any questions, please feel free to reach out. We will post the results of subtask 3 as soon as we have the aggregate data together after the submission deadline tomorrow.

Best,

Sasha

--
You received this message because you are subscribed to the Google Groups "DeftEval 2020" group.
To unsubscribe from this group and stop receiving emails from it, send an email to semeval-2020-task...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/semeval-2020-task-6-all/e1927cc1-f0a9-4ab0-a7df-bc7e5b05b2bf%40googlegroups.com.

[hx...@georgetown.edu]

Hi Sasha,

Thanks for clarifying. But it seems like that the ranking result of mine doesn't match my score output.

Kind regards,

Haotian Xue

To unsubscribe from this group and stop receiving emails from it, send an email to semeval-2020-task-6-all+unsub...@googlegroups.com.

[Amr Keleg]

Hi Sasha,

I have made a submission for subtask 1 under the tab Subtask 1&2.

My account name on codalab is AMR-KELEG and I can't find a entry for my system in the scoreboard.

Could you please tell me what went wrong?

Thanks and looking forward to hearing from you.

All the best,

Amr

--

You received this message because you are subscribed to the Google Groups "DeftEval 2020" group.

To unsubscribe from this group and stop receiving emails from it, send an email to semeval-2020-task...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/semeval-2020-task-6-all/7E51B244-173C-4B08-B552-F0B3689FB64A%40adobe.com.