Descriptions of the submissions

[Andrey Kutuzov]

Dear AXOLOTL'24 participants,

We remind you that the shared task is now at the test phase.
You have time until April 9 to submit your system's predictions (it is
possible to make several submissions, if you have several systems, or
use different hyper-parameters).

Some participants have already submitted their test set predictions to
Codalab. It's great, but please remember it's very important that you
accompany your submission with a brief description of your system:
several sentences are enough. We will use this description in the paper
describing AXOLOTL'24 shared task.

We also of course encourage you to submit papers describing your systems
and their performance to the LChange workshop, of which AXOLOTL'24 is a
part. See the details here:
https://www.changeiskey.org/event/2024-acl-lchange/

--
Andrey
Language Technology Group (LTG)
University of Oslo

[Leixin Zhang]

Dear Andrey,

May I ask which of the submissions will be counted in the final ranking if a team has more than one submission? the last submission / the one with the best score / or we can choose ourselves when results are published?

Thanks a lot for your clarification in advance.

Best,

Leixin

--
You received this message because you are subscribed to the Google Groups "AXOLOTL-24" group.
To unsubscribe from this group and stop receiving emails from it, send an email to axolotl-24+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/axolotl-24/fcdaf840-ef72-4b28-a282-10ef278680df%40ifi.uio.no.

[Andrey Kutuzov]

Dear Leixin,

By default, we will use the best submission from each team.

> <mailto:axolotl-24%2Bunsu...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/axolotl-24/fcdaf840-ef72-4b28-a282-10ef278680df%40ifi.uio.no <https://groups.google.com/d/msgid/axolotl-24/fcdaf840-ef72-4b28-a282-10ef278680df%40ifi.uio.no>.

>
> --
> You received this message because you are subscribed to the Google
> Groups "AXOLOTL-24" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to axolotl-24+...@googlegroups.com

> <mailto:axolotl-24+...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/axolotl-24/CAFaFujHMLn_2MfY9Xp9Y4PP3eiWT5KC41Zt0mFD3d2GJvdXeZA%40mail.gmail.com <https://groups.google.com/d/msgid/axolotl-24/CAFaFujHMLn_2MfY9Xp9Y4PP3eiWT5KC41Zt0mFD3d2GJvdXeZA%40mail.gmail.com?utm_medium=email&utm_source=footer>.

[Денис Кокосинский]

Hello! 

In the final ranking we see that our last submission was used instead of the best one. We definetely saw higher scores in the outputs of the scorer for our other test submissions. As I see from the discussion above, the best result among all the submissions per team should be in the leaderboard, not the last one. Could you please clarify the situation?

Best regards,
Denis Kokosinskii

четверг, 4 апреля 2024 г. в 00:51:35 UTC+3, Andrey Kutuzov:

[Tony Santiago Montes Buitrago]

same here, happened the same to us

To unsubscribe from this group and stop receiving emails from it, send an email to axolotl-24+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/axolotl-24/c14e99b5-be88-440b-8d99-8fc734a664e5n%40googlegroups.com.

[Andrey Kutuzov]

Dear Denis and Tony,

Indeed, it was a Codalab misconfiguration. Sorry for this, it's fixed
now, and the leaderboards in the Github repository use the best
submission for each team (where "best" means the highest average score
across three languages).

In Codalab, you are welcome to choose any of your submissions to push
them to the public leaderboard.

Sorry again, and thanks for your patience!

On 11.04.2024 02:43, Tony Santiago Montes Buitrago wrote:
> same here, happened the same to us
>
> El mié, 10 abr 2024 a las 15:04, Денис Кокосинский

> (<deniskok...@gmail.com <mailto:deniskok...@gmail.com>>)

> https://groups.google.com/d/msgid/axolotl-24/fcdaf840-ef72-4b28-a282-10ef278680df%40ifi.uio.no <https://groups.google.com/d/msgid/axolotl-24/fcdaf840-ef72-4b28-a282-10ef278680df%40ifi.uio.no> <https://groups.google.com/d/msgid/axolotl-24/fcdaf840-ef72-4b28-a282-10ef278680df%40ifi.uio.no <https://groups.google.com/d/msgid/axolotl-24/fcdaf840-ef72-4b28-a282-10ef278680df%40ifi.uio.no>>.

> >
> > --
> > You received this message because you are subscribed to the
> Google
> > Groups "AXOLOTL-24" group.
> > To unsubscribe from this group and stop receiving emails from
> it, send
> > an email to axolotl-24+...@googlegroups.com
> > <mailto:axolotl-24+...@googlegroups.com>.
> > To view this discussion on the web visit
> >

> https://groups.google.com/d/msgid/axolotl-24/CAFaFujHMLn_2MfY9Xp9Y4PP3eiWT5KC41Zt0mFD3d2GJvdXeZA%40mail.gmail.com <https://groups.google.com/d/msgid/axolotl-24/CAFaFujHMLn_2MfY9Xp9Y4PP3eiWT5KC41Zt0mFD3d2GJvdXeZA%40mail.gmail.com> <https://groups.google.com/d/msgid/axolotl-24/CAFaFujHMLn_2MfY9Xp9Y4PP3eiWT5KC41Zt0mFD3d2GJvdXeZA%40mail.gmail.com?utm_medium=email&utm_source=footer <https://groups.google.com/d/msgid/axolotl-24/CAFaFujHMLn_2MfY9Xp9Y4PP3eiWT5KC41Zt0mFD3d2GJvdXeZA%40mail.gmail.com?utm_medium=email&utm_source=footer>>.

>
> --
> Andrey
> Language Technology Group (LTG)
> University of Oslo
>
> --
> You received this message because you are subscribed to the Google
> Groups "AXOLOTL-24" group.
> To unsubscribe from this group and stop receiving emails from it,

> send an email to axolotl-24+...@googlegroups.com
> <mailto:axolotl-24+...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/axolotl-24/c14e99b5-be88-440b-8d99-8fc734a664e5n%40googlegroups.com <https://groups.google.com/d/msgid/axolotl-24/c14e99b5-be88-440b-8d99-8fc734a664e5n%40googlegroups.com?utm_medium=email&utm_source=footer>.

>
> --
> You received this message because you are subscribed to the Google
> Groups "AXOLOTL-24" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to axolotl-24+...@googlegroups.com
> <mailto:axolotl-24+...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/axolotl-24/CAMdCj1qKaddJJAWYBFH2-PuoCaCPfgXTD9jq1CxB8eaHJq5zXw%40mail.gmail.com <https://groups.google.com/d/msgid/axolotl-24/CAMdCj1qKaddJJAWYBFH2-PuoCaCPfgXTD9jq1CxB8eaHJq5zXw%40mail.gmail.com?utm_medium=email&utm_source=footer>.