Question about the test set

[Taido Purason]

Hi,

I noticed that some examples in the HSB QA test set (such as C1.1.L1) do not include any context, and their "possible_answers" field contains non-numerical keys. Could this be a formatting issue?

Also, are we allowed to use examples from the development set for few-shot prompting when generating the final outputs?

Best regards,
Taido Purason

[LLMs with Limited Resources for Slavic Languages 2025]

Hi!

Thank you for pointing out, we are on fixing this issue.

Best,

Organisers

воскресенье, 13 июля 2025 г. в 13:57:01 UTC+2, taidop...@gmail.com:

[OKABE Shu]

Dear Taido Purason,

Thank you very much for finding this typo about the non-numerical key!
The HSB C1 test dataset has now been updated on the GitHub repository.

On the questions without context at the C1 level, this is normal (to make that clear, the data now contains an empty string instead of null in the JSON file).

For the few-shot prompting, you are indeed allowed to use the development set.

Good luck with your submission!

Best regards,

Shu Okabe for the organisers

2025年7月13日(日) 13:57 Taido Purason <taidop...@gmail.com>:

--
Вы получили это сообщение, поскольку подписаны на группу "LLMs with Limited Resources for Slavic Languages 2025".
Чтобы отменить подписку на эту группу и больше не получать от нее сообщения, отправьте письмо на электронный адрес slavic-llms-mt2...@googlegroups.com.
Чтобы посмотреть обсуждение, перейдите по ссылке https://groups.google.com/d/msgid/slavic-llms-mt2025/96613ba4-5f98-4c7b-aa3b-64d975602b39n%40googlegroups.com.
Чтобы настроить другие параметры, перейдите по ссылке https://groups.google.com/d/optout.