Inquiry Regarding Participation in BrainTeaser Task and Dataset Usage

[John Joseph Z]

Hi Organizers,

I hope this email finds you in good spirits. My name is John Joseph, and I am currently pursuing my Master's degree in Computer Science. As part of this semester's curriculum, I am enrolled in a Natural Language Processing course, and I am eager to undertake the BrainTeaser Task as my project for the subject.

While I understand that the deadline for participation in the competition has passed, I have registered for the event and am eagerly awaiting confirmation. Having thoroughly reviewed the problem statement and examined the provided datasets for both training and testing purposes, I find the project both challenging and enriching.

In light of this, I am seeking additional information regarding the evaluation process:

1. Is the evaluation script publicly available? I would greatly appreciate access to any resources or documentation related to the evaluation methodology.

2. How can I obtain the scores for my model? Will there be an opportunity to submit my solution for evaluation within the competition framework?

Any clarification or guidance you could provide regarding these inquiries would be immensely helpful as I proceed with my project.

Thank you for your time and consideration. I look forward to your response.

Warm regards,
John Joseph

[yifan jiang]

Hi John,

Here is a short answer to your question.
1. The evaluation script is not available right now. But you can write your own evaluation script and submit it to Codalab to verify it.
2. Again, the post-evaluation phase never ends on Codalab, so you can submit your system result to it.

Best,

Yifan Jiang