[Announcement] Paper Submission Guidelines for the LLMs4OL Challenge

[LLMs4OL Challenge]

Dear Participants,

I hope you are doing well.

We are excited to announce the official paper submission guidelines for the LLMs4OL challenge. Please find the detailed guidelines here https://docs.google.com/document/d/1yOWC5j_8wn0rgtMLuac-Gzg3pvuUMbSluZUukzWKW1k/edit?usp=sharing

The guideline includes a paper submission link via EasyChair, a link to the paper template, paper lengths, title prefix instructions, and more. The due date of paper submissions is set to July 28, 2024. Please do not hesitate to reach out if you have any questions or need further clarification regarding the submission process.

We look forward to receiving your contributions!

Best Regards,
Hamed
On behalf of the LLMs4OL Challenge Organization Team

[Carick Atezong]

Dear all,

I am experiencing issues with the evaluation of the GeoNames, cellular, biological, and molecular tasks in the LLMs4OL challenge.

1. I annotated more than 300,000 data points for GeoNames, but received a score of 0.0000 during the evaluation.
2. I submitted again and got the same score.
3. I used another model, but still received a score of 0.0000.
4. Upon manually reviewing the file, I observed that many terms were correctly predicted with the appropriate types. However, I am unsure why my score remains 0.0000.

Could you please provide some insights or guidance on what might be causing this issue? 

Additionally, could you explain the evaluation process and criteria used for scoring?

Thank you for your assistance.

Best regards,

Carick Appolinaire ATEZONG YMELE

[pankaj Goyal]

sir same thing happening with me in the A.5 zero shot task as identified my term type of most of the terms are correct but still it is giving output to be zero.

--
You received this message because you are subscribed to the Google Groups "LLMs4OL Challenge" group.
To unsubscribe from this group and stop receiving emails from it, send an email to llms4ol-challe...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/llms4ol-challenge/7ef072b7-da94-4319-92ce-da45cbd78cf1n%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.