Task C Held Out Datasets

[Ryn Gray]

Hello,

We are working on Task C, and are wondering if you can provide which of the test sets is in which domain so we can leave out the correct data for cross-domain training.

E.g.  The Biora dataset is probably from the Biology and Life Sciences domain

Confirmation for which domain each test dataset is in would be very helpful.

Thanks,
Ryn

[Hamed Babaei]

Dear Ryn,

Thanks for your question.

Unfortunately, specify the held out ontologies is not possible due to the task objective. Moreover, the challenge policy mentions that it is not allowed to use the test domain ontologies for training for this task. 

To be more specific, the goal is to achieve a transferable taxonomy learner model. So making the test cases known is not in this direction. Moreover, in the task web page we specified the domains that should be ignored from training, while we will select one or multiple ontologies from those domains for testing. And including any ontology from those domains for training even if it is for one domain submission, it conflicts with challenge policy. So, if you trained the model, we recommend to run the inference on all the tests and submit it.

I hope this clarifies your question.

Thank you for bringing this up and good luck with your submission. If you have any questions  please don’t hesitate to contact us.

Best Regards

Hamed

On behalf of the LLMs4OL challenge organization team

--
You received this message because you are subscribed to the Google Groups "LLMs4OL Challenge" group.
To unsubscribe from this group and stop receiving emails from it, send an email to llms4ol-challe...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/llms4ol-challenge/7e35e12f-813e-47ae-b769-2cca8ab73a0bn%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.