Clarification on dataset transferlearning
96 views
Skip to first unread message
Martin Sondermann
Apr 8, 2025, 10:09:08 AMApr 8
to physionet-challenges
Dear Challenge Team,
While the FAQ mentions that transfer learning from external data sources is encouraged, I'd like to understand a following example scenario is permitted:
1. During development, training and preparing an embedding extractor on the CODE-15 dataset
2. Including this pre-trained model in the submission, along with the pre-training code, but not the goal to run it as it's precomputed.
3. During Challenge evaluation, the main training code would load this pre-trained model and continue training on the Challenge training data as per usual
The key difference from typical transfer learning is that pre-training on part of the Challenge data itself rather than external datasets.
My specific question: Does this approach qualify as permitted "transfer learning," and more importantly, does the time spent pre-training on the CODE-15 dataset before submission count toward the 72-hour computation limit, when it's following a different methodology than the later training steps.
Thank you for clarifying,
Martin
While the FAQ mentions that transfer learning from external data sources is encouraged, I'd like to understand a following example scenario is permitted:
1. During development, training and preparing an embedding extractor on the CODE-15 dataset
2. Including this pre-trained model in the submission, along with the pre-training code, but not the goal to run it as it's precomputed.
3. During Challenge evaluation, the main training code would load this pre-trained model and continue training on the Challenge training data as per usual
The key difference from typical transfer learning is that pre-training on part of the Challenge data itself rather than external datasets.
My specific question: Does this approach qualify as permitted "transfer learning," and more importantly, does the time spent pre-training on the CODE-15 dataset before submission count toward the 72-hour computation limit, when it's following a different methodology than the later training steps.
Thank you for clarifying,
Martin
PhysioNet Challenge
Apr 8, 2025, 10:12:07 AMApr 8
to Martin Sondermann, physionet-challenges
Dear Martin
You may pre-train your model on any data (except our hidden test data - which we hope you don't have!), including the CODE-15 data.
While we encourage parsimonious algorithms (and training of them) to minimize your carbon footprint, the computational restriction on our end is largely a practical limitation - we have a finite compute capacity. The time spent pretraining your model does not count towards your 72-hour limit.
The key is that you provide code that performs some meaningful training and improves your performance so others can re-implement and copy your work.
We will perform tests on your code to ensure that the training actually changes the performance of your code and that you are not just uploading a fully trained model.
All the best
Gari
(On behalf of the Challenge team)
https://physionetchallenges.org/
Please post questions and comments in the forum. However, if your question reveals information about your entry, then please email in...@physionetchallenges.org. We may post parts of our reply publicly if we feel that all Challengers should benefit from it. We will not answer emails about the Challenge to any other address. This email is maintained by a group. Please do not email us individually.
---
--
You received this message because you are subscribed to the Google Groups "physionet-challenges" group.
To unsubscribe from this group and stop receiving emails from it, send an email to physionet-challe...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/physionet-challenges/eb3fe34e-4a57-4312-b710-8e99835c419an%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages