Launching the official phase for the PhysioNet Challenge 2025

[PhysioNet Challenge]

Dear Challengers,

We are pleased to announce the beginning of the official phase of the George B. Moody PhysioNet Challenge 2025: Detection of Chagas Disease from the ECG:

https://physionetchallenges.org/2025/

For the official phase of this year’s Challenge, we have made many changes to improve the accessibility of the Challenge, including:

• a larger training set with more positive cases of Chagas disease;
• a larger and more realistic validation set that better assesses the robustness and generalizability of your approaches, and, hopefully, helps approaches to succeed on the test set;
• simplifications to the example code; and
• simplifications to the scoring code, including fixes to a minor edge case for “tied” records, but is otherwise the same as the official phase.

These changes, made in response to our observations and your feedback during the unofficial phase, help to improve the Challenge, but we have not changed the fundamental task of the Challenge: using ECGs to prioritize patients for confirmatory serological testing given a limited testing capacity.

Your code from the unofficial phase may not run as-is in the official phase, but you should be able to run it with minor updates. You do not need to re-download the raw data for the training set, but you should reprocess the raw data for your training code. Please see the example and scoring code for updates:
https://github.com/physionetchallenges/python-example-2025
https://github.com/physionetchallenges/matlab-example-2025
https://github.com/physionetchallenges/evaluation-2025

We will start evaluating official phase submissions after about a week to allow for questions and comments about these changes (and potential bug fixes), so please share any feedback that you may have as soon as possible.

Please see the Challenge webpage for details, update and double check your code to include all of the promising ideas that you've had since the end of the unofficial phase, and submit it for evaluation on the validation set when ready (but after a week or so to allow for feedback and potential changes in response to feedback):
https://physionetchallenges.org/2025/submissions/

The reviews of your Computing in Cardiology abstracts are nearly finished. We look forward to sharing the results with you in the coming weeks and seeing you and discussing your work in Brazil in September.

Best,
The PhysioNet Challenge Organizers

---

Please post questions and comments in the forum. However, if your question reveals information about your entry, then please email info at physionetchallenge.org. We may post parts of our reply publicly if we feel that all Challengers should benefit from it. We will not answer emails about the Challenge to any other address. This email is maintained by a group. Please do not email us individually.

[Angus Nicolson]

Hi,

Thank you for the update! Looking forward to taking part in the official phase. 

I have a question regarding the updated validation set. In the training set, the strong positive and negative labels come from different datasets (SamiTrop and PTB-XL, respectively). Could you clarify whether the same is true for the validation set, or whether both positive and negative cases are drawn from the same underlying dataset(s)?

This distinction could have a substantial impact on the difficulty of the validation task and on how well validation performance predicts test performance. Especially as distinguishing between datasets can be easier than detecting label-specific features within a single source, so knowing the dataset composition is important for interpreting results.

Best,

Angus Nicolson

[PhysioNet Challenge]

Dear Angus,

Thanks for the kind words and good observations.

For the official phase, the data in the validation and test sets are from different sources than the data in the training set. In particular, the validation and test sets do not contain data from the CODE-15%, the SaMi-Trop, or the PTB-XL datasets, which comprise the training set. Like you note, it can be easier to distinguish between datasets than to detect relevant features within a dataset, and learning and using differences between specific datasets may or may not generalize well to the new datasets in the validation and test sets.

Best,
Matt
(On behalf of the Challenge team.)

Please post questions and comments in the forum. However, if your question reveals information about your entry, then please email info at physionetchallenge.org. We may post parts of our reply publicly if we feel that all Challengers should benefit from it. We will not answer emails about the Challenge to any other address. This email is maintained by a group. Please do not email us individually.

[Angus Nicolson]

Hi Matt,

Thank you for the response, but I think I need to clarify my question slightly. 

Is the validation set composed of multiple datasets? If so, do the positive and negative samples come from different datasets? 

Likewise, it would be good to know for the test set (if you can share that information).

Best,

Angus

[PhysioNet Challenge]

Dear Angus,

For the official phase, the validation and test data have multiple sources, and each source has both positive and negative records. The training set has sources that only contain positive or negative records, and teams should think about how to use them effectively, but teams should not expect the same from the validation or test set.

The validation data for the official phase is different from the validation data for the unofficial phase, and it is now more representative of the test data.

Best,

Matt

(On behalf of the Challenge team.)

Please post questions and comments in the forum. However, if your question reveals information about your entry, then please email info at physionetchallenge.org. We may post parts of our reply publicly if we feel that all Challengers should benefit from it. We will not answer emails about the Challenge to any other address. This email is maintained by a group. Please do not email us individually.

[Tianzheng Dong]

Hi,

I am a bit confused about your statement regarding the modification of the validation set. My understanding was that during the training phase, available data is provided to the training function (data_folder), and then the data is split into the training set and validation set by the splitting logic in my team's code, which is used for the subsequent training.

Could you please explain how you define and use the validation set you mentioned?

Thank you for your clarification.

Best regards,
Tianzheng

[PhysioNet Challenge]

Dear Tianzheng,

People sometimes use the terms "training set", "validation set", and "test set" differently.

For the Challenge, we share the training set publicly, and we keep the validation and test sets privately so that we can evaluate the entries in a principled way. We score each entry on the validation set during the Challenge, and we score at most one entry from each team on the test set at the end of the Challenge. If you see information that suggests otherwise, then please let us know so that we can clarify it.

Please see the description of the data on the Challenge website for more information about training, validation, and test sets:
https://moody-challenge.physionet.org/2025/#data

Your team can use the training set in however you would to train your model, including by splitting the training set into multiple subsets to help choose model parameters or hyper parameters. In fact, we encourage teams to cross-validated performance on the training set. However, to reduce confusion with the actual validation and test sets, we ask teams not to refer to any part of the training set as a "validation set" or "test set", e.g., an "internal validation set" or "local test set", when they describe their approaches.

Best,
Matt
(On behalf of the Challenge team.)

Please post questions and comments in the forum. However, if your question reveals information about your entry, then please email info at physionetchallenge.org. We may post parts of our reply publicly if we feel that all Challengers should benefit from it. We will not answer emails about the Challenge to any other address. This email is maintained by a group. Please do not email us individually.