Scoring metrics for this year's challenge

[Ivan H]

Hi, 

I saw this in the abstract guidelines: "using the scoring metrics for this year’s Challenge "

I wanted to confirm that this is AUROC. Also, for the cross-validation — do we do cross-site validation, or just overall folds?

Thank you for your guidance

[PhysioNet Challenge]

Hi Ivan,

Yes, the current metric for the unofficial phase of the Challenge is AUROC. Please report your results on the hidden validation set. It would also be good to report cross-site validated results on the training set as well, but 5-fold cross validation, for example, would be fine for the abstract. As always, please be clear about what you're reporting.

As always, we use the unofficial phase to beta test the Challenge ahead of the official phase. If you observed something interest about the data, the evaluation metric, or your methods, then please share it with us directly and/or in your abstract. (In some cases, we may already be aware, or did it deliberately for the unofficial phase, but please share all the same!) The data, metric, and your methods are likely to change before and during the official phase, but you will still be able to update your work and your description of your work.

The CinC abstract submission deadline is 15th April 2026. Please see our website for advice on writing a strong abstract, and good luck!

http://physionetchallenges.org/2026/#abstracts

Best,
Matt
(On behalf of the Challenge team.)

Please post questions and comments in the forum. However, if your question reveals information about your entry, then please email info at physionetchallenge.org. We may post parts of our reply publicly if we feel that all Challengers should benefit from it. We will not answer emails about the Challenge to any other address. This email is maintained by a group. Please do not email us individually.