Regarding evaluation of official phase

[Shyamal Dharia]

Hi,

I hope you’re doing well. I have a couple of questions about the official phase of the challenge:

1. With the abstract decision release date (mid-June) approaching, will teams that scored low in the unofficial phase still be allowed to submit entries for evaluation, or only those whose abstracts are accepted?

2. Has evaluation of the official phase already begun, or will it start after the abstract decisions are released?

Thank you for your time.

Sincerely,

Shyamal

[PhysioNet Challenge]

Dear Shyamal,

Teams with high, low, or no scores from the unofficial phase are welcome to submit entries during the official phase, and teams with or without accepted abstracts are welcome to submit entries during the official phase.

The unofficial phase is a "beta test" of the Challenge that helps us to improve it for the official phase. We updated the validation set for the official phase to better assess model robustness and generalizability to unseen data, so many teams are likely to see different, and more realistic, scores for their entries during the official phase. In particular, we encourage teams with "low" scores from the unofficial phase to continue to develop their approaches and submit entries early and often during the official phase.

We have launched official phase, and we are not waiting for the abstract decisions to evaluate entries. We asked teams to wait a week before submitting entries to give the participants the chance to provide feedback, and to make it easier for us to make quick changes as needed, so please share any observations, concerns, and questions as soon as possible.

Best,
Matt
(On behalf of the Challenge team.)

Please post questions and comments in the forum. However, if your question reveals information about your entry, then please email info at physionetchallenge.org. We may post parts of our reply publicly if we feel that all Challengers should benefit from it. We will not answer emails about the Challenge to any other address. This email is maintained by a group. Please do not email us individually.