Announcing the 2023 PhysioNet Challenge

[PhysioNet Challenge]

Dear Community,

We are delighted to announce the opening of the George B. Moody PhysioNet Challenge 2023. This year’s Challenge invites teams to use electroencephalogram (EEG) recordings to predict the neurological recovery of patients from coma after cardiac arrest.

We have shared example code and scoring code in both MATLAB and Python, and we will open the scoring system later in February, so please be on the lookout for an announcement soon!

See the Challenge website for more information, rules and deadlines: https://physionetchallenges.org/2023/

As in previous years, we have divided the Challenge into two phases: an unofficial phase and an official phase. The unofficial phase solicits feedback from the research community (i.e., you) to help us to improve the Challenge for the official phase, so we require teams to register and participate in the unofficial phase of the Challenge to be eligible for a prize. Please enter early and often – we need you to look for quirks in our data, our scoring system, and otherwise. We are imperfect (and bandwidth-limited), so we rely on the community to help us to improve the quality of the Challenge each year.

More information will be posted on the PhysioNet Challenge website and the Challenge forum as it becomes available. Please post questions and comments to the Challenge forum as well. However, if your question reveals information about your entry, then please email info [at] physionetchallenge.org instead to help us safeguard the diversity of approaches to the Challenge. We may post parts of our replies publicly if we feel that all Challengers should benefit from the information contained in our responses. We will not answer emails about the Challenge sent to other email addresses.

Many thanks again for your continued support of this event, and we hope that you enjoy the 2023 Challenge!

Best,
Matthew A. Reyna, Edilberto Amorim, Michel J. van Putten, Jeanette Hoffmeijer, Nicolas Gaspard, Adithya Sivaraju, Susan Herman, Jong Woo Lee, M. Brandon Westover, Gari D. Clifford
(On behalf of the Challenge team. )

[Felix Krones]

[Error or evaluate_model.py]

Dear Challenge team,

Thank you so much for organising the challenge again! We are already excited to participate again!

Is it possible that there is a small copy-paste error in the evaluate_model.py file?

For both, the TPR and the FRP in line 82 and 83, the same denominator is used float(tp[j] + fn[j])

Best Felix

[PhysioNet Challenge]

Dear Felix,

Yes, you are correct: I had a copy-and-paste error in the evaluate_model.py file. (If we had a prize for the first bug fix, then your team would win it!)

I have updated the evaluation code repository with the corrected formula. Please pull or download the repository for these updates:
https://github.com/physionetchallenges/evaluation-2023

For people who are new to the Challenge, we welcome (and need!) your feedback, whether it is to help with minor bug fixes or to raise larger questions or concerns. The unofficial phase of the Challenge helps us to "beta test" everything, include your code and ours.

On that note, what do people think of this year's scoring function? Please note that we are asking you to predict patient outcomes, but we are actually scoring the *probabilities* for your predictions. Is this a good idea?

We are aware that some of you are having trouble downloading the Challenge data, and we will host the data in a more convenient location soon -- please check back soon. If you are encountering a "PaxHeader" error, then this is an issue with how Windows decompresses the tar.gz file that we shared -- please delete the "PaxHeader" directory, and everything will work.

Best,
Matt

(On behalf of the Challenge team.)

Please post questions and comments in the forum. However, if your question reveals information about your entry, then please email info at physionetchallenge.org. We may post parts of our reply publicly if we feel that all Challengers should benefit from it. We will not answer emails about the Challenge to any other address. This email is maintained by a group. Please do not email us individually.

[ismail]

Hi,

Thank you for hosting this challenge! The dataset and problem is super interesting.

I had a couple of questions about the data, specifically about the .tsv file quality scores.

1. For the quality score, was it some automated process that looked for artifacts and created this score or was each 10 second epoch manually checked by a person to check for artifacts?

2. Will the .tsv quality scores be available in the test set? I don't see it stated anywhere on the challenge web page. I was thinking if this challenge is to mimic a real life practical application of some kind of model then we wouldn't have access to these quality scores unless we were to make our own automated artifact detection method? 

Sincerely,

Ismail E. 

[PhysioNet Challenge]

Dear Ismail,

Thank you for the kind words about the Challenge!

Yes, we used an automated process to score the recordings for artifacts, so we will provide the quality/artifact scores with the training, validation, and test sets. We will add more details about this process to the website.

Of course, you do not need to use the provided quality scores, and you are welcome to check for artifacts as part of your code.

Best,
Matt
(On behalf of the Challenge team.)

Please post questions and comments in the forum. However, if your question reveals information about your entry, then please email info at physionetchallenge.org. We may post parts of our reply publicly if we feel that all Challengers should benefit from it. We will not answer emails about the Challenge to any other address. This email is maintained by a group. Please do not email us individually.