Clarification Needed on Challenge Participation and Submission Process

[Zachary L]

Hi Matt,

I am writing to seek clarification regarding the participation process in the two distinct parts of the challenge: the digitization and the classification part.

Upon attempting to submit my entry, I noticed the submission form did not provide an option to select which part of the challenge my team intends to participate in. Could you please explain how the decision is made regarding which segment a team is assigned to?

Furthermore, I observed that the scoring script appears to only evaluate entries for the classification segment in the absence of a reconstructed signal. I am unsure if this is the method used to determine a team's entry category.

Given these uncertainties, would it be necessary for me to establish two separate repositories, one for each challenge segment, and submit them individually to participate fully in the challenge?

I appreciate your guidance on these matters to ensure my team's compliance with the challenge's requirements.

Thanks,

Zisheng 

[PhysioNet Challenge]

Dear Zisheng,

Thanks for these questions.

As you noted, this year's Challenge has two tasks: digitization and classification. Each team can complete either or both of these tasks.

As of the unofficial phase, we're not asking teams to indicate which task or tasks they're completing when they submit their entries. You can simply choose not to complete one of the tasks, e.g., by returning NaN values, and we won't score that task.

For the official phase, we may ask teams to indicate which tasks or tasks they're completing when they submit their entries, depending on our observations about the entries and your feedback about the Challenge. We may also change the structure of the code to make it easier for teams to complete one or both tasks.

Could you please clarify your observation about the scoring script? If your code does not complete one of the tasks for any of the records, then the scoring code is designed not to evaluate the performance of your model for that task.

Best,

Matt

(On behalf of the Challenge team.)

Please post questions and comments in the forum. However, if your question reveals information about your entry, then please email info at physionetchallenge.org. We may post parts of our reply publicly if we feel that all Challengers should benefit from it. We will not answer emails about the Challenge to any other address. This email is maintained by a group. Please do not email us individually.

[Zachary L]

Hi Matt,

Thank you so much for your clear and comprehensive explanation regarding my previous inquiries. It has significantly helped in understanding the participation process for the challenge.

I apologize if my previous description of my observations with the scoring script led to any confusion. My findings align with your explanation: "If your code does not complete one of the tasks for any of the records, then the scoring code is designed not to evaluate the performance of your model for that task."

I have one additional question. For teams interested in participating in both parts of the challenge during the official phase, is it necessary to submit an entry that successfully scores in both parts? Or is an entry that achieves a score in either part considered sufficient for participation?

Thank you once again for your guidance.

Best,
Zisheng

[PhysioNet Challenge]

Dear Zisheng,

If you are interested in participating in either or both tasks (digitization and classification) for the Challenge, then you are welcome but not required to complete both tasks in a single entry, and you may or may not find that digitizing an ECG helps you to classify it. A successful entry in either or both tasks during the unofficial phase satisfies the rankings and prize criterion for successful participation during the unofficial phase.

Best,
Matt
(On behalf of the Challenge team.)

Please post questions and comments in the forum. However, if your question reveals information about your entry, then please email info at physionetchallenge.org. We may post parts of our reply publicly if we feel that all Challengers should benefit from it. We will not answer emails about the Challenge to any other address. This email is maintained by a group. Please do not email us individually.