Other metrics

[Bjørn-Jostein Singstad]

Dear organizers

Would it be possible to get some scores from other metric based on the performance on the hidden validation set other than the challenge score? Would be nice to also look at some metrics that we know from before such as AUROC, accuracy, sensitivity and so on.

Best regards

Bjørn-J

[PhysioNet Challenge]

Dear Bjørn-J,

Yes, the scoring code computes other scoring metrics, but we have only been sharing the Challenge scoring metric for now to emphasize Challenge objective.

We will share more traditional scores later in the Challenge for your papers, but the easiest way to get the other scores right now is to perform cross-validation on the training data and run the scoring code on the results.

Best,
Matt
(On behalf of the Challenge team.)

https://PhysioNetChallenges.org/
https://PhysioNet.org/

Please post questions and comments in the forum. However, if your question reveals information about your entry, then please email challenge at physionet.org. We may post parts of our reply publicly if we feel that all Challengers should benefit from it. We will not answer emails about the Challenge to any other address. This email is maintained by a group. Please do not email us individually.

[Bjørn-Jostein Singstad]

Great!

Yes, I see the point why you want to focus on the Challenge metric.

We had a case were the score on the validation set was a lot better than what we achieved during 10-fold CV on the training set. 

Would be interesting to see if this also would be the same if we used AUC, accuracy or other metrics.

Look forward to get the other metrics in the offical phase then.

BR

Bjørn-J