Good question! Yes, these variables are correct but confusing. I will add a comment to the code.
For the official phase, we are using two scoring metrics for the two different tasks: a weighted accuracy metric for murmur identification and a cost metric for clinic outcome identification.
However, we can define and compute both scoring metrics for both tasks, but we are only using the clinical outcomes to compute the cost metric for both tasks because the clinical outcome labels are directly related to the treatments and treatment costs for the patients. In this sense, the cost metric effectively evaluates both classifiers' ability to refer patients to a human expert. The clinical outcomes are based on the clinical history, physical examination, analog auscultation, and echocardiogram of each patient, so they are more directly related to whether or not a patient receives treatment than the murmur labels and therefore more directly associated with costs.
(On behalf of the Challenge team.)https://PhysioNetChallenges.org/https://PhysioNet.org/
Please post questions and comments in the forum. However, if your question reveals information about your entry, then please email info at physionetchallenge.org
. We may post parts of our reply publicly if we feel that all Challengers should benefit from it. We will not answer emails about the Challenge to any other address. This email is maintained by a group. Please do not email us individually.