How to create label directory

[Jason Wang]

Dear organizers,

Based on the description on https://github.com/physionetchallenges/evaluation-2021, I should create a directory for label files for each recording?

Is the file in csv and names after the record ID?

And the files should like this?

```

#Record ID 164889003, 270492004, 164909002, 426783006, 59118001, 284470004, 164884008, 429622005, 164931005 1, 1, 0, 0, 0, 0, 0, 0, 0 0.9, 0.6, 0.2, 0.05, 0.2, 0.35, 0.35, 0.1, 0.1

```

Thanks,

Jason

[PhysioNet Challenge]

Dear Jason,

The evaluation scripts load labels from the header files (*.hea). The header files for each of the training databases contain labels. You do not need to modify the header files or remove the signal files (*.mat) from the folders with the header files.

For example, if you have two folders, dataset_1 and dataset_2, that each have header and signal files from the training data, then you can run the following commands to train your classifier on dataset_1, run your trained classifier on dataset_2, and evaluate your classifier outputs using the labels from dataset_2:

Python commands:
    python train_model.py dataset_1 model
    python test_model.py model dataset_2 outputs
    python evaluate_model.py dataset_2 outputs

MATLAB commands:
    train_model('dataset_1', 'model')
    test_model('model', 'dataset_2', 'outputs')
    evaluate_model('dataset_2', 'outputs')

If you have trouble running these commands with your code, then please try to run the example code first:
https://github.com/physionetchallenges/python-classifier-2021
https://github.com/physionetchallenges/matlab-classifier-2021

Best,
Matt

(On behalf of the Challenge team.)

https://PhysioNetChallenges.org/
https://PhysioNet.org/

Please post questions and comments in the forum. However, if your question reveals information about your entry, then please email challenge at physionet.org. We may post parts of our reply publicly if we feel that all Challengers should benefit from it. We will not answer emails about the Challenge to any other address. This email is maintained by a group. Please do not email us individually.

[Jason Wang]

Thanks Matt. I think my question is if I tested the model on a local cross-validation set and my model was not the default random forest (neural network instead for example), how could I create the labels and used the evaluation script?

Also for some labels, they both include the normal case ("426783006") and other morbid cases. Is that normal?

Jason

[PhysioNet Challenge]

Dear Jason,

To perform cross-validation on the training data, you can put recordings (signal and header files) in different folders. You do not need to create labels for the training databases, which already contain labels. (The evaluation script simply extracts the labels on the line of each header file that starts with "#Dx:", so if you wanted to create new labels for some reason, then you could change this line or create a file with this single line.)

It sounds like you may also be asking about how to create a new model instead of using or adapting the models in the example code. You can modify the team_code.py script or the team_training_code.m and team_testing_code.m scripts to create a new model. Please do not modify the train_model or test_model scripts, which ensure that your code loads data and formats results correctly for evaluation. The README files of these repositories provide details about how to write your own code:
https://github.com/physionetchallenges/python-classifier-2021
https://github.com/physionetchallenges/matlab-classifier-2021

You are welcome and encouraged to try new models. You do not need to use the models in the example code, which we provide as examples of how to write code for the Challenge so that we can run it. These examples are designed to run quickly but not to perform well.

The labels for some recordings do include sinus rhythms and other diagnoses. Please see this discussion for more details:
https://groups.google.com/g/physionet-challenges/c/eNc26q2luM4/m/gLUu0vRlBgAJ

Best,
Matt

(On behalf of the Challenge team.)

https://PhysioNetChallenges.org/
https://PhysioNet.org/

Please post questions and comments in the forum. However, if your question reveals information about your entry, then please email challenge at physionet.org. We may post parts of our reply publicly if we feel that all Challengers should benefit from it. We will not answer emails about the Challenge to any other address. This email is maintained by a group. Please do not email us individually.