Simulab

[Bjørn-Jostein Singstad]

Dear Organizers

Since our team got "No" under the header "Working training code?" in the summary file I assume that the problem with our  code was that this is what happened:

If your training code crashed when we modified the labels, then it appeared that your training code was very sensitive to the choice of training set (or had intentionally or unintentionally hard-coded some aspect of the training set).

I think it might be because we hard-coded the labels in team_code.py. I agree that this is not the most elegant way to do it, but I also think you should have been more transparent about how you were going to test the code and give us a better warning that you were going to mix up the labels in the test stage.

Your challenge setup is already quite advanced compared to many others (using docker, etc.) and therefore requires quite a lot from the participants. To foresee that you were going to test something that was outside the scope of the challenge is simply too much to ask for in my opinion.

My team, Simulab, managed to achieve a score on the training set, validation set and test set and therefore I think you should consider moving us back to the official score board.

Best regards

Bjørn-Jostein