Evaluation of rotated cropped signals
147 views
Skip to first unread message
Felix Krones
Mar 17, 2024, 10:57:33 PMMar 17
to physionet-challenges
Hi Matt,
Sorry for having another one...
I noticed that when images are rotated or cropped, sometimes part of the signals are cut-off if not enough margin is added to the sides. However, the signal files are, I think, not corrected again like before where the signals are filled with nans if, e.g., only the first 2.5 seconds are plotted.
So I was wondering how evaluation will work in that case?
Best regards and thank you as always
Felix
Sorry for having another one...
I noticed that when images are rotated or cropped, sometimes part of the signals are cut-off if not enough margin is added to the sides. However, the signal files are, I think, not corrected again like before where the signals are filled with nans if, e.g., only the first 2.5 seconds are plotted.
So I was wondering how evaluation will work in that case?
Best regards and thank you as always
Felix
PhysioNet Challenge
Mar 17, 2024, 11:02:05 PMMar 17
to physionet-challenges
Dear Felix,
Good question!
We will evaluate the entries based on the printed parts of the signals, which may not always be easy or possible to see, but the entries should still try to reconstruct and/or classify the signals (and not crash) in these cases. It may even be helpful for teams to generate such cases deliberately for training and testing.
In general, parts of some of the signals may be difficult or impossible to see in both real and synthetic ECG printouts because of rotated or cropped papers as well as creases, shadows, writing, and stains on the papers, blurriness of the photographs or scans, etc., so this is a realistic problem that may frustrate most algorithms (and people) at times. However, as you noted, this only happens sometimes, and, of course, each team would have to tackle the same problem.
Best,
Good question!
We will evaluate the entries based on the printed parts of the signals, which may not always be easy or possible to see, but the entries should still try to reconstruct and/or classify the signals (and not crash) in these cases. It may even be helpful for teams to generate such cases deliberately for training and testing.
In general, parts of some of the signals may be difficult or impossible to see in both real and synthetic ECG printouts because of rotated or cropped papers as well as creases, shadows, writing, and stains on the papers, blurriness of the photographs or scans, etc., so this is a realistic problem that may frustrate most algorithms (and people) at times. However, as you noted, this only happens sometimes, and, of course, each team would have to tackle the same problem.
Best,
Matt
(On behalf of the Challenge team.)
Please post questions and comments in the forum. However, if your question reveals information about your entry, then please email info at physionetchallenge.org. We may post parts of our reply publicly if we feel that all Challengers should benefit from it. We will not answer emails about the Challenge to any other address. This email is maintained by a group. Please do not email us individually.
Felix Krones
Mar 18, 2024, 9:43:06 AMMar 18
to physionet-challenges
Hi Matt,
Thank you! Completely clear!
My question was more into the direction of how it will be defined what can be seen and what not.
E.g., at the moment, when I create images, the signal files will be filled with nans if we e.g. trim from 10sec to 2.5sec.
But I didn't see something like that for cases where images are rotated/cropped/shadow is too strong etc.
Since I think basically the difference between signals is used for evaluation, small misalignments in case the beginning of a signal is cropped could make huge differences.
Best Felix
PhysioNet Challenge
Mar 18, 2024, 9:45:09 AMMar 18
to physionet-challenges
Dear Felix,
Yes, for better or worse, we're evaluating the parts of the signal that are printed on the ECG paper (whether real or synthetic), not the parts of the signal that are visible in the image of the ECG paper.
If an image is rotated or cropped so that part of the signal is not visible, then we will not replace these parts of the signal with NaN values, and we will still evaluate these parts of the signal. At the same time, the goal of the Challenge is not to reconstruct or classify invisible parts of ECG signals.
Of course, if you think that the image generation code is producing something that it shouldn't, then please do share it with us directly so that we can take a look.
Best,
Yes, for better or worse, we're evaluating the parts of the signal that are printed on the ECG paper (whether real or synthetic), not the parts of the signal that are visible in the image of the ECG paper.
If an image is rotated or cropped so that part of the signal is not visible, then we will not replace these parts of the signal with NaN values, and we will still evaluate these parts of the signal. At the same time, the goal of the Challenge is not to reconstruct or classify invisible parts of ECG signals.
Of course, if you think that the image generation code is producing something that it shouldn't, then please do share it with us directly so that we can take a look.
Best,
Matt
(On behalf of the Challenge team.)
Please post questions and comments in the forum. However, if your question reveals information about your entry, then please email info at physionetchallenge.org. We may post parts of our reply publicly if we feel that all Challengers should benefit from it. We will not answer emails about the Challenge to any other address. This email is maintained by a group. Please do not email us individually.
Reply all
Reply to author
Forward
0 new messages