Digital Pathology and Image compression formats

[Thomas Schmidt]

Hi everyone.

We've recently acquired a set of scanners that natively stores WSI's using DICOM. Right now we've started looking at what format to store compressed images in and just found out the scanners produce images using JPEG 8-bit Lossy transfer-syntax (1.2.840.10008.1.2.4.50). This is somewhat a concern for us, as we've usually consider lossy compression formats prone to data quality issues. But we are very aware that our experience is from working with radiology for some 15 years almost. Pathology and Radiology are different in many concerns. So my question to all you experts on the forum: Is JPEG 8-bit Lossy transfer-syntax (1.2.840.10008.1.2.4.50) considered "safe" regarding Diagnostic quality for our Clinicians?

Kind regards,

Thomas

[David Clunie]

Hi Thomas

In short, yes, if that is what the regulatory agencies approved/cleared based on the scanner manufacturers' submitted clinical and technical data.

AFAIK, all of the currently marketed WSI scanner devices use some form of lossy compression (whether it be baseline JPEG, irreversible J2K, or some proprietary compression scheme). You can check their 510(k) summaries, though some may not provide much detail. E.g.:

- https://www.accessdata.fda.gov/cdrh_docs/reviews/K232202.pdf

This is similar to the case of Ultrasound carts that lossy compress their cine loops (the standard of care to which they were compared when they were originally introduced was VHS tape).

This is different from the question of retrospectively compressing (or further compressing) what was acquired (from an approved/cleared device).

It also does not mean that this lossy compression off the scanner has no impact on human performance (or machine performance in terms of quantitative evaluation, or AI performance), only that it was deemed to be of sufficient fidelity to get approved/cleared for the intended purpose.

There is a body of literature on the effect of compression on WSI, but such experiments are hampered by the near universal use of already lossy compressed WSI, so many experiments are actually of recompression, which is problematic. It is actually hard to find WSI that have never been compressed to test with (even though some scanners have the capability of acquiring these in "raw" form in research configurations). 

David

PS. Since you probably don't want to be deviating from what was approved/cleared, you need to make sure that your archives store and regurgitate exactly what was acquired.

PPS. Many but not all "DICOM converters" of WSI proprietary formats are capable of "losslessly" converting already lossy compressed images in baseline JPEG or irreversible J2K, by re-using the compressed bitstream and just "rearranging" it into DICOM form. If using such a converter, take extreme care to activate its lossless mode so as to not further degrade the image by decompressing and then recompressing. Further, validate your conversion pipeline by comparing the pixel data obtained from decompressing the proprietary and DICOM-converted formats. There should be no mathematical difference.

PPS. Also, take extreme care with the Photometric Interpretation; if it says RGB not YBR_FULL_422 but is baseline JPEG, that really means the color components have not been converted. One of the well known scanner vendors does this.

[Thomas Schmidt]

Hi David.

Thank you very much for your thorough and well-explained answer (as always I should point out). It provided exactly the insights I was looking for.

Many interesting points.

Kind regards,

Thomas