Adding additional stains to multicolor IHC

[Finn]

Hi

I have a project with 4 colors (hematoxylin + 3). I am able to set vectors for all of these (setColorDeconvolutionStains() ), but I'm unclear if QuPath is actually using the fourth stain for anything (deconvolution/measurement), as it doesn't seem to show up anywhere. I was hoping someone might clarify if this is possible/appropriate, and what a best practices for additional stains might be. Thanks!

[micros...@gmail.com]

The color vectors are a group of 3 that ideally take the place of the RGB values.  These values do not necessarily add up correctly in the case of the Brightfield(Other) image type, or if you assign all three vectors, as normally the third "residual" vector is calculated based on the other two.

The best way I have found to handle 3+ colors is to do my best to isolate the color I am interested in, apply measurements to the cells based on that color vector set, then change to focusing on another color vector.  For example, when dealing with just H-DAB-AP, I would set one color vector to AP (red) and the second would be a mix of H-DAB, with the residual balancing everything out.  I would then calculate what features I wanted to study for each cell (frequently Analyze->Calculate Features->Add Intensity Features) before changing the first color vector to DAB, the second to a mix of H-AP, and leaving the third to again balance the calculations.  It was very useful to use the 1-2-3-4 keys during this process to quickly check how well I was isolating my dye of interest.  DAB was, of course, frequently a problem I had to work around since dark brown is also most colors.  You can try assigning all three color vectors to try to pin down your color of interest for a given Add intensity features run, but expect... odd behavior if you do that.  I am also not 100% certain what the data that results from this actually means.  Pete probably knows more.

Exporting each cell to ImageJ in a macro might also give you access to a greater variety of color deconvolution or thresholding tools.

Some references:
http://www.mecourse.com/landinig/software/cdeconv/cdeconv.html
https://petebankhead.gitbooks.io/imagej-intro/content/chapters/colors/colors.html

[Finn]

Thanks!

I'm still new to image analysis so haven't quite pinned this all down yet. I hadn't considered that 3 colors would be fundamentally different from 4, but I guess it makes sense. I've essentially been doing what you've recommended, looking at H + 1 or 2 stains at a time, but I've been unsure that this is optimal, as I know that commercial platforms are unmixing more stains... I'll have to look into it a bit more...

[micros...@gmail.com]

I am still somewhat new myself, so if you do find anything with code or a reference to a larger number of channels in brightfield, please post it!  I took a quick look around for something that would do more than 3 channels (outside of fluorescence, of course) but was not able to find anything.  I suspect if you had a detector that picked up more than 3 channels that unmixing into more channels would be far easier; I think the issue is most brightfield images being analyzed start with only 3 channels (at the detector).  So attempting to "create" more than 3 channels ends up just being an exercise in 3 color deconvolution and then thresholding, which is basically what you are doing now with isolating values for a "best" color vector of interest.  Also, when outputting the colors to the screen, they tend to go back into an RGB/HLS format.

Of course, I could be missing something key and be way off!

[Pete]

I'm afraid there isn't much in QuPath directly to help with brightfield images having more than three stains... and even three is a bit of a stretch.  There is an issue on GitHub about improving support for the third stain: https://github.com/qupath/qupath/issues/73

One part of the explanation is that when developing QuPath I never had a project to work on in which the images had more than two stains... indeed, for brightfield I worked pretty much exclusively with H&E or hematoxylin & DAB.

I think support for more stains is one of the main areas needing improvement in the relatively short term.  In the meantime, I'm afraid that creative workarounds such as those described above are required to handle the images in QuPath.

Somewhat relatedly, I'm currently thinking about the more general question of 'what's next for QuPath?'  Plans are forming for the new year. I intend to write more about this soon, but in the meantime feel free to send me an email if you'd like to discuss further... or influence the thoughts.

[Finn]

Thanks both of you --

Micros -- you've inspired me to go back to the literature a bit to try to get a better handle on this. I think you're right about the challenges with more than 3 stains, but it looks like solutions to this are starting to emerge (http://stmi2014.ece.cornell.edu/papers/STMI-O-2.pdf) -- I guess it remains to be seen how good they are. Multispectral images, I would think, are a separate issue which I wouldn't expect QuPath to support anytime soon.

Pete -- thanks for your willingness to help on this! I definitely have some ideas for things that would be very helpful (at least for my work) -- I'll start to keep some notes and get in touch down the road. 

[micros...@gmail.com]

That is a great find!  It has interesting similarities on page 2 to what we were discussing, although they apply the differing color vectors pixel by pixel, rather than across the whole image multiple times. 

Also it looks like what they are performing the a similar analysis to I was recommending on a pixel by pixel basis again (page 3/4), rather than on a cell by cell basis. The cell by cell measurements are more or less the result of what I recommended above when you cycle through each color vector set and apply the stain intensity measurements in turn.  If you only keep the stain measurements that are "appropriate" for a given portion of the cell, it might be pretty close.  I'll need to read more about the group sparsity unmixing though.  Thanks again!