Pipeline for brightfield 2-plex immunoquantification - Estimate stain vectors

[Carlos Moro]

Hello,

First of all many thanks and congratulations to the whole team for the great, most needed and useful development that is QuPath!!

I would like to set up a pipeline for immunoquantification based on brightfield 2-plex slides, stained with: hematoxylin (counterstaining), DAB, and alkaline phosphatase (AP, red).

The idea is to use DAB for nuclear staining (Ki67) and AP for cytoplasmic (CK19), in order to assess the nuclear staining in cocultures of epihelial (CK19+) and stromal (CK19-) cells.

I read and followed the great documentation but as it's mostly focused on brightfield in single staining (H-DAB) I would be very grateful if you could kindly help me to optimize the pipeline for 2-plex.

In a first step, I'd like to ask about estimating staining vectors. As images are most illustrative I've taken screenshots of every step I followed, attached as pdf.

What do you think about this presented setting? 

Page 3: Just after opening image

Page 4: Image type > "Brightfield (other)"

Page 5: Estimate vectors (initial)

Page 6: Estimate vectors (after clicking "Auto"): In the third scattered the lines are very close and don't include most of points.

Page 7: Script with "Auto" generated  deconvolution vectors

Page 8: First staining, may represent hematoxylin (?)

Page 9: Second, the left panel says DAB, but probably is AP ?

Page 10: Third staining, residuals, may represent DAB (quite faint)?

This is a first attempt by strictly following the documentation. What do you think about its suitability for this 2-plex immunostaining? Would there be possibilities of improving this critical step?

Very grateful for your expert advice.

With very best wishes,

Carlos Fernández Moro

Department of Clinical Pathology,

Karolinska University Hospital/Institute

[Pete]

Hi Carlos,

Thanks very much for your post and very clear description of your application.

At the moment, QuPath does not have great support for more than 2 brightfield stains - I not yet had a similar project to work on, or example data to use to build out this functionality.  Nevertheless, you may still be able to put together something with the current tools.

I cropped out a part of the image from your PDF to try it out.  I put it into ImageJ and set the pixel size in microns based on the scalebar, and then imported it into QuPath.  The resulting image will not exactly match the raw data, but it gives some idea.

I have attached an image showing the results of counting DAB positive and negative nuclei.

'Estimate stain vectors’ can really only handle two stains (usually H&E or hematoxylin & DAB), however there is another way to get stain vectors.  If you have ‘Brightfield: Other’ selected, then you can zoom in to the highest magnification and draw a very small annotation around a region that looks like it contains just one color, then double-click on the stain within the table (several entries below ‘Brightfield: Other’).  You should see a prompt: ‘Set stain vector from ROI?’

If you click ‘Yes’, then the stain is updated based on the pixels in the selected region.  If you click ‘No’, then you have the option to enter the the numbers for the stain vector directly - and also the name of the stain.  Therefore you might want to do both… firstly set the stain from the ROI, secondly set the name (but leave the values unchanged).

That should allow you to enter 3 sets of stain values, but not yet do anything very useful with them.

The conventional ‘Cell detection’ command doesn’t work well with your stains; the red overwhelms the blue.  However, I was able to get ok results using the ‘Fast cell counts’ command applied to the sum of Hematoyxlin + DAB.

The settings I used are below:

setImageType('BRIGHTFIELD_OTHER');

setColorDeconvolutionStains('{"Name" : "H-DAB-AP", "Stain 1" : "Hematoxylin", "Values 1" : "0.54936 0.71051 0.43975 ", "Stain 2" : "DAB", "Values 2" : "0.3976 0.6146 0.68131 ", "Stain 3" : "AP", "Values 3" : "0.12439 0.7547 0.64417 ", "Background" : " 255 255 255 "}’);

runPlugin('qupath.opencv.CellCountsCV', '{"stainChannel": "Hematoxylin + DAB",  "gaussianSigmaMicrons": 2.5,  "backgroundRadiusMicrons": 0.0,  "doDoG": false,  "threshold": 0.2,  "thresholdDAB": 0.6,  "detectionDiameter": 10.0}');

Also, because the default ‘positive’ and ‘negative’ detection colors (red and blue) are not very clear with this staining, I changed them with the following script:

// Change colors

positiveColor = getColorRGB(20, 255, 20)

getPathClass("Positive").setColor(positiveColor)

negativeColor = getColorRGB(20, 200, 255)

getPathClass("Negative").setColor(negativeColor)

fireHierarchyUpdate()

I hope that is of some use to get started.  I also hope to be able to improve QuPath’s handling of multiple stains in brightfield images in the future.  However, I should probably mention that the accuracy of any intensity quantification is already limited in brightfield, and additional stains makes this even more difficult; therefore I would suggest that it is best to restrict any measurements to be counts or densities that can be verified visually, and retain some skepticism about any intensity-based measurements that can be made.

Best wishes,

Pete

[Carlos Moro]

Hi Pete,

Thank you very much for the kind and most useful response!

I'll work on the stain vectors and analysis approach you indicated and will write back with updated results/status and possibly further questions ;)

Best wishes,

Carlos

[micros...@gmail.com]

Oooh, I hadn't played with fast cell counts, the fact that there is a DAB+Hema detection could be very useful in the future.  Any chance we can get that for the regular cell detection? ^_^

My usual pathways for multiple stains ( and I have been dealing with this a lot recently, will try to post something more complete once I am off vacation) include either splitting the project into two and merging the two sets of exported cell detections (one for hematoxylin nuclei, one for DAB) in R, or creating a middle vector that is half hematoxylin and half DAB in order to pick up both sets of nuclei with the normal cell detection command.  

For the latter, I then reset the color vectors to a second set that better represents the DAB and AP (usually using the H&E setting under images since QuPath stores each set of vectors per image type for me, and I can just jump back to HDAB), and run the Subcellular detection command to find regions of positivity within each cell.  Then I use a script to determine whether a given subcellular detection is within the nucleus (though it looks like for Carlos' project that may be unnecessary) and classify the cells according to dye X subcellular detections being in the nucleus or not, and dye Y being in the cytoplasm or not.  Change nuclear size and cytoplasmic expansion to taste.

While this method can work very well for a large set of images and can all be run as a single script, the programming overhead might be a little rough for a small project.

[Carlos Moro]

Hi,

Thank you very much for the great feedback. this is really improving!

Attached next version of the process to deconvolute hematoxylin, DAB and AP, please see attached pdf.

In brief:

Slide 1: Initial image type (Image type: Brighfield H-DAB) and stain vectors.

Slide 2-3: Hematoxylin and DAB+AP stainings

Slide 4: Selected area with (visually) only hematoxylin and DAB > Estimate staining vectors > Auto

Slide 5-6: Selected a small area where (visually) only AP (red)

Slide 7: Image type changed to "Brightfield (other)". Before changing image type I needed to copy to a text file from "Automate > Create command history script" the previously derived vectors for hematoxylin and DAB, because when selecting "Brightfield (other)" they were lost and reset to the initial in slide 1. @Pete, wouldn't it be better to just keep the newly set instead?

              Then as very usefully indicated by Pete's response double click on "Slide 3" in the left panel > "Set stain vector from ROI" > Yes. And voilá it reflected AP. 

              Then double click again on "Slide 3" in the left panel > "Set stain vector from ROI" > No. And without changing the values just change the staining label to "AP".

Slide 8: Result of previous step. Now AP's vector and name are correctly set in the row "Stain 3"  in the left panel.

Slide 9-10: Hematoxylin, DAB, and separate AP stainings    :)

What do you think about this process and results? Any ideas to further improve stain separation? E.g. in Slide 11 (AP) there is some faint staining outside the "true" keratin/cytoplasmic area, there is faint staining in the nuclei of the AP negative cells. Could this be improved in some way?

One possible approach would be perform on these cell samples three separate/individual stainings: 1 slide only with hematoxylin, 1 slide only with DAB and 1 slide only with AP. Then we could try obtain more "pure" staining vectors from these individual slides.

As you well pointed out, the final aim is to differentially count the nuclear staining (like Ki67) differentially in the AP positive and the AP negative cells.

For clarity I will post very soon a separate thread for that task, I think there are already some promising preliminary results that would be great to discuss with you, hopefully useful to other researchers for their pipelines + some simple ideas to improve the great development that is QuPath  ;)

Best wishes,

Carlos

[Pete]

The approach and results so far look good to me.

Using individual stainings to get ‘pure’ stain vectors as described at http://www.mecourse.com/landinig/software/cdeconv/cdeconv.html should help.  I would be really interested to know the extent to which this makes a difference, but I don’t know how much effort is involved in testing that.  Still, I’m not sure if the stain separation could be improved much more using color deconvolution in QuPath (the biggest gains would likely require fluorescence).  In the end, the limitations in accuracy are so many and varied (from the sample, the staining, the scanner, the conversion to 8-bit RGB, the image compression...) that I suspect the improvements over what you now have would be small… but I’d be happy to be proven wrong on that.

Regarding keeping the newly-set stain even when changing image type, that probably would be better… the fact that it changed annoyed me too when I tried setting stains with your image.

I suspect the whole approach to setting/selecting stains needs another look.  I’ve created a GitHub issue to remind me, and give a place to list the potential improvements (including the hematoxylin + DAB cell detection one): https://github.com/qupath/qupath/issues/73 

I’m interested in the next steps of the analysis :)

[Carlos Moro]

Hi Pete,

Thanks a lot! 

I agree, the current status of stain separation is very good  :)

Also curious if setting the vectors in individual stains achieves or not some gain. 

Technically it'd be simple to test. I'll perform the three individual stainings (may take a while) at the same lab, scan with the same scanner and obtain the vectors using the ROI method. I'll then post the results in this thread.

Best wishes,

Carlos

[micros...@gmail.com]

A built in list of stains would be nice, though it is also the sort of thing that, in the mean time, we can build ourselves through saving the script line that sets it. Possibly two lines to set the image type as well.  The project scripts section keeps this nice and clean between sample sets.

I am looking forward to many of the things on that list!  Though I would want, even more than just calculating the third stain vector values for brightfeild other, modifying the Add Intensity Features to allow cytoplasmic and nuclear values so we can freely classify based on new stain vectors without regenerating all of the cells!

[Thomas Kilvær]

Sorry to post in this old thread

-If someone runs into the same problem as me: for some reason the set stain vector from roi only works for a rectangle roi. not from a circle or polygon

And so to my question. Is there any easy way to add cell measurements for stain 2 and 3? DAB seems to be hardcoded without any easy way to change to "Name of stain 1", "Name of stain 2" etc. Any thoughts?

[Pete]

It does!  I hadn't noticed that (I'd always drawn rectangles...).  I'll look into it...

Under the 'Image' tab, if you double-click on the 'Image type' entry you should be able to change to 'Brightfield (other)'.  This then allows you to change the names of the stains by double-clicking on them (unless you have a rectangle ROI, in which case it will offer to set the stain instead...).

Coincidentally, as you wrote this I was finally fixing the annoyance that this thread started with... namely that QuPath's cell detection only makes intensity measurements for Hematoxylin, Eosin or DAB - and not more than two of those, even if a third stain has been set.  It may take a bit of time before it makes it into a new QuPath release, but if you're interested the code for the fix is here.

[Thomas Kilvær]

Thank you a bunch.

A dummies question: do I have to set up eclipse and recompile the entire project or can I just add the class to the process-ij.jar somehow?

I tried to compile just the class, but obviously charged right into dependency issues...

[Pete]

I'm afraid it's the whole thing... at least, that's the easiest way I can think of.  Although if you can somehow compile the relevant jar file (qupath-processing-ij-0.1.2.jar) and replace the old jar in an existing QuPath installation it would probably work for that specific change (although only because I haven't updated the version number yet - that was the next thing to do).

But it is quite handy having it all set up in Eclipse.  Running it in debug mode through Eclipse, you can try things out by changing the source code while the application is still running.  Which is satisfying, when it isn't disastrous.

In my fork (in the same 'bugfixes' branch) I've also updated the stain vectors thing you mentioned previously, so it will now give a more informative message if the shape isn't a rectangle.  But that bit is in a different jar...

You can see the fixes here.  I think it covers most of what was flagged as 'bug' on GitHub.

[Thomas Kilvær]

Wow that worked perfectly after I was able to compile the jars and change them. However. Due to sad programming skills I was not able to set up qupath in eclipse properly. Kept getting an error stating that an import from org.sun.javafx did not work properly. Was able to compile after I just changed it to javafx.... But I was not able to deploy. Will work some more on that.

[micros...@gmail.com]

I too was able to get QuPath to compile in Eclipse, but never got around to figuring out anything past that.  Actually edited the cell detection to use percentages of the various color vectors so that it was easier to detect DAB stained nuclei when there were other colors (AP) present.  So if you figure it out, please share!

[Elijah Edmondson]

I realize this response is very late but I wanted to quickly share a method I used with existing QuPath features to accomplish quantification of two colors mainly because I started in this thread... and could not get the color method to work.  

The two-plex IHC for my project was CD3+ (AP) and CD45R+ (DAB); for the interest of this project, these two markers do not overlap. 

I started by detecting all cells. Then used single cell annotations (about 10 cells from each category: 1 - red, 2 - brown, 3 - neither) and machine learning to separate and quantify populations. The attached screenshot shows representative staining with and without image analysis mask, the 4 lines of code used, and annotation results. 

Hope this is helpful for someone. 

https://drive.google.com/file/d/14exbCAjQpQ1lKstJ88J38eVIKfibASDL/view?usp=sharing

https://drive.google.com/file/d/1fRuCLW-IWNuPlRUlb8jSD1E_kBxv9ix6/view?usp=sharing

- Elijah

[micros...@gmail.com]

Great!  Glad you got it working, and the staining in the first image looks very clean.  You may want to look into adding measurements from more color vectors if your stains are ever more difficult to resolve.  I think I ended up using something like 7 or 8 sets of color vector measurements for 5 stains+hematoxylin (black, teal, AP, purple, DAB).  Then you can either manually design the classifier, or give the automated classifiers more information to work with.  So many options :)