Hi, here are a few more thoughts/tips related to parameters that might help with the detection:
- If one nucleus is being wrongly subdivided into multiple nuclei, try increasing either the median filter or the sigma. Both smooth the image, but in slightly different way (the median filter removes outliers a bit more cleanly, while the sigma refers to a Gaussian filter that tends to be a bit less artefact-y)
- If too many nuclei are being detected, try increasing the Threshold value and/or the Minimum area
- If you have very large/clustered nuclei, you might need to increase the Maximum area and you could also try increasing the background radius - or set it to 0 to avoid trying to subtract background at all
Doing that, you can compare the hematoxylin and DAB images. If you can see all the nuclei clearly in the hematoxylin image, then I'd try setting the detection image (top parameter in cell detection, drop-down box) to be Hematoxylin OD; otherwise Optical density sum can help because it will just use all staining - regardless of color. It's worth trying both anyway.
Finally, in your case I think you might want to decrease sigma and increase the median radius - in addition to adjusting the threshold. It looks like lots of small 'brown things' are being detected as nuclei - but with large white spaces outside, which suggests they have been blurred out a lot before detection (which is where the sigma comes in).
If you find that is unavoidable, there is the option to remove cells based on specific measurements - or combinations of measurements - in a script. For example, I would guess that these objects have low values for both hematoxylin and for DAB. Here's a script that would remove cells with low values of both:
detections = getDetectionObjects()
print 'Number of detections before: ' + detections.size()
lowDAB = detections.findAll {return measurement(it, 'Nucleus: Hematoxylin OD mean') < 0.1 && measurement(it, 'Nucleus: DAB OD mean') < 0.1}
removeObjects(lowDAB, false)
detections = getDetectionObjects()
print 'Number of detections after: ' + detections.size()
Of course the cutoff values I've chosen here (0.1 in both cases) could be very wrong... it could require some use of File -> Save before and File -> Revert afterwards to play around with alternative values.