Clustering of an image by taking into account the spatial context of each pixel (besides its intensity)

[Hakim Benoudjit]

Hi,

Is there a clustering algorithm implemented in scikit-image that perform the image clustering by taking into account the spatial context of the clustered pixel (its neighbourhood), besides its pixel brightness?

For the time being, I'm clustering images by reshaping them as vectors of pixels intensities distributions, and then performing the K-means or Gaussian mixture models implemented in scikit-learn. But, I'm looking for a image clustering technique implemented (or could be implemented) in scikit-image that would consider the neighbourhood of a pixel when classifying it.

Thanks.

[Stéfan van der Walt]

Hi Hakim

Are you looking for a metric? Perhaps consider structural similarity index.

Regards
Stéfan

--
You received this message because you are subscribed to the Google Groups "scikit-image" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image...@googlegroups.com.
To post to this group, send email to scikit...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/scikit-image/d6eeb2b6-2abc-40c0-9c15-17185731f414%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Hakim Benoudjit]

Hi Stéfan,

Thanks for your reponse.
What I'm looking for is a spatial criteria that encourages the clustering algorithm (K-means or others) to group together similar neighbouring pixels inside the same cluster. This will help avoid having persistent noise inside a cluster.

[Juan Nunez-Iglesias]

Hey Hakim,

The right answer here depends on your ultimate goal. If you're after denoising, non-local means denoising (recently added to skimage) sounds like exactly what you're after.

Juan.

--

You received this message because you are subscribed to the Google Groups "scikit-image" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image...@googlegroups.com.
To post to this group, send email to scikit...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/scikit-image/0aad2045-b9da-442c-97bc-06c596b0469e%40googlegroups.com.

[Hakim Benoudjit]

Hi Juan,

Thanks for your answer, this seems to be a nice algorithm for the denoising of speckle.
But actually I'm looking for an image clustering (segmentation) technique instead (that would take into consideration the spatial context of pixels).

[Jonas Wulff]

Hi Hakim,

Have you tried just adding the coordinates of a pixel to its features? For each pixel, the features would then be R,G,B,X,Y. From your description, that seems what you're looking for.

So if you have an RGB image I (so that I.shape = (height,width,3)), you can do:

y,x = np.mgrid[:height,:width]

I_stacked = np.dstack((I,x,y))

data = I_stacked.reshape((-1,5))

... and then use "data" as input to your clustering algorithm.

You might want to scale / normalize the coordinates to fit the general range of your color values -- but in general, this should do what I think you're looking for.

Cheers,

-Jonas

To view this discussion on the web, visit https://groups.google.com/d/msgid/scikit-image/a2895510-2490-4ccf-a70a-20d67c74d2cd%40googlegroups.com.

[Hakim Benoudjit]

Hi Jonas,

Thanks for your response.
That's exactly what I've tried this week-end, by adding the (x, y) to gray-level intensity and giving the matrix of 3-components vector as input to k-means.
As for the normalization, I applied this formula to each column (intensity, x, y): (value - mean) / std_dev.
But, even with this normalization step, adding the (x, y) coordinates will influence the pixels on the left (resp. right) to be grouped together (See http://imgur.com/HxfkRig and original image taken from http://uk.mathworks.com/help/images/texture-segmentation-using-gabor-filters.html?refresh=true).

Maybe I will need to find another normalization to apply of the (intensity, x, y) space.

[Juan Nunez-Iglesias]

Incidentally, it seems you are just doing SLIC on a non-RGB image... Which SLIC supports. (skimage.segmentation.slic). The "compactness" parameter changes the weighting of intensity and space.

To view this discussion on the web, visit https://groups.google.com/d/msgid/scikit-image/5e41ffef-c3a3-421f-b6e4-d5566b5c37c0%40googlegroups.com.

[Hakim Benoudjit]

Thanks Juan, I think you're right.

I might have to read the paper on SLIC algorithm to understand how to tune the "compactness" parameter.

[Michael Aye]

As SLIC uses K-means where one has to provide a number of clusters, I wonder what a SLIC implementation with DBSCAN could do, considering that it is free from the burden of defining the number of clusters. One would have to come up with a method of constraining `eps` and `min_samples`, but maybe that could be quite powerful.

[Juan Nunez-Iglesias]

Cool idea!

To view this discussion on the web, visit https://groups.google.com/d/msgid/scikit-image/cb5869d3-0b93-421b-969d-45a1db088f36%40googlegroups.com.

[Stefan van der Walt]

On 2015-11-24 01:47:00, Hakim Benoudjit <h.ben...@gmail.com> wrote:
> Thanks Juan, I think you're right.
>
> I might have to read the paper on SLIC algorithm to understand how to tune
> the "compactness" parameter.

You can also use SLIC to label the image, and then compute features of
each SLIC region. Or perhaps that is what is being suggested already, I
wasn't sure.

Stéfan

[Hakim Benoudjit]

Hi Stéfan,

Thanks for the suggestion.
I didn't try SLIC algorithm yet, but it seems to me (from the resulting images) that it segment images into small regions (superpixels) that could belong (visually) to the same object.
In my case (ideally), these superpixels need to be merged afterwards.
Do you have an idea on how to achieve the subsequent merging?

[Emmanuelle Gouillart]

Hi Hakim,

I think this example from the gallery does what you want: merging slic
superpixels.
http://scikit-image.org/docs/dev/auto_examples/plot_rag_merge.html#example-plot-rag-merge-py

Best,
Emma

[Hakim Benoudjit]

Thanks Emma, that's exactely what I was looking for.