Interesting to include a the Scattering transform in sci-kit image?
301 views
Skip to first unread message
Sira Ferradans
Jun 13, 2016, 9:02:16 AM6/13/16
to scikit-image
Dear Sci-kit image developers,
at the ENS (Paris) we are planning on implementing a Python version of the Scattering transform 2D and 1D. The scattering transform has proven to be very powerful as a descriptor for image classification and signal analysis. We thought that it may be useful to integrate the 2D version in the sci-kit image package, since it aligns well with the software package and the community. The idea would be to implement the functionalities following as much as possible the APIs you already have for similar functions. More specifically, we will need the following (approximately):
-Morlet wavelet: Closely related to the Gabor wavelet, so it should take into account its API
-Scattering transform: output the scattering transform coefficients, either for display or (as a vector) for learning purposes.
We are attaching a small tutorial (better visualized if you download it) that compares the performance of the first order scattering coefficients computed with Gabor filters, versus the coefficients you extract in the 'Gabor filter banks for texture classification' tutorial. The goal of this ipython notebook is to show that the implementation can be easily integrated in you library while providing a powerful tool for image analysis.
If you think this is a good idea, please let us know. Moreover, it would be great if you could give us some guidelines that you think would make the process easier. We will be adhering to the instructions given on the contributions page, but please don't hesitate to give feedback on our PR!
Best Regards,
Michael Eickenberg and Sira Ferradans.
at the ENS (Paris) we are planning on implementing a Python version of the Scattering transform 2D and 1D. The scattering transform has proven to be very powerful as a descriptor for image classification and signal analysis. We thought that it may be useful to integrate the 2D version in the sci-kit image package, since it aligns well with the software package and the community. The idea would be to implement the functionalities following as much as possible the APIs you already have for similar functions. More specifically, we will need the following (approximately):
-Morlet wavelet: Closely related to the Gabor wavelet, so it should take into account its API
-Scattering transform: output the scattering transform coefficients, either for display or (as a vector) for learning purposes.
We are attaching a small tutorial (better visualized if you download it) that compares the performance of the first order scattering coefficients computed with Gabor filters, versus the coefficients you extract in the 'Gabor filter banks for texture classification' tutorial. The goal of this ipython notebook is to show that the implementation can be easily integrated in you library while providing a powerful tool for image analysis.
If you think this is a good idea, please let us know. Moreover, it would be great if you could give us some guidelines that you think would make the process easier. We will be adhering to the instructions given on the contributions page, but please don't hesitate to give feedback on our PR!
Best Regards,
Michael Eickenberg and Sira Ferradans.
Stéfan van der Walt
Jun 14, 2016, 5:30:03 PM6/14/16
to scikit-image
Hi Sira, Michael
This looks very interesting! The papers you mention are well cited, and the classification results promising (I did some texture classification on rock slices during my masters, and even then it seemed like wavelets would come to dominate this domain).I look forward to your PR & gallery example.
--
You received this message because you are subscribed to the Google Groups "scikit-image" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image...@googlegroups.com.
To post to this group, send email to scikit...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/scikit-image/ad063b83-f44d-463a-85fe-159ba77733a8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Sira Ferradans
Jun 16, 2016, 12:18:15 PM6/16/16
to scikit-image
Hello,
I have one implementation question now: In order to implement the scattering, we need the wavelet transform, with morlet filters. I saw that scikit-image has support for pywt, but morlet filters are not supported now. This leaves the only option of implementing the wavelet transform directly. I thought that a good idea would be to define a list of list for all the filters, that would be saved in the fourier domain. In this context the convolution of the signal with the filter would be implemented as a multiplication in the Fourier domain.
Is this implementation ok? do you have any comments, maybe references where to see a similar code already available in scikit-image?
Thanks a lot for the possible comments and recommendations.
I have one implementation question now: In order to implement the scattering, we need the wavelet transform, with morlet filters. I saw that scikit-image has support for pywt, but morlet filters are not supported now. This leaves the only option of implementing the wavelet transform directly. I thought that a good idea would be to define a list of list for all the filters, that would be saved in the fourier domain. In this context the convolution of the signal with the filter would be implemented as a multiplication in the Fourier domain.
Is this implementation ok? do you have any comments, maybe references where to see a similar code already available in scikit-image?
Thanks a lot for the possible comments and recommendations.
François Boulogne
Jun 16, 2016, 4:32:44 PM6/16/16
to scikit...@googlegroups.com
Le 16/06/2016 à 18:18, Sira Ferradans a écrit :
> Hello,
> I have one implementation question now: In order to implement the
> scattering, we need the wavelet transform, with morlet filters. I saw
> that scikit-image has support for pywt,
Not yet afaik. We have two main PR discussing this:
> Hello,
> I have one implementation question now: In order to implement the
> scattering, we need the wavelet transform, with morlet filters. I saw
> that scikit-image has support for pywt,
https://github.com/scikit-image/scikit-image/pull/1837 and
https://github.com/scikit-image/scikit-image/pull/1833
In #1837, @JDWarner voted in favor of pywavelet for 0.13. I would also
be happy to see these features coming in 0.13; I convinced Ralf to put
pywavelet on a new floating boat a couple of years ago, and scikit-image
was a motivation for me.
IMO, this could be also a killing feature for users who still hesitate
with scikit-image :)
--
François Boulogne.
http://www.sciunto.org
GPG: 32D5F22F
Josh Warner
Jun 17, 2016, 8:58:01 PM6/17/16
to scikit-image, fbou...@sciunto.org
I'll come out in favor of a required wavelet dependency again here. We need that domain.
Wavelet coefficients are features (should be present in skimage.feature), their coefficients sparsely describe images which is useful both for compression and denoising (skimage.restoration), and that's just off the top of my head. There are immediate uses we just can't add until we can depend on wavelet functionality.
IMHO of course.
Josh
Reply all
Reply to author
Forward
0 new messages