EBSD data smoothing for idiots

[Omero Felipe Orlandini]

Hello, all! 

Dr. Hielscher recently gave a very good talk at EBSD 2018 in Ann Arbor comparing methods of EBSD data smoothing in MTEX, and I would like to continue that discussion a little in this forum. I am convinced that EBSD orientation data probably has some degree of noise, and that in some cases (probably a lot of cases considering that this is the signal-to-noise ratio of the data that makes EBSD fundamentally useful) reducing that noise as responsibly as possible is good scientific practice. I feel that I understand the smoothing effect that calculating an ODF requires, but I am neither a mathematician, statistician, nor have any experience in the mathematics of image de-noising. I am hoping to find a practical and responsible understanding of the techniques on this page: https://mtex-toolbox.github.io/files/doc/EBSDSmoothing.html

Dr. Hielscher, my notes from your talk indicate that while there were pros and cons for each technique, something called the 'edge-preserving total variation minimization function' filter performed the most robustly across the widest variety of scenarios. Is this something that is currently accessible through the general release of MTEX, and would you agree with that interpretation?

Dr. Kilian, in your 2017 paper (https://search.proquest.com/docview/1954920739?pq-origsite=gscholar), you describe using the halfQuadratic filter to be edge-preserving above a 1.3 degree threshold. Could I please ask you to describe your logic for choosing that value, and how one executes that using MTEX? I've read and re-read Bergmann et al 2016 (https://arxiv.org/abs/1505.07029) describing the half-quadratic technique, but I have to admit that I get almost nothing practically useful out of that paper.

I would be curious to know what experiences or thoughts that all of the users on this forum have on this topic, and I apologize if I've missed other publications. What I would really like to come out of this is a sense of best-practices guidance in applying these EBSD smoothing filters, because it seems reasonable that EBSD orientation data will commonly be noisy and that noise is undesirable. I accept the possibility that these are just not tools that can be safely used by people who can't fully understand a paper like Bergmann et al 2016 - but that would be helpful to know, also! 

Best,

Phil

[ruediger Kilian]

Hi Phil,

here are a few thoughts on noise filtering.

If one is interested in low angle boundaries with fairly small angles it might be useful to get rid of noise in the sense of (probably random) local, pixel-to-pixel, unsystematic orientations variations and to be able by e.g. structural filtering to preserve boundaries but disregard single pixel entities. In those cases, one should probably not use a filter which isn't edge preserving such as the splineFilter os such which introduces or modifies edges such as the medianFilter or a Kuwahara filter (in some vendor implementations). The halfQuadratic filter preserves edges above a user specified threshold and our choice of e.g. 1.3 degree was simply based on the quality of the maps and finding that it might be a conservative choice and still one is only left with "true" low angle boundaries and single pixel entities which can be omitted.

I read here quite often that people use the splineFilter - which for some purposes might be adequate - but at the other hand, it seems people actually like to look into features such as low angle boundaries and then this is certainly a questionable choice. Also note that the splineFilter as well as the halfQuadraticFilter come with rather intense default values (e.g. alpha), so it's certainly worth experimenting with those.
So, I totally agree with you that blindly using those tools isn't most likely producing the desired effect.

Cheers,
Rüdiger

[Håkon Wiik Ånes]

I would be interested in seeing those notes!

Humphreys, Bate and Hurley have an enlightening article from 2001, Orientation averaging of electron backscattered diffraction data (https://dx.doi.org/10.1046/j.1365-2818.2001.00777.x), where they use a (modified) Kuwahara filter on a real subgrain microstructure in aluminium. An excerpt from their introduction:

Orientation averaging is only valid if the true orientation is constant within each grain or subgrain, and this should be considered before such procedures are used. In annealed aluminium, such as the sample in Fig. 1, this is known to be a good assumption. However, some materials may contatin many free dislocations and only poorly developed boundaries, in which case orientation averaging may not be appropriate. Wherever possible, the nature of the microstructure should be checked using other techniques such as TEM- or SEM-based channeling contrast imaging (Newbury et al., 1986; Prior et al., 1996, Wilkinson & Hirsch, 1997), before orientation averaging is used.

Håkon

[MTEXNewbie]

Hi Rüdiger,

Is there any plan to implement Edge preserving Bilateral Filter in MTEX?

Best Regards

[ruediger Kilian]

Hi,
not that I'm aware of. Doesn't the halfQuadraticFilter work for you?

Cheers,
Rüdiger

[MTEXNewbie]

Hi Rüdiger,

Bilateral Filter is widely used in image processing, and it does not require any coefficient value to be input, unlike halfQuadraticFilter.

It would be better to have this filter in MTEX. Below are two MATLAB implementations of Bilateral Filter -

1. https://www.mathworks.com/matlabcentral/fileexchange/12191-bilateral-filtering

2. https://github.com/GKalliatakis/Bilateral-Filtering

Regards

[ruediger Kilian]

Hi Rashed,

that is actually not correct, one has also to supply parameters for both kernels in the bilateral filter.

Could you elaborate why it would be better to have this filter?

Cheers,
Rüdiger

[MTEXNewbie]

Hi Rüdiger,

Yes, need to set averaging window pixels in Bilateral Filter, but not coefficient like in halfQuadraticFilter.

Median and Spline filter is over-smoothing (loss of sharp detail), the aforementioned halfQuadraticFilter is quite new to most users whereas Bilateral filter is as well known as Median yet better.

I have opened an issue to address this(feature request), the rest up to the devs - https://github.com/mtex-toolbox/mtex/issues/346