Issue with Neighbor Orientation Comparison (Bad Data) on 2D EBSD Data

[Maliesha Kalutotage]

Hi,

I'm trying to use the Neighbor Orientation Comparison (Bad Data) filter to clean up my 2D EBSD data. However, this filter requires cell data called "Quaternions," which I don't have available for my EBSD point data. I believe this type of data is not applicable to 2D EBSD data. How can I use the Neighbor Orientation Comparison (Bad Data) filter on my 2D data?

Thanks,
Maliesha

[stopk...@gmail.com]

Hi Maliesha,

Quaternions are just another method to represent orientation of crystals. You can use the filter "Convert Orientation Representation" to convert Euler Angles from your EBSD data to quaternions. You can place this filter directly after you read in your EBSD data.

Best regards,

Krzysztof Stopka

[Maliesha Kalutotage]

Thanks Stopka, I used "Convert Orientation Representation" filter to generate Quaternions. Thank you!

I have another question regarding the "Neighbor Orientation Comparison (Bad Data)" filter. I used the filter to clean up my data. Then, I applied IPFs both before and after using the filter to compare the results. However, I noticed that instead of converting the bad data (black points) into the corresponding IPF color, they are all represented in red after applying the filter.

Here are the IPF Maps before and after applying the filter:

Before applying the filter

After applying the filter (You can see that all converted bad data into good data has represented by red color)

Is this the correct output I should expect, or should I expect a map where the converted bad data is represented with the corresponding IPF color? 

In the tutorials, I noticed that the converted bad data is represented by the corresponding IPF color instead of just red.

Below, I have included the before and after maps from the tutorials for reference.

Before

After

Could anyone help me with this?

Thanks!

[stopk...@gmail.com]

Hi Maliesha,

That is an interesting observation. Could you post your data set and .json pipeline here for debugging?

Best regards,

Krzysztof Stopka

[Maliesha Kalutotage]

Hi Krzysztof,

I'm not sure how to save the pipeline as a .json file, so I've attached the pipeline in .d3dpipeline format along with the EBSD data file (I will attached the data file in the next email since it's a bit large). If it's necessary to have the pipeline as a .json file, could you provide guidance on how to do so? 

Thank you!

[Maliesha Kalutotage]

Hi Krzysztof,

Cannot attach the EBSD Data file due to the large file size. Here is the GDrive link - https://drive.google.com/file/d/1H7ja4J-_Wq1cWW6rkEvP-WFeX_CsYXcx/view?usp=drive_link

Thanks!

[Maliesha Kalutotage]

In the pipeline, please change 

1. Misorientation Tolerance (Degrees) = 30 and

2. Required Number of Neighbors = 1 

in the "Neighbor Orientation Comparison (Bad Data)" filter to get my output. 

Thanks!

[stopk...@gmail.com]

Hi Maliesha,

Thank you for uploading your data set and DREAM.3D pipeline. I misspoke when I mentioned ".json" pipeline because this was how pipelines were saved in DREAM.3D before version 7.

I was able to reproduce the two IPF filters maps as you posted in the third message above. I think the parameters you specify in the "Neighbor Orientation Comparison (Bad Data)" filter are too extreme for clean up. I believe a misorientation tolerance of 5 degrees is typically used here, so 30 degrees may be too large.

It appears that your data set contains many pixels that are of low quality/high error, evident by the black region in the first IPF map you posted. I think the "(04) Small IN100 prebuilt pipeline" demonstrates how a few bad pixels at grain boundaries can be cleaned up, but this filter may be inappropriate for your scenario. Another way to clean this up and get rid of the "bad" regions quickly is to erode the "bad data", determined using a mask. Please see the attached pipeline and ITK map generated using the "Erode/Dilate Bad Data" filter with the options set to Erode in the drop-down, 10 iterations, and only the X and Y boxes checked (since this is a 2D data set). This must be done after grains are segmented, and as the bad data is eroded, the "good data", i.e., the segmented grains, are grown/tessellated to fill the space.

Best regards,

Kris

Krzysztof Stopka

[stopk...@gmail.com]

I forgot the "Rotate Sample Reference Frame" filter in the earlier post. Attached here are the files, so now the IPF color map should match what you initially posted.

[Maliesha Kalutotage]

Hi Krzysztof,

Yeyy!! This is exactly what I was looking for. Thank you so much!

I have 2 other basic questions:

1. How can we determine the perfect misorientation tolerance corresponding to a given EBSD dataset?

2. Let's say I don't use the "Erode/Dilate Bad Data" filter. That means I have bad points in my data. When I use this data to obtain morphological or crystallographic statistics (e.g., using the "Find Feature Sizes" filter), does it consider the bad data, or does it ignore the bad data points?

Best regards,

Maliesha

[Maliesha Kalutotage]

And I need to have the statistics of individual grains as well as the average statistics of the grains. I know I can use "Find Feature Sizes" and similar filters to get the statistics of individual grains, but are there filters that we can use to directly output the average grain statistics?

[stopk...@gmail.com]

Hi Maliesha,

No problem, happy to help!

1) Do you mean what is an acceptable misorientation tolerance to use when segmenting grains in an EBSD data set? I think a good rule of thumb is 5 degrees, but a good exercise is to perhaps try a series of values (0.5, 1.0, 5.0, 10.0 degrees?) and assess the sensitivity of the tolerance on the results. If your material is heavily textured (i.e., many grains are aligned in the same orientation), then you might have to drop from 5 degrees to ~0.5-1.0 degrees. However, you do not want to go so low such that the intragranular misorientation within a single grain results in multiple segmented grains. This also depends on what you will do with the data. For example, if you will be performing a crystal plasticity simulation, the difference between simulating one grain with a uniform orientation vs. two neighboring grains with very similar orientations will be very small, likely negligible.

2) The bad data is ignored in this case. But a good way to test this would be to process your .ctf file (in excel, MatLab, Python, etc.) and sum the total number of "good" voxels. You can then compare this to the total "NumElements" in the features.csv file written by the "Write Feature Data as CSV File" filter. 

As another note, when grain segmentation takes place, all the "bad data" is lumped into FeatureID = 0, as shown in the screenshot below of your data without the use of the "Erode/Dilate Bad Data" filter. The threshold filter in ParaView was used to view only FeatureIDs = 0.

 

3) I think the best way to determine average grain statistics is to use the "Write Feature Data as CSV File" filter to write the data of each feature ID into a .csv file.

Another point on the calculation of microstructure statistics using your EBSD scan: it may be more appropriate to compute the statistics before you perform the eroding operation, since growing the good regions will bias the metrics you compute. But you can try after eroding the bad data as well; it would be interesting to see how this influences the statistics!

Best regards,

Krzysztof Stopka

[Maliesha Kalutotage]

Hi Krzysztof,

Thank you so much for the explanation. This will help me a lot.

Another thing - I just noticed that you haven't checked the "Convert Hexagonal X-Axis to EDAX Standard" option in the "Import Oxford Instr. Ebsd Data" filter. Could you explain the reason for this? Is it possible to determine this using something in the .ctf file?

Thanks

[Michael Jackson]

Dear Maliesha,

   A comment here on your data. There are 328,517 total scan points. Of those points, ONLY 120,320 or about 36.6% of the points were valid according to Oxford’s indexing algorithm. My criteria is a simple count of the number of scan points where the “ERROR” value = 0, which means that Oxfords indexing algorithm found a suitable value. 36% isn’t very high. Normally you want well into the 80% and above. To be honest I am not sure what kind of grain statistics you can get from this data set. I certainly would not trust any statistics that I did calculate as you are going to be filling in 64% of the data with values that are sort of “guessed” based on clean up criteria? I would go back to your EBSD technician and ask for a better EBSD scan. If you saved all of the EBSD Patterns, there are software libraries available that might be able to do a better job of indexing the points: EMSoft and PyEBSDIndex both come to mind. 

--

Mike Jackson                    mike.j...@bluequartz.net
BlueQuartz Software         www.bluequartz.net

President/Owner               Dayton, Ohio

Principal DREAM.3D Developer

--
You received this message because you are subscribed to the Google Groups "dream3d-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dream3d-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dream3d-users/38549976-beb6-4b47-ac97-aa53c573b431n%40googlegroups.com.

[Maliesha Kalutotage]

Hi Michael,

Thank you for the comment. I will talk with our EBSD technician about this issue. However, in the dataset I provided, it seems like there are 208,197 indexed points (ERROR = 0), and 120,320 is the number of unindexed points (ERROR > 0), which means 36.6% of the points are bad and 63.4% are good. However, I will still discuss this with the technician since 63% is also not a good number. Thank you for pointing this out.

Hi Krzysztof,

Can I have an answer to the previous email's question? It will help me to obtain accurate IPFs.

Thanks again!

Best regards,
Maliesha

[stopk...@gmail.com]

Hi Maliesha,

The "Convert Hexagonal X-Axis to EDAX Standard" has to do with different standards for EBSD systems when characterizing HCP crystal structures. You need to read through the documentation for that filter by clicking on the question mark in the top right side of the filter option screen. But since you have an FCC structure, you will not need to worry about this.

Best regards,

Krzysztof Stopka

[Maliesha Kalutotage]

Hi Krzysztof,

Understood the concept. Thank you so much!

Regarding the Python package: I installed "simplnx" library using the following commands

conda config --add channels conda-forge
conda config --set channel_priority strict
conda create -n nxpython python=3.10 -y
conda activate nxpython
conda install -c bluequartzsoftware dream3dnx

Then I tried to import the following packages:

import simplnx as nx
import itkimageprocessing as nxitk
import orientationanalysis as nxor
import simplnx_test_dirs as nxtest

But this gave me the following error:

ModuleNotFoundError: No module named 'simplnx_test_dirs'

Other 3 commands run without any issues. Issue is only with the "simplnx_test_dirs" package. 

How can I resolve this?

Thanks and regards,

Maliesha

[stopk...@gmail.com]

Hi Maliesha,

I am yet to use DREAM.3D through Python so Mike Jackson would be better suited to answer this.

@Mike, can you look into this?

Best regards,

Krzysztof Stopka