Grain ID (Feature ID) Grouping based on Orientation & Neighboring Grains

711 views
Skip to first unread message

Issac Loo

unread,
Jun 19, 2018, 1:28:05 PM6/19/18
to dream3d-users
I want to group Grain IDs together from an HDF5 file. I have converted the HDF5 file into a Dream.3D file with this Data Structure:
==> ImageDataContainer
  ==> CellData
    ==> Confidence Index
    ==> GrainId
    ==> Phases
    ==> Quats
  ==> CellEnsembleData
    ==> CrystalStructure
The GrainId is just used to compare with the number of FeatureIds I get. I've tried using filters Threshold Objects, Neighbor Orientation Comparison (Bad Data), Neighbor Orientation Correlation, and Segment Features (Misorientation), but I continue to get unique Feature Ids that are greater than the unique Grain Ids (which are obtained through our DCT tool) even with extremely high misorientation tolerance.

My questions:

1) Are there any other filters that would be helpful in isolating and grouping Feature Ids?

2) What filters (that I am currently using) are not beneficial?

3) How can I use some of the current filters I have in a better way?

Michael Jackson

unread,
Jun 19, 2018, 5:24:09 PM6/19/18
to Issac Loo, dream3d-users

Let’s take a step back and figure out what data you have and in what format? DREAM.3D is able to import ASCII files, raw binary files, text files and HDF5 files that are NOT organized how DREAM.3D expects. What form of input data do you have.

 

Once we know that then please let us know what you are wanting to do with the data. Are you wanting to segment the data? Did you make up the Crystal Structure data yourself? Are you sure it is correct? If there is a mismatch between the crystal structure and the actual data that may show up as incorrect results.

The existing GrainId values: Do they start at zero and are contiguous to the maximum value? Actually, they _could_ start at 1 but they MUST be contiguous from 1 to the max value with NO gaps.

There is a filter called “Segment Grains (Scalar)” that might work better for you since you already have GrainIds. Running this filter would ensure that the FeatureIds are contiguous and start at the proper value. What format are your quaternions? w<x,y,z> or <x,y,z>w. DREAM3D only likes <x,y,z>w style of Quats. You would have to convert them if they are w<x,y,z>

 

 

--

Michael Jackson | Owner, President

      BlueQuartz Software

[e] mike.j...@bluequartz.net

[w] www.bluequartz.net

--
You received this message because you are subscribed to the Google Groups "dream3d-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dream3d-user...@googlegroups.com.
To post to this group, send email to dream3...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dream3d-users/3b1cb04a-f063-4c24-a3ac-eb562760d114%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Issac Loo

unread,
Jun 20, 2018, 7:27:40 AM6/20/18
to dream3d-users
Hi Michael,

Thanks for your response.

To answer your initial questions:
The data I have is from a Lab DCT experiment, which outputs a HDF5 file. 

The HDF5 file contains the groups: AbsorptionCT, LabDCT, PhaseInfo, etc. 
    The groups of interest are LabDCT & PhaseInfo
        Within the LabDCT group, it contains the datasets of interest:
        Completeness: 243x235x180
        GrainId: 243x235x180
        PhaseId: 243x235x180
        Quat: 4x243x235x180
        Rodrigues: 3x243x235x180
        *Note: I converted Rodrigues to Quat through matlab and appended it to the HDF5 file under the LabDCT group. The Quat was originally w<xyz>, but I altered the columns so that it would be <xyz>w according to Dream.3D desired settings (thanks to your feedback).
        Within the PhaseInfo group, it contains the datasets of possible interest:
        SpaceGroup: 1
        UnitCell: 6
        UniversalHermannMauguin: 1

I've attached the .json file Pipeline which I used to convert the HDF5 to Dream.3D and will use another .json Pipeline further read in the Dream.3d file and process the data. The overall goal is to segment the data by combining neighbor GrainIds that have minimal misorientations and outputting less GrainIds.

I used the SpaceGroup dataset from the LabDCT group as the Crystal Structure Data in Dream.3D. It is the value 225. I am not sure it is correct nor do I know the format in which the Crystal Structure Data in Dream.3D is read.

The existing GrainIds start at zero and are contiguous to the maximum value.

I've attached the other .json Pipeline that I've tried to use in order to segment the data.

Let me know if you need any other information to better understand my issue.

Thanks,

Issac

On Tuesday, June 19, 2018 at 4:24:09 PM UTC-5, Michael Jackson wrote:

Let’s take a step back and figure out what data you have and in what format? DREAM.3D is able to import ASCII files, raw binary files, text files and HDF5 files that are NOT organized how DREAM.3D expects. What form of input data do you have.

 

Once we know that then please let us know what you are wanting to do with the data. Are you wanting to segment the data? Did you make up the Crystal Structure data yourself? Are you sure it is correct? If there is a mismatch between the crystal structure and the actual data that may show up as incorrect results.

The existing GrainId values: Do they start at zero and are contiguous to the maximum value? Actually, they _could_ start at 1 but they MUST be contiguous from 1 to the max value with NO gaps.

There is a filter called “Segment Grains (Scalar)” that might work better for you since you already have GrainIds. Running this filter would ensure that the FeatureIds are contiguous and start at the proper value. What format are your quaternions? w<x,y,z> or <x,y,z>w. DREAM3D only likes <x,y,z>w style of Quats. You would have to convert them if they are w<x,y,z>

 

 

--

Michael Jackson | Owner, President

      BlueQuartz Software

HDF5_Dream3D.json
GID_Ori_CleanUp.json

Michael Jackson

unread,
Jun 20, 2018, 9:10:57 AM6/20/18
to Issac Loo, dream3d-users

I would be interested in obtaining a sample file in the hopes that we could find funding to write a proper reader for the data or at least have an example pipeline showing how to get the data into/out-of DREAM.3D

 

Now to answer your questions: I would skip the Phase information and use the “Create Ensemble Info” filter to manually generate the phase information so that you know it is correct. Please refer to http://www.dream3d.io/#Filters/OrientationAnalysis_Filters/CreateEnsembleInfo/index.html#create-ensemble-info-createensembleinfo for the Laue Group class values that DREAM.3D uses. For space group 225 you would want Cubic-High.

 

What algorithm did you use to convert the Rodrigues to Quaternions? Was your reference frame correct when you did the conversion? DREAM.3D can do the conversion for you using the “Convert Orientation Representation” filter which is thoroughly tested and vetted among an international group of authors. http://muri.materials.cmu.edu/wp-content/uploads/2015/06/RotationPaperRevised.pdf See Appendix A. The conversions can be found here: https://github.com/BlueQuartzSoftware/DREAM3D/blob/develop/Source/OrientationLib/OrientationMath/OrientationTransforms.hpp

 

My guess is that using 225 for the Laue group was messing up the segmentation. Once you get the proper phase information you should be able to use the Segment by Misorientation to create a new set of Grain Ids.

 

--

Michael Jackson | Owner, President

      BlueQuartz Software

Michael Jackson

unread,
Jun 29, 2018, 7:13:50 PM6/29/18
to Issac Loo, dream3d-users
To get back to this after working with the data you have a data set that the following qualities:

26,133,822 voxels that have 59,340 unique “GrainId” values which are clustered into 175,846 features.

This can happen when distinct groups or clusters of voxels that are NOT connected to each other share the same “GrainId” value. I was thinking initially to have you just create the proper Feature AttributeMatrix but DREAM.3D assumes that each cluster will have a unique FeatureId so using the “GrainId” data is just not going to work. If I instead use the Rodrigues vector as the orientation data, after correcting for proper sample-crystal transformation and some cleanup where I force grains to have at least 25 voxels I get around 20,000 grains which at first glance seems reasonable. Once you get a segmentation and pre/post processing figured out you can run both morphological and crystallographic statistics and see what that produces.

I will include a starting pipeline that has some of these filters prepopulated. You will need to adjust the paths in the json file to suit your system. You will also need to use a nightly build of DREAM.3D starting on JUNE 28 2018. Those can be found at http://dream3d.bluequartz.net/binaries/experimental/ as I developed a few new filters while I was analyzing the data that are related to the orientation transformation.

--
Mike Jackson
LabDCT_GrainID_Analysis.json
Reply all
Reply to author
Forward
0 new messages