Import ascii data

[Abhishek Telang]

Hello all,

I am trying to import ascii data in Dream 3D and I get an error (code 103) saying that the last line has an inconsistent number of columns. Expecting 10 but found 0.

I do have 10 columns and this error is similar to the one in the link below.

https://groups.google.com/d/msg/dream3d-users/H8W2vKvLcEw/DhV7SRF0BQAJ

I did manually set the number of tuples to (total number of lines - 1) and ran the pipeline but I still get this error. 

Attached is the pipeline and the ascii file (tab delimited).

Dream3D version: 6.4.78 (Windows)

Thanks for your help.

Abhishek

[Michael Jackson]

Looking at your pipeline you have created an attribute matrix that is
too large for the file being imported. The easiest way to get this
correct is to let the Import ASCII Data filter create the Attribute
Matrix for you. I am attaching a pair of screen shots that show the
settings for the small file that you sent. The key pieces of the answer
are in the "Start import at row: 2" and "My data has headers at Line #
1". The switch tabs to teh "Attribute Matrix Settings" tab and let
DREAM.3D create the AttributeMatrix for you. The cell dimension _should_
be correct.

--
Michael A. Jackson
BlueQuartz Software, LLC
[e]: mike.j...@bluequartz.net

> --
> You received this message because you are subscribed to the Google
> Groups "dream3d-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to dream3d-user...@googlegroups.com
> <mailto:dream3d-user...@googlegroups.com>.
> To post to this group, send email to dream3...@googlegroups.com
> <mailto:dream3...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/dream3d-users/ed8d1221-983f-4c40-bc27-d35561f6a3f4%40googlegroups.com
> <https://groups.google.com/d/msgid/dream3d-users/ed8d1221-983f-4c40-bc27-d35561f6a3f4%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

[Michael Jackson]

Dear Abhishek,

    After you import the ASCII data you will need to use the "Create Geometry (Image)" filter to create the appropriate Image Geometry. You will need to "Combine Attribute Arrays" to combine the Euler Angles together into a single 3 component AttributeArray. Once you have the proper Image Geometry created for your DataContainer, you can proceed with applying various cleanups to your data to fill in the gaps.

 

After the data has been cleaned up you can use the Quick Surface Mesh to create a surface mesh, compuate statistics on the surface mesh and eventually calculate the GBCD.

 

If you look in the "Prebuilt Pipelines->Workshop->Reconstruction" there are various pipelines in there that might help you develop your own pipeline for your specific data.

 

Did this original data come from a simulation? Normally you would import the data straight from the EBSD files themselves.

 

If you look in the "Prebuilt Pipelines->Workshop->SurfaceMeshing" bookmark folder the pipelines for "03" and "04" can be used as a template for generating a GBCD. Running those pipelines assumes that the data is properly cleaned up.

 

I hope that helps

--

Mike Jackson

 

 

From: Abhishek Telang <amte...@gmail.com>
Date: Wednesday, August 16, 2017 at 9:16 PM
To: Michael Jackson <mike.j...@bluequartz.net>
Subject: Re: Import ascii data

 

Hi Michael,

 

Thanks for the clarification. The file was large so it took a while to load. I was able to create a 3D grain map colored per the IPF scheme. However, the image has "gaps" and/or points with low confidence index though I did not use a threshold filter in this case.

I wanted to clean up the data and then analyze this data set further to extract GBCD, and other statistics.

 

It seems like I need to create an "image" either from the .dream3D file that I created earlier or an "image-data-container". So I used the quick surface mesh filter but I had to specify the geometry earlier and this geometry needs to be specified. I selected the rectilinear grid but Dream3D crashes when the pipeline execution reaches this filter (Pipeline: AR1_new1.json). 

This may be because my data contains information in the vertex format (X, Y, Z co-ordinates of points). But the quick surface mesh filter is not compatible with the vertex geometry.

 

To try and get around this issue, I also tried to read the dream3D file with a different pipeline (mesh.json) but the data is recognized as vertex data and the pipeline cannot be executed.

 

My objectives are the following:

Cleanup the data by confidence index, replace neighbor data points based on the clean-up, extract GBCD, meshing the data set to extract other useful information from the data set.

 

Can you suggest a way to accomplish these objectives in Dream3D?

 

Thanks,

Abhishek

 

 

 

On Sun, Aug 13, 2017 at 3:07 PM, Michael Jackson <mike.j...@bluequartz.net> wrote:

The actual data is in the .dream3d file which is an hdf5 file.

 

--
Michael A. Jackson 400 S. Pioneer Blvd
Owner, President Springboro, Ohio 45066
BlueQuartz Software, LLC EMail: mike.j...@bluequartz.net
Voice: 937-790-1601 Web: http://www.bluequartz.net
Fax: 937-746-0783

---

From: Abhishek Telang <amte...@gmail.com>
Sent: Saturday, August 12, 2017 4:36:52 PM
To: Michael Jackson
Subject: Re: Import ascii data

 

Hello Michael,

 

Thanks for you reply. That change worked and I can now import the data.

I am trying to create an grain map with IPF coloring and created the AR1_new.json pipeline for this purpose.

The pipeline does run without errors but the output AR1.xdmf file is only a few bytes and lacks any data.

 

All files are in this folder (link below).

https://drive.google.com/drive/folders/0B8ttdz9aH8DXQnZrb20tLUxPYXM?usp=sharing

 

 

Is there something that I'm missing here? 

 

Thanks,

Abhishek