XYZ files to grid/DEM

[Mick]

Hi,

I have multiple ascii files (.xyz) containing lidar elevation data for
1km x 1km tiles of a large region. I'm using MI 9.5 and Encom
Engage3D.

I'd like to know if there is a better way to pull the xyz files
together to create a single grid for elevation/slope analysis. At the
moment I go through this process....

1. rename files to .txt
2. merge text files together using txtcollector
3. open in MI
4. create points
5. create grid
etc....

which is OK for just a few files, but as the xyz files are pretty big
(e.g. 30Mb each), the whole thing gets out of hand pretty quickly. Any
suggestions?

Thanks

- Mick

[Eric Gagnon]

Mick,

that's what I would do.. Process the single file once.

For merging the txt files use a DOS command.

Put all the .xyz file on one directory open the cmd window go in that
directory and type in the following:

type *.xyz > allpoints.txt

That's it..

Now for processing.. I use Vertical Mapper. I did one time process 18
million points on my workstation.
But if you have to much points you could always do a Data Aggregation.
This will shrink down your file size.

Hope this helps.

Eric

> __________ Information from ESET NOD32 Antivirus, version of virus signature database 3692 (20081215) __________
>
> The message was checked by ESET NOD32 Antivirus.
>
> http://www.eset.com
>
>
>
>
>

[Andrew Hare]

Mick

Append all the ascii files together into one large ascii file

Under the Engage file menu
Select table and workspace utilities
Select Advanced Ascii Import

The points should be create automatically

Or if the Ascii file becomes to large
Make 3 or 4 files
And later after creating the grids
Join all the grids together

The other way is to
Under the EnageSurfaces file menu
Select import grid file
Select Ascii Grid...

Hope this is helpful

Regards,

Andrew Hare
GIS Administrator
Waimate District Council

______________________________________________________________________________

This email has been scrubbed for your protection by SMX.
For more information visit http://smx.co.nz
______________________________________________________________________________

[Mick]

Eric, thanks for this. It's been so long since I've had a call to use
DOS that I'd completely forgotten the usefulness of these tools (in
any case, I never actually never knew you could TYPE into a file
name). Works a treat. I'm still a learner with GIS but will try the
data aggregation along with other suggestions from the list and
provide a summary when I get through it. I have about 60 million
points, and it seems my system doesn't fully recover if I try to do
anything with more than 10M at a time.

I'll try data aggregation along with a few other techniques as
suggested in the other responses to my post and will send a summary of
my experiences when I'm through it all.

Regards

- Mick

> >http://www.eset.com- Hide quoted text -
>
> - Show quoted text -

[Mick]

Andrew, it was interesting to compare importing the ascii files via
the engage menu vs through the normal MI "open". In the end it was
about 2x faster to open through MI, then create points than it was
open through engage - is this normal? I can see engage offers a bit
more flexibility though.

I tried merging grids, but ended up with obvious merge lines on the
boundaries - is there a technique to achieving a seamless merge? My
ideal solution would be to create a heap of hi-res grids for my 1km
tiles then merge them all neatly together.

Regards

- Mick

> ___________________________________________________________________________­___

>
> This email has been scrubbed for your protection by SMX.
> For more information visithttp://smx.co.nz

> ___________________________________________________________________________­___- Hide quoted text -

[Andrew Hare]

Mick

I must admit I'm a bit of a newbie with engage 3d. I still need to find some time and have a real good play.

I haven't compared the two with regards to speed but engage does offer more options. The import Ascii under the EnageSurfaces menu may be faster then the Advanced Ascii Import. Anyway start the process and go and have some lunch hopefully the import will be finished by the time you return.

I would think in the long run combining all the Ascii files into one and use Engage 3d to create one grid would be faster than creating a number of grids and then having to combine then all. However the large grid may lead to computer performance issues.

After creating a large grid you could always trim the grid into smaller manageable grids.

Any best of luck with it

______________________________________________________________________________

This email has been scrubbed for your protection by SMX.
For more information visit http://smx.co.nz

______________________________________________________________________________

[Mick]

Thanks for eveyone's help with this... here's a summary of what worked
for me:

* used the type *.xyz > newfile.txt" command to merge the lidar xyz
tiles
* I created combined files for the largest combined xyz files that my
system would handle (seemed to be around 20 million points) making
sure that the adjoining edges of the separate grids would overlap
* used mapinfo to open the files and create the points (which was
quicker than engage3D)
* used Engage3D to create the grids
* trimmed the grids exactly to the common boundaries (using engage
grid utilities). (I did this rather than overlap and average the grids
as this would end up with an obvious smoother boundary)
* then merged the grids (engage grid utilities)

the output was very satisfactory with no boundaries, and good
resolution, etc.

Regards

- Mick