using merge with las2dem

[Ty Kennedy-Bowdoin]

Martin,

I am currently testing the lastile and las2dem executibles.

• I have started by tiling a single strip of las data  (3.5gb) into 1000m buffered tiles.
• I would like to interpolate ground and surface dems using las2dem. It was easy to do a las2dem -i *.las -step 1.12 -keep_class 2 3 -otif, but I would like to assemble these into dems by strip using the -merge keyword.
• when I use merge it las2dem gives me an error: cannot alloc point_buffer for 180301612 points.

I would assume that is only one of the tiles, and it doesnt seem like too many points. I can output the dems into tiles and then merge them in envi, but I was hoping to avoid using envi.

Is this possible in your tools?

Thanks!

Ty

[Michel Wolters]

Hi,

Have you tried to use blast2dem? It can work with far more points than las2dem.

Regards,

Michel Wolters

On Friday, May 11, 2012 7:50:21 PM UTC+2, Ty Kennedy-Bowdoin wrote:
> <span>Martin,</span>
>
> </div>
> I am currently testing the lastile and las2dem executibles.</div>
> <ul><li style="margin-left:15px">I have started by tiling a single strip of las data  (3.5gb) into 1000m buffered tiles.</li>
> <li style="margin-left:15px">I would like to interpolate ground and surface dems using las2dem. It was easy to do a las2dem -i *.las -step 1.12 -keep_class 2 3 -otif, but I would like to assemble these into dems by strip using the -merge keyword.</li>
> <li style="margin-left:15px">when I use merge it las2dem gives me an error: cannot alloc point_buffer for 180301612 points.</li></ul>
>
> </div>I would assume that is only one of the tiles, and it doesnt seem like too many points. I can output the dems into tiles and then merge them in envi, but I was hoping to avoid using envi.</div>
>
>
> </div>
> Is this possible in your tools?</div>
>
> </div>
> Thanks!</div><span><font color="#888888">Ty</font></span>
>
> </div>

[Martin Isenburg]

Hello Ty,

when las2dem is run with the input options '-i *.laz -merged' then
it's usually a wrong. (-: las2dem is the tool to be used for smaller
data sets (up to 25-30 million points that participate in the
triangulation step) and usually in a tile-based batch processing
environment where each tile will result into a separate DEM that are
put back together in the end by some other tool (gdal, grass, ...).
here is how the command line should then look like:

lastile hugestrip.las -tile_size 1000 -buffer 50 -o tiles\tiles -olaz
las2dem -i tiles\tiles*.laz -keep_class 2 3 -extra_pass -step 1.12 -
use_tile_bb -odir rasters -otif -cores 3

note the '-use_tile_bb' which assures that only the interior of each
tile is rastered and not the buffer. and you probably have a nice
computer with multiple cores, so why not put them to work with '-olaz'
and '-cores 3'. if you have a single strip of 3.5 GB that is already
classfied and all you need to do is create a DEM via TIN interpolation
then simply run

blast2dem -verbose -i hugestrip.las -keep_class 2 3 -step 1.12 -otif

and you get one big TIFF file without memory issues.

Cheers,

Martin @lastools

PS: For an example of a complete tile-based batch-processing pipeline
check the last few slides of this presentation: http://lastools.org/download/lastools.ppt

On May 11, 10:50 am, Ty Kennedy-Bowdoin <tbowd...@stanford.edu> wrote:
> Martin,
>
> I am currently testing the lastile and las2dem executibles.
>

>    - I have started by tiling a single strip of las data  (3.5gb) into
>    1000m buffered tiles.
>    - I would like to interpolate ground and surface dems using las2dem. It

>    was easy to do a las2dem -i *.las -step 1.12 -keep_class 2 3 -otif, but I
>    would like to assemble these into dems by strip using the -merge keyword.

>    - when I use merge it las2dem gives me an error: cannot alloc

[Ty Kennedy-Bowdoin]

Thanks Michel and Martin!

blast2dem is indeed the tool I was looking for ;)

I made a batch file that runs two instances in windows command line (one for a ground and another for a surface dem). This works on most las files, and takes around 4mb for each process. Occasionally, i get an error that says that las2tin failed and the command line gives me the attached error message. The resulting DEMs will not open in envi. 

I dont see any large usage of memory or cpu, and the files that have failed dont seem to be particularly large (~3gb). 

Can you think of any reason that this would happen?

thanks again!

Ty

--
Download LAStools at
http://lastools.org/
Visit the LAStools group at
http://groups.google.com/group/lastools/
Be social with LAStools at
http://www.facebook.com/LAStools
http://www.twitter.com/LAStools

--
Ty Kennedy-Bowdoin
Carnegie Institution for Science
260 Panama St.
Stanford, CA 94305
USA
+1 831 227-3885
Skype: Ty.bowdoin

[Martin Isenburg]

Hello,

> blast2dem is indeed the tool I was looking for ;)
>
> I made a batch file that runs two instances in windows command line (one
> for a ground and another for a surface dem). This works on most las files,
> and takes around 4mb for each process.

So you are saying that you repeatedly observe that you can triangulate
and raster a 3 GB file using no more than 4MB of main memory with
blast2dem.exe? I know blast2dem.exe (aka the BLAST extension pack for
LAStools) is *extremely* memory-efficient and therefore scalable to
very very large point clouds but I would have expected maybe 15 MB or
20MB being used. (-:

> Occasionally, i get an error that says that las2tin failed and the

> command line gives me the attached error message. Can you

> think of any reason that this would happen?

Yes i can. Looking at the size of the raster (ncols * nrows) i see a
32 bit integer overflow that makes the number negative (hence small)
so that the BufferInMemory option is chosen. Stupid little bug. I need
to upcast in place and put a 64 bit integer compare into that place.

Can you send me a teeny weeny bit of that LAS file by running

laszip -i large_SLOTH_that_crashes.las -o tiny_file.laz

and then pressing CTRL-C as soon as you can? This assures I get the
same LAS header without all the LAS points. Please send that file to
my private email address and I will fix the issue.

Cheers,

Martin @lastools