Effect of the warning "on-the-fly merged file contain too many points for a single LAS1.2 file"

[christian...@gmail.com]

Hi,

We are calculating point density statistics using LasTools+OGR/GDAL and for some larger projects I see the following warning in the processing logs:

"WARNING: on-the-fly merged LAS 1.2 files contain too many points (19048499460) for single LAS 1.2 file"

The workflow: *.laz -> lasmerge.exe -> lasgrid.exe -> GDAL operations...

How does lasmerge deal with this limitation? 

Does the software stop adding points to the merged laz or is the warning there just to inform that we are not adhering to the LAS 1.2 specifications.

Thanks,

Christian

[Martin Isenburg]

Hello,

lasmerge uses the LAS version and the point type of the very first file. If that file is a LAS 1.2 file then you should not merge more than 4 billion points (or 4,294,967,295 to be exact) into one file. I would argue that you should never (or almost never) merge billions of points into a single file when you already have them as a selection of other files. You can always merge on-the-fly without creating a redundant copy. And an on-the-fly merge of LAS 1.2 files will probably be consumed correctly by lasgrid (unless you run out of memory). 

When lasmerge writes more than 4,294,967,295 into one file then the counters in the LASheader will overflow and go back to zero. That means that the point count will be set to 0 if you write 4,294,967,296 points,  will be set to 1 if you write 4,294,967,297 points, will be set to 11,00,000 if you write 4,295,967,296 points, and will be set to  100,000,000 if you write 4,394,967,296 points. In other words, don't do it.

If you really want to merge that many points from LAS 1.2 files into one merged file then make sure to use the LAS 1.4 format instead. Either by converting the first file of all files in the list of lasmerge from LAS 1.2 to LAS 1.4 or by using las2las instead:

las2las -i *.laz -merged -set_version 1.4 -o merged_14.laz

But all that said I would never do any of the above but rather work with a tile-based workflow so that you can also run on multiple cores

lastile -i *.laz -merged -tile_size 1000 -odir temp -o temp.laz

lasgrid -i temp\temp*.laz -density -step 10 -odir temp1 -obil -cores 8

gdalxxxx -fdsfsdfdf -fdfsd -fdsfdds -fsdfdf

Note that lasgrid does *not* need buffered tiles (*). As long as the tile size is a multiple of the step size  lasgrid will not generate edge artifacts (like lasground, lasheight, lasclassify, las2dem, ...)  as it neither interpolates nor uses temporary data structures that rely on or are more correct when there are complete neighborhoods. 

Regards,

Martin

(*) https://rapidlasso.com/2015/08/07/use-buffers-when-processing-lidar-in-tiles/

--
Download LAStools at
http://lastools.org
http://rapidlasso.com
Be social with LAStools at
http://facebook.com/LAStools
http://twitter.com/LAStools
http://linkedin.com/groups/LAStools-4408378
Manage your settings at
http://groups.google.com/group/lastools/subscribe

[christian...@gmail.com]

Thank you for a very thorough answer.

This step in our QC flow is done via a C++ gui and I'll have a look at the sourcecode in order to learn exactly how the lastool call is implementet.

Thanks,

Christian

PS: We are currently acquiring data for a new nationwide LiDAR model of Norway and LasTools is essential in the QC flow. Mighty impressed with your software!