"on-the-fly buffering" in LAStools (with '-buffered 30')
Martin Isenburg
once in a blue moon I release some funky new LAStools functionality
that is not fully tested yet and quick a look into the night sky above
Germany suggested that I tell you about the new 'on-the-fly' buffering
mechanism in LAStools ... (-:
Assume you have a 3 by 3 tiling of LAZ tiles that is named as shown
below where '30_55.laz' is the center tile. The first thing we want to
do is run 'lasindex -i *.laz' to create a LAX file for every LAZ file.
This is not necessary but can significantly speed up the on-the-fly
buffering
25_50.laz 30_50.laz 35_50.laz
25_55.laz 30_55.laz 35_55.laz
25_60.laz 30_60.laz 35_60.laz
Here is how we create an on-the-fly buffer of 50 meters for
'30_55.laz' and look at it:
lasview -i 30_55.laz -neighbors *.laz -buffered 50
Similarly we can use this 50 meter buffer to ground-classify the
points while avoiding edge artifacts:
lasground -i 30_55.laz -neighbors *.laz -buffered 50 -o ground
\30_55.laz
By default lasground will not write the buffered points into the
output file. However, we can ask lasground to do this by adding the
command line option '-remain_buffered' and later use the buffer to
create DEMs without edge artifacts. However, we can also do this on
the fly with
las2dem -i 30_55.laz -neighbors *.laz -buffered 50 -first_only -
hillshade -use_orig_bb -o dsm\30_55.png
and here the '-use_orig_bb' command line option is important. It tells
las2dem to raster only the area that corresponds to the original
bounding box of the file (i.e. without the buffer). Internally las2dem
uses the buffered points to create a TIN that is much larger than the
rastered area and thereby avoids edge artifacts.
I have described how to buffer a single LAS file. How does that work
in batch mode? Well, the '-neighbors' option is no longer needed as
the other files in the batch call are considered the neighbors.
lasground -i *.laz -buffered 50 -odir ground -olaz
lasindex -i ground\*.laz
lasheight -i ground\*.laz -buffered 50 -odir height -olaz
lasindex -i height\*.laz
lasclassify -i height\*.laz -buffered 50 -odir classify -olaz
It is *REALLY* important to be careful with the '*.laz' input when you
work in one directory. I suggest to keep inputs and outputs in
differen directories as you may end up buffering one result tile
together with the original tile on a subsequent call. That happened
several times to me, which is why I would recommend to use on-the-fly
buffering always with distinct input and output directories. Here how
to batch generate bare-earth DTMs
las2dem -i ground\*.laz -buffered 50 -hillshade -keep_class 2 -
use_orig_bb -odir dtm -opng
The on-the-fly technology is still new and may need a few more tweeks
to integrate more smoothly with the rest. Please let me know if you
find any issues.
Regards,
Martin @rapidlasso
Veldman, Edgar
I had my own work-arounds to do buffering on the fly, but this is
better, setting up the processing batches should become a lot easier
now.
Great work!
Edgar
Download LAStools at
http://lastools.org/
Visit the LAStools group at
http://groups.google.com/group/lastools/
Be social with LAStools at
http://www.facebook.com/LAStools
http://www.twitter.com/LAStools
Martin Isenburg
here a quick example on how to do buffering in LAStools - both old-style and new-style. Imagine you have a large LAS file that you want to first tile, then run ground classify, and finally create a DTM tiling without edge-effects. Then you can choose from 3 different ways to buffer that should not be mixed together. Here are the three possible work-flows to accomplish the task:
(a) create buffers of 80 meter on disk during lastile and use them until the end (already lastile writes buffered points with each tile to disk and they are read and written again by lasground)
lastile -i large.laz -buffer 80 -tile_size 1000 -o tile -olaz
mkdir ground
lasground -i tile*.laz -city -odir ground -olaz -cores 6
las2dem -i ground\tile*.laz -keep_class 2 -use_tile_bb -otif -cores 6
(b) create on-the-fly buffers of 100 during lasground and remain buffered in the output (so that lasground writes buffered points with each tile to disk)
lastile -i large.laz -tile_size 1000 -o tile -olaz
lasindex -i tile*.laz
mkdir ground
lasground -i tile*.laz -buffered 100 -city -remain_buffered -odir ground -olaz -cores 6
las2dem -i ground\tile*.laz -keep_class 2 -use_tile_bb -otif -cores 6
(c) create on-the-fly buffers of 100 during lasground and of 150 during las2dem and therefore never store any buffered points to disk
lastile -i large.laz -tile_size 1000 -o tile -olaz
lasindex -i tile*.laz
mkdir ground
lasground -i tile*.laz -buffered 100 -city -odir ground -olaz -cores 6
lasindex -i ground\tile*.laz
las2dem -i ground\tile*.laz -buffered 150 -keep_class 2 -use_tile_bb -otif -cores 6
The on-the-fly buffering is still new technology in beta, so please use it with care. I appreciate feedback about erroneous behavior, but expect me to provide enough details to recreate any issue on my end. When using on-the-fly buffering it is advisable to create LAX files with lasindex to create these buffers more efficiently.
Cheers,
Martin
Veldman, Edgar
Hi Martin,
This question is somehow related to this subject.
What do you use to differentiate ‘buffer’ points from ‘normal’ points in a lasfile, so that they can be removed later from that lasfile?
eg. when using lastile I have the option to include buffered data ('-buffer 50') and at a later stage I can remove that with -remove_buffer.
So I assume there an extra field with each LAS point you use to attribute these points, something that flags them as ‘buffer’ points?
What I want to use it for (for example) is:
Lasground -i 630000-5757000.las -buffered 50 -neighbors *.las -o 630000-5757000_ground.las -remain_buffered
(and then later on remove that buffer from the output, but not yet, because I want to use the buffered output to do some more analysis)
or
las2las -i 630000-5757000.las -buffered 50 -neighbors *.las -o 630000-5757000_buff.las -remain_buffered
(and then later on remove that buffer from the output, but not yet)
Can I remove that buffered section at a later stage as well?
Do those buffer points have attributes (or something like that) which I can use to separate them from the rest?
Ofcourse I can process the data twice, and output one dataset with the buffer remained and another one without the buffer remained, but that seems inefficient.
I have experimented with using pipe_on but that keeps the input data buffered ‘in the pipe’, and not the result of my process.
Thanks,
Edgar
From: last...@googlegroups.com [mailto:last...@googlegroups.com] On Behalf Of Martin Isenburg
Sent: Saturday, 13 October 2012 8:32 AM
To: LAStools - efficient tools for LiDAR processing
Martin Isenburg
What do you use to differentiate ‘buffer’ points from ‘normal’ points in a lasfile, so that they can be removed later from that lasfile?
There are two ways in LAStools to add buffer points to LAS/LAZ tiles:
(1) by re-tiling the data with buffer on-disk using lastile. This is the option you should choose if you are starting out fresh from the original flight strips. A typical way of turning 273 flight strips into 1000m by 1000m tiles with a 50 m buffer would be
lastile -i strips\*.laz -files_are_lightlines ^
-tile_size 1000 -buffer 50 ^
-o tiles\tile -olaz
the following pipeline would then usually be something like:
lasground -i tiles_raw\tile*.laz ^
-town -extra_fine ^
-odir tiles_classified -olaz ^
-cores 7
las2dem -i tiles_classified\tile*.laz ^
-keep_class 2 -extra_pass -use_tile_bb ^
-odir dtm -otif ^
-cores 7
las2dem -i tiles_classified\tile*.laz ^
-first_only -extra_pass -use_tile_bb ^
-odir dsm -otif ^
-cores 7
and if you need to deliver the classified tiles without buffer to the client or want to host them on a server you can remove the buffers with
lastile -i tiles_classified\tile*.laz ^
-remove_buffer ^
-odir tiles_final -olaz
the way this is done internally is by deleting all points whose x and y coordinate fall outside of the tile. Because the tiling was created by lastile the tool not only knows the exact extends of each tile but also knows which points to keep and which points to toss to assure that the original tiling is reversed exactly so that each point will be in exactly one tile.
(2) by on-the-fly buffering the data using the '-buffered 50' option. In the latter case it can be done only in-memory or also on-disk. This option is used when your data is already in tiles that were not produced by lastile. For example when you downloaded the tiles from the DNR Minnesota that provides almost the entire state of Minnesota as already classified LAZ tiles without buffer (*). In this case you can create the buffering either on-the-fly or permanently on disk:
las2dem -i minnesota_tiles\*.laz -buffered 50 ^
-keep_class 2 -extra_pass -use_orig_bb ^
-odir dtm -otif ^
-cores 7
las2dem -i minnesota_tiles\*.laz -buffered 50 ^
-first_only -extra_pass -use_tile_bb ^
-odir dsm -otif ^
-cores 7
(*) ftp://ftp.lmic.state.mn.us/pub/data/elevation/lidar/
You can also add the on-he-fly buffer to an on-disk representation. You can do this either while you compute something else by telling lasground, lasheight, or lasclassify to '-remain_buffered' or by using las2las.
lasheight -i minnesota_tiles\*.laz -buffered 50 ^
-replace_z -remain_buffered ^
-odir minnesota_heights_buffered -olaz ^
-cores 7
or
las2las -i minnesota_tiles\*.laz -buffered 50 ^
-remain_buffered ^
-odir minnesota_buffered -olaz ^
-cores 7
So I assume there an extra field with each LAS point you use to attribute these points, something that flags them as ‘buffer’ points?
No extra per-point field is used. Instead temporary VLR stores the original number of points. Because all buffer points are added at the end, stripping the LAS/LAZ file of the on-the-fly buffer is done by simply omitting the excess points at the end. Unlike for the buffers generated with lastile we cannot do this with bounding box comparisons because we do not know how the original tilings were prepared (e.g. were they correct? which tile do points on the edges belong to?).
What I want to use it for (for example) is:
Lasground -i 630000-5757000.las -buffered 50 -neighbors *.las -o 630000-5757000_ground.las -remain_buffered
(and then later on remove that buffer from the output, but not yet, because I want to use the buffered output to do some more analysis)
Can I remove that buffered section at a later stage as well?
Yes. Currently you need to use either lasground, lasheight, or lasclassify because only those can "remove" a buffer. Admitting this is a bit odd and certainly a "work-around". This will change eventually but as with all new LAStools functionality sometimes needs a few iterations to converge on the best possible set of command-line options according to user needs that are learned from emails such as the one you just sent ... (-:
lasheight -i minnesota_buffered\*.laz ^
-odir minnesota_unbuffered -olaz ^
-cores 7
will remove the buffer (but also compute the height of each point and store it in decimeter the user data field).
Cheers,
Martin
Veldman, Edgar
Hi Martin,
Thank you for the reply, very comprehensive.
I assume that your last example is missing the -remove_buffer function, correct?
lasheight -i minnesota_buffered\*.laz ^
-odir minnesota_unbuffered -olaz ^
-cores 7 ^
-remove_buffer
will remove the buffer (but also compute the height of each point and store it in decimeter the user data field).
I didn’t know that -remove_buffer would also work for buffers added by ‘-buffered 50 -neighbors *.las’. if that is so, that is great.
I will test this and some of your other suggestions on our data tomorrow, and get back to you.
thanks,
Edgar
From: last...@googlegroups.com [mailto:last...@googlegroups.com] On Behalf Of Martin Isenburg
Sent: Thursday, 1 November 2012 1:50 PM
To: last...@googlegroups.com
Subject: Re: [LAStools] "on-the-fly buffering" in LAStools (with '-buffered 30')
Hello Edgar,
--