Removing noise from a big LiDAR Dataset

[MJ Tube]

Hello,

I have a big LiDAR dataset (>66 GB) including 3,670 LAS files that I want to use to create a DEM and a DSM, to ultimately extract height for buildings. I have identified low noise and high noise that I want to remove before processing the LASD to create elevation surfaces. Following previous discussions, it seems that the LASNoise tool doesn't accommodate the LASD format but LAS and LAZ.

Is there another tool I can use on the LASD that can remove the anomalies given the size? 

Thanks for your help!

[Martin Isenburg]

Hello,

The LASD file format is a proprietary file by ESRI acting as a contarier for sets of LAS files. Simply disregard it and start with a folder of LAS files. Are the files classified already? If yes, then you just need to create the DSM and the DTM. Make sure you use buffers! 

https://rapidlasso.com/2015/08/07/use-buffers-when-processing-lidar-in-tiles/

Regards,

Martin

--
Download LAStools at
http://lastools.org
http://rapidlasso.com
Be social with LAStools at
http://facebook.com/LAStools
http://twitter.com/LAStools
http://linkedin.com/groups/LAStools-4408378
Manage your settings at
http://groups.google.com/group/lastools/subscribe
---
You received this message because you are subscribed to the Google Groups "LAStools - efficient tools for LiDAR processing" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lastools+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lastools/1a7cd815-f112-4bc0-9575-88b132c6a343n%40googlegroups.com.

[Myrtho Joseph]

I have done that before. Unfortunately there outputs (DEM and DSM) have many outliers that affect the resulting height. What I actually want is to remove the outliers before I process the DEM and the DSM.

 

From: last...@googlegroups.com <last...@googlegroups.com> On Behalf Of Martin Isenburg
Sent: Wednesday, March 17, 2021 7:48 AM
To: LAStools - efficient command line tools for LIDAR processing <last...@googlegroups.com>
Subject: Re: [LAStools] Removing noise from a big LiDAR Dataset

 

*****EXTERNAL EMAIL: Please do not click on links or attachments unless you know the content is safe.*****

 

To view this discussion on the web visit https://groups.google.com/d/msgid/lastools/CABSWR-FD_1%2BU04KKirP4jmoR1ivrd2Yc3tv%3DuL5K9whP_Mhepg%40mail.gmail.com.

[Terje Mathisen]

Myrtho Joseph wrote:

I have done that before. Unfortunately there outputs (DEM and DSM) have many outliers that affect the resulting height. What I actually want is to remove the outliers before I process the DEM and the DSM.

The terrain model should only depend on the actual ground (class = 2) points, all the rest are ignored.

For your vegetation/building/whatever surface you do need to skip past any spurious high noise points, and you probably also need to avoid holes.

Have you tried Martin's Spike Free pipeline?

Terje

To view this discussion on the web visit https://groups.google.com/d/msgid/lastools/688ae4bc1fdb434da12a37c8ead6433e%40HDExchange01.brgov.net.

-- 
- <Terje.M...@tmsw.no>
"almost all programming can be viewed as an exercise in caching"

[George S. Mercier]

So are you using ESRI to remove the noise? and is that why you have ended up with a LASD (esri's LAS dataset)?  Maybe try removing the noise with LASTools and keeping your data in a open and standard format like LAS or LAZ.  Here is a link to a post where Martin talks about removing noise with LASTools.  https://groups.google.com/g/lastools/c/_nWgp0s-Hls/m/kBCjmVQ9AwAJ

Respectfully,

George S. Mercier

[Myrtho Joseph]

I have a large number of LAS files. To facilitate their display and processing, I converted them to LASD using a 3X4 fishnet (As a result I have 12 datasets for the area, since it took a lot of time to run the entire dataset for the whole area). This is how I ended up with the LASD. I created 2 different sets, one for the DEM, with ground data exclusively, and another set for the DSM with the first return values. I used the statistic methods, removing first the global outliers, then local outliers with the formula suggested. However, I still find local outliers. So the statistical methods is not efficient for my purpose. 

 

My question is how (or) can I use LASNoise on the indivual LASD to remove the outliers? Or can I use this tool providing the folder that contains all the 3500 individual LAS files? If yes, how do I set the parameters? What is the expected outcome?

 

I am sure I will find someone in the group with great expertise in this matter. I am desperate to find help.

To view this discussion on the web visit https://groups.google.com/d/msgid/lastools/ecb5edec-1a1c-4e94-aec4-34179b34eecen%40googlegroups.com.

[Martin Isenburg]

Hello,

in future messages to the LAStools group please refrain from mentioning the proprietary *.lasd files from ESRI. Those were "invented" shortly after ESRI joined the LAS Working Group. Rather than using their own needs to contribute, for example a good standardized container class, to the open LAS standard and proposing it as an optional addition to the LAS format, the management at ESRI spewed out a whole bunch of their own closed formats that sound like being related to the open LAS format (*.lasd, *.zlas, and *.lasx). This muddied the tropical waters of our previously so clear and paradisiacal LiDAR beaches lie an oil tanker wrecking into a coral reef and led to the great clone wars of 2014 - 2016 that was eventually won with the emscriptem of jedi scripts.

https://wiki.osgeo.org/wiki/LIDAR_Format_Letter,

What do you have?  A folder of LAS ot LAZ files? Are they tiles (likely) or flight lines? You will need to create buffered tiles (to avoid edge artifacts) and then run lasnoise. There are a number of blog articles on running lasnoise and you should also find a few previous discussions in this forum.

https://rapidlasso.com/?s=lasnoise 

Yes, you can use lasnoise on your folder with 3500 LAS files. Make sure you use buffers, though, otherwise you will find more noise along the tile edges and even more noise in the tile corners. That said ... depending on your application you might be able to live with these tile boundary effects as they might be less drastic for lasnoise then they are for lasground and lasheight. However, as you will need to use buffers for your DEM creation step you may just go ahead and retile once with buffer.

Regards,

Martin

To view this discussion on the web visit https://groups.google.com/d/msgid/lastools/21ff37461c344177bbb10c74ecb19949%40HDExchange01.brgov.net.