Help me understanding how Lastools (blas2them works)

[Daniel Perez Perez]

Hi everyone and Martin!

Hope you are all having good health!

I´m sending some screenshots to make my question easier....Hope you can help me understand how Lastools works for the pulses and the algorithm to create DSM.

We are measuring areas of the crowns from olive trees, all of them from open data at a very low density (0,5 points/square metre).

The purpose of this research is to evaluate how accurate is the open lidar data, available at the moment.  To do so, we have done a few drone flights over the fields with no Ground Control Points measured with GPS coordinates, just the drone´s gps itself, so there is an error of more or less 0,5 meters in real location.

As you can see in the screenshot, a vector layer has been digitalised over the RGB orthomosaic in order to get the most realistic area as possible.

In the next image you can see the vector layer over the DSM made, using Lastools performing a pixel of 2 square metre (please notice  the geolocation error mentioned above, but imagine that the vector is just perfect over the DSM).

Given the DSM, the value of the pixels is as it follows:

So, taking the values in consideration, we can see that the pulses have registered different heights, and the target is to understand how the algorithm from Lastools works... So as I see it, there are different possibilities as shown in the next screenshot:

Option A: we get one pulse and it gives the value of the height of the first return and considered class (3,4 or 5) or multiple returns and then, what value is it considering?

Option B: we get two pulses on the ground that are considered class 2 and are both last returns or one first return and the other could not be last return? and also, both should not have multiple returns as there is bare soil...?

Option C: we get only one pulse on the ground and it is considered last return and then it is classified as class 2?

Option D: we get one pulse on the tree and the other on the ground. The one on the tree could be single return or multiple return but in any case is first return?. The other pulse is last return and should be single return?

Option E: we get 2 pulses on the tree and are both classified as class 2 and could be single or multiple returns?

So:

- In option A the DSM should show a tree with the highest value in case of multiple return, is the value of the pixel the maximum z value?

- In option B the DSM should show ground as both pulses are last returns and the value of the pixel is the average value between both pulses?

-In option C the DSM should show class 2 as ground and it is a last return?

- Option D and the million dollar question: Is the value of the pixel the average of both pulses? How does Lastools work with these situations?

- Option E, is the value of the pixel calculated the same way as option B?

Hope the images are helping to understand....

Thanks in advance for your time and hope you can help me with the understanding of this matter.

Any information will be appreciated and also any references to publications or books will be very welcome.

Thanks again and I look forward to hearing any responses!

--

Daniel Pérez Pérez

Responsable de operaciones RPA´s

Contacto: 661350660

Email: cultiva...@gmail.com

This email and any attachments to it contain confidential information and are intended solely for the use of the individual to whom it is addressed.If you are not the intended recipient or receive it accidentally, please immediately notify the sender by e-mail and delete the message and any attachments from your computer system, and destroy all hard copies. If any, please be advised that any unauthorized disclosure, copying, distribution or any action taken or omitted in reliance on this, is illegal and prohibited. Thank you for your cooperation.

[Martin Isenburg]

Hello Daniel.

sorry for the delayed response but things have been a bit hectic lately with the ridiculous SLANDER and DEFAMATION attack by US DoI / BLM / USGS on LAStools, my company and my person.

There are a number of different ways with LAStools to produce a DSM and which DSM you get will depend on the method you use.

When you use either las2dem or blast2dem you will create a triangulated surface of the elevation points and the elevation values are interpolated linearly across each facet. The elevation value used is the one where a "needle" that you lower in the center of the grid cell until it touches the TIN makes contact, hence for 2 by 2 meter cells, the needles will be lowered on every odd 1, 3, 5, 7, and 9 meter coordinates in x and y. You can actually visualize this in lasview. Can you send me the sample data for this scene above?  

When you use lasgrid or lascanopy you can create either a maximum, minimum, average, mean, or a certain percentile DSM based on the points falling into each cell - no matter what their exact x/y coordinates are.

Then you can get creative and use combinations of lasthin / lasgrid / las2dem to create particular DSM.

Which DSM you get depends on what modules you run with which parameters.

https://rapidlasso.com/2016/02/03/generating-spike-free-digital-surface-models-from-lidar/

https://rapidlasso.com/2014/11/04/rasterizing-perfect-canopy-height-models-from-lidar/

Regards,

Martin

--
Download LAStools at
http://lastools.org
http://rapidlasso.com
Be social with LAStools at
http://facebook.com/LAStools
http://twitter.com/LAStools
http://linkedin.com/groups/LAStools-4408378
Manage your settings at
http://groups.google.com/group/lastools/subscribe
---
You received this message because you are subscribed to the Google Groups "LAStools - efficient tools for LiDAR processing" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lastools+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lastools/CAJjMPF%2B8BMnVP1sWJGW8cWWGx8_hycktMVnQxNoC0DOF1CxeAw%40mail.gmail.com.

[Daniel Perez Perez]

Hey Martin!

Thanks for your answer, I have been reading the emails that you´ve shared in the Lastools group about defamation. 

It is difficult for someone like me to give any advice to someone like you but I will say that being positive and strong minded always help. I have to agree with what different people suggested here before, having an audit for the software will clear any doubts, so getting a certification for Q/A sounds very helpful from the outside view.

I have attached the laz archive, clipped from the 2x2 km TIN downloadable with the sample area. This was done by using Lasclip. 

The main target is to obtain the most accurate area of crown of the olive trees. 

We were advised to get a raster (2x2 metre cells)  for the DTM and DSM because of the 0.5 points/m2 LIDAR pointcloud so it could be possible to have a point in every cell of the raster. We are not sure that is necessary as that might result in an over-estimation of crown cover because of the pixel size.

If you don´t mind, it would be really welcome that you describe the path of what you consider that could bring the most accurate results.

In order to understand how LIDAR classification works I bought this book:

https://www.amazon.es/gp/product/1138747246/ref=ppx_yo_dt_b_asin_title_o02_s00?ie=UTF8&psc=1

Please, feel free to recommend articles or books that can help me learning how LIDAR works. It is not an easy subject to learn about.

I can send you any other info that you might need, just let me know what you need.

Thanks a lot for helping and for your time, I understand that you are very busy!

Look forward to hearing from you!!

Regards!

Dani.

To view this discussion on the web visit https://groups.google.com/d/msgid/lastools/CABSWR-HPrMxMUx4GjQvKW_h_cN0apF6mJDALpd2JrsvAXM%3DgOQ%40mail.gmail.com.

[Martin Isenburg]

Hi,

Thanks for your answer, I have been reading the emails that you´ve shared in the Lastools group about defamation. 

It is difficult for someone like me to give any advice to someone like you but I will say that being positive and strong minded always help. I have to agree with what different people suggested here before, having an audit for the software will clear any doubts, so getting a certification for Q/A sounds very helpful from the outside view.

Sounds reasonable. But is it really? Imagine someone were to spread the unsubstantiated rumor about me being a pedophile. Would you equally suggest that I should see a behavioural specialist in order to get a psychiatric audit that  certifies that my sexual habits are following societal norms and then publish those in detail to clear any doubts in my community about whether I lust after children or not? Obviously not. You would advise me to prosecute the slander and libel to the fullest extent on the law.

But maybe we have been conditioned that whenever the United States (!) government acts like a tough bully, they must be doing this for our own safety (or that of the entire human species as every Hollywood movie tells us). So we should oblige and obediently jump through whatever hoops they bother us with. It's for our own safety. No. I call total chicken poop on this.

This "Mister Gack Gack" has been in a "Bad Romance" with the US Government since 2010 and you would think they would eventually make amends.

https://youtube.com/watch?v=qrO4YZeyl0I

Make a teddy bear and a box of chocolates? There were some nice days, but a few months later the abuse always continued.

https://youtube.com/watch?v=uelHwf8o7_U

Should I like the way they hurt and love the way they lie?

(-;

Martin

PS: I will look at your data next. 

[Martin Isenburg]

Hello,

I had a closer look at the file you sent me using these commands:

lasview -i 1-LT3SB_clip_LIDAR.laz -kamera 0 -46.6406 -110.566 -0.0537109 0.0716146 0.282375
lasview -i 1-LT3SB_clip_LIDAR.laz -kamera 0 -34.4531 -109.687 0.0214844 0.0572917 0.0593165
lasview -i 1-LT3SB_clip_LIDAR.laz -kamera 0 -34.4531 -109.687 0.0214844 0.0572917 0.0593165 -grid 2

look at the README file for the options to create these visuals:

http://rapidlasso.com/lasview

http://lastools.org/download/lasview_README.txt

Below the picture comments as numbered:

(1) Only two multi-returns in the entire area. What LiDAR sensor produced this?
(2) Triangulated first returns. Wireframe view. Well separated tree crowns.

(3) Rough measurement of the diameter of a tree crown: 6 meters
(4) Rough measurement of the diameter of a tree crown: 8 meters

(5) Single tree across measurement: 8.3 meters

(6) Single tree height measurement: 3.7 meters

(7) Example ground sampling distance: 1.1 meters

(8) Example ground sampling distance: 1.7 meters

(9) Example ground sampling distance: 88 centimeters

(10) Example ground sampling distance: 55 centimeters

(11) The TIN with the "elevation probe needles" for a 2 meter grid

(12) The corresponding rasterized 2 meter grid

(13) The TIN with the "elevation probe needles" for a 1 meter grid  
(14) The corresponding rasterized 1 meter grid

Especially the last four images should help you decide whether using an interpolation approach based in Delaunay triangles and subsequent rasterization onto an elevation grid is a possible method for you to choose. Given that your olive trees have very little laser penetration (spikes) a simple first return might do for you. Otherwise you could also consider the spike-free algorithm.

https://rapidlasso.com/2016/02/03/generating-spike-free-digital-surface-models-from-lidar/

For comparison the options of using lasgrid to create elevation rasters and the "empty pixel" issue are discussed here:

https://rapidlasso.com/2014/11/04/rasterizing-perfect-canopy-height-models-from-lidar/

More questions to any of these visualizations?

Regards,

Martin @rapidlasso  

[Jorge Delgado García]

Hi all,

Martin, thank you very much for your great work and examples.

At the moment I am working on lines similar to this, and also with the same type of trees. The olive grove has the characteristic of being planted following a very regular pattern, in traditional crops 10m, in modern ones smaller. The truth is that I believe that although some information can be obtained, I would not risk proposing any rigorous analysis, from a LiDAR with that density.

We must not forget that in the end, the LiDAR does not measure points but rather measures "a certain surface on the ground" and that furthermore, the starting files are not actually the raw files (they are classified and surely subjected to other treatments, perhaps that is why multiple returns are missing). All this without actually considering the exactness of LiDAR, etc.

It is becoming increasingly clear to me that in the end, in each case we must be clear about the objective of this survey (to have a 5m national DEM), the rest is to force the data. Surely we can count how many trees there are but establish height - perhaps if we assume that the laser has hit the highest area of the tree, which does not have to be regular - or width. In my opinion, it could be complicated. Actually, we are working (with LAStools) processing high-resolution 3D clouds -obtained from a UAV system using SfM- for obtaining this type of information. I have some examples, I will try to compare my results (with cm spacing 3d clouds) with the national LiDAR program.

Jorge

--

Download LAStools at
http://lastools.org
http://rapidlasso.com
Be social with LAStools at
http://facebook.com/LAStools
http://twitter.com/LAStools
http://linkedin.com/groups/LAStools-4408378
Manage your settings at
http://groups.google.com/group/lastools/subscribe
---
You received this message because you are subscribed to the Google Groups "LAStools - efficient tools for LiDAR processing" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lastools+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lastools/CABSWR-E4jpRARWCgtjvgG4s5ZJy6avopOxQn%2Bd9FucL1sfde8Q%40mail.gmail.com.

--

[Daniel Perez Perez]

Hi again!

First of all, thank you very much to both Martin and Jorge!

Your help and opinions are very  much appreciated.

I understand that that data is the data, forcing the data won't bring better achievements. However, there are a couple of things that still don´t understand and maybe you can help me here...

1- Considering that there are areas where density is actually higher than 0,5 points/m2, and then, rasters smaller than 2x2 m2 can be performed, the points classified as vegetation must be vegetation. The question here is: If the density is, indeed, very low and a short number of points are shown, how or why are we obtaining over estimation of the area of the crowns instead of obtaining under estimation of it? 

2- Please, feel free to take a look at the attached archives from another crop:

As you can see, in the screenshot attached, Lasview shows a lot of multiples returns in this crop. Nevertheless, Lasinfo shows as it follows:

 number of first returns:        42706 (this should be vegetation)

number of intermediate returns: 0
number of last returns:         42714 (this should be ground)
number of single returns:       41441 (either if they are first or last returns)

- How is it possible that showing a big amount of multiple returns, Lasinfo is not taking them into consideration? Even when overlap covers a large number of returns, its classification should be appropriate and if it is meant to be multiple return it should be shown as a multiple return.

- Also, it has been observed that a lot of these multiple returns are over ground or soil area. If this lidar sensor was not able to go through olive leafs, what is it hitting over the soil, weeds? (probably dry low weeds as the flights were performed over the summer).

Hope these questions are not too boring for you as they will definitely help me understand a lot more of lidar data and its classification.   

Thanks again and I look forward to reading your answers. 

Have a good day!

Best regards!

Dani Perez

To view this discussion on the web visit https://groups.google.com/d/msgid/lastools/CADgtU__4xUZtQVa8Yf2PWckB2_J4yhiiECuemy8b%3DLVyiWOW-w%40mail.gmail.com.

[Martin Isenburg]

Hello, 

1- Considering that there are areas where density is actually higher than 0,5 points/m2, and then, rasters smaller than 2x2 m2 can be performed, the points classified as vegetation must be vegetation.

No. The vegetation classification in your sample data is garbage. Do not trust it. Half the vegetation points are marked overlap (class 12). Use all the non-ground points,

 

The question here is: If the density is, indeed, very low and a short number of points are shown, how or why are we obtaining over estimation of the area of the crowns instead of obtaining under estimation of it? 

That makes no sense indeed. I would need to look at such a case. Bomb craters in the ground classification? Steep areas / sloped terrains and wide overhanging tree crowns are the main way of over estimating tree height in height-normalized CHMs.

https://www.sciencedirect.com/science/article/abs/pii/S0924271615000581

 

2- Please, feel free to take a look at the attached archives from another crop:

That crop is more "penetratable" than the other one so here the "spike-free" option of las2dem will give nicer results. 

  

 number of first returns:        42706 (this should be vegetation)

WRONG: These are the first things the laser reflects from: roads, terrain, anywhere in trees, wires, birds, building roofs, cars 

The first returns are usually used to compute a DSM.

number of last returns:         42714 (this should be ground)

WRONG: These are the last things the laser reflects from: roads, terrain, trees, birds, building roofs, cars, and even wires (but more rarely).

A small subset of the last returns are usually used to compute a DTM.

number of single returns:       41441 (either if they are first or last returns)

These single returns are counted as both first and last returns. They happen when the laser has only one significant reflection. This happens for hits on roads, terrain, trees, (really big) birds, building roofs, dragons, cars, and even wires (but more rarely).

 

- How is it possible that showing a big amount of multiple returns, Lasinfo is not taking them into consideration?

It does. There are no triple returns. But there are 42706 - 41441 = 1265 first of two returns and 42714 - 41441 = 1273 last of two returns. Those are the ones you see as 1265 red points and 1273 blue points in lasview.

Even when overlap covers a large number of returns, its classification should be appropriate and if it is meant to be multiple return it should be shown as a multiple return.

It is. You were miss-reading the lasinfo output.

 

- Also, it has been observed that a lot of these multiple returns are over ground or soil area. If this lidar sensor was not able to go through olive leafs, what is it hitting over the soil, weeds? (probably dry low weeds as the flights were performed over the summer).

There should not be multiple returns over open ground and soil area. I do not see those in the data. Do you?

Regards,

Martin

[Daniel Perez Perez]

Hi! I tried to make the email as detailed as I could and it was too long, I crompresed everything and I hope it is clear to read.

 

-  I think I must have a few wrong concepts about classification and how lidar works.

 

- From the point cloud downloaded from CNIG, we did some processing in order to get ground and vegetation only (considering that in those crops there is nothing else apart from trees and soil):

1- Download point clouds from the source.

2- Merged 2x2 Km clouds and clipped them to obtain the sample crop polygons.

3- Remove all points with classification different from 2,3,4 or 5.

4- Overlap class points were removed with the rest of the classes (different to 2,3,4 & 5) as some sample areas are not completely covered by overlap points and some other sample areas don´t have overlap at all. Also, the same method must be applied for all sample crops    

5- Classes 2,3,4 & 5  were set to zero values.

6- Lasground to obtain ground class.

7- All the rest of points were considered as vegetation.

 

- About the quality of the lidar classification from PNOA for the vegetation classes; 

When I display the RGB (real drone flight) + the point vector layer including classes 3,4 & 5. It seems to be more or less ok. Considering that the RGB images were taken not long ago and the Lidar point cloud is from august 2014. As you say, the point cloud classification is probably not very efficient but it is what we have and as Jorge mentioned before, probably not good enough for this purpose.

- About the multiple returns display, I managed to display the vector layer for returns 2 & 3. They were only shown  2nd returns.  I displayed, over the DSM, the vector layer for the 2nd return (out of two returns). As you said, there are no multiple returns over the ground layer but they are mostly displayed over the branches in contact between trees.

It does make sense that two returns (multiple returns) occur where branches are not as dense as within the tree crown, doesn´t it?

 

- About the overestimation, the average slope within the 35 sample crops we have is 11%. Max slope= 24,75% Min slope= 1,58%

Probably there will be some affected and others will not be affected by the real position of the very top of the tree.

In our case, area is more important than height... 

 

- I believe that the overestimation comes from the pixel value and how blas2dem gives these values to the cells and  I think that you already gave the answer to this in your first email which goes quite well with Jorge´s recommendation about trying Lasgrid for the DSM.

  - After reading the links from your first email, probably using Lasgrid will work well for our target.

I think the overestimation occurs within the DSM, when there is a cell with low Z values (ground) but it has been affected by its neighbor because of the linear interpolation, giving a value higher than the soil and then converted into part of the crown of the tree by vectorizing those values higher than the ground.

If this is correct, then, using Lasgrid with the highest "Z" values, will stop contiguous cells being affected by the neighbors values?

Another way that comes to my mind is using the raster calculator and apliying a cut on the CHM over 0,7 metres or 1 metres to avoid this overestimation.

Let me know your thoughts!

Thank you again for your time to Martin and Jorge!

Have a good day!

Dani Perez

El jue, 19 nov 2020 a las 19:27, Daniel Perez Perez (<cultiva...@gmail.com>) escribió:

Hello!

Sorry for my late reply!

Because my email is too large and I wanted to include some screenshots, I have sent a word document attached to this email.

Sorry for the inconvenience... hope you understand.

Please, take a look and let me know what do you think!

Best regards!

--

Download LAStools at
http://lastools.org
http://rapidlasso.com
Be social with LAStools at
http://facebook.com/LAStools
http://twitter.com/LAStools
http://linkedin.com/groups/LAStools-4408378
Manage your settings at
http://groups.google.com/group/lastools/subscribe
---
You received this message because you are subscribed to the Google Groups "LAStools - efficient tools for LiDAR processing" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lastools+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lastools/CABSWR-G2KcmV8JfAJ0hsa8_oBprqWOhFLowRF%3DEn3ThFv98QGQ%40mail.gmail.com.

--

Daniel Pérez Pérez

Responsable de operaciones RPA´s

Contacto: 661350660

Email: cultiva...@gmail.com

This email and any attachments to it contain confidential information and are intended solely for the use of the individual to whom it is addressed.If you are not the intended recipient or receive it accidentally, please immediately notify the sender by e-mail and delete the message and any attachments from your computer system, and destroy all hard copies. If any, please be advised that any unauthorized disclosure, copying, distribution or any action taken or omitted in reliance on this, is illegal and prohibited. Thank you for your cooperation.