Classify noise between flightlines

[James Carracher]

Hi,

I'm just wondering if something is possible in lastools. I've never personally come across anything but you never know. I do a lot of MLS processing and in it there can be a lot of noise from passing cars or pedestrians etc. Being that I will have at least two flight lines one a road (either side of the road), is there a way to classify groups of points that only exist in one flight line? Ie a passing or parked car was in one pass, but it wasn't in the second. In this case I would classify the points only in one scan as noise.

Do you think this is possible in the current suite?

What I currently do is make a buffer around the trajectory of where I have driven, and then classify based on that and a height above the ground. This works pretty well, but will only work where I have driven and won't find parked cars that have moved or pedestrians. So I was wondering if I could improve my method.

Thanks
James

[Michael Finlay]

Good suggestion James. This capability in LAStools would be very useful. I have wanted this for a while. 

I have wondered if this could be done by modifying lasnoise tool. 

In theory, instead of lasnoise looking for a minimum number of ALL points surrounding the candidate point, perhaps it could consider only surrounding points with a different point source ID different to that of the candidate point. OR, it could only consider points outside a timestamp threshold, like +/- more than 5 seconds relative to the candidate point. This I think could effectivly remove scanner noise and moving objects as long as it was scanned minimum twice if not multiple times. And work for mobile, terrestrial and rapidly growing SLAM LiDAR pointclouds in our favourite format (LAZ). 

Other software appears to use a method like voxel based statistics to mark or classify single source points. Counting different point sources inside each voxel and then re classifies the (original) points within. This would also be handy for a range of different applications, not noise. eg if there is the presence of 2nd, 3rd, 4th, nth, returns, then probably vegetation. My understanding is this is not how lasgrid or lasvoxel currently works. It would be good if it could classify original points based on grid or voxel results. 

LAStools has done wonderful things for aerial LiDAR, but it could evolve to be great at terrestrial, mobile, SLAM LiDAR too by re-thinking how some tools work. 

Regards

Michael 

[James Carracher]

I think you started to make me want it Michael when you told me about the idea years ago!

Using lasnoise in that way is something I hadn't considered before, I think that could work. I am pretty sure other software does use voxels like you said. I've never been able to work it out in that. Using it, or to a lesser extent lasgrid, you'd think that there would be a way to reclassify noise if a step/voxel doesn't have any overlap. Though it might need the flightline too so thinks too far from it (with a low density of points) don't get misclassified

Do you have any ideas Jochen/Silke?

James

[Jochen Rapidlasso]

Hi James and Michael,

we don't want to give too much away, but I think the new lascopy can handle this.

This is almost ready to deliver, you can do a copy/merge of 2 files not only by order or gps/channel but also by many other attributes, like xyz coordinates where you can separate or classify points just in one of both files.

Best,

Jochen @rapidlasso

[Processing Lidar]

"It would be a great idea if lastools manages to incorporate that feature. In my personal experience, when I need to remove noise points in terrestrial scans, what I do is use another software that allows me to separate those points using various techniques, which works 90% of the time. Occasionally, manual intervention has been necessary, but only in specific areas.

--
Download LAStools at
https://rapidlasso.de
Manage your settings at
https://groups.google.com/g/lastools/membership
---
You received this message because you are subscribed to the Google Groups "LAStools - efficient tools for LiDAR processing" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lastools+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lastools/31f43fc9-854b-4203-8813-8ac6cc2de21cn%40googlegroups.com.

[James Carracher]

Hi all,

I have had a play today with the new version of lascopy and I haven't been able to work out how to get it to do what I want do. Are you able to help Jochen?

This is a summary of what I'd like to do. Basically, I will have a single point cloud file which has ID's within it for each flightline. Of course I could separate these if needed, but it would be simpler if I was able to do this with one input file, and then one output file. If I was able to use a wildcard and read in a whole folder to process in parallel that would be awesome. I would have already tiled the files. Each tiled file would go to its own core. 

What I then want to do is look in a 3D search window (say 0.1x0.1x0.1). If the points in this box are only from one flightline, then I would classify the whole box as noise.

I'm not sure if that is possible with lascopy, at least I can't work it out

Regards,

[Jochen Rapidlasso]

Hi James,

here hopefully a solution to the problem.

Attached the generic testdata we used.

We have 90 points in our base flightline, then we copied the flightlines 2 and 3 which differ in
 2: xy 2 2 is missing
    xy 3 3 is missing
    xy 4 4 has delta z > 0.1
    xy 5 5 has delta z > 0.1
 3: xy 2 2 is missing
    xy 4 4 has delta z > 0.1
    xy 6 6 is missing
    xy 7 7 has delta z > 0.1
    4 points differ < 0.1

we expect as class 18(noise):
    xy 2 2 for fl 1
    xy 4 4 for fl 1,2,3
    xy 5 5 for fl 2
    xy 7 7 for fl 3 

This is the script we run:

:: first we convert our sample data to a LAZ file
txt2las64 -i sample_data.csv -o tmp_raw.laz
...
Column description detected. Parse string is xyzp

(266 pts)
...

:: then we split by flightlines, this is mandatory, because lascopy64 always work with source and target files to match
md tmpfl
lassplit64 -i tmp_raw.laz -o tmp -olaz -odir tmpfl

we get
  tmpfl\tmp_0000001.laz
  tmpfl\tmp_0000002.laz
  tmpfl\tmp_0000003.laz
for our 3 flightlines (90/88/88 pts)

:: next we set the classification to each future target to noise. this noise will remain if the other flightlines does not match(>copy)
md tmp18
las2las64 -i tmpfl\*.laz -odir tmp18 -olaz -set_classification 18

:: now we need to merge all source combinations to match against a target
:: for many flightlines this can be done using a looping script or a python snippet
lasmerge64 -i tmpfl\tmp_0000001.laz tmpfl\tmp_0000002.laz -o tmpfl\tmp_00000012.laz
lasmerge64 -i tmpfl\tmp_0000001.laz tmpfl\tmp_0000003.laz -o tmpfl\tmp_00000013.laz
lasmerge64 -i tmpfl\tmp_0000002.laz tmpfl\tmp_0000003.laz -o tmpfl\tmp_00000023.laz

:: now we can copy each flightline as source against the noise classified target. also a script maybe help to do this on large filesets.
md tmpcp
lascopy64 -i tmpfl\tmp_00000023.laz -i tmp18\tmp_0000001.laz -odir tmpcp -olaz -match_xyz 0.1 -copy_classification
lascopy64 -i tmpfl\tmp_00000013.laz -i tmp18\tmp_0000002.laz -odir tmpcp -olaz -match_xyz 0.1 -copy_classification
lascopy64 -i tmpfl\tmp_00000012.laz -i tmp18\tmp_0000003.laz -odir tmpcp -olaz -match_xyz 0.1 -copy_classification

:: we extract all noise to separate files
md tmp18only
las2las64 -i tmpcp\*.laz -olaz -odir tmp18only -cores 16 -keep_class 18
  (2 surviving points each)

:: we extract all matching points out of one copied file - which should have class 0
md tmpcommon
las2las64 -i tmpcp\tmp_0000001.laz -olaz -odir tmpcommon -keep_class 0
  (22 surviving points)

:: we merge the common points with the noise points
lasmerge -i tmpcommon\*.laz -i tmp18only\*.laz -o final.laz
  (94 points)

:: finally we check the result
lasinfo64 -i final.laz

histogram of classification of points:
              88  never classified (0)
               6  Reserved for ASPRS Definition (18)

las2txt64 -i final.laz -o final.txt -parse xyzpc -coldesc

all points has class 0 excpt this:
2.0000000 2.0000000 3.56 1 18
4.0000000 4.0000000 0.38 1 18
4.0000000 4.0000000 0.58 2 18
5.0000000 5.0000000 2.17 2 18
4.0000000 4.0000000 0.68 3 18
7.0000000 7.0000000 2.18 3 18


:: optional cleanup of all temp files
del tmp*.*
rmdir /s /q tmpcommon
rmdir /s /q tmp18only
rmdir /s /q tmpcp
rmdir /s /q tmpfl
rmdir /s /q tmp18


This is done by classes itself.
If the classes are populated already you can use a flag as marker and set noise at the final end using the marker (e.g. synthetic flag).
Also this is done by distance from point to point (which make sense in our opinion).
If you really want to use 3d boxes/voxels you need to assign a voxel to each point before and then work with this voxel IDs.
Probably lasvoxel -compute_IDs_and_voxel_table could help you there.

Cheers,

Jochen @rapidlasso

[James Carracher]

Hi Jochen,

Thanks for the very comprehensive reply.

I can replicate your results for your test file. I'll just have to work out how to make all the comparisons loop (my python is terrible but I think I'll be able to do is easily in FME). I would be slotting this procedure into another script after some classification has happened when I get it to work. So thanks for the tip about the synthetic flags, I've never used them so hopefully I can work it out
I agree that you're method is better, I was just thinking that would be an easy solution (but a clunky result)
Regards

[James Carracher]

Hi Jochen,

I've been on leave for a while so sorry to comment on an old post. I thought everyone might like to know that I did get your method to work. But it is extremely slow and computationally expensive for even three MLS records/flightlines that are side by side. So not really useful in practice. If I followed your instructions correctly
James