LASTile and Lots of Empty Tiles

[Mark Prenter]

Hi, I'm just trying out LasTile to retile some LAZ data with a buffer and I'm finding it's going through a huge number of empty tiles....generating them and then removing them because they have 0 points.  This looks like it's going to take quite a long time.  I looked at the tile naming convention that LasTile uses for it's output tile names and it appears to be using the coordinates of a corner to name the output tile.  For example, when I look at the file it's currently working on, I see a number like 5942000_2092000.  This would be well outside my dataset.  Off to the west and south of where my data falls.  My dataset starts around 5994000,2151000.  That's about 52000 feet to the west and 59000 feet to the south.  Any idea why it would be trying to clip data so far outside the dataset.  It looks like it's going to go through hundreds or even thousands of tiles before it gets to where my data starts.  Ideas?

I originally thought I may have some "stray" data out there so I used LasBoundary to generate shape files of my source data and there's nothing way out there.

I also thought maybe I could use LasIndex to generate statistics and thought that might fix or at least speed up the process, but it didn't seem to help.

Any ideas?

Thanks,

Mark

[Martin Isenburg]

Hello,

lastile merges all the bounding box info from the LAS gile headers to decide on the area to tile. lasboundary encloses the actual extend of the points. maybe some LAS header contains wrong information?

lasboundary -i *.laz -use_bb -oshp

will create shapefiles using the header info only

lasvalidate -i *.laz -no_CRS_fail -o report.xml

will inform about any abnormal LAS files

lasinfo -i *.laz -repair_bb

will set the LAS header bounding box to the correct values

Martin

--
Download LAStools at
http://lastools.org
http://rapidlasso.com
Be social with LAStools at
http://facebook.com/LAStools
http://twitter.com/LAStools
http://linkedin.com/groups/LAStools-4408378
Manage your settings at
http://groups.google.com/group/lastools/subscribe

[Mark Prenter]

It's the "-cores" option.  I was using "-cores 8" in an effort to make processing go faster.  If I leave it off the command line the tiling works normally.

Mark

[Mark Prenter]

Hello Martin, I have used lasboundary to generate shapefiles and this all looks normal.  I ran lasvalidate and the warnings that I see in the report file all have to do with the fact that these are autocorrelated points, generated by Pix4Dmapper.  Such as all points have a return number of 0, Intensity of all points is 0, scan angle rank of all points is 0, etc...

I then ran lasinfo the with repair_bb option but the same issue exists.  The tiling starts around 5936000,2086000 when the LL corner of my data is more like 5994000,2151000.

I ran lastile before on a much smaller test dataset and I didn't recall seeing any issues.  This is a pretty large test data set (7.8 billion points in 6175 LAZ files) which I'm using to try to prove to my boss that we need to purchase this software.  So far, it's been running for 20 hours using 8 cores and still hasn't reached the valid data area yet.  :-(

Perhaps I'm doing something incorrectly?  This is my runstring:

lastile -i *.laz -tile_size 1000 -buffer 100 -flag_as_withheld -odir Tiled/ -o Test1a.laz -feet -elevation_feet -cores 8

Thanks for your help,

Mark

On Tuesday, October 27, 2015 at 4:51:09 PM UTC-7, Mark Prenter wrote:

[Martin Isenburg]

Hello,

the ability to run lastile on multiple '-cores 4' was a recent addition.

http://lastools.org/download/lastile_README.txt

http://rapidlasso.com/lastile

You *MUST* keep in mind that this only makes sense for spatially indexed input ... otherwise you will slow down your operation significantly as each process will have to completely read all input LAS/LAZ files that the tile it is in charge for is overlapping with. However, the behaviour your report sounds surprisingly slow. Any chance you could provide us with some way to debug this on your data set?

Regards,

Martin @rapidlasso

[Mark Prenter]

Hello Martin, here is a link to a very simplified data set.  The full data set is over 17 gigs.

https://drive.google.com/file/d/0B-_RklSLkSRZY1UyVVZnTnlmZ2c/view?usp=sharing

I included 1 tile on the west-most edge, 1 tile on the north-most edge, 1 tile on the east-most edge and 1 tile on the south-most edge, along with my batch file containing the command line.  When you run the batch, you will notice that the images go by on the screen really quickly.  This is NOT the case with the full data set.  It takes 15 to 30 seconds for each group of 8 files to go by....but you can still see the problem.

The first tile it tries to output is 5932000_2085000.  The sample data starts at 5994579, 2151016.  So you see, the software is starting the tiling roughly 62,000 feet to the west and 66,000 feet to the south of my dataset.  Without the -cores 8, this doesn't happen, it begins outputting valid data right away.

I hope this helps....

Mark Prenter

[Floris Groesz]

Any progress on this issue?
I think that I have the same problem.
I have 30 000 laz files (230 GB) that I need to tile. It has been running for a day now and it has created 145 tiles with a total of 3 GB. This is much slower than anything I tiled before.
I have indexed the files and I defined a lower-left coordinate for the first tile. I also see that quite a number of empty tiles are created and removed.
Could it be that the large amount of laz files in the input folder is slowing the process down?

Floris

[Mark Prenter]

I found that it worked fine as long as I didn't use the -cores option and used -extra_pass in my command line.

Mark