Navarra's LiDAR data from SPL100 in FTP site

Paski

unread,

Sep 12, 2018, 7:46:53 AM9/12/18

to LAStools - efficient tools for LiDAR processing

Hi all.

Finally we have in our FTP site (ftp://ftp.cartografia.navarra.es/) for free download all LiDAR data from Single Photon Lidar (ftp://ftp.cartografia.navarra.es/5_LIDAR/5_4_2017_NAV_ca_EPSG25830/).

I announced this intention in a discussion on January: https://groups.google.com/d/topic/lastools/0idqUZwpYPA/discussion

There (ftp://ftp.cartografia.navarra.es/5_LIDAR/) you have data from conventional LiDAR (2011-2012), data from SPL100 LiDAR (2017) in laz 1.4 format, an application to transform data (LiDAR Converter) and, very important, a "readme" document.

We are delighted to offer these data and to have your feddback too.

Don't hesitate to tell us your impresssions.

Thank you

Roberto Pascual

Cartography Section

Navarra's Government

Martin Isenburg

unread,

Sep 12, 2018, 10:38:34 AM9/12/18

to LAStools - efficient command line tools for LIDAR processing

Hello,

thank you for sharing. This is really awesome. That's the first open data release of SPL LiDAR that comes to my mind right now. And when they say that SPL is noisy, they are really not kidding. Have a look at the attached screenshot of the one tile that I have downloaded.

You can improve your data with the following command:

las2las -i original/*.laz ^

-rescale 0.01 0.01 0.01 ^

-auto_reoffset ^

-set_global_encoding_gps_bit 1 ^

-remove_vlr 1 ^

-odir fixed -olaz ^

-cores 4

What will those options do?

-rescale 0.01 0.01 0.01

There is "resolution fluff" because the points are given mm (millimeter) resolution for storage but they actually only have cm (centimeter) accuracy. I can tell from the lasinfo report. How? Well, all coordinates have the exact same digits in their millimeter. All x coordinates have a digit of 9, all y coordinates have a 0, and all z coordinates have a 4. This essentially translates all coordinates by a (0.009, 0.000, 0.004) vector but this is not actually measured information but an artifact of how the centimeter quantized points were written to millimeters somewhere during processing.

-auto_reoffset

Fixes the rather odd offset of 0 4716500 0 in the header to something more suitable such as 500000 4700000 0. Because the offset of x is zero you find very large X coordinates stored in the point records that are ranging from 536000009 to 536999999 compare to the Y coordinates that are ranging from -499990 to 499990.

-set_global_encoding_gps_bit 1

As the time stamp are obviously in Adjusted Standard GPS time and not in GPS week time this bit needs to be set.

-remove_vlr 1

There is a redundant copy of the OGC WKT CRS that is from the days before this was standardized. Now there are two copies of the OGC WKT CRS and the second one (whose *index* is 1) is only meaningful to the old libLAS library and can safely be removed.

A few other things - namely the legacy counters that should be zero - will automatically get fixed. You see the resulting lasinfo report after running this command on that one file below and you will notice that all of the WARNINGS have disappeared.

===============================================================================

C:\LAStools\bin>las2las -i las_ca_536_4717_2017_NAV_EPSG25830.laz ^

-rescale 0.01 0.01 0.01 ^

-auto_reoffset ^

-set_global_encoding_gps_bit 1 ^

-remove_vlr 1 ^

-o las_ca_536_4717_2017_NAV_EPSG25830_fixed.laz

If in addition you run lasoptimize which mainly *reorders* the points for better compression and better spatial indexing you can reduce the file size by another 21 percent as you can see below. Given the amount of data you are hosting that may well be worthwhile. It also means that all future downloads are completed in 20% faster time.

C:\LAStools\bin>lasoptimize.exe -i las_ca_536_4717_2017_NAV_EPSG25830_fixed.laz ^

-o las_ca_536_4717_2017_NAV_EPSG25830_optimized.laz

46.763 secs to write 178360342 bytes for 'las_ca_536_4717_2017_NAV_EPSG25830_optimized.laz' with 20723793 points of type 8

225,347,643 las_ca_536_4717_2017_NAV_EPSG25830.laz

216,991,939 las_ca_536_4717_2017_NAV_EPSG25830_fixed.laz

178,360,342 las_ca_536_4717_2017_NAV_EPSG25830_optimized.laz

===============================================================================

C:\LAStools\bin>lasinfo -i las_ca_536_4717_2017_NAV_EPSG25830.laz

lasinfo (180911) report for 'las_ca_536_4717_2017_NAV_EPSG25830.laz'

reporting all LAS header entries:

file signature: 'LASF'

file source ID: 0

global_encoding: 16

project ID GUID data 1-4: 00000000-0000-0000-0000-000000000000

version major.minor: 1.4

system identifier: 'PDAL'

generating software: 'PDAL 1.7.1 (bbc047)'

file creation day/year: 147/2018

header size: 375

offset to point data: 1797

number var. length records: 2

point data format: 8

point data record length: 38

number of point records: 20723793

number of points by return: 19154236 1491841 74946 2716 54

scale factor x y z: 0.001 0.001 0.001

offset x y z: 0 4716500 0

min x y z: 536000.009 4716000.010 396.794

max x y z: 536999.999 4716999.990 1210.004

start of waveform data packet record: 0

start of first extended variable length record: 0

number of extended_variable length records: 0

extended number of point records: 20723793

extended number of points by return: 19154236 1491841 74946 2716 54 0 0 0 0 0 0 0 0 0 0

variable length header record 1 of 2:

reserved 0

user ID 'LASF_Projection'

record ID 2112

length after header 657

description 'OGC Transformation Record'

WKT OGC COORDINATE SYSTEM:

PROJCS["ETRS89 / UTM zone 30N",GEOGCS["ETRS89",DATUM["European_Terrestrial_Reference_System_1989",SPHEROID["GRS 1980",6378137,298.257222101,AUTHORITY["EPSG"

,"7019"]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY["EPSG","6258"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","912

2"]],AUTHORITY["EPSG","4258"]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",-3],PARAMETER["scale_factor",0.9

996],PARAMETER["false_easting",500000],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH],AUTHOR

ITY["EPSG","25830"]]

variable length header record 2 of 2:

reserved 0

user ID 'liblas'

record ID 2112

length after header 657

description 'OGR variant of OpenGIS WKT SRS'

LASzip compression (version 3.2r2 c3 50000): POINT14 3 RGBNIR14 3

reporting minimum and maximum for all LAS point record entries ...

X 536000009 536999999

Y -499990 499990

Z 396794 1210004

intensity 0 65535

return_number 1 5

number_of_returns 1 5

edge_of_flight_line 0 0

scan_direction_flag 0 1

classification 1 28

scan_angle_rank -127 127

user_data 1 100

point_source_ID 2265 2266

gps_time 189322439.910865 189322898.589792

WARNING: range violates GPS week time specified by global encoding bit 0

Color R 1792 60672

G 4096 61696

B 1792 61952

NIR 0 55040

extended_return_number 1 5

extended_number_of_returns 1 5

extended_classification 1 28

extended_scan_angle -21167 21167

extended_scanner_channel 0 0

WARNING: there is coordinate resolution fluff (x10) in XYZ

number of first returns: 19154236

number of intermediate returns: 78336

number of last returns: 19136717

number of single returns: 17645496

WARNING: point type is 8 but (legacy) number of point records in header is 20723793 instead zero.

WARNING: point type is 8 but (legacy) number of points by return [1] in header is 19154236 instead zero.

WARNING: point type is 8 but (legacy) number of points by return [2] in header is 1491841 instead zero.

WARNING: point type is 8 but (legacy) number of points by return [3] in header is 74946 instead zero.

WARNING: point type is 8 but (legacy) number of points by return [4] in header is 2716 instead zero.

WARNING: point type is 8 but (legacy) number of points by return [5] in header is 54 instead zero.

overview over extended number of returns of given pulse: 17645496 2848776 218492 10759 270 0 0 0 0 0 0 0 0 0 0

histogram of classification of points:

2160704 unclassified (1)

1937735 ground (2)

2920190 low vegetation (3)

2904555 medium vegetation (4)

4231324 high vegetation (5)

13106 building (6)

169660 noise (7)

1775360 Reserved for ASPRS Definition (18)

383874 Reserved for ASPRS Definition (21)

124011 Reserved for ASPRS Definition (22)

634555 Reserved for ASPRS Definition (23)

707615 Reserved for ASPRS Definition (24)

2222590 Reserved for ASPRS Definition (25)

6524 Reserved for ASPRS Definition (26)

38294 Reserved for ASPRS Definition (27)

493696 Reserved for ASPRS Definition (28)

===============================================================================

C:\LAStools\bin>lasinfo -i las_ca_536_4717_2017_NAV_EPSG25830_fixed.laz

lasinfo (180911) report for 'las_ca_536_4717_2017_NAV_EPSG25830_fixed.laz'

reporting all LAS header entries:

file signature: 'LASF'

file source ID: 0

global_encoding: 17

project ID GUID data 1-4: 00000000-0000-0000-0000-000000000000

version major.minor: 1.4

system identifier: 'LAStools (c) by rapidlasso GmbH'

generating software: 'las2las (version 180911)'

file creation day/year: 147/2018

header size: 375

offset to point data: 1086

number var. length records: 1

point data format: 8

point data record length: 38

number of point records: 0

number of points by return: 0 0 0 0 0

scale factor x y z: 0.01 0.01 0.01

offset x y z: 500000 4700000 0

min x y z: 536000.01 4716000.01 396.79

max x y z: 537000.00 4716999.99 1210.00

start of waveform data packet record: 0

start of first extended variable length record: 0

number of extended_variable length records: 0

extended number of point records: 20723793

extended number of points by return: 19154236 1491841 74946 2716 54 0 0 0 0 0 0 0 0 0 0

variable length header record 1 of 1:

reserved 0

user ID 'LASF_Projection'

record ID 2112

length after header 657

description 'OGC Transformation Record'

WKT OGC COORDINATE SYSTEM:

PROJCS["ETRS89 / UTM zone 30N",GEOGCS["ETRS89",DATUM["European_Terrestrial_Reference_System_1989",SPHEROID["GRS 1980",6378137,298.257222101,AUTHORITY["EPSG"

,"7019"]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY["EPSG","6258"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","912

2"]],AUTHORITY["EPSG","4258"]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",-3],PARAMETER["scale_factor",0.9

996],PARAMETER["false_easting",500000],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH],AUTHOR

ITY["EPSG","25830"]]

LASzip compression (version 3.2r4 c3 50000): POINT14 3 RGBNIR14 3

reporting minimum and maximum for all LAS point record entries ...

X 3600001 3700000

Y 1600001 1699999

Z 39679 121000

intensity 0 65535

return_number 1 5

number_of_returns 1 5

edge_of_flight_line 0 0

scan_direction_flag 0 1

classification 1 28

scan_angle_rank -127 127

user_data 1 100

point_source_ID 2265 2266

gps_time 189322439.910865 189322898.589792

Color R 1792 60672

G 4096 61696

B 1792 61952

NIR 0 55040

extended_return_number 1 5

extended_number_of_returns 1 5

extended_classification 1 28

extended_scan_angle -21167 21167

extended_scanner_channel 0 0

number of first returns: 19154236

number of intermediate returns: 78336

number of last returns: 19136717

number of single returns: 17645496

overview over extended number of returns of given pulse: 17645496 2848776 218492 10759 270 0 0 0 0 0 0 0 0 0 0

histogram of classification of points:

2160704 unclassified (1)

1937735 ground (2)

2920190 low vegetation (3)

2904555 medium vegetation (4)

4231324 high vegetation (5)

13106 building (6)

169660 noise (7)

1775360 Reserved for ASPRS Definition (18)

383874 Reserved for ASPRS Definition (21)

124011 Reserved for ASPRS Definition (22)

634555 Reserved for ASPRS Definition (23)

707615 Reserved for ASPRS Definition (24)

2222590 Reserved for ASPRS Definition (25)

6524 Reserved for ASPRS Definition (26)

38294 Reserved for ASPRS Definition (27)

493696 Reserved for ASPRS Definition (28)

--
Download LAStools at
http://lastools.org
http://rapidlasso.com
Be social with LAStools at
http://facebook.com/LAStools
http://twitter.com/LAStools
http://linkedin.com/groups/LAStools-4408378
Manage your settings at
http://groups.google.com/group/lastools/subscribe

Navarra_Open_LiDAR_SPL.jpg

Stoker, Jason

unread,

Sep 12, 2018, 11:15:31 AM9/12/18

to <lastools@googlegroups.com>

Hi all- thought we'd also share that some of our SPL data is publicly-available as well, since most here don't follow too closely what we're doing in the US for 3DEP. Over 3 TB of laz data for people to check out. This is for a large area in South Dakota, USA, broken in to 3 blocks:

ftp://rockyftp.cr.usgs.gov/vdelivery/Datasets/Staged/Elevation/LPC/Projects/USGS_LPC_SD_MORiver_Woolpert_B1_2016_LAS_2018/

ftp://rockyftp.cr.usgs.gov/vdelivery/Datasets/Staged/Elevation/LPC/Projects/USGS_LPC_SD_MORiver_Woolpert_B2_2016_LAS_2018/

ftp://rockyftp.cr.usgs.gov/vdelivery/Datasets/Staged/Elevation/LPC/Projects/USGS_LPC_SD_MORiver_Woolpert_B3_2016_LAS_2018/

or, if you are on AWS, we have it mirrored in a S3 requester-pays bucket, so if you have an AWS account and $ you can work with the data easier/faster (note you'll need to get at it using cloudberry or some other cloud explorer application- these links won't work directly):

https://usgs-lidar.s3-us-west-2.amazonaws.com/Projects/USGS_LPC_SD_MORiver_Woolpert_B1_2016_LAS_2018/

https://usgs-lidar.s3-us-west-2.amazonaws.com/Projects/USGS_LPC_SD_MORiver_Woolpert_B2_2016_LAS_2018/

https://usgs-lidar.s3-us-west-2.amazonaws.com/Projects/USGS_LPC_SD_MORiver_Woolpert_B3_2016_LAS_2018/

Would be interested to see how these data compare with the Navarra data, as we're learning how to best handle these new types of data as well.

Cheers,
Jason

Jason M. Stoker, Ph.D
US Geological Survey
National Geospatial Program
Office: 970-226-9227
Cell: 605-496-3513
My USGS Profile

Martin Isenburg

unread,

Sep 12, 2018, 12:09:01 PM9/12/18

to LAStools - efficient command line tools for LIDAR processing

Hello Jason,

looks good. Can you tell us what the

extended_scanner_channel 0 3

field is encoding? When I copy the scanner channel field into the point source ID field and then map it to different colors

C:\LAStools\bin>lasview -i USGS_LPC_SD_MORiver_Woolpert_B1_2016_14TNP120310_LAS_2018.laz ^

-copy_scanner_channel_into_point_source ^

-color_by_flightline

then I get a pattern that reminds me of these magic eye pictures of the 90th. I see a sailboat, and you (see attached)?

C:\LAStools\bin>lasinfo -i USGS_LPC_SD_MORiver_Woolpert_B1_2016_14TNP120310_LAS_2018.laz

lasinfo (180911) report for 'USGS_LPC_SD_MORiver_Woolpert_B1_2016_14TNP120310_LAS_2018.laz'

reporting all LAS header entries:

file signature: 'LASF'

file source ID: 0

global_encoding: 17

project ID GUID data 1-4: 194774FA-35FE-4591-D484-010AFA13F6D9

version major.minor: 1.4

system identifier: 'Woolpert LAS'

generating software: 'GeoCue LAS Updater'

file creation day/year: 332/2017

header size: 375

offset to point data: 1376

number var. length records: 1

point data format: 6

point data record length: 30

number of point records: 0

number of points by return: 0 0 0 0 0

scale factor x y z: 0.01 0.01 0.01

offset x y z: 0 0 0

min x y z: 512000.00 4831000.00 490.52

max x y z: 512999.99 4831999.99 864.77

start of waveform data packet record: 0

start of first extended variable length record: 0

number of extended_variable length records: 0

extended number of point records: 37973595

extended number of points by return: 31810075 5717990 428450 16724 356 0 0 0 0 0 0 0 0 0 0

variable length header record 1 of 1:

reserved 0

user ID 'LASF_Projection'

record ID 2112

length after header 943

description 'OGC WKT Coordinate System'

WKT OGC COORDINATE SYSTEM:

COMPD_CS["NAD83(2011) / UTM zone 14N + NAVD88 height - Geoid12B (metre)",PROJCS["NAD83(2011) / UTM zone 14N",GEOGCS["NAD83(2011)",DATUM["NAD83_National_Spat

ial_Reference_System_2011",SPHEROID["GRS 1980",6378137,298.257222101,AUTHORITY["EPSG","7019"]],AUTHORITY["EPSG","1116"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","

8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","6318"]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0]

,PARAMETER["central_meridian",-99],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",500000],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EP

SG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH],AUTHORITY["EPSG","6343"]],VERT_CS["NAVD88 height - Geoid12B (metre)",VERT_DATUM["North American Vertica

l Datum 1988",2005,AUTHORITY["EPSG","5103"]],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Gravity-related height",UP],AUTHORITY["EPSG","5703"]]]

the header is followed by 4 user-defined bytes

LASzip compression (version 3.1r0 c3 50000): POINT14 3

reporting minimum and maximum for all LAS point record entries ...

X 51200000 51299999

Y 483100000 483199999

Z 49052 86477

intensity 3139 12341

return_number 1 5

number_of_returns 1 5

edge_of_flight_line 0 0

scan_direction_flag 0 1

classification 1 10

scan_angle_rank -127 127

user_data 1 100

point_source_ID 1061 2003

gps_time 176475478.674116 176478540.635619

extended_return_number 1 5

extended_number_of_returns 1 5

extended_classification 1 10

extended_scan_angle -21167 21167

extended_scanner_channel 0 3

number of first returns: 31810075

number of intermediate returns: 54

number of last returns: 37973169

number of single returns: 31809703

overview over extended number of returns of given pulse: 31809703 5718314 428492 16730 356 0 0 0 0 0 0 0 0 0 0

histogram of classification of points:

13606181 unclassified (1)

22967251 ground (2)

1164317 noise (7)

233073 water (9)

2773 rail (10)

+-> flagged as withheld: 1164317

+-> flagged as extended overlap: 24863425

USGS_Woolpert_Open_LiDAR_SPL.jpg

Gottfried Mandlburger

unread,

Sep 12, 2018, 12:14:58 PM9/12/18

to last...@googlegroups.com

Martin, Jason, Roberto, all, Great to have so many SPL data available from sites in the US and now also Europe! ad Martin: Evidently there are pros and cons of SPL vs MPL (multi-photon lidar). SPL: Higher acquisition speed (km2/h), lower ranging accuracy, occurrence of clutter points; MPL: cleaner point cloud, higher ranging precision, lower coverage performance. I had the chance to evaluate the Navarra SPL dataset (thanks Roberto!!) in comparison to a topo-bathymetric (green lidar) dataset with the same point density (both: 14 pts/m2). The results w.r.t. local spread of point cloud and achievable strip fitting, both measured at impenetrable, smooth surfaces: SPL MPL (topo-bathy) Precision/roughness [cm] 3.0 1.3 Strip fitting precision [cm] 3.9 1.2 More details about the above study in an ISPRS Archives paper written for the TC2 midterm symposium in Karlsruhe (next month). And, SPL requires more robust data processing strategies, which is a challenge for lidar processing software. Kind regards, Gottfried

On 12.09.2018 17:12, Stoker, Jason wrote:

Hi all- thought we'd also share that some of our SPL data is publicly-available as well, since most here don't follow too closely what we're doing in the US for 3DEP. Over 3 TB of laz data for people to check out. This is for a large area in South Dakota, USA, broken in to 3 blocks:

ftp://rockyftp.cr.usgs.gov/vdelivery/Datasets/Staged/Elevation/LPC/Projects/USGS_LPC_SD_MORiver_Woolpert_B1_2016_LAS_2018/

ftp://rockyftp.cr.usgs.gov/vdelivery/Datasets/Staged/Elevation/LPC/Projects/USGS_LPC_SD_MORiver_Woolpert_B2_2016_LAS_2018/

ftp://rockyftp.cr.usgs.gov/vdelivery/Datasets/Staged/Elevation/LPC/Projects/USGS_LPC_SD_MORiver_Woolpert_B3_2016_LAS_2018/

or, if you are on AWS, we have it mirrored in a S3 requester-pays bucket, so if you have an AWS account and $ you can work with the data easier/faster (note you'll need to get at it using cloudberry or some other cloud explorer application- these links won't work directly):

https://usgs-lidar.s3-us-west-2.amazonaws.com/Projects/USGS_LPC_SD_MORiver_Woolpert_B1_2016_LAS_2018/

https://usgs-lidar.s3-us-west-2.amazonaws.com/Projects/USGS_LPC_SD_MORiver_Woolpert_B2_2016_LAS_2018/

https://usgs-lidar.s3-us-west-2.amazonaws.com/Projects/USGS_LPC_SD_MORiver_Woolpert_B3_2016_LAS_2018/

Would be interested to see how these data compare with the Navarra data, as we're learning how to best handle these new types of data as well.

Cheers,
Jason

-- 
Dr. Gottfried Mandlburger

Tel.: +43 1 58801 12235
Fax.: +43 1 58801 12299
http://www.ipf.tuwien.ac.at
    _____ _____ _____
   /____// ___//    /  TU Wien
  // __ / /__ / // /  Department of Geodesy and Geoinformation
 //__/// /__ / // /  Research Group Photogrammetry
/____//____//____/  Gusshausstrasse 27-29, A-1040 Vienna

Stoker, Jason

unread,

Sep 12, 2018, 5:52:49 PM9/12/18

to <lastools@googlegroups.com>

Hi Martin- working to get clarification, but from what I understand for SPL100, because they use 100 beamlets to collect data rather than one or two, displaying the data using this field isn’t ideal. So when a user views the data by channel you see the points displayed with their unique channel value of 1-100, scaled to two bits so it looks a little messy. Sounds like we may need to change how that field works in LAS for this type of system?

Jason M. Stoker, Ph.D
US Geological Survey
National Geospatial Program
Office: 970-226-9227
Cell: 605-496-3513
My USGS Profile

Martin Isenburg

unread,

Sep 14, 2018, 5:14:37 PM9/14/18

to LAStools - efficient command line tools for LIDAR processing

Hello Jason.

I had a closer look at the SPL data. In the blog post below I show how simple reordering and clever remapping of single photon LiDAR data can reduce file size by a whopping 50%. I also show that there is at least one Leica’s SPL100 sensor out there that should be called SPL99 because one of its 100 beamlets (the one with beamlet ID 53) does not seem to produce any data … (-:

http://rapidlasso.com/2018/09/14/scrutinizing-lidar-data-from-leicas-single-photon-scanner-spl100-aka-spl99/

Regards.

Martin @rapidlasso

Jason Stoker

unread,

Sep 14, 2018, 6:03:09 PM9/14/18

to last...@googlegroups.com

Thanks Martin! A great example of how making data available to the public for free can reap benefits not expected- even on the data provider and data steward side.

Jason

--

Martin Isenburg

unread,

Sep 16, 2018, 1:11:20 PM9/16/18

to LAStools - efficient command line tools for LIDAR processing

Hello,

first a word of warning. By default lasoptimize will set the user data field to zero. Read the README. For SPL100 data where the beamlet ID is stored to the user data field you should probably use lasoptimize with the option '-do_not_zero_user_data'. Or better follow the more complex procedure outlined here. But first some more investigations on the Navarra data that is much more "raw" than the USGS data so if you really want to see what raw SPL100 data looks like the Navarra data is a better choice (yes, SPL100 not SPL99 because all beamlets are firing).

First I cut a quarter-second slice from the downloaded tile:

las2las -i las_ca_536_4717_2017_NAV_EPSG25830.laz ^

-keep_gps_time 189322449 189322449.25 ^

-ocut 10 -odix _quartersec_189322449 -olaz

We see how much noise there is (see first two attached pictures) when the data is raw. Here all the first returns are still there. Most of them are missing in the USGS data as most of them are noise.

lasview -i las_ca_536_4717_2017_NAV_quartersec_189322449.laz

Now we sort this, first by GPS time to group together all returns from one beamlet shot, then by user data to group together all returns of each beamlet, and then by return to make sure the first return appears before the second return.

lassort -i las_ca_536_4717_2017_NAV_quartersec_189322449.laz ^

-gps_time ^

-user_data ^

-return_number ^

-odix _sorted -olaz

Because first and last returns are now back to back in the point order we can use the "pulses ..." popup menu of lasview and draw a pulse of x meters from the last return to and through the first return. Where all pulses converge is where the scanner was. We see that this happens about 4000 meters above ground for this data set as evidenced by the converging cone of pulses (the next three attached pictures).

lasview -i las_ca_536_4717_2017_NAV_quartersec_189322449_sorted.laz

Now we want to look at the time stamps of each individual and the time stamps differences between every shot of 100 beamlets.

las2txt -i las_ca_536_4717_2017_NAV_quartersec_189322449_sorted.laz -parse turnxyz -stdout | more

We find that the time stamps are 189322449.018789, 189322449.018809, 189322449.018829, 189322449.018849, 189322449.018869, ... and to cut out the one shot of 100 beamlets at time stamp 189322449.018849 we use this command:

las2las -i las_ca_536_4717_2017_NAV_quartersec_189322449_sorted.laz ^

-keep_gps_time 189322449.018839 189322449.018859 ^

-o las_ca_536_4717_2017_NAV_beamlet_189322449_018849.laz

And now we can print out this one shot of 100 beamlets at time 189322449.018849.

las2txt -i las_ca_536_4717_2017_NAV_beamlet_189322449_018849.laz -parse turnxyz -stdout

Note how the millimeter digit of each coordinate really is identical as we had pointed out in the first post. Also note that three individual beamlets that generated two returns.

189322449.018849 1 1 1 536204.749 4716004.810 863.674

189322449.018849 2 1 1 536205.159 4716007.130 863.114

189322449.018849 3 1 1 536205.639 4716006.150 863.654

189322449.018849 4 1 1 536206.149 4716005.040 863.974

189322449.018849 5 1 1 536202.689 4716006.490 863.364

189322449.018849 9 1 1 536208.839 4716006.390 862.924

189322449.018849 11 1 1 536208.689 4716006.890 863.024

189322449.018849 13 1 1 536210.029 4716006.660 872.804

189322449.018849 14 1 1 536203.899 4716005.170 863.564

189322449.018849 15 1 1 536204.859 4716006.010 863.154

189322449.018849 16 1 1 536207.179 4716005.740 876.384

189322449.018849 18 1 1 536207.589 4716007.750 862.674

189322449.018849 19 1 1 536207.209 4716005.440 863.254

189322449.018849 20 1 1 536206.659 4716006.960 863.394

189322449.018849 23 1 1 536205.589 4716007.340 862.424

189322449.018849 24 1 1 536206.349 4716006.290 863.844

189322449.018849 26 1 1 536205.409 4716004.930 863.684

189322449.018849 27 1 1 536205.289 4716006.680 863.064

189322449.018849 28 1 1 536207.449 4716005.660 869.584

189322449.018849 30 1 1 536203.089 4716006.720 862.704

189322449.018849 31 1 1 536204.409 4716007.030 862.824

189322449.018849 33 1 1 536204.679 4716006.490 863.044

189322449.018849 34 1 1 536208.209 4716006.260 862.954

189322449.018849 37 1 1 536204.589 4716005.260 863.644

189322449.018849 38 1 1 536206.749 4716006.430 862.834

189322449.018849 39 1 1 536203.339 4716004.990 863.834

189322449.018849 41 1 1 536203.089 4716005.520 863.644

189322449.018849 43 1 2 536220.079 4716008.190 906.814

189322449.018849 43 2 2 536208.149 4716007.950 862.494

189322449.018849 48 1 1 536206.079 4716006.790 863.454

189322449.018849 49 1 1 536206.509 4716005.770 863.644

189322449.018849 51 1 1 536202.479 4716007.020 863.294

189322449.018849 52 1 1 536203.569 4716007.370 862.674

189322449.018849 53 1 1 536202.189 4716007.590 862.884

189322449.018849 54 1 1 536215.379 4716008.110 907.184

189322449.018849 55 1 1 536207.289 4716010.000 862.044

189322449.018849 56 1 1 536205.899 4716009.730 861.734

189322449.018849 57 1 1 536207.469 4716009.520 862.084

189322449.018849 58 1 1 536206.319 4716009.180 862.434

189322449.018849 59 1 1 536202.639 4716009.430 862.074

189322449.018849 60 1 1 536204.809 4716007.690 862.604

189322449.018849 62 1 1 536205.089 4716010.070 861.714

189322449.018849 63 1 1 536203.959 4716009.730 862.194

189322449.018849 64 1 1 536206.389 4716010.370 861.754

189322449.018849 66 1 1 536205.019 4716008.810 862.464

189322449.018849 67 1 2 536205.619 4716009.110 870.044

189322449.018849 67 2 2 536203.529 4716009.060 862.204

189322449.018849 68 1 1 536202.249 4716008.750 862.284

189322449.018849 69 1 1 536206.079 4716008.010 862.534

189322449.018849 70 1 1 536203.779 4716008.560 862.514

189322449.018849 72 1 1 536202.659 4716008.240 863.134

189322449.018849 73 1 1 536205.899 4716008.510 862.544

189322449.018849 74 1 1 536207.179 4716008.790 862.574

189322449.018849 75 1 1 536202.599 4716009.300 864.314

189322449.018849 79 1 1 536203.879 4716008.070 862.264

189322449.018849 80 1 1 536206.659 4716009.860 861.984

189322449.018849 81 1 1 536205.319 4716009.540 861.844

189322449.018849 82 1 1 536206.879 4716009.330 862.164

189322449.018849 83 1 1 536205.569 4716009.020 862.034

189322449.018849 84 1 1 536203.259 4716009.590 861.974

189322449.018849 86 1 1 536204.239 4716009.230 862.564

189322449.018849 87 1 1 536205.709 4716010.230 861.644

189322449.018849 88 1 1 536204.529 4716009.910 862.024

189322449.018849 89 1 1 536206.979 4716010.560 861.684

189322449.018849 90 1 2 536208.949 4716008.440 876.304

189322449.018849 90 2 2 536205.209 4716008.370 862.454

189322449.018849 91 1 1 536204.729 4716009.380 862.074

189322449.018849 94 1 1 536206.729 4716008.160 862.594

189322449.018849 95 1 1 536203.059 4716008.430 862.304

189322449.018849 96 1 1 536207.879 4716008.480 862.184

189322449.018849 99 1 1 536207.839 4716008.960 862.714

lasview -i las_ca_536_4717_2017_NAV_beamlet_189322449_018849.laz ^

-scale_user_data 2.5

lasview -i las_ca_536_4717_2017_NAV_beamlet_189322449_018849.laz ^

-map_user_data beamlet_ID_map.txt ^

-scale_user_data 2.5

In the attached images you can see how the two double returns are connected. You can also see that the low intensity returns are likely the noisy ones. Finally you can see that the SPL used in Navarra uses the same beamlet ID order. The same mapping table (attached) can be used to make the order more spatially coherent.

Finally we compress the original files putting all these ideas together:

las2las -i las_ca_536_4717_2017_NAV_EPSG25830.laz ^

-rescale 0.01 0.01 0.01 ^

-auto_reoffset ^

-set_global_encoding_gps_bit 1 ^

-remove_vlr 1 ^

-map_user_data beamlet_ID_map.txt ^

-o las_ca_536_4717_2017_NAV_EPSG25830_fixed.laz

lassort -i las_ca_536_4717_2017_NAV_EPSG25830_fixed.laz ^

-gps_time ^

-user_data ^

-return_number ^

-o las_ca_536_4717_2017_NAV_EPSG25830_sorted.laz

Here the resulting file size:

225,347,643 las_ca_536_4717_2017_NAV_EPSG25830.laz

216,947,678 las_ca_536_4717_2017_NAV_EPSG25830_fixed.laz

172,185,365 las_ca_536_4717_2017_NAV_EPSG25830_sorted.laz

Regards,

Martin @rapidlasso

PS: Note that some of the commands used here will only be available in the next LAStools release.

beamlet_ID_map.txt

Navarra_Open_LiDAR_SPL_beamlet_189322449_018849_user_data_remapped.jpg

Navarra_Open_LiDAR_SPL_quartersecond_189322449_classification.jpg

Navarra_Open_LiDAR_SPL_quartersecond_189322449_return_type.jpg

Navarra_Open_LiDAR_SPL_quartersecond_189322449_pulses_2000meter.jpg

Navarra_Open_LiDAR_SPL_quartersecond_189322449_pulses_3000meter.jpg

Navarra_Open_LiDAR_SPL_quartersecond_189322449_pulses_4000meter.jpg

Navarra_Open_LiDAR_SPL_beamlet_189322449_018849_return_type.jpg

Navarra_Open_LiDAR_SPL_beamlet_189322449_018849_intensity.jpg

Navarra_Open_LiDAR_SPL_beamlet_189322449_018849_user_data.jpg

Reply all

Reply to author

Forward