-waveform_with_map laszip option

85 views
Skip to first unread message

Mauro Assis

unread,
Oct 9, 2017, 11:35:21 PM10/9/17
to LAStools - efficient tools for LiDAR processing
Hi,

I was packing some full waveform clouds using laszip -waveform option and I got a message asking me to use -waveform_with_map. I tried and got a Windows APPCRASH error message.

What could be happening? What this option means (is not documented)?

Below the lasinfo output:

reporting all LAS header entries:
  file signature:             'LASF'
  file source ID:             580
  global_encoding:            4
  project ID GUID data 1-4:   4198F16B-5135-A34A-D3B2-DEEC04E72183
  version major.minor:        1.3
  system identifier:          'EBA Project'
  generating software:        'Trimble LPMaster'
  file creation day/year:     279/2016
  header size:                235
  offset to point data:       1123
  number var. length records: 11
  point data format:          4
  point data record length:   57
  number of point records:    53780097
  number of points by return: 39712769 12286010 1686661 92523 2107
  scale factor x y z:         0.001 0.001 0.001
  offset x y z:               500000 280000 0
  min x y z:                  784122.068 279752.433 16.940
  max x y z:                  799278.617 281222.465 233.766
  start of waveform data packet record: 0
variable length header record 1 of 11:
  reserved             0
  user ID              'LASF_Projection'
  record ID            34735
  length after header  32
  description          'plane metric system'
    GeoKeyDirectoryTag version 1.1.0 number of keys 3
      key 1024 tiff_tag_location 0 count 1 value_offset 1 - GTModelTypeGeoKey: ModelTypeProjected
      key 3072 tiff_tag_location 0 count 1 value_offset 31974 - ProjectedCSTypeGeoKey: SIRGAS 2000 / UTM zone 20N
      key 3076 tiff_tag_location 0 count 1 value_offset 9001 - ProjLinearUnitsGeoKey: Linear_Meter
variable length header record 2 of 11:
  reserved             0
  user ID              'LASF_Spec'
  record ID            100
  length after header  26
  description          'WDP_1'
  index 1 bits/sample 16 compression 0 samples 80 temporal 1000 gain 1, offset 0
variable length header record 3 of 11:
  reserved             0
  user ID              'LASF_Spec'
  record ID            101
  length after header  26
  description          'WDP_2'
  index 2 bits/sample 16 compression 0 samples 160 temporal 1000 gain 1, offset 0
variable length header record 4 of 11:
  reserved             0
  user ID              'LASF_Spec'
  record ID            102
  length after header  26
  description          'WDP_3'
  index 3 bits/sample 16 compression 0 samples 240 temporal 1000 gain 1, offset 0
variable length header record 5 of 11:
  reserved             0
  user ID              'LASF_Spec'
  record ID            103
  length after header  26
  description          'WDP_4'
  index 4 bits/sample 16 compression 0 samples 0 temporal 1000 gain 1, offset 0
variable length header record 6 of 11:
  reserved             0
  user ID              'LASF_Spec'
  record ID            104
  length after header  26
  description          'WDP_5'
  index 5 bits/sample 16 compression 0 samples 0 temporal 1000 gain 1, offset 0
variable length header record 7 of 11:
  reserved             0
  user ID              'LASF_Spec'
  record ID            105
  length after header  26
  description          'WDP_6'
  index 6 bits/sample 16 compression 0 samples 0 temporal 1000 gain 1, offset 0
variable length header record 8 of 11:
  reserved             0
  user ID              'LASF_Spec'
  record ID            106
  length after header  26
  description          'WDP_7'
  index 7 bits/sample 16 compression 0 samples 0 temporal 1000 gain 1, offset 0
variable length header record 9 of 11:
  reserved             0
  user ID              'LASF_Spec'
  record ID            107
  length after header  26
  description          'WDP_8'
  index 8 bits/sample 16 compression 0 samples 0 temporal 1000 gain 1, offset 0
variable length header record 10 of 11:
  reserved             0
  user ID              'LASF_Spec'
  record ID            108
  length after header  26
  description          'WDP_9'
  index 9 bits/sample 16 compression 0 samples 0 temporal 1000 gain 1, offset 0
variable length header record 11 of 11:
  reserved             0
  user ID              'LASF_Spec'
  record ID            109
  length after header  26
  description          'WDP_10'
  index 10 bits/sample 16 compression 0 samples 0 temporal 1000 gain 1, offset 0
the header is followed by 2 user-defined bytes
reporting minimum and maximum for all LAS point record entries ...
  X           284122068  299278617
  Y             -247566    1222465
  Z               16940     233766
  intensity           5      65535
  return_number       1          6
  number_of_returns   1          6
  edge_of_flight_line 0          1
  scan_direction_flag 1          1
  classification      0          4
  scan_angle_rank   -21         21
  user_data           0          0
  point_source_ID     1          1
  gps_time 487690.562712 487959.006252
  Wavepacket Index    1 3
             Offset   0 9451661600
             Size     160 480
             Location 1566.86 228567
             Xt       -1.66028e-005 1.1569e-005
             Yt       -8.4808e-005 7.79343e-005
             Zt       0.000123622 0.000149931
WARNING: 2 points outside of header bounding box
number of first returns:        39712769
number of intermediate returns: 1781318
number of last returns:         39712769
number of single returns:       27426759
WARNING: there are 27 points with return number 6
overview over number of returns of given pulse: 27426759 21198698 4782414 361664 10400 162 0
histogram of classification of points:
            1260  never classified (0)
        21547516  ground (2)
        32231321  medium vegetation (4)
real max x larger than header max x by 0.000287
real max y larger than header max y by 0.000322


Martin Isenburg

unread,
Oct 10, 2017, 10:25:16 AM10/10/17
to LAStools - efficient command line tools for LIDAR processing
Hello,

the "waveform compression" part of LASzip is more or less experimental. I really dislike the FWF part of LAS 1.3 / LAS 1.4 (aka point types 4, 5, 9, and 10) which is what spurred the PulseWaves effort. I suggest you simply do not compress the waveforms. Compression is not that great anyways (1:2 or so). However, I'd be interested in debugging this eventually so if you can put your file online somewhere and send me a link?

What happens is that if the digitized waveforms are not stored in the same order as the compressed returns that reference the waveforms then I need to reorder the waveforms for compression purposes to match the order of the points. That is why it is asking you to use this 'waveform_with_map' option and why that fails requires debugging. It's rare that people try to compress the waveforms that are part of the FWF extension of LAS 1.3/1.4 I guess ...

Regards,

Martin @rapidlasso

PS: Want to meet rapidlasso GmbH in person and talk LiDAR, photogrammetry points and LAStools? See our upcoming event schedule here: http://rapidlasso.com/events

Mauro Assis

unread,
Oct 10, 2017, 9:20:55 PM10/10/17
to LAStools - efficient tools for LiDAR processing
Tks, Martin.

I´ll put the cloud in Mega and send you the corresponding link.

In fact, the compression I´m achieving with laszip is 1:4, what´s nice for our backup purposes.

Cheers,

Mauro

Susana Gonzalez

unread,
Aug 29, 2018, 12:30:13 AM8/29/18
to last...@googlegroups.com

Hi Martin,

I got some full waveform exported directly into LAZ from LPMaster, but some of them are in LAS with wdp files, uploaded one, what would be the best line of code to compress the data? Or as you said before, we leave it.

Thanks

Susana

Martin Isenburg

unread,
Sep 4, 2018, 8:59:36 AM9/4/18
to LAStools - efficient command line tools for LIDAR processing
Hello,

it seems  compress nicely with

laszip -i cld_170604_01_042610_018.las ^
          -waveform ^
          -o cld_170604_01_042610_018_new.laz
WARNING: number of samples for wave packet descr 4 is zero
WARNING: number of samples for wave packet descr 5 is zero
WARNING: number of samples for wave packet descr 6 is zero
WARNING: number of samples for wave packet descr 7 is zero
WARNING: number of samples for wave packet descr 8 is zero
WARNING: number of samples for wave packet descr 9 is zero
WARNING: number of samples for wave packet descr 10 is zero
WARNING: gap in waveform offsets.
WARNING: last offset plus size was 60 but new offset is 160 (for point 2)

However, note that the '-waveform' extension to the external WDP files of the (unloved) LAS FWF extension has seen much less testing than the core LASzip compressor. No guarantees but if you make a large scale test that would be useful. But the savings are significant.

BEFORE:
  53,740,648 cld_170604_01_042610_018.las
 151,487,840 cld_170604_01_042610_018.wdp

AFTER:
  13,371,956 cld_170604_01_042610_018_new.laz
  39,172,226 cld_170604_01_042610_018_new.wdz

At first the visuals are nice as the attached images show. But closer inspection reveals that the waveforms directions do not properly align nicely with the discrete returns. The red and the blue dot that are the first and the last return of a laser pulse generating two returns should lie exactly on one line but they do not as the attached images demonstrate. Did you maybe reproject your points from one CRS to another CRS? This would require also changing the waveform direction vectors. One reason I have fallen so out of love with the LAS FWF extension. (-:

lasview -i cld_170604_01_042610_018_new.laz

However, I'd first tackle the resolution and the bounding box values as well as the non-zero legacy counts before archiving this data with

las2las -i cld_170604_01_042610_018.las ^
            -rescale 0.01 0.01 0.01 ^
            -remove_vlrs_from_to 4 10 ^
            -o cld_170604_01_042610_018_nicer.las

as those seems to be a little ugly as lasinfo complains (see below all the red marked entries in the lasinfo report before and how much nicer it looks after). Careful removing the fixed range of VLRs in case there are other files where there are more proper WavePacketDescriptors because different digitization sample numbers are in use. And the resulting "nicer" LAS files compress even better:

BEFORE:  
13,371,956 cld_170604_01_042610_018_new.laz

AFTER:  
11,894,792 cld_170604_01_042610_018_nicer.laz

============================================================================= 
 
lasinfo -i cld_170604_01_042610_018.las 

lasinfo (180831) report for 'cld_170604_01_042610_018.las'
reporting all LAS header entries:
  file signature:             'LASF'
  file source ID:             18
  global_encoding:            21
  project ID GUID data 1-4:   01045231-0000-0000-6F42-746F0000724E
  version major.minor:        1.4
  system identifier:          'AX60_180-901 EXTRACTION'
  generating software:        'Trimble LPMaster'
  file creation day/year:     23/2018
  header size:                375
  offset to point data:       1913
  number var. length records: 11
  point data format:          9
  point data record length:   59
  number of point records:    910825
  number of points by return: 738856 140862 27810 3129 162
  scale factor x y z:         0.001 0.001 0.001
  offset x y z:               1600000 5745000 0
  min x y z:                  1925836.458 5744947.024 181.659
  max x y z:                  1926829.675 5745560.160 240.288
WARNING: stored resolution of min_x not compatible with x_offset and x_scale_factor: 1925836.4580397061
WARNING: stored resolution of min_y not compatible with y_offset and y_scale_factor: 5744947.0237300051
WARNING: stored resolution of min_z not compatible with z_offset and z_scale_factor: 181.65890502929687
WARNING: stored resolution of max_x not compatible with x_offset and x_scale_factor: 1926829.6747145108
WARNING: stored resolution of max_y not compatible with y_offset and y_scale_factor: 5745560.1595842494
WARNING: stored resolution of max_z not compatible with z_offset and z_scale_factor: 240.28761291503906
  start of waveform data packet record: 0
  start of first extended variable length record: 0
  number of extended_variable length records: 0
  extended number of point records: 910825
  extended number of points by return: 738856 140862 27810 3129 162 6 0 0 0 0 0 0 0 0 0
variable length header record 1 of 11:
  reserved             0
  user ID              'LASF_Projection'
  record ID            2112
  length after header  682
  description          'WKT description'
    WKT OGC COORDINATE SYSTEM:
    PROJCS["NZGD2000 / New Zealand Transverse Mercator 2000",GEOGCS["NZGD2000",DATUM["New_Zealand_Geodetic_Datum_2000",SPHEROID["GRS 1980",6378137,298.257222101
,AUTHORITY["EPSG","7019"]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY["EPSG","6167"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.01745329251994328,AUT
HORITY["EPSG","9122"]],AUTHORITY["EPSG","4167"]],UNIT["metre",1,AUTHORITY["EPSG","9001"]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PA
RAMETER["central_meridian",173],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",1600000],PARAMETER["false_northing",10000000],AUTHORITY["EPSG","2193"
],AXIS["Easting",EAST],AXIS["Northing",NORTH]]
  X           325836458  326829675
  Y              -52975     560160
  Z              181659     240288
  intensity           3      16983
  return_number       1          6
  number_of_returns   1          6
  edge_of_flight_line 0          1
  scan_direction_flag 1          1
  classification      0          4
  scan_angle_rank   -25         30
  user_data          17        255
  point_source_ID    18         18
  gps_time 180585572.845342 180585577.175763
  Wavepacket Index    1 3
             Offset   0 151487680
             Size     160 480
             Location 639.596 175640
             Xt       -3.17766e-005 0.000122964
             Yt       -7.55894e-005 1.00387e-005
             Zt       1.59413e-005 0.00027238
  extended_return_number          1      6
  extended_number_of_returns      1      6
  extended_classification         0      4
  extended_scan_angle         -4232   4985
  extended_scanner_channel        0      0
WARNING: 3 points outside of header bounding box
number of first returns:        738856
number of intermediate returns: 31107
number of last returns:         738856
number of single returns:       597994
WARNING: point type is 9 but (legacy) number of point records in header is 910825 instead zero.
WARNING: point type is 9 but (legacy) number of points by return [1] in header is 738856 instead zero.
WARNING: point type is 9 but (legacy) number of points by return [2] in header is 140862 instead zero.
WARNING: point type is 9 but (legacy) number of points by return [3] in header is 27810 instead zero.
WARNING: point type is 9 but (legacy) number of points by return [4] in header is 3129 instead zero.
WARNING: point type is 9 but (legacy) number of points by return [5] in header is 162 instead zero.
overview over extended number of returns of given pulse: 597994 226104 74043 11868 780 36 0 0 0 0 0 0 0 0 0
histogram of classification of points:
            1509  never classified (0)
          518583  ground (2)
          390733  medium vegetation (4)
WARNING: real max x larger than header max x by 0.000285
WARNING: real max y larger than header max y by 0.000416
WARNING: real max z larger than header max z by 0.000387

=============================================================================

lasinfo -i cld_170604_01_042610_018_nicer.las

lasinfo (180831) report for 'cld_170604_01_042610_018_nicer.las'
reporting all LAS header entries:
  file signature:             'LASF'
  file source ID:             18
  global_encoding:            21
  project ID GUID data 1-4:   01045231-0000-0000-6F42-746F0000724E
  version major.minor:        1.4
  system identifier:          'LAStools (c) by rapidlasso GmbH'
  generating software:        'las2las (version 180831)'
  file creation day/year:     23/2018
  header size:                375
  offset to point data:       1353
  number var. length records: 4
  point data format:          9
  point data record length:   59
  number of point records:    0
  number of points by return: 0 0 0 0 0
  scale factor x y z:         0.01 0.01 0.01
  offset x y z:               1600000 5745000 0
  min x y z:                  1925836.46 5744947.02 181.66
  max x y z:                  1926829.68 5745560.16 240.29
  start of waveform data packet record: 0
  start of first extended variable length record: 0
  number of extended_variable length records: 0
  extended number of point records: 910825
  extended number of points by return: 738856 140862 27810 3129 162 6 0 0 0 0 0 0 0 0 0
variable length header record 1 of 4:
  reserved             0
  user ID              'LASF_Projection'
  record ID            2112
  length after header  682
  description          'WKT description'
    WKT OGC COORDINATE SYSTEM:
    PROJCS["NZGD2000 / New Zealand Transverse Mercator 2000",GEOGCS["NZGD2000",DATUM["New_Zealand_Geodetic_Datum_2000",SPHEROID["GRS 1980",6378137,298.257222101
,AUTHORITY["EPSG","7019"]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY["EPSG","6167"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.01745329251994328,AUT
HORITY["EPSG","9122"]],AUTHORITY["EPSG","4167"]],UNIT["metre",1,AUTHORITY["EPSG","9001"]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PA
RAMETER["central_meridian",173],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",1600000],PARAMETER["false_northing",10000000],AUTHORITY["EPSG","2193"
],AXIS["Easting",EAST],AXIS["Northing",NORTH]]
variable length header record 2 of 4:
  reserved             0
  user ID              'LASF_Spec'
  record ID            100
  length after header  26
  description          'WDP_1'
  index 1 bits/sample 16 compression 0 samples 80 temporal 1000 gain 1, offset 0
variable length header record 3 of 4:
  reserved             0
  user ID              'LASF_Spec'
  record ID            101
  length after header  26
  description          'WDP_2'
  index 2 bits/sample 16 compression 0 samples 160 temporal 1000 gain 1, offset 0
variable length header record 4 of 4:
  reserved             0
  user ID              'LASF_Spec'
  record ID            102
  length after header  26
  description          'WDP_3'
  index 3 bits/sample 16 compression 0 samples 240 temporal 1000 gain 1, offset 0
the header is followed by 2 user-defined bytes
reporting minimum and maximum for all LAS point record entries ...
  X            32583646   32682968
  Y               -5298      56016
  Z               18166      24029
  intensity           3      16983
  return_number       1          6
  number_of_returns   1          6
  edge_of_flight_line 0          1
  scan_direction_flag 1          1
  classification      0          4
  scan_angle_rank   -25         30
  user_data          17        255
  point_source_ID    18         18
  gps_time 180585572.845342 180585577.175763
  Wavepacket Index    1 3
             Offset   0 151487680
             Size     160 480
             Location 639.596 175640
             Xt       -3.17766e-005 0.000122964
             Yt       -7.55894e-005 1.00387e-005
             Zt       1.59413e-005 0.00027238
  extended_return_number          1      6
  extended_number_of_returns      1      6
  extended_classification         0      4
  extended_scan_angle         -4232   4985
  extended_scanner_channel        0      0
number of first returns:        738856
number of intermediate returns: 31107
number of last returns:         738856
number of single returns:       597994
overview over extended number of returns of given pulse: 597994 226104 74043 11868 780 36 0 0 0 0 0 0 0 0 0
histogram of classification of points:
            1509  never classified (0)
          518583  ground (2)
          390733  medium vegetation (4)
full_waveform_LAS14_point_type_9_optech_3.jpg
full_waveform_LAS14_point_type_9_optech_4.jpg
full_waveform_LAS14_point_type_9_optech_5.jpg
full_waveform_LAS14_point_type_9_optech_6.jpg
Reply all
Reply to author
Forward
0 new messages