trouble with laszip on upconverted las 1.4 file
130 views
Skip to first unread message
Jeffrey Osborne
Sep 13, 2016, 12:41:58 PM9/13/16
to LAStools - efficient tools for LiDAR processing
Hi,
The 1.4 las file is fine but the laz file has no records.
I'm getting laz files with no records after running laszip on upconverted las 1.4 data. My original las file is version 1.1 with point type 1.
original.las lasinfo:
reporting all LAS header entries: file signature: 'LASF' file source ID: 0 global_encoding: 0 project ID GUID data 1-4: 00000000-0000-0000-F500-DD41E34B89C2 version major.minor: 1.1 system identifier: 'Virtual Geomatics' generating software: 'VG4D' file creation day/year: 247/2014 header size: 227 offset to point data: 1280 number var. length records: 2 point data format: 1 point data record length: 28 number of point records: 8095202 number of points by return: 7001538 808595 233731 45175 6163 scale factor x y z: 0.0001 0.0001 0.0001 offset x y z: 579515.64319999993 5052294.3870999999 0 min x y z: 579514.9329 5051218.2561 0.3999 max x y z: 581098.4030 5052359.2813 66.9280variable length header record 1 of 2: reserved 43707 user ID 'LASF_Projection' record ID 34735 length after header 48 description 'GeoKeyDirectoryTag (mandatory)' GeoKeyDirectoryTag version 1.1.0 number of keys 5 key 1024 tiff_tag_location 0 count 1 value_offset 1 - GTModelTypeGeoKey: ModelTypeProjected key 1026 tiff_tag_location 34737 count 35 value_offset 0 - GTCitationGeoKey: key 2052 tiff_tag_location 0 count 1 value_offset 9001 - GeogLinearUnitsGeoKey: Linear_Meter key 2054 tiff_tag_location 0 count 1 value_offset 9102 - GeogAngularUnitsGeoKey: Angular_Degree key 3076 tiff_tag_location 0 count 1 value_offset 9001 - ProjLinearUnitsGeoKey: Linear_Metervariable length header record 2 of 2: reserved 43707 user ID 'LASF_Projection' record ID 34737 length after header 35 description 'GeoASCIIParamsTag (optional)' GeoAsciiParamsTag (number of characters 35) the header is followed by 862 user-defined bytesreporting minimum and maximum for all LAS point record entries ... X -7103 15827598 Y -10761310 648941 Z 3999 669279 intensity 8 65520 return_number 1 7 number_of_returns 1 7 edge_of_flight_line 0 0 scan_direction_flag 0 0 classification 1 31 scan_angle_rank -23 25 user_data 205 205 point_source_ID 0 3 gps_time 567369.656098 568774.437919number of first returns: 7001538number of intermediate returns: 287217number of last returns: 6983027number of single returns: 6176580WARNING: for return 5 real number of points by return (5731) is different from header entry (6163).WARNING: there are 417 points with return number 6WARNING: there are 15 points with return number 7overview over number of returns of given pulse: 6176580 1155340 571515 161219 27695 2688 165histogram of classification of points: 2127 unclassified (1) 5865989 ground (2) 497044 low vegetation (3) 386198 medium vegetation (4) 1341057 high vegetation (5) 2787 Reserved for ASPRS Definition (31)
I realize that aside from the return number there is nothing to gain by converting my original file to 1.4, this is an exercise only for testing the las 1.4 compatibility mode of the compressor on my data.
I've converted it to 1.4 and point type 6 following Martin's guide:
las2las -i original.las -set_version 1.4 -set_point_type 6 -rescale 0.01 0.01 0.01 -auto_reoffset -remove_all_vlrs -remove_padding -week_to_adjusted 1818 -epsg 2961 -vertical_cgvd28 -set_ogc_wkt -o las_14.laslas_14.las lasinfo:
reporting all LAS header entries: file signature: 'LASF' file source ID: 0 global_encoding: 17 project ID GUID data 1-4: 00000000-0000-0000-F500-DD41E34B89C2 version major.minor: 1.4 system identifier: 'LAStools (c) by rapidlasso GmbH' generating software: 'las2las (version 160910)' file creation day/year: 247/2014 header size: 375 offset to point data: 1376 number var. length records: 2 point data format: 6 point data record length: 30 number of point records: 0 number of points by return: 0 0 0 0 0 scale factor x y z: 0.01 0.01 0.01 offset x y z: 500000 5000000 0 min x y z: 579514.93 5051218.26 0.40 max x y z: 581098.40 5052359.28 66.93 start of waveform data packet record: 0 start of first extended variable length record: 0 number of extended_variable length records: 0 extended number of point records: 8095202 extended number of points by return: 7001538 808595 233731 45175 5731 417 15 0 0 0 0 0 0 0 0variable length header record 1 of 2: reserved 0 user ID 'LASF_Projection' record ID 34735 length after header 48 description 'by LAStools of rapidlasso GmbH' GeoKeyDirectoryTag version 1.1.0 number of keys 5 key 1024 tiff_tag_location 0 count 1 value_offset 1 - GTModelTypeGeoKey: ModelTypeProjected key 3072 tiff_tag_location 0 count 1 value_offset 2961 - ProjectedCSTypeGeoKey: NAD83(CSRS) / UTM zone 20N key 3076 tiff_tag_location 0 count 1 value_offset 9001 - ProjLinearUnitsGeoKey: Linear_Meter key 4099 tiff_tag_location 0 count 1 value_offset 9001 - VerticalUnitsGeoKey: Linear_Meter key 4096 tiff_tag_location 0 count 1 value_offset 5114 - VerticalCSTypeGeoKey: VertCS_Canadian_Geodetic_Vertical_Datum_1928variable length header record 2 of 2: reserved 0 user ID 'LASF_Projection' record ID 2112 length after header 845 description 'by LAStools of rapidlasso GmbH' WKT OGC COORDINATE SYSTEM: PROJCS["NAD83(CSRS) / UTM zone 20N",GEOGCS["NAD83(CSRS)",DATUM["NAD83_Canadian_Spatial_Reference_System",SPHEROID["GRS 1980",6378137,298.257222101,AUTHORITY["EPSG","7019"]],AUTHORITY["EPSG","6140"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.01745329251994328,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4617"]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",-63],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",500000],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH],AUTHORITY["EPSG","2961"]]VERT_CS["CGVD28",VERT_DATUM["Canadian Geodetic Vertical Datum of 1928",2005,AUTHORITY["EPSG","5114"]],UNIT["metre",1.0,AUTHORITY["EPSG","9001"]]AXIS["Gravity-related height",UP],AUTHORITY["EPSG","5713"]]reporting minimum and maximum for all LAS point record entries ... X 7951493 8109840 Y 5121826 5235928 Z 40 6693 intensity 8 65520 return_number 1 7 number_of_returns 1 7 edge_of_flight_line 0 0 scan_direction_flag 0 0 classification 1 31 scan_angle_rank -23 25 user_data 205 205 point_source_ID 0 3 gps_time 100093769.656098 100095174.437919 extended_return_number 1 7 extended_number_of_returns 1 7 extended_classification 1 31 extended_scan_angle -3833 4167 extended_scanner_channel 0 0number of first returns: 7001538number of intermediate returns: 287217number of last returns: 6983027number of single returns: 6176580overview over extended number of returns of given pulse: 6176580 1155340 571515 161219 27695 2688 165 0 0 0 0 0 0 0 0histogram of classification of points: 2127 unclassified (1) 5865989 ground (2) 497044 low vegetation (3) 386198 medium vegetation (4) 1341057 high vegetation (5) 2787 Reserved for ASPRS Definition (31)
The 1.4 las file is fine but the laz file has no records.
laszip las_14.las
las_14.laz lasinfo:
reporting all LAS header entries: file signature: 'LASF' file source ID: 0 global_encoding: 1 project ID GUID data 1-4: 00000000-0000-0000-F500-DD41E34B89C2 version major.minor: 1.2 system identifier: 'LAStools (c) by rapidlasso GmbH' generating software: 'las2las (version 160910)' file creation day/year: 247/2014 header size: 227 offset to point data: 2260 number var. length records: 4 point data format: 1 point data record length: 33 number of point records: 0 number of points by return: 0 0 0 0 0 scale factor x y z: 0.01 0.01 0.01 offset x y z: 500000 5000000 0 min x y z: 579514.93 5051218.26 0.40 max x y z: 581098.40 5052359.28 66.93variable length header record 1 of 4: reserved 0 user ID 'LASF_Projection' record ID 34735 length after header 48 description 'by LAStools of rapidlasso GmbH' GeoKeyDirectoryTag version 1.1.0 number of keys 5 key 1024 tiff_tag_location 0 count 1 value_offset 1 - GTModelTypeGeoKey: ModelTypeProjected key 3072 tiff_tag_location 0 count 1 value_offset 2961 - ProjectedCSTypeGeoKey: NAD83(CSRS) / UTM zone 20N key 3076 tiff_tag_location 0 count 1 value_offset 9001 - ProjLinearUnitsGeoKey: Linear_Meter key 4099 tiff_tag_location 0 count 1 value_offset 9001 - VerticalUnitsGeoKey: Linear_Meter key 4096 tiff_tag_location 0 count 1 value_offset 5114 - VerticalCSTypeGeoKey: VertCS_Canadian_Geodetic_Vertical_Datum_1928variable length header record 2 of 4: reserved 0 user ID 'LASF_Projection' record ID 2112 length after header 845 description 'by LAStools of rapidlasso GmbH' WKT OGC COORDINATE SYSTEM: PROJCS["NAD83(CSRS) / UTM zone 20N",GEOGCS["NAD83(CSRS)",DATUM["NAD83_Canadian_Spatial_Reference_System",SPHEROID["GRS 1980",6378137,298.257222101,AUTHORITY["EPSG","7019"]],AUTHORITY["EPSG","6140"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.01745329251994328,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4617"]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",-63],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",500000],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH],AUTHORITY["EPSG","2961"]]VERT_CS["CGVD28",VERT_DATUM["Canadian Geodetic Vertical Datum of 1928",2005,AUTHORITY["EPSG","5114"]],UNIT["metre",1.0,AUTHORITY["EPSG","9001"]]AXIS["Gravity-related height",UP],AUTHORITY["EPSG","5713"]]variable length header record 3 of 4: reserved 0 user ID 'lascompatible' record ID 22204 length after header 156 description 'by LAStools of rapidlasso GmbH'variable length header record 4 of 4: reserved 0 user ID 'LASF_Spec' record ID 4 length after header 768 description 'by LAStools of rapidlasso GmbH' Extra Byte Descriptions data type: 4 (short), name "LAS 1.4 scan angle", description: "additional attributes", scale: 0.006, offset: 0 (not set) data type: 1 (unsigned char), name "LAS 1.4 extended returns", description: "additional attributes", scale: 1 (not set), offset: 0 (not set) data type: 1 (unsigned char), name "LAS 1.4 classification", description: "additional attributes", scale: 1 (not set), offset: 0 (not set) data type: 1 (unsigned char), name "LAS 1.4 flags and channel", description: "additional attributes", scale: 1 (not set), offset: 0 (not set)LASzip compression (version 2.4r1 c2 50000): POINT10 2 GPSTIME11 2 BYTE 2reporting minimum and maximum for all LAS point record entries ... X 0 0 Y 0 0 Z 0 0 intensity 0 0 return_number 1 1 number_of_returns 1 1 edge_of_flight_line 0 0 scan_direction_flag 0 0 classification 0 0 scan_angle_rank 0 0 user_data 0 0 point_source_ID 0 0 gps_time 0.000000 0.000000number of first returns: 0number of intermediate returns: 0number of last returns: 0number of single returns: 0real min x smaller than header min x by 79514.930000real min y smaller than header min y by 51218.260000real min z smaller than header min z by 0.400000
Does anyone see what I'm doing wrong? I've attached a small clip of my original file. Thanks
Evon Silvia
Sep 14, 2016, 4:35:20 AM9/14/16
to last...@googlegroups.com
When I open the las_14_clip.laz file that you attached, I'm seeing 8095 points. However, this is stored in the LAS 1.4 extended point tallies instead of the LAS 1.2 legacy point tallies, so that might be confusing lasinfo. I believe that LAZ files should be using a LAS 1.2 style header, not a LAS 1.4 style header.
I'm guessing that Martin's bug fix here might've introduced a bug that results in this behavior.
Evon
--
Download LAStools at
http://lastools.org
http://rapidlasso.com
Be social with LAStools at
http://facebook.com/LAStools
http://twitter.com/LAStools
http://linkedin.com/groups/LAStools-4408378
Manage your settings at
http://groups.google.com/group/lastools/subscribe
Martin Isenburg
Sep 14, 2016, 5:31:24 AM9/14/16
to LAStools - efficient command line tools for LIDAR processing
Hello Jeffrey,
turns out that Evon is correct. The "compatible mode" has been developed and tested only with "specification-violating" LAS 1.4 files where both the legacy counters as well as the extended counters were populated for the new point types 6 and up. Initially LAStools (and other software) was writing the new content for LAS 1.4 in this fashion. This was fixed when the "LASvalidator" was made more "strict" to check for this violation. Since then I had never tried to compress a LAS 1.4 file with new point types and properly zeroed legacy counters (most of the content I have come across - such as the LAS 1.4 content reported on in this discussion here - still populates both counters.
Thank you for pointing this out. It required just a minor fix to LASzip-compress LAS 1.4 content with new point types and properly zeroed legacy counters using the NOAA-sponsored compatibility mode as you can see below. I can send you the fixed version if you need it today. Otherwise it will be in the next release.
Regards,
Martin @rapidlasso
=================================
E:\LAStools\bin>las2las -i original_clip.las -set_version 1.4 -set_point_type 6 -rescale 0.01 0.01 0.01 -auto_reoffset -o original_clip14.las
E:\LAStools\bin>lasinfo original_clip14.las
lasinfo (160914) report for original_clip14.las
reporting all LAS header entries:
file signature: 'LASF'
file source ID: 0
global_encoding: 0
project ID GUID data 1-4: 00000000-0000-0000-F500-DD41E34B89C2
version major.minor: 1.4
system identifier: 'LAStools (c) by rapidlasso GmbH'
generating software: 'las2las (version 160910)'
file creation day/year: 247/2014
header size: 375
offset to point data: 1428
number var. length records: 2
point data format: 6
point data record length: 30
number of point records: 0
number of points by return: 0 0 0 0 0
scale factor x y z: 0.01 0.01 0.01
offset x y z: 500000 5000000 0
min x y z: 579815.13 5051573.16 8.91
max x y z: 579863.79 5051617.94 27.11
start of waveform data packet record: 0
start of first extended variable length record: 0
number of extended_variable length records: 0
extended number of point records: 8095
extended number of points by return: 7332 617 127 17 2 0 0 0 0 0 0 0 0 0 0
variable length header record 1 of 2:
reserved 43707
user ID 'LASF_Projection'
record ID 34735
length after header 48
description 'GeoKeyDirectoryTag (mandatory)'
GeoKeyDirectoryTag version 1.1.0 number of keys 5
key 1024 tiff_tag_location 0 count 1 value_offset 1 - GTModelTypeGeoKey: ModelTypeProjected
key 1026 tiff_tag_location 34737 count 35 value_offset 0 - GTCitationGeoKey:
key 2052 tiff_tag_location 0 count 1 value_offset 9001 - GeogLinearUnitsGeoKey: Linear_Meter
key 2054 tiff_tag_location 0 count 1 value_offset 9102 - GeogAngularUnitsGeoKey: Angular_Degree
key 3076 tiff_tag_location 0 count 1 value_offset 9001 - ProjLinearUnitsGeoKey: Linear_Meter
variable length header record 2 of 2:
reserved 43707
user ID 'LASF_Projection'
record ID 34737
length after header 35
description 'GeoASCIIParamsTag (optional)'
GeoAsciiParamsTag (number of characters 35)
the header is followed by 862 user-defined bytes
reporting minimum and maximum for all LAS point record entries ...
X 7981513 7986379
Y 5157316 5161794
Z 891 2711
intensity 9 1192
return_number 1 5
number_of_returns 1 6
edge_of_flight_line 0 0
scan_direction_flag 0 0
classification 1 5
scan_angle_rank 7 11
user_data 205 205
point_source_ID 1 1
gps_time 567711.360144 568062.272339
extended_return_number 1 5
extended_number_of_returns 1 6
extended_classification 1 5
extended_scan_angle 1167 1833
extended_scanner_channel 0 0
number of first returns: 7332
number of intermediate returns: 291
number of last returns: 5892
number of single returns: 5420
overview over extended number of returns of given pulse: 5420 1289 926 367 79 14 0 0 0 0 0 0 0 0 0
histogram of classification of points:
2 unclassified (1)
5428 ground (2)
674 low vegetation (3)
558 medium vegetation (4)
1433 high vegetation (5)
E:\LAStools\bin>laszip -v -i original_clip14.las
0 secs to write 38622 bytes for 'original_clip14.laz' with 8095 points of type 1
E:\LAStools\bin>lasinfo original_clip14.laz
lasinfo (160914) report for original_clip14.laz
reporting all LAS header entries:
file signature: 'LASF'
file source ID: 0
global_encoding: 0
project ID GUID data 1-4: 00000000-0000-0000-F500-DD41E34B89C2
version major.minor: 1.2
system identifier: 'LAStools (c) by rapidlasso GmbH'
generating software: 'las2las (version 160910)'
file creation day/year: 247/2014
header size: 227
offset to point data: 2312
number var. length records: 4
point data format: 1
point data record length: 33
number of point records: 8095
number of points by return: 7332 617 127 17 2
scale factor x y z: 0.01 0.01 0.01
offset x y z: 500000 5000000 0
min x y z: 579815.13 5051573.16 8.91
max x y z: 579863.79 5051617.94 27.11
variable length header record 1 of 4:
reserved 43707
user ID 'LASF_Projection'
record ID 34735
length after header 48
description 'GeoKeyDirectoryTag (mandatory)'
GeoKeyDirectoryTag version 1.1.0 number of keys 5
key 1024 tiff_tag_location 0 count 1 value_offset 1 - GTModelTypeGeoKey: ModelTypeProjected
key 1026 tiff_tag_location 34737 count 35 value_offset 0 - GTCitationGeoKey:
key 2052 tiff_tag_location 0 count 1 value_offset 9001 - GeogLinearUnitsGeoKey: Linear_Meter
key 2054 tiff_tag_location 0 count 1 value_offset 9102 - GeogAngularUnitsGeoKey: Angular_Degree
key 3076 tiff_tag_location 0 count 1 value_offset 9001 - ProjLinearUnitsGeoKey: Linear_Meter
variable length header record 2 of 4:
reserved 43707
user ID 'LASF_Projection'
record ID 34737
length after header 35
description 'GeoASCIIParamsTag (optional)'
GeoAsciiParamsTag (number of characters 35)
variable length header record 3 of 4:
reserved 0
user ID 'lascompatible'
record ID 22204
length after header 156
description 'by LAStools of rapidlasso GmbH'
variable length header record 4 of 4:
reserved 0
user ID 'LASF_Spec'
record ID 4
length after header 768
description 'by LAStools of rapidlasso GmbH'
Extra Byte Descriptions
data type: 4 (short), name "LAS 1.4 scan angle", description: "additional attributes", scale: 0.006, offset: 0 (not set)
data type: 1 (unsigned char), name "LAS 1.4 extended returns", description: "additional attributes", scale: 1 (not set), offset: 0 (not set)
data type: 1 (unsigned char), name "LAS 1.4 classification", description: "additional attributes", scale: 1 (not set), offset: 0 (not set)
data type: 1 (unsigned char), name "LAS 1.4 flags and channel", description: "additional attributes", scale: 1 (not set), offset: 0 (not set)
the header is followed by 862 user-defined bytes
LASzip compression (version 2.4r1 c2 50000): POINT10 2 GPSTIME11 2 BYTE 2
reporting minimum and maximum for all LAS point record entries ...
X 7981513 7986379
Y 5157316 5161794
Z 891 2711
intensity 9 1192
return_number 1 5
number_of_returns 1 6
edge_of_flight_line 0 0
scan_direction_flag 0 0
classification 1 5
scan_angle_rank 7 11
user_data 205 205
point_source_ID 1 1
gps_time 567711.360144 568062.272339
number of first returns: 7332
number of intermediate returns: 291
number of last returns: 5892
number of single returns: 5420
overview over number of returns of given pulse: 5420 1289 926 367 79 14 0
histogram of classification of points:
2 unclassified (1)
5428 ground (2)
674 low vegetation (3)
558 medium vegetation (4)
1433 high vegetation (5)
Jeffrey Osborne
Sep 14, 2016, 9:09:23 AM9/14/16
to LAStools - efficient tools for LiDAR processing
Hi Martin,
That makes perfect sense. I had noticed that the filesize of my laz files was much greater than they should be if the header info was correct. My software was properly reading the header and not loading any data. I'll grab the updated version next release but since you're here could I get a copy of laspulse?
Thanks Evon and Martin.
Martin Isenburg
Sep 14, 2016, 4:18:59 PM9/14/16
to LAStools - efficient command line tools for LIDAR processing
Note ... the new "compatibility mode" for the new point types of LAS 1.4 of LASzip will not entirely be lossless as it will :"repair" the header and set the legacy counters to zero as shown below:
E:\LAStools\bin>laszip -i DouglasDaly2016_C2_754500_8447500.LAS
WARNING: legacy number_of_point_records in header of LAS 1.4 file should be zero.
WARNING: legacy number_of_points_by_return[0] in header of LAS 1.4 file should be zero.
WARNING: legacy number_of_points_by_return[1] in header of LAS 1.4 file should be zero.
WARNING: legacy number_of_points_by_return[2] in header of LAS 1.4 file should be zero.
WARNING: legacy number_of_points_by_return[3] in header of LAS 1.4 file should be zero.
WARNING: legacy number_of_points_by_return[4] in header of LAS 1.4 file should be zero.
E:\LAStools\bin>laszip -i DouglasDaly2016_C2_754500_8447500.laz -o DouglasDaly2016_C2_754500_8447500_dec.las
E:\LAStools\bin>lasdiff -i DouglasDaly2016_C2_754500_8447500.LAS -i DouglasDaly2016_C2_754500_8447500_dec.las
checking 'DouglasDaly2016_C2_754500_8447500.LAS' against 'DouglasDaly2016_C2_754500_8447500_dec.las'
different number_of_point_records: 3646281 0
different number_of_points_by_return[0]: 2977551 0
different number_of_points_by_return[1]: 577609 0
different number_of_points_by_return[2]: 85157 0
different number_of_points_by_return[3]: 5749 0
different number_of_points_by_return[4]: 212 0
headers have 6 differences.
raw points are identical.
both have 3646281 points. took 1.718 secs.
Jeffrey. The "laspulse" tool was renamed to "lasreturn" ...
Martin
Martin Isenburg
Sep 21, 2016, 2:43:50 AM9/21/16
to LAStools - efficient command line tools for LIDAR processing
Hello Jeffrey.
This issue should be fixed in today's release (160921). See example below.
las2las -i ..\data\fusa.laz ^
-set_version 1.4 -set_point_type 6 ^
-set_extended_scanner_channel 2 ^
-set_ogc_wkt ^
-o fusa14.las
laszip -i fusa14.las
laszip -i fusa14.laz -o fusa14_dec.las
dir fusa14*
8,328,333 fusa14.las
1,093,259 fusa14.laz
8,328,333 fusa14_dec.las
lasdiff -i fusa14.las -i fusa14_dec.las
checking 'fusa14.las' against 'fusa14_dec.las'
headers are identical.
raw points are identical.
files are identical. both have 277573 points. took 0.154 secs.
Regards,
Martin @rapidlasso
Reply all
Reply to author
Forward
0 new messages