RE: [LAStools] Laz in Qt Reader

[Michael....@csiro.au]

Hi Michael,

 

As far as I’m aware the QT software from Applied Imagery does not support the opening of LAZ files. In the newer versions of QT Modeler you can compress your  LAS files into LAZ format, however it does not enable you to open LAZ files in either QT Reader or Modeler. I hope this helps.

 

Cheers,

Mick Schaefer.

 

From: last...@googlegroups.com [mailto:last...@googlegroups.com] On Behalf Of Michael Lefsky
Sent: Tuesday, 12 March 2013 9:46 AM
To: last...@googlegroups.com
Subject: [LAStools] Laz in Qt Reader

 

I've been trying to get LAZ files to open in QT Reader (not Modeler) but each time I do, it crashes. QTR works fine with LAS files, and the crashes occur under the 32 and 64 bit versions. 

 

Has anyone else had this problem? Alternatively, have you been able to use LAZ  with QT Reader? 

Also, do LAZ files work properly with QT Modeler?

 

M 

--
Download LAStools at
http://lastools.org
http://rapidlasso.com
Be social with LAStools at
http://facebook.com/LAStools
http://twitter.com/LAStools
http://linkedin.com/groups/LAStools-4408378
Manage your settings at
http://groups.google.com/group/lastools/subscribe
 
 

[Martin Isenburg]

Hello,

after a little off-line discussion, here a summary about QT Modeller
and the LAZ format as well as some issues with LAS 1.3 files. The main
conclusion is that I recommend not to use the QT Modeller as a LiDAR
compressor or decompressor (e.g. to convert LAS files to LAZ files or
vice-versa) without triple-checking the results until they have
resolved the issues listed below. In general I advise to only use
laszip.exe for *purely* lossless compression and decompression since
it seems that most other software makes some changes when loading
LAS/LAS and writing LAS/LAZ to the contents, be it projection VLRs,
scaling factors, or just an overwriting of identifying strings such as
generating software, etc ...

In detail, QTM

* fails to load LAZ files with extra bytes

Under the hood QT Modeller seems to use PDAL whenever they are reading
or writing a LAZ file but use their own readers and writers for LAS
files. Because PDAL does not support "extra bytes" in LAZ (or LAS)
files it will crash if you open LAZ files containing "extra bytes".
This is trouble-some because "extra bytes" are exported to the LAZ
format, for example, by RIEGL's RiPROCESS to store attributes such as
echo width and normalized reflectance values.

* loads LAS files with extra bytes but omits them when writing to LAZ

It seems that the QTM loader can handle "extra bytes" with its native
LAS reader but it omits these "extra bytes" when using PDAL to export
those LAS files to LAZ.

* corrupts the global encoding bit when converting LAS files to LAZ

In case the bit was 1 in the LAS file it will be 0 in the LAZ file and
wrongly turn Adjusted Standard GPS time into GPS week time.

* corrupts loaded LAS 1.3 files when exporting them to LAS 1.2

It seems that the QTM writer does not properly handle the removal of
the 8 zero bytes that describe the "start of waveform data" when it
changes the version number from 1.3 down to 1.2 on export. Hence the
VLR reading starts 8 zero bytes too early which makes them all
corrupt.

* changes the projection VLRs (but apparently still correct) when you
load LAS and save LAZ (amybe also LAS?).

Thanks to Ty for helping with tracking down these issues. Rest assured
that Mike Umansky from Applied Imagery is already on to fixing these
issues. He promised to have a fixed version out as soon as possible.

Regards,

Martin @rapidlasso

--
http://rapidlasso.com - lossless tools for compressing LiDARs

[Evon Silvia]

Because PDAL does not support "extra bytes" in LAZ (or LAS) files it will crash if you open LAZ files containing "extra bytes". This is trouble-some because "extra bytes" are exported to the LAZ format, for example, by RIEGL's RiPROCESS to store attributes such as echo width and normalized reflectance values.

But why would PDAL support extra bytes for LAS 1.2 and 1.3?

As something of a newcomer to the LAS world, I have always found the idea of "extra bytes" rather confusing. Until LAS 1.4, the LAS specifications say nothing about including extra bytes with the point data, instead indicating that extra information should be stored in the VLR and EVLR records. They seem to imply that point records should be of a standard, fixed length, and so, by my interpretation, the inclusion of "extra bytes" would actually be a violation of the LAS specification for versions 1.0 through 1.3.

Is this not the case in standard practice?

Evon

[Martin Isenburg]

Hello Evon,

I am cc-ing "the LAS room" on this one as this would be the right
place to continue this discussion should it be necessary.

>> Because PDAL does not support "extra bytes" in LAZ (or LAS) files it will
>> crash if you open LAZ files containing "extra bytes". This is trouble-some
>> because "extra bytes" are exported to the LAZ format, for example, by
>> RIEGL's RiPROCESS to store attributes such as echo width and normalized
>> reflectance values.
>
> But why would PDAL support extra bytes for LAS 1.2 and 1.3?

Extra bytes have been with the LAS format since day 0. In the LAS
header there is a field that specifies the point format being used (0,
1, 2, 3) and then there is a field that specifies the size of each
point record. Should the point size implied by the point type be
smaller than the specified point record size any reader had to either
skip those "extra bytes" if it did not know how to interpret them or
load them into a separate attribute field assuming there was some way
for it to know what they meant.

The only thing that is new with LAS 1.4 is that there is now a way to
descriptive those extra bytes in the specification. Hence, unknown
arrays of "extra bytes" become suddenly meaningful to anyone via the
LASF_Spec VLR with user_ID 4. Because it does not hurt to put unknown
VLRs into earlier versions of the specification (they could safely be
ignored but may as well be parsed if known) RIEGL also put their
"extra bytes" descriptions into earlier versions. And so does LAStools
and probably a few other softwares ...

> As something of a newcomer to the LAS world, I have always found the idea of
> "extra bytes" rather confusing. Until LAS 1.4, the LAS specifications say
> nothing about including extra bytes with the point data, instead indicating
> that extra information should be stored in the VLR and EVLR records.

If you have extra information that is per point such as an "echo
width" you need something like "extra bytes" to avoid IO hampering
file seeks. VLRs and EVLRs are better suited to store information that
is per file such as projection information, spatial indexing,
compression parameters, additional mission info ...

> They
> seem to imply that point records should be of a standard, fixed length, and
> so, by my interpretation, the inclusion of "extra bytes" would actually be a
> violation of the LAS specification for versions 1.0 through 1.3.

If point records were meant to be a standard fixed size as implied by
the point type then the inclusion of the point record size would have
not been a redundant source for error. While the concept of "extra
bytes" per point is not explicitely stated in the spec in earlier
versions it has always been there ... just like the potential presence
of this additional information that does not clash with anything
written in the specification:

// if the header_size is larger than implied by the version
U32 user_data_in_header_size;
U8* user_data_in_header;

// if the offset_to_point_data is larger than the combination of
header size and all VLR sizes
U32 user_data_after_header_size;
U8* user_data_after_header;

Cheers,

Martin @rapidlasso

--
http://rapidlasso.com - fast tools to play with large LiDARs

[Martin Isenburg]

Hello,

I am forwarding the below message from Lewis Graham (with his
permission) as we figured it will be reassuring for everyone involved
with LAS to hear that we are both in full agreement on this ... (-;

Regards,

Martin @rapidlasso

http://rapidlasso.com - fast tools for surviving LiDAR wars (-;

---------- Forwarded message ----------
From: Lewis Graham <lgr...@geocue.com>
Date: Fri, Apr 5, 2013 at 2:59 AM
Subject: RE: [LAStools] Re: Laz in Qt Reader
To: jeewe...@gmail.com
Cc: martin....@gmail.com

Hello Jan,

I am responding directly because I have somehow broken my Google links
to the forums (I will get around to fixing this one of these days….).

I have been the chair of the LAS working group since before the format
was “donated” to the ASPRS (since around 1998).

The situation is exactly as Martin describes. From the very first
draft of the format, we intended for users to be able to pack anything
they wanted in the Point Data Records. We only specified what was
intended as the “well known exchange payload” for the first few fields
(as defined in the various Point Data Record Formats, PDRF). For
example, you can even put double precision X, Y, Z coordinates in the
record so long as you provide the integer equivalents in the required
X, Y, Z fields.

As Martin says, this is why both the format number (0 -> 10) is
required as well as the length of the payload. In fact, in every
article we have ever written on the format, we warn programmers not to
use the size of the required fields as the size of the payload. See,
for example, page 207 of the “ASPRS Manual of Airborne Topographic
LIDAR.” Paragraph 3 begins “It is critically important for
programmers to understand that a point data record in LAS can contain
fields beyond those defined by the PDRF.” It goes on to explain that
we added the ability to formalize extensions via Dr. Isenburg’s
contributions to LAS 1.4 but using the Extra Bytes facility is not
mandatory.

So the bottom line is that it has always been perfectly fine to extend
the point payloads in LAS and this is, in fact, very frequently done.

I hope this information is helpful.

Best Regards,

Lewis
______________________________
_

Lewis Graham
President/CTO
GeoCue Group
9668 Madison Blvd., Suite 202
Madison, AL 35758
(256) 461-8289 Telephone
(256) 461-8249 Fax
www.geocue.com
www.qcoherent.com

Complete LIDAR Solutions
Integrating the Geospatial Workplace...

On Thu, Apr 4, 2013 at 6:38 PM, Jan Willem de Pater
<jeewe...@gmail.com> wrote:
> hi there,
>
> as a matter of fact I have wrestled precisely with this theme for quite some
> time, sometimes. Are or are not extra bytes per point allowed in Las 1.0 -
> Las 1.3 . Martin (who is certainly not just someone in the world of LAS)
> says: absolutely. My conclusion, after several times studying the ASPRS
> specs, is: it is just not clear. Martin anyhow is right that the Point
> Record Length in the lasheader would be redundant if there were no extra
> bytes per point. Maybe the International Court of Justice in The Hague can
> sentence in this case. (Be aware that there may be several Revision specs of
> one LAS version).
> I think, if you implement Extra Bytes for 1.4 in some software, implement it
> for previous versions as well, if it is not too much pain.
>
> regards, Jan Willem

[Evon Silvia]

Thank you to both Martin and Lewis for the clarification. Perhaps the lastools mailing list wasn't the best place to clarify this point, but I figured you'd know the answer if anyone does. :-)

Evon

[Mike Umansky]

Dear LASTools Community,

 

It was brought to my attention that QT Reader was crashing when certain kinds of LAZ files were opened.  We have addressed those issues and should have a new release shortly.

 

Thank you,

Mike

 

Michael Umansky

301-589-4047

www.appliedimagery.com

 

From: last...@googlegroups.com [mailto:last...@googlegroups.com] On Behalf Of Jan Willem de Pater
Sent: Friday, April 05, 2013 7:17 PM
To: last...@googlegroups.com
Subject: [LAStools] Re: Laz in Qt Reader

 

Hello Lewis, thanks, and Martin for forwarding, because I would not have noticed your answer in this gmail address that I had completely forgotten about....

It is a helpful answer in that it gives some interesting insight in the history of Las.

I can only say that anyhow it would have been a good idea if the phenomena "Extra Bytes Per Point" would have been described more explicitly in ASPRS specs 1.0 to 1.3, as now was done in ASPRS specs 1.4. Basing myself purely on the specs, ignorant of this phenomena and the Manual, I certainly did not include this in my code at the time, and thought there was some redundancy indeed.

 

best regards, Jan Willem