Re: [LAStools] Re: Laz in Qt Reader

[Martin Isenburg]

Hello Evon,

I am cc-ing "the LAS room" on this one as this would be the right
place to continue this discussion should it be necessary.

>> Because PDAL does not support "extra bytes" in LAZ (or LAS) files it will
>> crash if you open LAZ files containing "extra bytes". This is trouble-some
>> because "extra bytes" are exported to the LAZ format, for example, by
>> RIEGL's RiPROCESS to store attributes such as echo width and normalized
>> reflectance values.
>
> But why would PDAL support extra bytes for LAS 1.2 and 1.3?

Extra bytes have been with the LAS format since day 0. In the LAS
header there is a field that specifies the point format being used (0,
1, 2, 3) and then there is a field that specifies the size of each
point record. Should the point size implied by the point type be
smaller than the specified point record size any reader had to either
skip those "extra bytes" if it did not know how to interpret them or
load them into a separate attribute field assuming there was some way
for it to know what they meant.

The only thing that is new with LAS 1.4 is that there is now a way to
descriptive those extra bytes in the specification. Hence, unknown
arrays of "extra bytes" become suddenly meaningful to anyone via the
LASF_Spec VLR with user_ID 4. Because it does not hurt to put unknown
VLRs into earlier versions of the specification (they could safely be
ignored but may as well be parsed if known) RIEGL also put their
"extra bytes" descriptions into earlier versions. And so does LAStools
and probably a few other softwares ...

> As something of a newcomer to the LAS world, I have always found the idea of
> "extra bytes" rather confusing. Until LAS 1.4, the LAS specifications say
> nothing about including extra bytes with the point data, instead indicating
> that extra information should be stored in the VLR and EVLR records.

If you have extra information that is per point such as an "echo
width" you need something like "extra bytes" to avoid IO hampering
file seeks. VLRs and EVLRs are better suited to store information that
is per file such as projection information, spatial indexing,
compression parameters, additional mission info ...

> They
> seem to imply that point records should be of a standard, fixed length, and
> so, by my interpretation, the inclusion of "extra bytes" would actually be a
> violation of the LAS specification for versions 1.0 through 1.3.

If point records were meant to be a standard fixed size as implied by
the point type then the inclusion of the point record size would have
not been a redundant source for error. While the concept of "extra
bytes" per point is not explicitely stated in the spec in earlier
versions it has always been there ... just like the potential presence
of this additional information that does not clash with anything
written in the specification:

// if the header_size is larger than implied by the version
U32 user_data_in_header_size;
U8* user_data_in_header;

// if the offset_to_point_data is larger than the combination of
header size and all VLR sizes
U32 user_data_after_header_size;
U8* user_data_after_header;

Cheers,

Martin @rapidlasso

--
http://rapidlasso.com - fast tools to play with large LiDARs

[Martin Isenburg]

Hello,

This is the remainder of the discussion that happened in the LAStools
user forum including Lewis Graham's (concluding) remarks on this topic
(see below)"

On Thu, Apr 4, 2013 at 6:38 PM, Jan Willem de Pater wrote:
> hi there,
>
> as a matter of fact I have wrestled precisely with this theme for quite some
> time, sometimes. Are or are not extra bytes per point allowed in Las 1.0 -
> Las 1.3 . Martin (who is certainly not just someone in the world of LAS)
> says: absolutely. My conclusion, after several times studying the ASPRS
> specs, is: it is just not clear. Martin anyhow is right that the Point
> Record Length in the lasheader would be redundant if there were no extra
> bytes per point. Maybe the International Court of Justice in The Hague can
> sentence in this case. (Be aware that there may be several Revision specs of
> one LAS version).
> I think, if you implement Extra Bytes for 1.4 in some software, implement it for previous versions as well, if it is not too much pain.
>
> regards, Jan Willem

Hello,

I am forwarding the below message from Lewis Graham (with his
permission) as we figured it will be reassuring for everyone involved
with LAS to hear that we are both in full agreement on this ... (-;

Regards,

Martin @rapidlasso

http://rapidlasso.com - fast tools for surviving LiDAR wars (-;

---------- Forwarded message ----------
From: Lewis Graham <lgr...@geocue.com>
Date: Fri, Apr 5, 2013 at 2:59 AM
Subject: RE: [LAStools] Re: Laz in Qt Reader
To: jeewe...@gmail.com
Cc: martin....@gmail.com

Hello Jan,

I am responding directly because I have somehow broken my Google links
to the forums (I will get around to fixing this one of these days….).

I have been the chair of the LAS working group since before the format
was “donated” to the ASPRS (since around 1998).

The situation is exactly as Martin describes. From the very first
draft of the format, we intended for users to be able to pack anything
they wanted in the Point Data Records. We only specified what was
intended as the “well known exchange payload” for the first few fields
(as defined in the various Point Data Record Formats, PDRF). For
example, you can even put double precision X, Y, Z coordinates in the
record so long as you provide the integer equivalents in the required
X, Y, Z fields.

As Martin says, this is why both the format number (0 -> 10) is
required as well as the length of the payload. In fact, in every
article we have ever written on the format, we warn programmers not to
use the size of the required fields as the size of the payload. See,
for example, page 207 of the “ASPRS Manual of Airborne Topographic
LIDAR.” Paragraph 3 begins “It is critically important for
programmers to understand that a point data record in LAS can contain
fields beyond those defined by the PDRF.” It goes on to explain that
we added the ability to formalize extensions via Dr. Isenburg’s
contributions to LAS 1.4 but using the Extra Bytes facility is not
mandatory.

So the bottom line is that it has always been perfectly fine to extend
the point payloads in LAS and this is, in fact, very frequently done.

I hope this information is helpful.

Best Regards,

Lewis
______________________________
_

Lewis Graham
President/CTO
GeoCue Group
9668 Madison Blvd., Suite 202
Madison, AL 35758
(256) 461-8289 Telephone
(256) 461-8249 Fax
www.geocue.com
www.qcoherent.com

Complete LIDAR Solutions
Integrating the Geospatial Workplace...