"official Extra Bytes" in the LAS specification
549 views
Skip to first unread message
Martin Isenburg
Dec 30, 2012, 8:45:27 AM12/30/12
to LAStools - efficient command line tools for LIDAR processing
The message below is a cross-posting with "The LAS room". Should you wish to comment, please do so in "The LAS room" directly. You can enter "The LAS room" here: http://groups.google.com/group/lasroom/
---
Dear LAS friends,
maybe you have noticed that RIEGL have started to use the concept of
"Extra Bytes" to describe and store additional attributes with each
return. For example, they store the normalized reflectivity, the pulse
width, or other attributes depending on the sensor employed. It should
be noted that they can also do this for exports to LAS 1.1, LAS 1.2,
or LAS 1.3 since the "extra bytes" concept is forward compatible.
For the pulse width, for example, RIEGL uses the "Extra Bytes" data
type 4 - a scaled unsigned integer with a 2 byte representation. For
more details on how they use "Extra Bytes" see their whitepaper:
http://lastools.org/las14/Whitepaper_LAS_extrabytes_implementation_in_Riegl_software.pdf
Also OPALS has started to use "Extra Bytes" for pretty much the same
items but now uses a slightly different terminology to refer to them.
Before we continue to diverge in the "extra bytes" name space I
suggest we come together as a community and agree on a set of
"accepted names" that could possibly become "official Extra Bytes"
that are describes in an addendum to the LAS 1.4 specification.
When I suggested the addition of "Extra Bytes" to the specification I
was hoping that one day certain "Extra Bytes" would be agreed upon by
all and could then become part of an addendum to the specification. To
realize this I had reserved the first two bytes of the "Extra Bytes"
struct. Look at the definition of the EXTRA BYTES VLR:
struct EXTRA_BYTES
{
unsigned char reserved[2]; // 2 bytes
unsigned char data_type; // 1 byte
unsigned char options; // 1 byte
char name[32]; // 32 bytes
unsigned char unused[4]; // 4 bytes
anytype no_data[3]; // 24 = 3*8 bytes
anytype min[3]; // 24 = 3*8 bytes
anytype max[3]; // 24 = 3*8 bytes
double scale[3]; // 24 = 3*8 bytes
double offset[3]; // 24 = 3*8 bytes
char description[32]; // 32 bytes
}; // total of 192
My intention with reserving the first two bytes and forcing them to be
set as 0 in the specification was to allow the specification to
support "first-class" EXTRA BYTES at a later stage. So, for example,
we could add some of the new topo fields that RIEGLS has suggested and
is aready using and enumerate them
1 - Normalized Reflectivity
2 - Pulse Width
3 - Shape Pulse Deviation
4 - ...
and similar for some bathymetric properties that I have been chatting
about with several folks recently we could add:
256 - Calibrated intensity
257 - Sigma_x, Sigma_y, Sigma_z
258 - Water Column Optical Depth
259 - ....
and put these descriptions into the spec. This way we do not need to
mess with the current spec but we have "native" support for all the
new items that are starting tp emerge and various softwares could
implement a quick and reliable check to see whether the topo/bathy
points have the whatever extra information they were hoping to use.
Using extra bytes makes everything backwards compatible. Older
software will still be able to read the resulting points out of the
LAS files (albeit without being able to interpret the new topo/bathy
fields).
My questions:
(1) should we agree on unique "extra bytes"?
(2) should we merely agree on a unique naming?
(3) should we also agree on a enumeration and ask ASPRS to give it its
blessing?
(4) should we also specify the data type?
The latter is prevent folks from storing, for example, the "pulse
width" with a bloated 8 byte floating-point number where a scaled two
byte integer would already serve the same purpose (like done in the
RIEGL white paper).
I wish you a Happy New Year and may many billions of LiDAR points
cross your path in 2013 ... (-:
Martin @rapidlasso
--
http://rapidlasso.com - fast tools to catch reality
--
You are subscribed to the LAS room - a friendly place to discuss the specifications of the LAS format. This is a public forum for those who want LAS to succeed as an open standard. Here you can go on record with bug reports, suggestions, and concerns about the current and proposed specifications.
Visit this group at http://groups.google.com/group/lasroom
Post to this group with an email to las...@googlegroups.com
Unsubscribe by email to lasroom+u...@googlegroups.com
---
Dear LAS friends,
maybe you have noticed that RIEGL have started to use the concept of
"Extra Bytes" to describe and store additional attributes with each
return. For example, they store the normalized reflectivity, the pulse
width, or other attributes depending on the sensor employed. It should
be noted that they can also do this for exports to LAS 1.1, LAS 1.2,
or LAS 1.3 since the "extra bytes" concept is forward compatible.
For the pulse width, for example, RIEGL uses the "Extra Bytes" data
type 4 - a scaled unsigned integer with a 2 byte representation. For
more details on how they use "Extra Bytes" see their whitepaper:
http://lastools.org/las14/Whitepaper_LAS_extrabytes_implementation_in_Riegl_software.pdf
Also OPALS has started to use "Extra Bytes" for pretty much the same
items but now uses a slightly different terminology to refer to them.
Before we continue to diverge in the "extra bytes" name space I
suggest we come together as a community and agree on a set of
"accepted names" that could possibly become "official Extra Bytes"
that are describes in an addendum to the LAS 1.4 specification.
When I suggested the addition of "Extra Bytes" to the specification I
was hoping that one day certain "Extra Bytes" would be agreed upon by
all and could then become part of an addendum to the specification. To
realize this I had reserved the first two bytes of the "Extra Bytes"
struct. Look at the definition of the EXTRA BYTES VLR:
struct EXTRA_BYTES
{
unsigned char reserved[2]; // 2 bytes
unsigned char data_type; // 1 byte
unsigned char options; // 1 byte
char name[32]; // 32 bytes
unsigned char unused[4]; // 4 bytes
anytype no_data[3]; // 24 = 3*8 bytes
anytype min[3]; // 24 = 3*8 bytes
anytype max[3]; // 24 = 3*8 bytes
double scale[3]; // 24 = 3*8 bytes
double offset[3]; // 24 = 3*8 bytes
char description[32]; // 32 bytes
}; // total of 192
My intention with reserving the first two bytes and forcing them to be
set as 0 in the specification was to allow the specification to
support "first-class" EXTRA BYTES at a later stage. So, for example,
we could add some of the new topo fields that RIEGLS has suggested and
is aready using and enumerate them
1 - Normalized Reflectivity
2 - Pulse Width
3 - Shape Pulse Deviation
4 - ...
and similar for some bathymetric properties that I have been chatting
about with several folks recently we could add:
256 - Calibrated intensity
257 - Sigma_x, Sigma_y, Sigma_z
258 - Water Column Optical Depth
259 - ....
and put these descriptions into the spec. This way we do not need to
mess with the current spec but we have "native" support for all the
new items that are starting tp emerge and various softwares could
implement a quick and reliable check to see whether the topo/bathy
points have the whatever extra information they were hoping to use.
Using extra bytes makes everything backwards compatible. Older
software will still be able to read the resulting points out of the
LAS files (albeit without being able to interpret the new topo/bathy
fields).
My questions:
(1) should we agree on unique "extra bytes"?
(2) should we merely agree on a unique naming?
(3) should we also agree on a enumeration and ask ASPRS to give it its
blessing?
(4) should we also specify the data type?
The latter is prevent folks from storing, for example, the "pulse
width" with a bloated 8 byte floating-point number where a scaled two
byte integer would already serve the same purpose (like done in the
RIEGL white paper).
I wish you a Happy New Year and may many billions of LiDAR points
cross your path in 2013 ... (-:
Martin @rapidlasso
--
http://rapidlasso.com - fast tools to catch reality
--
You are subscribed to the LAS room - a friendly place to discuss the specifications of the LAS format. This is a public forum for those who want LAS to succeed as an open standard. Here you can go on record with bug reports, suggestions, and concerns about the current and proposed specifications.
Visit this group at http://groups.google.com/group/lasroom
Post to this group with an email to las...@googlegroups.com
Unsubscribe by email to lasroom+u...@googlegroups.com
Gottfried Mandlburger
Jan 4, 2013, 3:41:07 PM1/4/13
to last...@googlegroups.com, las...@googlegroups.com, Martin Isenburg
Martin, LAS folks,
Let me first wish you all a happy and peaceful New Year!
On 30.12.2012 14:45, Martin Isenburg wrote:
> Dear LAS friends,
is for scientists, software produces, and end-users. Thus, you can count
on the support of the OPALS team (TU Vienna, GEO, Department of Geodesy
and Geoinformation).
Some remarks:
> My questions:
>
> (1) should we agree on unique "extra bytes"?
> (2) should we merely agree on a unique naming?
> (3) should we also agree on a enumeration and ask ASPRS to give it its
> blessing?
We think, it is most important to provide the following information for
each "standard extra byte":
a) a full description of the SEMANTICS
b) a proper UNIT
ad a) Two examples:
.) What is "pulse width"? Riegl stores the pulse width as FWHW (full
width at half maximum). However, there are other possibilities to
describe the width of a pulse (e.g. sigma (std.dev.) in case of Gaussian
decomposition). Thus, it is absolutely mandatory to know the exact
content/meaning of an extra byte point attribute.
By the way, we would rather prefer "echo width" for a clear separation
of the outgoing laser pulse and the received/recorded echo. And in case
of LAS, we are dealing with echoes only, aren't we?
.) Intensity: The LAS specification contains "intensity" as a standard
point attribute since the beginning and in all point record format
types. However, some data providers rather store peak power (amplitude)
than signal intensity (i.e. area under the waveform curve). This is a
very unsatisfactory situation, and thus, special emphasis should be laid
on a proper definition of the underlying semantics, whenever a new
standard extra byte is created/added.
ad b) Example "Slope": Slopes can be expressed in percent, grades,
radians, degrees. Thus, a proper definition of the underlying unit
(preferably using SI units) is obligatory.
> (4) should we also specify the data type?
We don't think that this is a good idea. Think of the evolution of the
"scan angle rank" attribute. It was first (and still is) a 1-byte-signed
integer, limiting the resolution to 1 (degree). Then, there was a demand
for a higher resolution, thus, the standard had to be adopted (LAS 1.4:
point record format 6, scan angle = 2 byte signed integer, increment =
0.006 degrees). So, whenever you can provide an attribute with a higher
resolution (and you can bet that new sensors will provide that sooner or
later), either the standard needs to be changed or a new attribute needs
to be created (eg., scan angle rank (1-byte)--> scan angle (2-byte)). It
is one of the strengths of the extra byte concept itself, that the type
is defined within the the extra byte VLR.
Kind regards,
Gottfried (for the entire OPALS developer team)
See us on: www.ipf.tuwien.ac.at/opals
>
> The latter is prevent folks from storing, for example, the "pulse
> width" with a bloated 8 byte floating-point number where a scaled two
> byte integer would already serve the same purpose (like done in the
> RIEGL white paper).
>
> I wish you a Happy New Year and may many billions of LiDAR points
> cross your path in 2013 ... (-:
>
> Martin @rapidlasso
>
> --
> http://rapidlasso.com - fast tools to catch reality
>
> --
> You are subscribed to the LAS room - a friendly place to discuss the
> specifications of the LAS format. This is a public forum for those who
> want LAS to succeed as an open standard. Here you can go on record with
> bug reports, suggestions, and concerns about the current and proposed
> specifications.
>
> Visit this group at http://groups.google.com/group/lasroom
> Post to this group with an email to las...@googlegroups.com
> <mailto:las...@googlegroups.com>
>
> --
> Download LAStools at
> http://lastools.org/
> Visit the LAStools group at
> http://groups.google.com/group/lastools/
> Be social with LAStools at
> http://www.facebook.com/LAStools
> http://www.twitter.com/LAStools
--
------------------------------------------------------------------------
Dr. Gottfried Mandlburger
Research groups Photogrammetry and Remote Sensing
Department of Geodesy and Geoinformation
Vienna University of Technology
Gusshausstrasse 27-29, A-1040 Wien
email: g...@ipf.tuwien.ac.at Tel (++43 1) 58801 12235
www: http://www.ipf.tuwien.ac.at Fax (++43 1) 58801 12299
------------------------------------------------------------------------
Let me first wish you all a happy and peaceful New Year!
On 30.12.2012 14:45, Martin Isenburg wrote:
> Dear LAS friends,
> When I suggested the addition of "Extra Bytes" to the specification I
> was hoping that one day certain "Extra Bytes" would be agreed upon by
> all and could then become part of an addendum to the specification.
This is a charming idea. The more standardised things are the better it
> was hoping that one day certain "Extra Bytes" would be agreed upon by
> all and could then become part of an addendum to the specification.
is for scientists, software produces, and end-users. Thus, you can count
on the support of the OPALS team (TU Vienna, GEO, Department of Geodesy
and Geoinformation).
Some remarks:
> My questions:
>
> (1) should we agree on unique "extra bytes"?
> (2) should we merely agree on a unique naming?
> (3) should we also agree on a enumeration and ask ASPRS to give it its
> blessing?
each "standard extra byte":
a) a full description of the SEMANTICS
b) a proper UNIT
ad a) Two examples:
.) What is "pulse width"? Riegl stores the pulse width as FWHW (full
width at half maximum). However, there are other possibilities to
describe the width of a pulse (e.g. sigma (std.dev.) in case of Gaussian
decomposition). Thus, it is absolutely mandatory to know the exact
content/meaning of an extra byte point attribute.
By the way, we would rather prefer "echo width" for a clear separation
of the outgoing laser pulse and the received/recorded echo. And in case
of LAS, we are dealing with echoes only, aren't we?
.) Intensity: The LAS specification contains "intensity" as a standard
point attribute since the beginning and in all point record format
types. However, some data providers rather store peak power (amplitude)
than signal intensity (i.e. area under the waveform curve). This is a
very unsatisfactory situation, and thus, special emphasis should be laid
on a proper definition of the underlying semantics, whenever a new
standard extra byte is created/added.
ad b) Example "Slope": Slopes can be expressed in percent, grades,
radians, degrees. Thus, a proper definition of the underlying unit
(preferably using SI units) is obligatory.
> (4) should we also specify the data type?
"scan angle rank" attribute. It was first (and still is) a 1-byte-signed
integer, limiting the resolution to 1 (degree). Then, there was a demand
for a higher resolution, thus, the standard had to be adopted (LAS 1.4:
point record format 6, scan angle = 2 byte signed integer, increment =
0.006 degrees). So, whenever you can provide an attribute with a higher
resolution (and you can bet that new sensors will provide that sooner or
later), either the standard needs to be changed or a new attribute needs
to be created (eg., scan angle rank (1-byte)--> scan angle (2-byte)). It
is one of the strengths of the extra byte concept itself, that the type
is defined within the the extra byte VLR.
Kind regards,
Gottfried (for the entire OPALS developer team)
See us on: www.ipf.tuwien.ac.at/opals
>
> The latter is prevent folks from storing, for example, the "pulse
> width" with a bloated 8 byte floating-point number where a scaled two
> byte integer would already serve the same purpose (like done in the
> RIEGL white paper).
>
> I wish you a Happy New Year and may many billions of LiDAR points
> cross your path in 2013 ... (-:
>
> Martin @rapidlasso
>
> --
> http://rapidlasso.com - fast tools to catch reality
>
> --
> You are subscribed to the LAS room - a friendly place to discuss the
> specifications of the LAS format. This is a public forum for those who
> want LAS to succeed as an open standard. Here you can go on record with
> bug reports, suggestions, and concerns about the current and proposed
> specifications.
>
> Visit this group at http://groups.google.com/group/lasroom
> Post to this group with an email to las...@googlegroups.com
> Unsubscribe by email to lasroom+u...@googlegroups.com
> <mailto:lasroom%2Bunsu...@googlegroups.com>
>
> --
> Download LAStools at
> http://lastools.org/
> Visit the LAStools group at
> http://groups.google.com/group/lastools/
> Be social with LAStools at
> http://www.facebook.com/LAStools
> http://www.twitter.com/LAStools
--
------------------------------------------------------------------------
Dr. Gottfried Mandlburger
Research groups Photogrammetry and Remote Sensing
Department of Geodesy and Geoinformation
Vienna University of Technology
Gusshausstrasse 27-29, A-1040 Wien
email: g...@ipf.tuwien.ac.at Tel (++43 1) 58801 12235
www: http://www.ipf.tuwien.ac.at Fax (++43 1) 58801 12299
------------------------------------------------------------------------
Sevcik Christian
Feb 6, 2013, 1:21:58 PM2/6/13
to last...@googlegroups.com, las...@googlegroups.com, Martin Isenburg
Hello,
I am not sure if this thread is still alive, but we are open for any discussion regarding unique extra bytes and naming.
The Whitepaper Martin mentioned is a first start and if there is a suggestion how to improve the usability, we are happy to support it.
Best Regards
Christian Sevcik
________________________________________
_____________________________________________
RIEGL Laser Measurement Systems GmbH
Riedenburgstra�e 48, 3580 Horn, Austria
Registered at Landesgericht Krems, FN 40233 t
________________________________ Disclaimer _______________________________
This email and any files attached are intended for the addressee and may
contain information of a confidential nature. If you are not the intended
recipient, be aware that this email was sent to you in error and you should
not disclose, distribute, print, copy or make other use of this email or
its attachments. In that case please notify us by return email, and erase
all copies of the message and attachments. Thank you.
RIEGL reserves the right to monitor (and examine for viruses) all emails
and email attachments, both inbound and outbound. Email communications and
their attachments may not be secure or error- or virus- free and the com-
pany does not accept liability or responsibility for such matters or the
consequences thereof.
I am not sure if this thread is still alive, but we are open for any discussion regarding unique extra bytes and naming.
The Whitepaper Martin mentioned is a first start and if there is a suggestion how to improve the usability, we are happy to support it.
Best Regards
Christian Sevcik
________________________________________
RIEGL Laser Measurement Systems GmbH
Riedenburgstra�e 48, 3580 Horn, Austria
Registered at Landesgericht Krems, FN 40233 t
________________________________ Disclaimer _______________________________
This email and any files attached are intended for the addressee and may
contain information of a confidential nature. If you are not the intended
recipient, be aware that this email was sent to you in error and you should
not disclose, distribute, print, copy or make other use of this email or
its attachments. In that case please notify us by return email, and erase
all copies of the message and attachments. Thank you.
RIEGL reserves the right to monitor (and examine for viruses) all emails
and email attachments, both inbound and outbound. Email communications and
their attachments may not be secure or error- or virus- free and the com-
pany does not accept liability or responsibility for such matters or the
consequences thereof.
Reply all
Reply to author
Forward
0 new messages