how to use project GUID of LAS header?

[Martin Isenburg]

Hello,

I have been asked to add a function to lasinfo that can modify (aka set) the "project GUID" of the LAS header (that is completely set to zero in all examples I have at hand). Does anyone's software make use of these project GUIDs? Are you using them in your workflows? If yes, how would you like to be able to set them?

lasinfo -i my.las -set_project_GUID_data1 xxxx1 ^

                          -set_project_GUID_data2 xxxx2 ^

                          -set_project_GUID_data3 xxxx3 ^

                          -set_project_GUID_data4 xxxx4

lasinfo -i my.las -set_project_GUID_data1 xxxx1 ^

                          -set_project_GUID_data2 xxxx2 ^

                          -set_project_GUID_data3 xxxx3 ^

                          -set_project_GUID_data4 xxxx4a xxxx4b

lasinfo -i my.las -set_project_GUID xxxx1 xxxx2 xxxx3 xxxx4

lasinfo -i my.las -set_project_GUID xxxx1 xxxx2 xxxx3 xxxx4a xxxx4b

and how should the parameters xxxx1 to xxxx4 be written down? As numbers? As strings? In hexadecimal notation?  Here is how the LAS 1.4 spec specifies them:

Project ID - GUID data 1 unsigned long 4 bytes

Project ID - GUID data 2 unsigned short 2 byte

Project ID - GUID data 3 unsigned short 2 byte

Project ID - GUID data 4 unsigned char[8] 8 bytes

And here is how the LAS 1.4 spec describes them:

Project ID (GUID data): The four fields that comprise a complete Globally Unique Identifier (GUID) are now reserved for use as a Project Identifier (Project ID). The field remains optional. The time of assignment of the Project ID is at the discretion of processing software. The Project ID should be the same for all files that are associated with a unique project. By assigning a Project ID and using a File Source ID (defined above) every file within a project and every point within a file can be uniquely identified, globally.

Regards,

Martin @rapidlasso

IMPORTANT: zLAS is a proprietary format. LAS and LAZ are open formats. Some people call LAZ "Martin's proprietary format". That is wrong. LAZ is *not* a proprietary format at all. What does "proprietary format" mean? Here is what Wikipedia [1] says:

A proprietary format is a file format of a company, organization, or individual

• that contains data that is ordered and stored according to a particular encoding-scheme, designed by the company or organization to be secret, such that the decoding and interpretation of this stored data is only easily accomplished with particular software or hardware that the company itself has developed. The specification of the data encoding format is not released, or underlies non-disclosure agreements.

In contrast an open format is a file format that is published and free to be used by everybody.

[1] http://en.wikipedia.org/wiki/Proprietary_format

[Evon Silvia]

I know of no centralized system to lookup or assign globally-unique GUIDs, so assigning one would be completely arbitrary and only useful to the agency that assigned it. Therefore, I opted to simply use the 4th field as a place to store an 8-character abbreviation of the project name.

We have kicked around several other options (date, Unique vendor ID# + projectID# + delivery#, completely random GUID) but this one seemed to be the most obvious and simple, even if it doesn't technically result in a GUID per se.

It'll be interesting to see how other utilize that field, as it's always been something of an enigma to me.

Evon

--
--
You are subscribed to "The LAS room - a friendly place to discuss the the LAS or LAZ formats" for those who want to see LAS or LAZ succeed as open standards. Go on record with bug reports, suggestions, and concerns about current and proposed specifications.
 
Visit this group at http://groups.google.com/group/lasroom
Post to this group with an email to las...@googlegroups.com
Unsubscribe by email to lasroom+u...@googlegroups.com
---
You received this message because you are subscribed to the Google Groups "The LAS room - a friendly place to discuss the LAS and LAZ formats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lasroom+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Lewis Graham]

The GUID is infinitely useful.

It is guaranteed to be unique across the universe of all GUIDs.

You can generate one using any of a number of on-line GUID generators.

Remember, no one need know anything at all about the GUID so long as the same GUID is used for all files in a project. 

 

Thus if a GUID is properly assigned to a project, all data within that project (all the way down to single points) can be uniquely identified.

 

Lewis Graham

[Evon Silvia]

If I were to implement a GUID parser I would probably follow the hex-based convention set forth in this wikipedia article so that it could accept GUIDs generated (as Lewis mentioned) by websites like http://www.guidgen.com/. Example syntax might be:

lasinfo -i my.las -set_project_guid xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

                                    | i1   | |i2| |i3| |4a| |    4b    |

Where i1, i2, and i3 are the first, second, and third IDs from the LAS specification, 4a is the first two bytes from the fourth ID, and 4b is the remaining six. The main issue would probably be to ensure you get the endianness right. It's unclear to me from the LAS specification whether the GUID convention of big-endian ordering should be followed for the final eight bytes or whether it should be LittleEndian like the rest of the LAS file - that is, whether 4a should be the first two bytes or the last two bytes.

It might also be helpful to have a command switch to have one generated for a set of LAS files by LASinfo.

Evon

--
Download LAStools at
http://lastools.org
http://rapidlasso.com
Be social with LAStools at
http://facebook.com/LAStools
http://twitter.com/LAStools
http://linkedin.com/groups/LAStools-4408378
Manage your settings at
http://groups.google.com/group/lastools/subscribe

[Jan Depner]

Lewis,

    Exactly *how* is it guaranteed to be unique across the universe of all GUIDs.  The implied question in the original post was, "where is the centralized system to lookup or assign globally-unique GUIDs".  Please explain how a GUID is, to quote you, "properly assigned to a project".  I can inadvertently (or on purpose for that matter) create a duplicate of any GUID that exists in any LAS file, therefore, it is *not* guaranteed to be unique across the universe of all GUIDs.  Please explain how we can make sure that we aren't using something that was previously used by some other project.  Can you provide a central clearing house for GUIDs?  Can *anyone* provide a central clearing house for GUIDs?  If so we would all like to know about it.  I'd love to be able to label every file I create as USACE JALBTCX or Naval Oceanographic Office JALBTCX or NOAA JALBTCX but I don't know if I'll be duplicating something that already exists.  The definition in the LAS v.1.4 specification is woefully inadequate.

V/r,
Jan Depner

--

[Lewis Graham]

Hi Jan,

 

GUIDs are unique by construction – that’s the entire purpose of the invention of GUIDs.  They solve the problem of uniqueness without the need for centralized arbitration.   

If you create a GUID (using a proper GUID generator which, again, does not require a lookup of precreated GUIDs), the chances of creating a duplicate are less than the chance you get hit in the head by a meteorite. 

 

We envisioned these as being used (in LAS) for managing data while still “near” the sensor.  For example, I could tag a bunch of LAS files that came from the same sortie with a GUID and then tag the related metadata (such as the trajectory) with that same GUID.  It is an optional field in the LAS header and can be freely used for project management schemes. 

 

We did not envision the GUID as being particularly useful once data hit the exploitation phase (for example, a county government taking delivery of some LAS data).  Once you have merged, at the tile level, a bunch of individual LAS files, the individual point tracking requires a local catalog (which can be stored in a VLR of the LAS file).  I am not aware of any implementations of this.

 

Thus the GUID is primarily used in production management.  It is particularly useful in multi-sensor, multi-contractor projects.

 

Regards,

Lewis

 

 

 

Lewis Graham

[Terje Mathisen]

Lewis Graham wrote:
>
> Hi Jan,
>
> GUIDs are unique by construction – that’s the entire purpose of the
> invention of GUIDs. They solve the problem of uniqueness without the
> need for centralized arbitration.
>
> If you create a GUID (using a proper GUID generator which, again, does
> not require a lookup of precreated GUIDs), the chances of creating a
> duplicate are less than the chance you get hit in the head by a
> meteorite.
>

The chances are _far_ less than that.

With 2^128 possible values, the Birthday paradox states that you need to
produce approximately 2^64 of them (in a single usage domain) before you
get the _first_ duplicate, i.e. since the surface area of the Earth is
about 5.1e8 sq km we need an average of 36170 surveys per square meter,
each with its own randomly generated GUID, before we can expect the
first collision.

BTW, those numbers include all the oceans; if we limit ourselves to land
surveys we get about 250 000 GUIDs per square meter or one for every 2x2
mm spot on the planet.

Even if you reserve part of the space for a vendor or application ID
string, most of that calculation will still hold. :-)

Terje

--
- <Terje.M...@tmsw.no>
"almost all programming can be viewed as an exercise in caching"

[Jan Depner]

Thanks Lewis,

    Now the next question is - where can I get a proper GUID generator ;-)  Is there one in LASlib?

    The second point though is covered here - http://news.nationalgeographic.com/news/2013/02/130220-russia-meteorite-ann-hodges-science-space-hit/

Cheers,
Jan

[Lewis Graham]

Hello Jan,

 

Here is an on-line generator:

http://www.guidgen.com/

 

 

Second part – she must have duplicated a doppelganger’s  GUID…..

 

Regards,

Lewis

 

 

 

Lewis Graham

AirGon LLC, small UAS Solutions

GeoCue Group

9668 Madison Blvd., Suite 202

Madison, AL USA 35758

01-256-461-8289

www.geocue.com

www.LP360.com

[Terje Mathisen]

Jan Depner wrote:
> Thanks Lewis,
>
> Now the next question is - where can I get a proper GUID generator
> ;-) Is there one in LASlib?

Which OS are you on?

Linux and most of the other unix derivatives have something like
/dev/random which combines a true random number generator (collecting
entropy from all available sources, including mouse and kbd
interactions, packet arrival times and any cpu hw source.

What kind of cpu hardware?

On a reasonably modern INtel chip you have the RDRAND instruction which
returns true hw random values of up to 64 bits:

Call it twice (checking the carry flag each time to verify that data was
available) and you have the 128 bits needed.

BTW, the minimum code needed to use this instruction requires either
inline asm or compiler intrinsics, you should first check that the
RDRAND opcode is available, which you do with the following code:

if (cpuid(0,abcd) >= 1 && (cpuid(1,abcd)) && (abcd[2] >> 30) & 1) {
// Safe to read the RDRAND register!

I can code up a minimal-size Win32 application to do just this, it
should be _very_ tiny! :-)

Terje

>
> The second point though is covered here -
> http://news.nationalgeographic.com/news/2013/02/130220-russia-meteorite-ann-hodges-science-space-hit/

Since we have far less than 2^64 people, the odds of being hit is
obviously significantly larger than those for accidentally getting
duplicate GUIDs. :-)

Terje

>> *From:*last...@googlegroups.com [mailto:last...@googlegroups.com]
>> *On Behalf Of *Jan Depner
>> *Sent:* Monday, May 11, 2015 21:24
>> *To:* last...@googlegroups.com
>> *Subject:* Re: [LAStools] RE: [The LAS room] how to use project GUID

>> *From:*las...@googlegroups.com <mailto:las...@googlegroups.com>
>> [mailto:las...@googlegroups.com] *On Behalf Of *Evon Silvia
>> *Sent:* Monday, May 11, 2015 10:38
>> *To:* las...@googlegroups.com <mailto:las...@googlegroups.com>
>> *Cc:* LAStools - efficient command line tools for LIDAR processing
>> *Subject:* Re: [The LAS room] how to use project GUID of LAS header?

>> A *proprietary format* is a file format
>> <http://en.wikipedia.org/wiki/File_format> of a company,

>> organization, or individual
>>
>> ·that contains data that is ordered and stored according to a
>> particular encoding-scheme, designed by the company or
>> organization to be secret, such that the decoding and

>> interpretation of this stored data is /only/ easily

>> accomplished with particular software or hardware that the
>> company itself has developed. The specification of the data
>> encoding format is not released, or underlies non-disclosure
>> agreements

>> <http://en.wikipedia.org/wiki/Non-disclosure_agreement>.

>>
>> In contrast an open format

>> <http://en.wikipedia.org/wiki/Open_format> is a file format

>> that is published and free to be used by everybody.
>>
>>
>> [1] http://en.wikipedia.org/wiki/Proprietary_format
>>
>> --
>> --
>> You are subscribed to "The LAS room - a friendly place to
>> discuss the the LAS or LAZ formats" for those who want to see
>> LAS or LAZ succeed as open standards. Go on record with bug
>> reports, suggestions, and concerns about current and proposed
>> specifications.
>>
>> Visit this group at http://groups.google.com/group/lasroom
>> <http://groups.google.com/group/lasroom>
>> Post to this group with an email to las...@googlegroups.com

>> <mailto:las...@googlegroups.com>

>> Unsubscribe by email to lasroom+u...@googlegroups.com

>> <mailto:lasroom%2Bunsu...@googlegroups.com>

>> ---
>> You received this message because you are subscribed to the
>> Google Groups "The LAS room - a friendly place to discuss the
>> LAS and LAZ formats" group.
>> To unsubscribe from this group and stop receiving emails from
>> it, send an email to lasroom+u...@googlegroups.com

>> <mailto:lasroom+u...@googlegroups.com>.

>> For more options, visit https://groups.google.com/d/optout

>> <https://groups.google.com/d/optout>.

>>
>> --
>> --
>> You are subscribed to "The LAS room - a friendly place to discuss
>> the the LAS or LAZ formats" for those who want to see LAS or LAZ
>> succeed as open standards. Go on record with bug reports,
>> suggestions, and concerns about current and proposed specifications.
>>
>> Visit this group at http://groups.google.com/group/lasroom
>> Post to this group with an email to las...@googlegroups.com

>> <mailto:las...@googlegroups.com>

>> Unsubscribe by email to lasroom+u...@googlegroups.com

>> <mailto:lasroom+u...@googlegroups.com>

>> ---
>> You received this message because you are subscribed to the
>> Google Groups "The LAS room - a friendly place to discuss the LAS
>> and LAZ formats" group.
>> To unsubscribe from this group and stop receiving emails from it,
>> send an email to lasroom+u...@googlegroups.com

>> <mailto:lasroom+u...@googlegroups.com>.

[Malcolm Williamson]

GUIDs (UUIDs) are documented and standardized - see
http://en.wikipedia.org/wiki/Universally_unique_identifier for an
overview. These are particularly useful in enterprise-level GIS
databases to support check-out/check-in editing and synchronization
capabilities.
-Malcolm

[Dennis Shimer]

Another idea.....

Ignore the word Linux in the title, it has nothing to do with Linux, but does use Python
https://stackoverflow.com/questions/16372433/linux-command-to-generate-new-guid/21191302#21191302

I just tested the following 2 line script and it seems to fit the ticket.

     import uuid
     print str(uuid.uuid1())


python new_uuid.py
c83a0330-f8bd-11e4-b7fa-001ec9404dd1

[Ramesh Sridharan]

Good for lewis, she didn't get hit in the head. :)

Regards, 

Ramesh 

[Jan Depner]

Since some of the JALBTCX surveys are done off-the-internet I can't use that link but I found an answer.  Since I'm using Qt I can just use QUuid and convert to a QString (using QUuid::toString) to get the pieces.  Now all I have to do is throw together a little app that they can use to generate one per survey (and of course keep track of it ;-)

[Martin Isenburg]

Hello,

okay. I am adding an option to lasinfo called '-set_GUID' that takes in a hexastring. That said, there is still an issue with endian-ness. The GUIDs seem to be defined as big endian [1] but the LAS specification states all fields are little endian [2]. Hence, I am not even sure the current way these strings are reported by lasinfo has been correct all these years.

Are all GUIDs reported by lasinfo in the wrong byte order? I am admittingly much more knowledgable about grilling SQUIDs than being grilled over GUIDs ... (-;

Martin

[1] http://en.wikipedia.org/wiki/Globally_unique_identifier#Binary_encoding

[2] http://www.asprs.org/a/society/committees/standards/LAS_1_4_r13.pdf (page 11)

--------------------------------------------------------------

D:\LAStools\bin>copy ..\data\fusa.laz .

        1 file(s) copied.

D:\LAStools\bin>lasinfo -i fusa.laz -set_GUID F794F8A4-A23E-421E-A134-ACF7754E1C54 -nc

lasinfo for fusa.laz

reporting all LAS header entries:

  file signature:             'LASF'

  file source ID:             0

  global_encoding:            0

  project ID GUID data 1-4:   00000000-0000-0000-0000-000000000000

  version major.minor:        1.1

  system identifier:          'LAStools (c) by Martin Isenburg'

  generating software:        'lasclassify (110920) unlicensed'

  file creation day/year:     40/2010

  header size:                227

  offset to point data:       321

  number var. length records: 1

  point data format:          1

  point data record length:   28

  number of point records:    277573

  number of points by return: 263413 13879 281 0 0

  scale factor x y z:         0.01 0.01 0.01

  offset x y z:               0 0 0

  min x y z:                  277750.00 6122250.00 42.21

  max x y z:                  277999.99 6122499.99 64.35

variable length header record 1 of 1:

  reserved             43707

  user ID              'LASF_Projection'

  record ID            34735

  length after header  40

  description          'by LAStools of Martin Isenburg'

    GeoKeyDirectoryTag version 1.1.0 number of keys 4

      key 1024 tiff_tag_location 0 count 1 value_offset 1 - GTModelTypeGeoKey: ModelTypeProjected

      key 3072 tiff_tag_location 0 count 1 value_offset 32754 - ProjectedCSTypeGeoKey: UTM 54 southern hemisphere

      key 3076 tiff_tag_location 0 count 1 value_offset 9001 - ProjLinearUnitsGeoKey: Linear_Meter

      key 4099 tiff_tag_location 0 count 1 value_offset 9001 - VerticalUnitsGeoKey: Linear_Meter

LASzip compression (version 2.0r1 c2 50000): POINT10 2 GPSTIME11 2

D:\LAStools\bin>lasinfo -i fusa.laz -nc

lasinfo for fusa.laz

reporting all LAS header entries:

  file signature:             'LASF'

  file source ID:             0

  global_encoding:            0

  project ID GUID data 1-4:   F794F8A4-A23E-421E-A134-ACF7754E1C54

  version major.minor:        1.1

  system identifier:          'LAStools (c) by Martin Isenburg'

  generating software:        'lasclassify (110920) unlicensed'

  file creation day/year:     40/2010

  header size:                227

  offset to point data:       321

  number var. length records: 1

  point data format:          1

  point data record length:   28

  number of point records:    277573

  number of points by return: 263413 13879 281 0 0

  scale factor x y z:         0.01 0.01 0.01

  offset x y z:               0 0 0

  min x y z:                  277750.00 6122250.00 42.21

  max x y z:                  277999.99 6122499.99 64.35

variable length header record 1 of 1:

  reserved             43707

  user ID              'LASF_Projection'

  record ID            34735

  length after header  40

  description          'by LAStools of Martin Isenburg'

    GeoKeyDirectoryTag version 1.1.0 number of keys 4

      key 1024 tiff_tag_location 0 count 1 value_offset 1 - GTModelTypeGeoKey: ModelTypeProjected

      key 3072 tiff_tag_location 0 count 1 value_offset 32754 - ProjectedCSTypeGeoKey: UTM 54 southern hemisphere

      key 3076 tiff_tag_location 0 count 1 value_offset 9001 - ProjLinearUnitsGeoKey: Linear_Meter

      key 4099 tiff_tag_location 0 count 1 value_offset 9001 - VerticalUnitsGeoKey: Linear_Meter

LASzip compression (version 2.0r1 c2 50000): POINT10 2 GPSTIME11 2

[Andrew Bell]

On Fri, May 15, 2015 at 10:54 AM, Martin Isenburg <martin....@gmail.com> wrote:

Hello,

okay. I am adding an option to lasinfo called '-set_GUID' that takes in a hexastring. That said, there is still an issue with endian-ness. The GUIDs seem to be defined as big endian [1] but the LAS specification states all fields are little endian [2]. Hence, I am not even sure the current way these strings are reported by lasinfo has been correct all these years.

Are all GUIDs reported by lasinfo in the wrong byte order? I am admittingly much more knowledgable about grilling SQUIDs than being grilled over GUIDs ... (-;

The UUID is a 4-part entity, made up of a 32-bit unsigned int, a 16-bit unsigned int and another 16-bit unsigned int, and eight single characters.  Each of the values should have the standard byte ordering for LAS files (little-endian) as far as I'm aware.  A packed structure representation in C is:

struct

{

    uint32_t uidPart1;

    uint16_t uidPart2;

    uint16_t uidPart3;

    char uidPart4[8];

};

In PDAL we read each of the parts from the file separately, properly applying byte ordering as we go.  Note that no byte ordering is necessary for part 4.

-- 

Andrew Bell
andrew....@gmail.com

[Martin Isenburg]

Hello,

while the definite answer whether GUIDs are big-endian or little-endian is not solved, you can now set the GUID with lasinfo as follows using the latest version of LAStools (150516),

copy ..\data\fusa.laz .

lasinfo -i fusa.laz -nc 

[...]

  project ID GUID data 1-4:   00000000-0000-0000-0000-000000000000

[...]

lasinfo -i fusa.laz -set_GUID F794F8A4-A23E-421E-A134-ACF7754E1C54 -nh -nv -nc 

lasinfo -i fusa.laz -nc 

[...]

  project ID GUID data 1-4:   F794F8A4-A23E-421E-A134-ACF7754E1C54

[...]

Regards,

Martin @rapidlasso

--

[tjorda01]

Martin,

In your example that sets the GUID, there are 5 fields separated by four '-' characters. The first three fields, highlighted in yellow below, map correctly to uidPart1, uidPart2, uidPart3 and are in the right byte order. 

    lasinfo -i fusa.laz -set_GUID F794F8A4-A23E-421E-A134-ACF7754E1C54 -nh -nv -nc 

However, I'm confused about he last two fields (highlighted in green). How do these map to the 8 character string uidPart4?  Since there are 16 hex digits, I assume each pair maps to one of the 8 uidPart4 characters in some way. Are these suppose to be utf-8 hex codes? If so, "A1 34 AC F7 75 4E 1C 54" appears to be invalid.

How would I set uidPart4 to 'MyProj01', such that lasinfo reports the following?

    [...]

    project ID GUID data 1-4:   F794F8A4-A23E-421E-MyProj01

    [...]

LAStools version = 170828

Thanks

Tom Jordan

[Evon Silvia]

After a close read of the spec, I'm getting the impression that it was originally intended to follow with the Microsoft-style of UUIDs, as described on the Wikipedia page.

Since I wasn't on the committee at that time, I can't be sure. If that's true, though, then your letters would be encoded in the human-readable order as defined in the 8-bit ASCII char table (example: http://www.asciitable.com/).

Evon

On Wed, Oct 11, 2017 at 1:45 PM, Evon Silvia <esi...@quantumspatial.com> wrote:

I, too, am interested to know how LAStools, LP360, TerraSolid, GeoCue, and similar tools implement this.

Also added lasroom back to this email chain.

Evon

[Evon Silvia]

I, too, am interested to know how LAStools, LP360, TerraSolid, GeoCue, and similar tools implement this.

Also added lasroom back to this email chain.

Evon

On Wed, Oct 11, 2017 at 8:22 AM, tjorda01 <tjor...@harris.com> wrote:

[Lewis Graham]

Yes, this is correct.

Sent from my iPad

On Oct 11, 2017, at 19:55, Evon Silvia <esi...@quantumspatial.com> wrote:

After a close read of the spec, I'm getting the impression that it was originally intended to follow with the Microsoft-style of UUIDs, as described on the Wikipedia page.

<image.png>

[Tom Jordan]

Thank you Evon and Lewis. But, returning to my original question: How would I set uidPart4 to 'MyProj01', such that lasinfo reports it as a string up to 8 characters, for instance "MyProj01"?

    [...]

    project ID GUID data 1-4:   F794F8A4-A23E-421E-MyProj01

    [...]

Thanks

Tom Jordan

[Evon Silvia]

I believe that lasinfo will always display the final 8 characters in their raw hexadecimal form. I was, however, able to get the 8 chars to read as "MyProj01" using other software with the following command:

lasinfo -i myfile.las -set_GUID F794F8A4-A23E-421E-794D-725031306A6F

...which results in:            2767754487-16034-07746-MyProj01 (BigEndian integers)

...or:                          4153735332-41534-16926-MyProj01 (LittleEndian integers)

Hope that helps.

Evon