lasvalidate.exe - the (in-)official LAS/LAZ validator from rapidlasso
821 views
Skip to first unread message
Martin Isenburg
Sep 10, 2013, 5:17:56 PM9/10/13
to LAStools - efficient command line tools for LIDAR processing, las...@googlegroups.com
Hello folks,
as promised ... here is lasvalidate.exe ... which will become the
(in-)official LAS/LAZ validator from rapidlasso and become an integral
part of LAStools with the next release scheduled for later this week.
It includes a GEOTIFF tag check for all the geo-encodings currently
supported by LAStools (e.g. UTM, state planes, generic transform
mercator, generic lambertian conic conformal, and longlat) with more
being added as your complaints stream in. It also supports reading the
compressed LAZ format now. If you are pre-LAStools-release curious ...
the direct link to the current executable is here (click "view raw" to
download)
https://github.com/LASvalidator/lasvalidate/blob/master/bin/lasvalidate.exe
Or get the complete package here ... the LINUX makefiles should work now.
git clone https://github.com/LASvalidator/LASread.git
cd LASread
make
cd ..
git clone https://github.com/LASvalidator/lasvalidate.git
cd lasvalidate
make
cd bin
lasvalidate -i your_file.las
Cheers,
Martin @rapidlasso
On Tue, Sep 3, 2013 at 7:17 PM, Martin Isenburg
<martin....@gmail.com> wrote:
> Hello folks,
>
> a few of you have inquired over the past weeks whatever happened to my
> bid for the ASPRS LAS Validation Suite. It's been ... aehhhm ...
> funny. I received a serious and professional looking letter marked
> "business confidential" that said that I was selected to do the work
> ... but only on the precondition that I sign a document that more or
> less said that I would only be able to talk about the development via
> an official ASPRS channel. Why was that funny? Well, if you read my
> bid then you know that at its very core was the intention to do the
> entire development in an open and transparent manner that would invite
> early and frequent community feedback. So it is still a mystery to me
> why my bid was selected in the first place. Zähneknirschend (fig.
> German) I decided to sign the "silencer" ... I was worried that the
> alternative might be worse. The next funny thing was that the ASPRS
> LAS committee members did not seem informed about all this. Instead a
> "secret" technical review panel was deciding things. But the funniest
> thing was that a couple of weeks after me signing the dreaded
> "silencer" I was told that the ASPRS was canning whole project.
> Shortly thereafter the ASPRS Executive Director announced his
> retirement and I personally like to think - just because it makes it
> even funnier - that this was somehow related to the LAS Validation
> Suite contract fiasko ... (-;
>
> Okay. Enough funny. Now serious. I have sunk many hours into this
> project and would like to complete its development in open souce. The
> main thing that is lacking for completion is the CRSscan library for
> the GeoTIFF versus OCG WKT validation. I was thinking just calling the
> right functions of GTIFF and GDAL could do this. I looked through
> Frank Wanderdam's code and it seems fairly straight forward. To keep
> the tool nice and small - like the original ASPRS LVS tender was
> calling for - it would be nice to "rip" only those parts out of GTIFF
> and GDAL that are needed to read the Geokey projection tags and the
> OCG WKT string. We would then initialize a projection once from the
> Geokey tags and once from the OGC WKT string and make sure they are
> the same. For the time being I'd be happy if we support 98% (?!?) of
> all cases by focusing on the most common projections such as UTM,
> generic TM, generic LCC, generic OM, stateplanes, and latlong.
>
> I just updated the github repository with all the necessary source
> code to compile the current version of lasvalidate.exe (the LINUX
> Makefile are untested and may still have a few errors).
>
> http://github.com/LASvalidator
>
> Regards,
>
> Martin @rapidlasso
>
> PS: Would be great to have some agency / company / institute funding the effort.
>
> --
> http://rapidlasso.com - fast tools for all you LiDARs
>
> On Tue, May 7, 2013 at 9:34 PM, Martin Isenburg
> <martin....@gmail.com> wrote:
>> Hello,
>>
>> to anyone who has followed what's been happening in these two
>> discussion groups it will probably not come as a surprise to hear that
>> rapidlasso GmbH has submitted a bid to the ASPRS call for proposals
>> for implementing the LAS Validation Suite (LVS). The fundamental
>> premise of our bid is to implement the LAS validation tool through an
>> open process that invites early feedback.
>>
>> Providing an "official correctness check" for LAS files and especially
>> that of the Coordinate Reference System is likely to spark emotional
>> discussions. We doubt that any one party will - on their own and
>> behind closed doors - be able to develop a comprehensive LAS
>> validation software that will find acceptance by an already "slightly
>> suspicious" LiDAR community. We believe an open approach will disarm
>> any notions that such a software could be used as a "lock-out"
>> mechanism, will make the validation tool quickly available for
>> professional use, and will find result in a acceptance by the LiDAR
>> community.
>>
>> To make our bid as strong as possible we have completed a partial
>> prototype of the LAS Validation Suite. With this we do not only want
>> to demonstrate our technical capability but also illustrates the way
>> we plan to inject transparency into the development process. The tool
>> is using the open source LASlibrary API which was designed
>> particularly for this bid. The LASlibrary package is an unrestricted
>> open source (LGPL 2.1) API to read LAS (optionally also LAZ) files. It
>> is a completely re-engineered subset of the LASlib API of LAStools
>> that has been drastically simplified to better suit the
>> being-light-weight-requirement of the ASPRS' request for proposals.
>>
>> You can find this prototype tool, example input and output, our bid,
>> the call for proposals, and a few other goodies here:
>>
>> http://github.com/LASvalidator
>>
>> I invite you to run the 'lasvalidate.exe' prototype tool on your own
>> data, report bugs and other findings, suggest improvements to the XML
>> output, and else ...
>>
>> prototype validation tool:
>> https://github.com/LASvalidator/lasvalidate/tree/master/bin
>>
>> example data:
>> https://github.com/LASvalidator/lasvalidate/tree/master/data
>>
>> collection unit tests (in progress):
>> https://github.com/LASvalidator/lasvalidate/tree/master/unit
>>
>> No CRS handling is implemented yet. This is likely to be the most
>> tricky part. Using the "gold standard" by combining GeoTIFF with GDAL
>> and proj4 seems the way to go but may require some code juggling if we
>> want to minimize the amount of code dependencies on other packages.
>>
>> Regards,
>>
>> Martin @rapidlasso
>>
>> PS: Please only post replies in "The LAS room" user forum since this
>> is not about LAStools but about the LAS format.
as promised ... here is lasvalidate.exe ... which will become the
(in-)official LAS/LAZ validator from rapidlasso and become an integral
part of LAStools with the next release scheduled for later this week.
It includes a GEOTIFF tag check for all the geo-encodings currently
supported by LAStools (e.g. UTM, state planes, generic transform
mercator, generic lambertian conic conformal, and longlat) with more
being added as your complaints stream in. It also supports reading the
compressed LAZ format now. If you are pre-LAStools-release curious ...
the direct link to the current executable is here (click "view raw" to
download)
https://github.com/LASvalidator/lasvalidate/blob/master/bin/lasvalidate.exe
Or get the complete package here ... the LINUX makefiles should work now.
git clone https://github.com/LASvalidator/LASread.git
cd LASread
make
cd ..
git clone https://github.com/LASvalidator/lasvalidate.git
cd lasvalidate
make
cd bin
lasvalidate -i your_file.las
Cheers,
Martin @rapidlasso
On Tue, Sep 3, 2013 at 7:17 PM, Martin Isenburg
<martin....@gmail.com> wrote:
> Hello folks,
>
> a few of you have inquired over the past weeks whatever happened to my
> bid for the ASPRS LAS Validation Suite. It's been ... aehhhm ...
> funny. I received a serious and professional looking letter marked
> "business confidential" that said that I was selected to do the work
> ... but only on the precondition that I sign a document that more or
> less said that I would only be able to talk about the development via
> an official ASPRS channel. Why was that funny? Well, if you read my
> bid then you know that at its very core was the intention to do the
> entire development in an open and transparent manner that would invite
> early and frequent community feedback. So it is still a mystery to me
> why my bid was selected in the first place. Zähneknirschend (fig.
> German) I decided to sign the "silencer" ... I was worried that the
> alternative might be worse. The next funny thing was that the ASPRS
> LAS committee members did not seem informed about all this. Instead a
> "secret" technical review panel was deciding things. But the funniest
> thing was that a couple of weeks after me signing the dreaded
> "silencer" I was told that the ASPRS was canning whole project.
> Shortly thereafter the ASPRS Executive Director announced his
> retirement and I personally like to think - just because it makes it
> even funnier - that this was somehow related to the LAS Validation
> Suite contract fiasko ... (-;
>
> Okay. Enough funny. Now serious. I have sunk many hours into this
> project and would like to complete its development in open souce. The
> main thing that is lacking for completion is the CRSscan library for
> the GeoTIFF versus OCG WKT validation. I was thinking just calling the
> right functions of GTIFF and GDAL could do this. I looked through
> Frank Wanderdam's code and it seems fairly straight forward. To keep
> the tool nice and small - like the original ASPRS LVS tender was
> calling for - it would be nice to "rip" only those parts out of GTIFF
> and GDAL that are needed to read the Geokey projection tags and the
> OCG WKT string. We would then initialize a projection once from the
> Geokey tags and once from the OGC WKT string and make sure they are
> the same. For the time being I'd be happy if we support 98% (?!?) of
> all cases by focusing on the most common projections such as UTM,
> generic TM, generic LCC, generic OM, stateplanes, and latlong.
>
> I just updated the github repository with all the necessary source
> code to compile the current version of lasvalidate.exe (the LINUX
> Makefile are untested and may still have a few errors).
>
> http://github.com/LASvalidator
>
> Regards,
>
> Martin @rapidlasso
>
> PS: Would be great to have some agency / company / institute funding the effort.
>
> --
> http://rapidlasso.com - fast tools for all you LiDARs
>
> On Tue, May 7, 2013 at 9:34 PM, Martin Isenburg
> <martin....@gmail.com> wrote:
>> Hello,
>>
>> to anyone who has followed what's been happening in these two
>> discussion groups it will probably not come as a surprise to hear that
>> rapidlasso GmbH has submitted a bid to the ASPRS call for proposals
>> for implementing the LAS Validation Suite (LVS). The fundamental
>> premise of our bid is to implement the LAS validation tool through an
>> open process that invites early feedback.
>>
>> Providing an "official correctness check" for LAS files and especially
>> that of the Coordinate Reference System is likely to spark emotional
>> discussions. We doubt that any one party will - on their own and
>> behind closed doors - be able to develop a comprehensive LAS
>> validation software that will find acceptance by an already "slightly
>> suspicious" LiDAR community. We believe an open approach will disarm
>> any notions that such a software could be used as a "lock-out"
>> mechanism, will make the validation tool quickly available for
>> professional use, and will find result in a acceptance by the LiDAR
>> community.
>>
>> To make our bid as strong as possible we have completed a partial
>> prototype of the LAS Validation Suite. With this we do not only want
>> to demonstrate our technical capability but also illustrates the way
>> we plan to inject transparency into the development process. The tool
>> is using the open source LASlibrary API which was designed
>> particularly for this bid. The LASlibrary package is an unrestricted
>> open source (LGPL 2.1) API to read LAS (optionally also LAZ) files. It
>> is a completely re-engineered subset of the LASlib API of LAStools
>> that has been drastically simplified to better suit the
>> being-light-weight-requirement of the ASPRS' request for proposals.
>>
>> You can find this prototype tool, example input and output, our bid,
>> the call for proposals, and a few other goodies here:
>>
>> http://github.com/LASvalidator
>>
>> I invite you to run the 'lasvalidate.exe' prototype tool on your own
>> data, report bugs and other findings, suggest improvements to the XML
>> output, and else ...
>>
>> prototype validation tool:
>> https://github.com/LASvalidator/lasvalidate/tree/master/bin
>>
>> example data:
>> https://github.com/LASvalidator/lasvalidate/tree/master/data
>>
>> collection unit tests (in progress):
>> https://github.com/LASvalidator/lasvalidate/tree/master/unit
>>
>> No CRS handling is implemented yet. This is likely to be the most
>> tricky part. Using the "gold standard" by combining GeoTIFF with GDAL
>> and proj4 seems the way to go but may require some code juggling if we
>> want to minimize the amount of code dependencies on other packages.
>>
>> Regards,
>>
>> Martin @rapidlasso
>>
>> PS: Please only post replies in "The LAS room" user forum since this
>> is not about LAStools but about the LAS format.
Martin Isenburg
Sep 26, 2013, 11:50:39 AM9/26/13
to LAStools - efficient command line tools for LIDAR processing, las...@googlegroups.com
Hello,
the LAS validator has been updated based on your comments and now also
checks for unpopulated intensity, scan angle rank, and point source ID
fields. For the latter it only warns if both the file source ID is
zero and all point source ID are zero. The direct link to
lasvalidate.exe is here:
https://github.com/LASvalidator/lasvalidate/blob/master/bin/lasvalidate.exe
(click on "view raw")
https://github.com/LASvalidator/lasvalidate/blob/master/bin/lasvalidate_README.txt
Simply run
lasvalidate -i *.las -i *.laz -o my_report.xml
and let me know how lasvalidate performs on your LAS / LAZ files.
Regards,
Martin @rapidlasso
On Tue, Sep 10, 2013 at 11:17 PM, Martin Isenburg
the LAS validator has been updated based on your comments and now also
checks for unpopulated intensity, scan angle rank, and point source ID
fields. For the latter it only warns if both the file source ID is
zero and all point source ID are zero. The direct link to
lasvalidate.exe is here:
https://github.com/LASvalidator/lasvalidate/blob/master/bin/lasvalidate.exe
(click on "view raw")
https://github.com/LASvalidator/lasvalidate/blob/master/bin/lasvalidate_README.txt
Simply run
lasvalidate -i *.las -i *.laz -o my_report.xml
and let me know how lasvalidate performs on your LAS / LAZ files.
Regards,
Martin @rapidlasso
On Tue, Sep 10, 2013 at 11:17 PM, Martin Isenburg
Martin Isenburg
Oct 10, 2013, 11:22:54 AM10/10/13
to las...@googlegroups.com, LAStools - efficient command line tools for LIDAR processing
Hello,
seems like the LAS validator is rapidly gaining acceptance as the bug reports come in from data producing and providing heavy-weights. One bug report just came in from RIEGL because the last version had an incorrect warning for all scan angles being zero for all new LAS 1.4 point types (e.g. point data format > 6). That was fixed now. However, the RIEGL exports are still failing the test because the CRS information is not populated correctly. After a few words with their engineers at INTERGEO I had the impression that the LAS validator will speed them up a little bit in addressing this in future releases because they surely did not seem to like this "fail" in their report card ...
The lasvalidator is now also available via the QGIS and the ArcGIS toolboxes of LAStools.
Regards,
Martin @rapidlasso
--
http://rapidlasso.com - fast tools to validate your LiDARs
On Thursday, September 26, 2013 5:50:39 PM UTC+2, Martin Isenburg wrote:
Hello,
the LAS validator has been updated based on your comments and now also
checks for unpopulated intensity, scan angle rank, and point source ID
fields. For the latter it only warns if both the file source ID is
zero and all point source ID are zero. The direct link to
lasvalidate.exe is here:
https://github.com/LASvalidator/lasvalidate/blob/master/bin/lasvalidate.exe
(click on "view raw")
https://github.com/LASvalidator/lasvalidate/blob/master/bin/lasvalidate_README.txt
Simply run
lasvalidate -i *.las -i *.laz -o my_report.xml
and let me know how lasvalidate performs on your LAS / LAZ files.
Regards,
Martin @rapidlasso
Loren
Oct 23, 2013, 4:46:50 PM10/23/13
to last...@googlegroups.com, las...@googlegroups.com
Hi Martin,
When I create a LAS file with EPSG code 26910 which is "NAD83 / UTM zone 10N" my file does not fail. However if I use EPSG code 3157 (NAD83(CSRS) / UTM zone 10N) I get a fail with "the 4 geokeys do not properly specify a Coordinate Reference System" as well as a command line (stdout) error "ProjectedCSTypeGeoKey: look-up for 3157 not implemented".
PLEASE do not fail EPSG codes based on your software's limited EPSG look-up. There are many non-GIS, non-technical or simply ignorant LAS consumers out there that will take your softwares "fails" as valid and revoke valid LAS deliveries based on lasvalidate's output. This can and will cost companys that produce LAS files time and money.
While you are at it, please consider accepting the code 32767 which has long been used in geotiff tags for "user-defined" or custom projections. We frequently produce data in "local" projections that have no EPSG code (and never will).
Martin Isenburg
Oct 23, 2013, 10:07:44 PM10/23/13
to LAStools - efficient command line tools for LIDAR processing, las...@googlegroups.com
Hello Loren,
than you for your feedback. This is very useful to improve the
LASvalidator. As clearly stated in every run of the software, please
contact me if the validation reports are questionable. See the typical
LASvalidator output below:
==========================
D:\lastools\bin>lasvalidate -i ..\data\fusa.laz
This is version 131015 of the LAS validator. Please contact
me at 'martin....@rapidlasso.com' if you disagree with
validation reports, want additional checks, or find bugs.
needed 0.33 sec for 'fusa.laz'
D:\lastools\bin>lasvalidate -version
This is version 131015 of the LAS validator. Please contact
me at 'martin....@rapidlasso.com' if you disagree with
validation reports, want additional checks, or find bugs.
lasvalidate 131015 with LASread (v 1.0) and LAScheck (v 0.0) by rapidlasso GmbH
==========================
One of the main weak spots of the LASvalidator is the incomplete CRS
checking. Feedback such as yours is motivating to improve the
LASvalidator to the point where it can be useful for everyone. I
certainly understand your concern where a customer gives "too much
credit" to my current LASvalidator prototype. I should make it more
obvious that the LASvalidator is still in development. Also I will try
to make the CRS checking more complete as my time allows. I did, in
fact, sit down with Frank Warmerdam (creator of GDAL & GeoTIFF) on
September 20th at FOSS4G to discuss how to best complete the handling
of all valid ESPG codes but have been too busy since to turn these
ideas into code.
In the past months I had asked several of my contacts, including
several big federal agencies, if they were willing to pick up the
slack of the ASPRS and help to fund the LASvalidator. As you may
recall, the LASvalidator is a volunteer effort that was released into
the open source after the ASPRS decided suddenly to cancel the
contract negotiations about funding the creation of an "official" LAS
validation suite that I was bidding for. So I encourage you to
complain to someone at the ASPRS for neither having provided a
reference implementation nor a validator for their LAS format ... (-;
Folks ... here again my request: please try out lasvalidate.exe on
your data and report any disagreements you find. I cannot complete the
validation checks without your help. It can only be done through a
community effort to find all the "wrong" fails as I do not have the
resources to perfect the validator on my own.
Regards,
Martin @rapidlasso
PS: I am happy to hear folks are starting to use the lasvalidator
already. I head from NOAA that they are planning to run it across
their 1.5 trillion LiDAR point holdings ... that should help to iron
out the remaining situations were lasvalidate.exe miss-behaves ... .
> --
> Download LAStools at
> http://lastools.org
> http://rapidlasso.com
> Be social with LAStools at
> http://facebook.com/LAStools
> http://twitter.com/LAStools
> http://linkedin.com/groups/LAStools-4408378
> Manage your settings at
> http://groups.google.com/group/lastools/subscribe
than you for your feedback. This is very useful to improve the
LASvalidator. As clearly stated in every run of the software, please
contact me if the validation reports are questionable. See the typical
LASvalidator output below:
==========================
D:\lastools\bin>lasvalidate -i ..\data\fusa.laz
This is version 131015 of the LAS validator. Please contact
me at 'martin....@rapidlasso.com' if you disagree with
validation reports, want additional checks, or find bugs.
needed 0.33 sec for 'fusa.laz'
D:\lastools\bin>lasvalidate -version
This is version 131015 of the LAS validator. Please contact
me at 'martin....@rapidlasso.com' if you disagree with
validation reports, want additional checks, or find bugs.
lasvalidate 131015 with LASread (v 1.0) and LAScheck (v 0.0) by rapidlasso GmbH
==========================
One of the main weak spots of the LASvalidator is the incomplete CRS
checking. Feedback such as yours is motivating to improve the
LASvalidator to the point where it can be useful for everyone. I
certainly understand your concern where a customer gives "too much
credit" to my current LASvalidator prototype. I should make it more
obvious that the LASvalidator is still in development. Also I will try
to make the CRS checking more complete as my time allows. I did, in
fact, sit down with Frank Warmerdam (creator of GDAL & GeoTIFF) on
September 20th at FOSS4G to discuss how to best complete the handling
of all valid ESPG codes but have been too busy since to turn these
ideas into code.
In the past months I had asked several of my contacts, including
several big federal agencies, if they were willing to pick up the
slack of the ASPRS and help to fund the LASvalidator. As you may
recall, the LASvalidator is a volunteer effort that was released into
the open source after the ASPRS decided suddenly to cancel the
contract negotiations about funding the creation of an "official" LAS
validation suite that I was bidding for. So I encourage you to
complain to someone at the ASPRS for neither having provided a
reference implementation nor a validator for their LAS format ... (-;
Folks ... here again my request: please try out lasvalidate.exe on
your data and report any disagreements you find. I cannot complete the
validation checks without your help. It can only be done through a
community effort to find all the "wrong" fails as I do not have the
resources to perfect the validator on my own.
Regards,
Martin @rapidlasso
PS: I am happy to hear folks are starting to use the lasvalidator
already. I head from NOAA that they are planning to run it across
their 1.5 trillion LiDAR point holdings ... that should help to iron
out the remaining situations were lasvalidate.exe miss-behaves ... .
> Download LAStools at
> http://lastools.org
> http://rapidlasso.com
> Be social with LAStools at
> http://facebook.com/LAStools
> http://twitter.com/LAStools
> http://linkedin.com/groups/LAStools-4408378
> Manage your settings at
> http://groups.google.com/group/lastools/subscribe
Martin Isenburg
Oct 25, 2013, 4:55:15 AM10/25/13
to las...@googlegroups.com, LAStools - efficient command line tools for LIDAR processing, Loren Dawe
Hello Loren (and all),
I have added the ESPG codes for the regions 3154 to 3160 which includes the one in question but cannot test it myself. Please try (or have your customer try) if the latest lasvalidate.exe now behaves as expected.
You can make sure to have the latest version by running
========================
lasvalidate -version
This is version 131025 of the LAS validator. Please contact
me at 'martin....@rapidlasso.com' if you disagree with
validation reports, want additional checks, or find bugs as
the software is still under development. Your feedback will
help to finish it sooner.
lasvalidate 131025 with LASread (v 1.0) and LAScheck (v 0.0) by rapidlasso GmbH
========================
Is anyone aware of a complete listing of all valid ESPG codes? The closest I have found so far is this speaksheet but is did not contain ESPG 3154 to 3160.
Of course I could manually step through all possibilities via this awesome page here but there got to be a complete listing somewhere ...?
Martin
On Thursday, October 24, 2013 4:07:44 AM UTC+2, Martin Isenburg wrote:
Hello Loren,
than you for your feedback. This is very useful to improve the
LASvalidator. As clearly stated in every run of the software, please
contact me if the validation reports are questionable. See the typical
LASvalidator output below:
==========================
D:\lastools\bin>lasvalidate -i ..\data\fusa.laz
This is version 131015 of the LAS validator. Please contact
me at 'martin.isenburg@rapidlasso.com' if you disagree with
validation reports, want additional checks, or find bugs.
needed 0.33 sec for 'fusa.laz'
D:\lastools\bin>lasvalidate -version
This is version 131015 of the LAS validator. Please contact
me at 'martin.isenburg@rapidlasso.com' if you disagree with
Loren
Oct 25, 2013, 1:10:19 PM10/25/13
to last...@googlegroups.com, las...@googlegroups.com, Loren Dawe
Hi Martin,
Awesome quick response! My UTM(CSRS) codes now do not fail.
As for the EPSG codes, I have always used spatialreference.org for my source. That said, it's probably not totally complete and/or always up to date. I dont believe a current list exists, if it does the folks at epsg.org may know or supply it some how.
However, I think that you may be jumping into the deep end attempting to add complete CRS validation to LASvalidate. I admire your attempt but perhaps you should take a more generic approach such as recognizing the standard/common projections like you do now and reporting errors/failing only for gross "LAS" formatting errors, not unrecognized projections.
I suggest you change your "fail: the 4 geokeys do not properly specify a Coordinate Reference System" to "warning: geokeys specify an uncommon or non-standard projection". This would be expected by those using an uncommon projection or their own local coordinate system and a red flag for those expecting that their LAS files are indeed in a standard projection.
Remember that the LAS spec does not even specify that the EPSG codes must be used or even which geokeys are required, simply that the GeoTiff specification must be used. Additionally with 1.4, geoTiff tags are replaced with WKT, another format in which valid projections can be specified in many different ways.
IMHO, Attempting to validate the numerous variations of geoTiff and WKT should probably be left to it's own open source project with partners such as Frank Warmerdam, ESRI, GRASS and other GIS interests.
Loren.
Awesome quick response! My UTM(CSRS) codes now do not fail.
As for the EPSG codes, I have always used spatialreference.org for my source. That said, it's probably not totally complete and/or always up to date. I dont believe a current list exists, if it does the folks at epsg.org may know or supply it some how.
However, I think that you may be jumping into the deep end attempting to add complete CRS validation to LASvalidate. I admire your attempt but perhaps you should take a more generic approach such as recognizing the standard/common projections like you do now and reporting errors/failing only for gross "LAS" formatting errors, not unrecognized projections.
I suggest you change your "fail: the 4 geokeys do not properly specify a Coordinate Reference System" to "warning: geokeys specify an uncommon or non-standard projection". This would be expected by those using an uncommon projection or their own local coordinate system and a red flag for those expecting that their LAS files are indeed in a standard projection.
Remember that the LAS spec does not even specify that the EPSG codes must be used or even which geokeys are required, simply that the GeoTiff specification must be used. Additionally with 1.4, geoTiff tags are replaced with WKT, another format in which valid projections can be specified in many different ways.
IMHO, Attempting to validate the numerous variations of geoTiff and WKT should probably be left to it's own open source project with partners such as Frank Warmerdam, ESRI, GRASS and other GIS interests.
Loren.
Evon Silvia
Oct 25, 2013, 11:46:55 AM10/25/13
to last...@googlegroups.com, las...@googlegroups.com
It is possible for one to register at http://www.epsg-registry.org/ and download the entire EPSG dataset in an XML format called GML. I have found quite a few EPSG codes on there that aren't officially listed in the GeoTIFF spec. Unfortunately, some popular LiDAR software packages choke and crash when they encounter more obscure EPSG codes, such as UTM projections on NAD83(HARN)—causing some clients to reject certain configurations—so my use of them has been fairly limited in scope.
Note that that site's JavaScript has some issues in internet explorer, so Chrome or Firefox work a little better.
Evon
--
Evon Silvia Geomatics Specialist
WSI Corvallis, OR WSI Portland, OR WSI Oakland, CA
517 SW 2nd St., Suite 400, Corvallis, OR 97333
mborish
Feb 4, 2014, 12:33:19 PM2/4/14
to last...@googlegroups.com, las...@googlegroups.com, esi...@watershedsciences.com
We have encountered a situation of suspected file corruptions which manifest as just a few points per file with classes well beyond those existing in out Terrascan .ptc (point class) file. For instance, after running a combination of automated and manual processes on several thousand files for which we are trying to end up with only building, ground, default, and high vegetation points, we somehow end up with a small number of files that have a few points with seemingly random point class values that we never use such as 184, 210, etc. We have had success using lasvalidate to identify other types of copy errors with points outside the bounding box, points with zero returns, etc., but the files with the strange point class values produce no warnings or failures. Any ideas?
Additionally, it would be great if there was a way to select which types of failures lasvalidate identifies. In relation to some of the other comments in this conversation, having the option to turn off CRS checking would be very useful.
Thanks!
Evon Silvia
Feb 4, 2014, 12:48:59 PM2/4/14
to mborish, last...@googlegroups.com
Assuming that your files are LAS 1.2 or 1.3, point class should never exceed a value of 31 (5bits), as the last three bits of the 8-bit classification field are reserved by the specification as flags to mark the point as "Synthetic, Key-Point, and Withheld." If your software strictly followed the specification, then a value of 210 would represent a point with class 18 that is flagged as a withheld key-point. Similarly, a value of 184 would have class 24 and be flagged as a withheld, synthetic point.
Now, I know of almost no software packages that use those three bits for their intended use - most vendors designate certain point classes to represent keypoints and withheld points - and I also know of several software packages that violate the LAS 1.0-1.3 specifications to extend the available class numbers to the full 8bit range. Frankly, I would consider this behavior worthy of a warning in lasvalidate.
All that said, you probably just have file corruption from copy errors. This happens most frequently (for us) when copying LAS files to USB drives on Windows machines. We always copy to USB drives using the TeraCopy software to verify that no copy errors occurred.
Evon
--
Evon Silvia Geomatics Specialist
WSI Corvallis, OR WSI Portland, OR WSI Oakland, CA
517 SW 2nd St., Suite 400, Corvallis, OR 97333
Martin Isenburg
Feb 8, 2014, 12:05:20 PM2/8/14
to LAStools - efficient command line tools for LIDAR processing, The LAS room - a friendly place to discuss specifications of the LAS format, mborish
Hello,
I added a new WARNING to lasvalidate that detects "resolution fluff"
in the coordinate values that i regularly come across. In the most
common case it means that the LAS file uses scaling factors 0.001
0.001 0.001 but has only centimeter resolution (e.g. all stored
coordinate values are multiples of 10). Below an example. There is no
verdict yet for WARNING about unusual 'classifications' (see quoted
discussion at end of message) that may really be result of correctly
set 'withheld', 'synthetic', or 'keypoint' flags. I am not sure how to
add WARNINGS that won't punish those who actually inerpret (and use)
the LAS specification correctly ...
D:\lastools\bin>lasvalidate -i 97750_183750.laz
This is version 140207 of the LAS validator. Please contact
The new warning is
<warning>
<variable>coordinate values</variable>
<note>resolution fluff (x10) in XYZ </note>
</warning>
The full validation report in XML format is attached.
Regards,
Martin @rapidlasso
>>> NAD83(HARN)--causing some clients to reject certain configurations--so my use
I added a new WARNING to lasvalidate that detects "resolution fluff"
in the coordinate values that i regularly come across. In the most
common case it means that the LAS file uses scaling factors 0.001
0.001 0.001 but has only centimeter resolution (e.g. all stored
coordinate values are multiples of 10). Below an example. There is no
verdict yet for WARNING about unusual 'classifications' (see quoted
discussion at end of message) that may really be result of correctly
set 'withheld', 'synthetic', or 'keypoint' flags. I am not sure how to
add WARNINGS that won't punish those who actually inerpret (and use)
the LAS specification correctly ...
D:\lastools\bin>lasvalidate -i 97750_183750.laz
This is version 140207 of the LAS validator. Please contact
me at 'martin....@rapidlasso.com' if you disagree with
validation reports, want additional checks, or find bugs as
the software is still under development. Your feedback will
help to finish it sooner.
needed 2.54 sec for '97750_183750.laz'
the software is still under development. Your feedback will
help to finish it sooner.
The new warning is
<warning>
<variable>coordinate values</variable>
<note>resolution fluff (x10) in XYZ </note>
</warning>
The full validation report in XML format is attached.
Regards,
Martin @rapidlasso
Reply all
Reply to author
Forward
0 new messages