point2dem / point2las: GdalIO: Unable to open EPSG support file gcs.csv error in ASP 3.0.0 (NOT 2.7.0!)

[M. Elias]

Hi @all,

I have been working with ASP for a few months now and was very happy about the new features in ASP 3.0.0 regarding the creation of point clouds or DEMs using 'point2las' and 'point2dem' 

(especially that I can now save the triangulation error per point in LAS point cloud generation

Unfortunately, I´m not able to run point2las or rather point2dem with the reference system "-t_srs" option since the upgrade from version 2.7.0 to 3.0.0. 

I get the (for GDAL users well known) error message:

"Error: GdalIO: Unable to open EPSG support file gcs.csv.  Try setting the GDAL_DATA environment variable to point to the directory containing EPSG csv files. (code = 4)

Failed to parse: "EPSG:4326"."

when calling the expression:

>> point2las merged_pc.tif --t_srs 'EPSG:4326'

Log:

Writing log info to: merged_pc-log-point2las-11-09-1206-5471.txt

Estimating the maximum valid triangulation error (outlier cutoff).

Error estim : [******************************************************] Complete!

Collected a sample of 357208 positive triangulation errors.

For this sample: min = 0.000976562, mean = 3.30303, stdev = 29.1566, max = 1307.14.

Error percentiles: Q1 (25%): 0.529297, Q2 (50%): 1.20312, Q3 (75%): 2.42676.

Using as outlier cutoff the 75 percentile times 3.

Found the maximum valid triangulation error (outlier cutoff): 7.28027.

Error: GdalIO: Unable to open EPSG support file gcs.csv.  Try setting the GDAL_DATA environment variable to point to the directory containing EPSG csv files. (code = 4)

Failed to parse: "EPSG:4326".

GDAL 2.4.4. is installed though and the environment variable GDAL_DATA is also set correctly.

However, if I call point2las or point2dem (the error occurs with both functions) calling the respective function from ASP 2.7.0, everything works perfectly:

>> point2las merged_pc.tif --t_srs 'EPSG:4326'

Log:

Writing log info to: merged_pc-log-point2las-11-09-1640-5937.txt

Using projection string: '+proj=longlat +datum=WGS84 +no_defs'

Statistics: [********************************************************] Complete!

Computing the point cloud bounding box.

        --> [********************************************************] Complete!

Writing LAS file: merged_pc.las

        --> [********************************************************] Complete!

How can I solve the problem? Many thanks in advance and all the best,
Mel.

[Alexandrov, Oleg (ARC-TI)[KBR Wyle Services, LLC]]

Thank you the report.

I just tried both point2las and point2dem with the ASP 3.0.0 official release, and they work. For example:

./StereoPipeline-3.0.0-2021-07-27-x86_64-Linux/bin/point2las --t_srs 'EPSG:4326' run/run-PC.tif

I also found the file that you get an error about, it is in ./StereoPipeline-3.0.0-2021-07-27-x86_64-Linux/share/gdal/gcs.csv. 

Which platform are you on? Mac maybe? Do you use the tarball or the conda build? 

Could you try fetching a brand new tarball, and try again? Can you check if your "share/gda" directory has gcs.csv?

Could something in your path or your environment stay in the way? ASP ships its own GDAL and sets its own library paths and data paths. It is unlikely something from your env would mess up with it, but it is a possibility.

---

From: ames-stereo-pi...@googlegroups.com <ames-stereo-pi...@googlegroups.com> on behalf of M. Elias <post.mela...@gmail.com>
Sent: Tuesday, November 9, 2021 8:10 AM
To: Ames Stereo Pipeline Support <ames-stereo-pi...@googlegroups.com>
Subject: [EXTERNAL] point2dem / point2las: GdalIO: Unable to open EPSG support file gcs.csv error in ASP 3.0.0 (NOT 2.7.0!)

 

--
You received this message because you are subscribed to the Google Groups "Ames Stereo Pipeline Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ames-stereo-pipeline...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ames-stereo-pipeline-support/cdf55d93-c1e6-4d95-a42d-8d78add1557an%40googlegroups.com.

[M. Elias]

Hi Oleg.

First of all, it's incredible how quickly you always get help from you here, even with the most unimpressive problems. That's great and a big thank you for that!

I run ASP with Ubuntu 20.04 installed in a virtual box (VMware workstation player 16). I installed ASP using the tarball option. The gcs.csv file is perfectly fine.

I recognized that I installed the nightly build and assumed that maybe something was wrong with this build. But also the installation of the release version from july 2021 did not bring success in solving the error.

Yes, I thought exactly the same that the reason could be a confict with any Conda environment. I have spent the last hours cleaning up all conda environments to rule out that no other gdal version can conflict (e.g. some gdal 3.x.x which, afaik, does not use anymore the gcs.csv file).
I also deactivated any Conda environment and cleaned up the bashrc and environment variables. Nothing worked. The error remains. 
I also tried to set ASPs gdal as GDAL_DATA environment variable... no success.

Finally I thought that maybe some old corrupt installation of gdal via pip or conda could be the problem and I set up a completely new clean virtual machine (xUbuntu 21.04). No conda, no pip, no gdal... everything is "pure" ;-)

I downloaded the tarball of the nightly build, unzipped  ASP and tried to run point2las with a sample point cloud... and got the same error when using the --t_srs option (without this option, point2las workes like a charm) !!! :-(

I am slowly running out of things to test.... do you have another idea what I can try to do?

Thank you once again and all the best!
Mel.

[Oleg Alexandrov]

Thank you for the very big effort to chase this down.

I was able to finally reproduce this. This is nasty. I spent more than an hour on it and with no luck. I summarize my findings here: https://github.com/NeoGeographyToolkit/StereoPipeline/issues/347

I will suggest for now you can avoid the EPSG option. That seems to point to the WGS84 datum, as I see here: https://epsg.io/4326.

You can try to use maybe instead --t_srs '+proj=longlat +datum=WGS84 +no_defs '. I am thinking it may have the same effect.

We don't have funding for ASP for now, and it looks that this won't be quick to solve. Could be an issue with GDAL's assumptions about what where the data should be, or we ship something incorrectly. Hopefully I will find time for this one day.

To view this discussion on the web visit https://groups.google.com/d/msgid/ames-stereo-pipeline-support/df2569a9-c7c0-4753-a6d0-4b4a720af0c2n%40googlegroups.com.

[M. Elias]

Hi Oleg,

thank you for your great efforts to reproduce this weird error! 

A few minutes before your message I also tried to use --t_srs with the proj4 stringkey instead of the EPSG code and indeed the ASP v3.0.0 tools seem to work perfectly fine!

I will add a small workaround so that I derive a proj4 key from a before computed EPSG code with respect to the AOI (as done by Shashank in his great skysat_stereo workflow) 

and use this one with the --t_srs option.

Nevertheless, thank you for opening a ticket and for the idea with the proj4 stringkey! 

I fully understand that fixing the EPSG problem will be time-consuming and cannot be implemented immediately without appropriate funding.

I think, creating a the proj4 stringkey and using this one instead of the EPSG code is a rather simple workaround and may also help other users for the moment (in case somebody run into the gdal problem too).

Best,

Mel.

[Alexandrov, Oleg (ARC-TI)[KBR Wyle Services, LLC]]

I actually found the problem, it was a bug I made, and I describe at at the issue I opened. It should be fixed in tomorrow's build.

---

From: ames-stereo-pi...@googlegroups.com <ames-stereo-pi...@googlegroups.com> on behalf of M. Elias <post.mela...@gmail.com>

Sent: Wednesday, November 10, 2021 12:03 PM

To: Ames Stereo Pipeline Support <ames-stereo-pi...@googlegroups.com>

Subject: Re: [EXTERNAL] point2dem / point2las: GdalIO: Unable to open EPSG support file gcs.csv error in ASP 3.0.0 (NOT 2.7.0!)

 

To view this discussion on the web visit https://groups.google.com/d/msgid/ames-stereo-pipeline-support/e9237a3f-ec04-43b4-8b1c-90691c19b870n%40googlegroups.com.