Re: arcsi.py undefined symbol error

Pete Bunting [pfb]

unread,

Jul 3, 2014, 5:06:23 PM7/3/14

to Dan Slayback, RSGISLib

Hi Dan,

Thanks for your email, I have accepted you to the google group. The conda build is relatively new and we haven’t widely tested but it was built on Centos 6.5 64bit and I’ve tried in on machines here in Aber with Debian derived Ubuntu, Minted and Trisquel and it seems to be working OK.

Can you confirm which packages you have installed, these are the packages I have installed in my test environment. Do you have the same? Do you have any other builds of gdal on your system?

Could you also try one of the gdal commands to see whether it is just a problem with the python binding? Try gdalinfo or gdal_translate.

Finally, did you set up the GDAL_DRIVER_PATH and GDAL_DATA variables as Dan Clewley detailed on his spectral differenced blog (http://spectraldifferences.wordpress.com/2014/05/27/arcsi/).

I am still testing the functionality but ARCSI also has functions to estimate AOT from the image data, either using dense dark veg targets and a relationship between the SWIR and blue as LEDAPS does or through a dark object subtraction (DOS).

The FMask algorithm for cloud masking is also implemented into the system, select the CLOUDS product.

Best wishes,

Pete

(test2)[pete@localhost ~]$ conda list

# packages in environment at /home/pete/miniconda3/envs/test2:

#

arcsi 0.5 py34_0

boost 1.55.0 0

cgal 4.4 0

cmake 2.8.10.2 0

dateutil 2.1 py34_2

fftw 3.3.4 0

freetype 2.4.10 0

gdal 1.11.0 np18py34_0

geos 3.4.2 0

gmp 5.1.2 2

gsl 1.16 0

hdf5 1.8.13 0

kealib 1.4.1 0

libpng 1.5.13 1

matplotlib 1.3.1 np18py34_1

mpfr 3.1.2 0

muparser 2.2.3 0

numpy 1.8.1 py34_0

openssl 1.0.1h 0

pandas 0.14.0 np18py34_0

proj4 4.8.0 0

py6s 1.5.0 np18py34_0

pyparsing 2.0.1 py34_0

pyqt 4.10.4 py34_0

pysolar 0.4.3 py34_0

python 3.4.1 0

pytz 2014.4 py34_0

qt 4.8.5 0

readline 6.2 2

rios 1.3.0 np18py34_0

rsgislib 2.2.867 np18py34_0

scipy 0.14.0 np18py34_0

setuptools 3.6 py34_0

sip 4.15.5 py34_0

six 1.7.3 py34_0

sixs 1.1 0

sqlite 3.8.4.1 0

system 5.8 1

tk 8.5.15 0

tuiview 1.1.0 np18py34_0

xerces-c 3.1.1 0

zlib 1.2.7 0

****************************************************
* Dr Pete Bunting
* Senior Lecturer in Remote Sensing
* Earth Observation and Ecosystem Dynamics Group
* Department of Geography and Earth Sciences
* Aberystwyth University
* Aberystwyth
* Ceredigion
* SY23 3DB
* UK
*
* Ph: +44 (0) 1970 622615
* Mob: +44 (0) 7917 842743
* Email: p...@aber.ac.uk
* ORCID: http://orcid.org/0000-0002-7435-0148
****************************************************

On 3 Jul 2014, at 21:28, Dan Slayback <dan.sl...@nasa.gov> wrote:

Hi Pete,

I ran across your arcsi / wordpress post recently, so thought I'd give it a try as I am looking to automate some processing code, and apply atmospheric correction to landsat. This looks like very useful software - thanks for putting it out there. I joined the google group, but membership is pending so thought I'd email you directly.

So i'm at square 1 (or 2 perhaps) - I've installed everything without complaint (as far as I can see) per the instructions in the wordpress post, but I get an undefined symbol error in libgdal.so.1 when I try to run anything. This is on a CentOS 6.5/x86-64 system. For example:

$ tuiview
Traceback (most recent call last):
File "/data/Tools/miniconda3/bin/tuiview", line 19, in <module>
   from tuiview import viewerapplication
File "/data/Tools/miniconda3/lib/python3.4/site-packages/tuiview/viewerapplication.py", line 26, in <module>
   from . import geolinkedviewers
File "/data/Tools/miniconda3/lib/python3.4/site-packages/tuiview/geolinkedviewers.py", line 26, in <module>
   from . import viewerwindow
File "/data/Tools/miniconda3/lib/python3.4/site-packages/tuiview/viewerwindow.py", line 36, in <module>
   from . import viewerwidget
File "/data/Tools/miniconda3/lib/python3.4/site-packages/tuiview/viewerwidget.py", line 30, in <module>
   from . import viewerlayers
File "/data/Tools/miniconda3/lib/python3.4/site-packages/tuiview/viewerlayers.py", line 25, in <module>
   from osgeo import gdal
File "/data/Tools/miniconda3/lib/python3.4/site-packages/osgeo/__init__.py", line 21, in <module>
   _gdal = swig_import_helper()
File "/data/Tools/miniconda3/lib/python3.4/site-packages/osgeo/__init__.py", line 17, in swig_import_helper
   _mod = imp.load_module('_gdal', fp, pathname, description)
File "/data/Tools/miniconda3/lib/python3.4/imp.py", line 243, in load_module
   return load_dynamic(name, filename, file)
ImportError: /data/Tools/miniconda3/lib/python3.4/site-packages/osgeo/../../../libgdal.so.1: undefined symbol: pj_ctx_alloc

$ arcsi.py
Traceback (most recent call last):
File "/data/Tools/miniconda3/bin/arcsi.py", line 52, in <module>
   from arcsilib.arcsiutils import ARCSIUtils
File "/data/Tools/miniconda3/lib/python3.4/site-packages/arcsilib/arcsiutils.py", line 42, in <module>
   import osgeo.gdal as gdal
File "/data/Tools/miniconda3/lib/python3.4/site-packages/osgeo/__init__.py", line 21, in <module>
   _gdal = swig_import_helper()
File "/data/Tools/miniconda3/lib/python3.4/site-packages/osgeo/__init__.py", line 17, in swig_import_helper
   _mod = imp.load_module('_gdal', fp, pathname, description)
File "/data/Tools/miniconda3/lib/python3.4/imp.py", line 243, in load_module
   return load_dynamic(name, filename, file)
ImportError: /data/Tools/miniconda3/lib/python3.4/site-packages/osgeo/../../../libgdal.so.1: undefined symbol: pj_ctx_alloc

I tried both the full anaconda, and the miniconda methods, but get the same errors as above.

Any clues?

Thanks very much,

Dan

--
___________________________________________________________________
Dan Slayback, PhD
Science Systems and Applications, Inc.          tel: (301) 614-6687
Biospheric Sciences Laboratory, Code 618        fax: (301) 614-6695
NASA Goddard Space Flight Center              dan.sl...@nasa.gov
Greenbelt, MD 20771                   or daniel....@ssaihq.com
___________________________________________________________________

Alexandra Mates

unread,

Jul 3, 2014, 6:13:14 PM7/3/14

to rsgislib...@googlegroups.com, dan.sl...@nasa.gov, p...@aber.ac.uk

Hello Dan,

I had a similar problem recently, see https://groups.google.com/d/msg/rsgislib-support/AVSI2r4CCng/RHH51XVGmBEJ

I haven't posted my answer yet to that thread (thank you for reminding me!), but the way I fixed it was to delete the ~/anaconda3/ folder, do the installation again, but remove the 3 at the end of the default install path i.e. from ~/home/alex/anaconda3 to ~/home/alex/anaconda. You should also not include the 3 for the GDAL_DRIVER and GDAL_PATH variables.

Hope this helps,

Alex

NASA Goddard Space Flight Center dan.slayback@nasa.gov

Greenbelt, MD 20771 or daniel....@ssaihq.com
___________________________________________________________________

RSPSoc 2014 - 2nd -5th September

Dan Slayback

unread,

Jul 7, 2014, 9:07:48 AM7/7/14

to rsgislib...@googlegroups.com, dan.sl...@nasa.gov, p...@aber.ac.uk

Thanks Pete, & Alex,

Unfortunately, I'm still having the same issue. I tried installing anaconda and miniconda, without a trailing '3' in the install path, but get the same errors re undefined symbol pj_ctx_alloc in libgdal.so.1.

I think I do have GDAL_DRIVER_PATH and GDAL_DATA set correctly:
$ env | grep GDAL
GDAL_DRIVER_PATH=/data/Tools/miniconda/gdalplugins
GDAL_DATA=/data/Tools/miniconda/share/gdal

And yes, any gdal commands fail with the anaconda/miniconda python 3.4 installed:

$ gdalinfo
gdalinfo: symbol lookup error: /data/Tools/miniconda/bin/../lib/libgdal.so.1: undefined symbol: pj_ctx_alloc

If I revert to the system python path (=python 2.6.6), then gdal commands do work (I have had gdal installed previously).
$ gdalinfo --version
GDAL 1.9.2, released 2012/10/08

So the python 3.4 gdal installed with conda is not happy for some reason...any clues?

My conda list looks similar to yours.

Thanks much,

Dan

arcsi                     0.5.061                  py34_0
boost                     1.54.0                        0
cgal                      4.4                           0
conda                     3.5.5                    py34_0

dateutil                  2.1                      py34_2
fftw                      3.3.4                         0
freetype                  2.4.10                        0
gdal                      1.11.0               np18py34_0
geos                      3.4.2                         0

gmp 5.1.2 0

gsl 1.16 0
hdf5 1.8.13 0

kealib 1.4.2 0

libpng                    1.5.13                        1
matplotlib                1.3.1                np18py34_1
mpfr                      3.1.2                         0
muparser                  2.2.3                         0
numpy                     1.8.1                    py34_0
openssl                   1.0.1h                        0
pandas                    0.14.0               np18py34_0
proj4                     4.8.0                         0
py6s                      1.5.0                np18py34_0

pycosat 0.6.1 py34_0

pyparsing                 2.0.1                    py34_0
pyqt                      4.10.4                   py34_0
pysolar                   0.4.3                    py34_0
python                    3.4.1                         0
pytz                      2014.4                   py34_0

pyyaml 3.11 py34_0

qt 4.8.5 0
readline 6.2 2

requests                  2.3.0                    py34_0
rios                      1.3.1                np18py34_0
rsgislib                  2.2.894              np18py34_0

scipy                     0.14.0               np18py34_0
setuptools                3.6                      py34_0
sip                       4.15.5                   py34_0
six                       1.7.3                    py34_0
sixs                      1.1                           0
sqlite                    3.8.4.1                       0
system                    5.8                           1
tk                        8.5.15                        0

tuiview                   1.1.2                np18py34_0
xerces-c                  3.1.1                         0
yaml                      0.1.4                         0
zlib                      1.2.7                         0

NASA Goddard Space Flight Center dan.sl...@nasa.gov

Greenbelt, MD 20771 or daniel....@ssaihq.com
___________________________________________________________________

Daniel Clewley

unread,

Jul 7, 2014, 9:31:05 AM7/7/14

to Dan Slayback, rsgislib...@googlegroups.com, dan.sl...@nasa.gov, Pete Bunting

Hi Dan,

I think the problem might be you have two versions of GDAL installed, and it's finding the wrong libraries. Are you able to remove the previous version of GDAL and Proj4.

Thanks,

Dan

--
You received this message because you are subscribed to the Google Groups "RSGISLib Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rsgislib-suppo...@googlegroups.com.
To post to this group, send email to rsgislib...@googlegroups.com.
Visit this group at http://groups.google.com/group/rsgislib-support.
For more options, visit https://groups.google.com/d/optout.

Dan Slayback

unread,

Jul 7, 2014, 10:19:19 AM7/7/14

to rsgislib...@googlegroups.com, dansl...@gmail.com, dan.sl...@nasa.gov, p...@aber.ac.uk

Thanks for the tip Dan, but still no dice. I've now removed the previously intstalled PROJ.4 and GDAL. With the new conda installs, PROJ.4 seems ok, but gdal still has the symbol issue:

$ which proj
/data/Tools/miniconda/bin/proj
$ proj
Rel. 4.6.1, 21 August 2008
usage: proj [ -beEfiIlormsStTvVwW [args] ] [ +opts[=arg] ] [ files ]

$ which gdalinfo
/data/Tools/miniconda/bin/gdalinfo

$ gdalinfo
gdalinfo: symbol lookup error: /data/Tools/miniconda/bin/../lib/libgdal.so.1: undefined symbol: pj_ctx_alloc

If I install python 2.7 via miniconda, and then use conda to install gdal, I DO get working gdalinfo, but of course I cant then install arcsi.

So there seems to be (on my system anyhow) some issue with gdal that comes with the python 3.4/conda installer.

I might try in a fresh CentOS 6.5 virtual machine to rule out any possible oddities in my system.

Dan

To unsubscribe from this group and stop receiving emails from it, send an email to rsgislib-support+unsub...@googlegroups.com.

Pete Bunting [pfb]

unread,

Jul 7, 2014, 10:20:49 AM7/7/14

to Dan Slayback, Dan Slayback, RSGISLib

Hi Dan,

I think Dan C. is right, it is getting confused between gdal versions. If you run 'which gdalinfo’ what path does is give?

Also, can you check your PATH environment variable? If it is not already then maybe set the PATH for conda to come before the rest. Also, check you LD_LIBRARY_PATH and again make sure the miniconda path comes ahead of everything else.

Cheers, Pete

****************************************************
* Dr Pete Bunting
* Senior Lecturer in Remote Sensing
* Earth Observation and Ecosystem Dynamics Group
* Department of Geography and Earth Sciences
* Aberystwyth University
* Aberystwyth
* Ceredigion
* SY23 3DB
* UK
*
* Ph: +44 (0) 1970 622615
* Mob: +44 (0) 7917 842743
* Email: p...@aber.ac.uk
* ORCID: http://orcid.org/0000-0002-7435-0148
****************************************************

Dan Slayback

unread,

Jul 7, 2014, 10:40:09 AM7/7/14

to rsgislib...@googlegroups.com, dan.sl...@nasa.gov, dansl...@gmail.com, p...@aber.ac.uk

Aha! LD_LIBRARY_PATH was the culprit. It was set for another software package; if I unset it, everything seems to be working fine now.

Although things appear to work (I've not run any actual processing, but tuiview starts up, and arcsi.py spits out a complaint about lack of input file), should LD_LIBRARY_PATH be set to something, and if so, what directory? the arcsi install does not appear to modify it.

thanks much,

Dan

To unsubscribe from this group and stop receiving emails from it, send an email to rsgislib-support+unsub...@googlegroups.com.

Daniel Clewley

unread,

Jul 7, 2014, 11:18:05 AM7/7/14

to Dan Slayback, Pete Bunting, rsgislib...@googlegroups.com, dan.sl...@nasa.gov

Hi Dan,

Great it's working now. You shouldn't need to set LD_LIBRARY_PATH for conda.

Thanks,

Dan

To unsubscribe from this group and stop receiving emails from it, send an email to rsgislib-suppo...@googlegroups.com.

Reply all

Reply to author

Forward