tomcat - gdal

[Davis Mendoza Paco]

hello 

I'm doing tests to import gpx files in mapfishapp and using the documentation

https://github.com/georchestra/georchestra/blob/master/INSTALL.md#gdal-for-geoserver-extractorapp--mapfishapp

I did all the steps, but does not work

what I did was add the repository of "debian jessie" and install  "libgdal-java"

mapfishapp logs show, as I can fix this problem

2014-05-30 16:51:34 DefaultListableBeanFactory [DEBUG] Creating instance of bean 'org.georchestra.mapfishapp.ws.UpLoadGeoFileController#0'

2014-05-30 16:51:34 upload [INFO] gdal/ogr is not available in the system

java.lang.UnsatisfiedLinkError: org.gdal.ogr.ogrJNI.GetDriverCount()I

at org.gdal.ogr.ogrJNI.GetDriverCount(Native Method)

at org.gdal.ogr.ogr.GetDriverCount(ogr.java:103)

at org.geotools.data.ogr.jni.JniOGR.<clinit>(JniOGR.java:27)

Perform the steps

install "java-libgdal"

apt-get install libgdal-java

creating the link

ln -s /usr/share/java/gdal.jar /var/lib/tomcat/lib/

edit the file "conf/catalina.properties" to share gdal.jar

shared.loader==${catalina.base}/lib/*.jar

check that the environment variable "LD_LIBRARY_PATH" /etc/default/tomcat

LD_LIBRARY_PATH=/lib:/usr/lib/:$LD_LIBRARY_PATH

[Sylvain Lesage]

El 30/05/14 17:25, Davis Mendoza Paco escribió:
>

> /2014-05-30 16:51:34 upload [INFO] gdal/ogr is not available in the system/

Maybe it's related to the new way to use GDAL / OGR with tomcat:
https://github.com/georchestra/georchestra/blob/master/INSTALL.md#gdal-for-geoserver-extractorapp--mapfishapp

[Pierre Mauduit]

Hi,

> > /2014-05-30 16:51:34 upload [INFO] gdal/ogr is not available in the system/
>
> Maybe it's related to the new way to use GDAL / OGR with tomcat:
> https://github.com/georchestra/georchestra/blob/master/INSTALL.md#gdal-for-geoserver-extractorapp--mapfishapp

@Davis: Thanks for testing the documentation, you seem to be pretty close to
finish your setup. But I think I've got the flaw in what you described earlier.
Could you provide the result of the following command ?:

dpkg -l | grep ii | grep gdal

What I suspect is that you are currently trying to use libgdal from debian
wheezy with libgdal-java from debian jessie. Note that the java bindings
provided by the libgdal-java package should come along with the same compiled
version of the binary library. One solution to fix it would be to use libgdal
from jessie, but it might force you to dist-upgrade (depending on if the library
in jessie has been linked against a newer version of the libc or other
underlying libs).

Please also note that the currently packaged version of libgdal would probably
not fix the MIF/MID extraction bug yet.

Regards,

-- Pierre

[Davis Mendoza Paco]

hi, the result of the command is

# dpkg -l | grep ii | grep gdal 

+ii  gdal-bin                                 1.10.1+dfsg-5+b1              amd64        Geospatial Data Abstraction Library - Utility programs

ii  libgdal-dev                              1.10.1+dfsg-5+b1              amd64        Geospatial Data Abstraction Library - Development files

ii  libgdal-java                             1.10.1+dfsg-5+b1              amd64        Java bindings to the Geospatial Data Abstraction Library

ii  libgdal1-dev                             1.9.0-3.1                     all          Geospatial Data Abstraction Library - Development files

ii  libgdal1h                                1.10.1+dfsg-5+b1              amd64        Geospatial Data Abstraction Library

ii  python-gdal                              1.10.1+dfsg-5+b1              amd64        Python bindings to the Geospatial Data Abstraction Library

[Pierre Mauduit]

Hi,

This is weird, since you seem to use the same version for libgdal /
libgdal-java, so I'm assuming that you are using bindings that are compatible
with the native lib (the -dev package is an older one, but should not interfere
with the others). Here are some debugging ways:

- Do you have other webapps in the tomcat hosting your mapfishapp ? (if one of
the other webapps ships a gdal java bindings - sometimes named gdal.jar or
imageio-ext-gdal-bindings... - it can lead to failures. You can inspect the
suspected jars into WEB-INF/lib using 'jar tf file.jar' ; if it lists
org.gdal.* classes, that is wrong.)

- Is there any other gdal setups defined elsewhere in your tomcat configuration
? (a LD_LIBRARY_PATH or java.library.path which could lead to resolve an other
version of GDAL ?)

- Are you able to launch CLI GDAL tools ? (ogr2ogr --formats, gdalinfo ...)
Without complaining about library not found / version issues ?

- Could you try deploying the following webapp into your tomcat:
http://sdi.georchestra.org/~pmauduit/gdalogr-java-bindings/jni-info.war
then call the 2 following services (adapt to suit your setup):

* http://yourtomcat:8080/jni-info/native.json
* http://yourtomcat:8080/jni-info/info.json

It gives some extra debugging information (native.json lists every packages
known to the current classloader, info.json tries to use GDAL/OGR bindings and
reports what is wrong with the setup)

Regards,

--
Pierre

[Fabrice Phung]

Le 31/05/2014 11:59, Pierre Mauduit a écrit :
> Hi,

Hi Pierre

Here are my results. I assume that the libs are OK but the classloader
gone wrong ?
Anyway your tool helps a lot, thanks.

native.json

"loaded_native_libraries": [
"/opt/jdk1.6.0_32-x64/jre/lib/amd64/libzip.so",
"/opt/jdk1.6.0_32-x64/jre/lib/amd64/libnet.so",
"/opt/jdk1.6.0_32-x64/jre/lib/amd64/libmanagement.so",
"/opt/jdk1.6.0_32-x64/jre/lib/amd64/libclib_jiio.so",
"/opt/gdal-georchestra/java/libogrjni.so",
"/opt/gdal-georchestra/java/libgdaljni.so",
"/opt/jdk1.6.0_32-x64/jre/lib/amd64/libnio.so",
"/opt/gdal-georchestra/java/libosrjni.so",
"/opt/jdk1.6.0_32-x64/jre/lib/amd64/headless/libmawt.so",
"/opt/jdk1.6.0_32-x64/jre/lib/amd64/libawt.so",
"/opt/jdk1.6.0_32-x64/jre/lib/amd64/libfontmanager.so",
"/opt/jdk1.6.0_32-x64/jre/lib/amd64/libmlib_jai.so"
],

info.json

{
"jai-imageio": {"status": "available"},
"geotools": {
"GeoTools": {"datastores": [
"Directory of spatial files (shapefiles)",
"Shapefile",
"Web Feature Server",
"OGR"
]},
"GDAL": {
"reason": "java.lang.NoClassDefFoundError: org/gdal/gdal/gdal",
"status": "unavailable",
"via-geotools": {"status": "unavailable"}
},
"OGR": {
"reason": "java.lang.NoClassDefFoundError: org/gdal/ogr/ogr",
"status": "unavailable",
"via-geotools": "unavailable"
}
},
"libturbojpeg": {
"reason": "Unable to find class
org.libjpegturbo.turbojpeg.TJCompressor, this is likely due to a
classpath misconfiguration.",
"status": "unavailable"
}
}

[Pierre Mauduit]

Fabrice Phung a écrit :

> Le 31/05/2014 11:59, Pierre Mauduit a écrit :
> >Hi,
>
> Hi Pierre
>
> Here are my results. I assume that the libs are OK but the
> classloader gone wrong ?
> Anyway your tool helps a lot, thanks.

I have some plans to improve it (checking the library path more thoroughly,
digging a bit more on the defined classpath, some other controls on the parent
classloaders), and maybe to integrate it into the concerned webapps ?

>
> native.json
>
> "loaded_native_libraries": [
> "/opt/jdk1.6.0_32-x64/jre/lib/amd64/libzip.so",
> "/opt/jdk1.6.0_32-x64/jre/lib/amd64/libnet.so",
> "/opt/jdk1.6.0_32-x64/jre/lib/amd64/libmanagement.so",
> "/opt/jdk1.6.0_32-x64/jre/lib/amd64/libclib_jiio.so",
> "/opt/gdal-georchestra/java/libogrjni.so",
> "/opt/gdal-georchestra/java/libgdaljni.so",
> "/opt/jdk1.6.0_32-x64/jre/lib/amd64/libnio.so",
> "/opt/gdal-georchestra/java/libosrjni.so",
> "/opt/jdk1.6.0_32-x64/jre/lib/amd64/headless/libmawt.so",
> "/opt/jdk1.6.0_32-x64/jre/lib/amd64/libawt.so",
> "/opt/jdk1.6.0_32-x64/jre/lib/amd64/libfontmanager.so",
> "/opt/jdk1.6.0_32-x64/jre/lib/amd64/libmlib_jai.so"
> ],

It sounds like the native libraries have been correctly loaded, so there must be
something in the classloader that took care of the System.loadLibrary call,
independently of the specific tomcat configuration (gdal.jar into
$catalina.base/lib). It is on GéoBretagne, right ? I could not find it on
sdi.georchestra.org.

> "GDAL": {
> "reason": "java.lang.NoClassDefFoundError: org/gdal/gdal/gdal",

> },
> "OGR": {
> "reason": "java.lang.NoClassDefFoundError: org/gdal/ogr/ogr",

It clearly shows that jni-info webapp is unable to resolve the gdal.jar java
package. that probably means that one of the webapp in the tomcat includes gdal
jar bindings, which has been responsible of loading the native libraries in
/opt/gdal-georchestra/... but is not available at a tomcat's scope.


--
Pierre

[Davis Mendoza Paco]

pmauduit:  Hello, I install from scratch "debian wheezy" and add repositories "debian jessie" and it works correctly importing files "GPX" 

Steps to perform 

apt-get install tomcat7

apt-get install -f libgdal-java

service tomcat7 stop

ln -s /usr/share/java/gdal.jar /usr/share/tomcat7/lib/

service tomcat7 start

I will continue testing the configuration on a "tomcat binary distributions"

[Pierre Mauduit]

Davis Mendoza Paco a écrit :

I did not get it, does it work yet ? The default shared.loader value in
catalina.properties for tomcat7 (at least in debian wheezy, I mean, from the
debian pakage) is:

shared.loader=/var/lib/tomcat7/shared/classes,/var/lib/tomcat7/shared/*.jar

So the gdal.jar symlink should go into /var/lib/tomcat7/shared/ instead ; you
have to make sure that the gdal.jar can be resolved by the shared classloader.

Regards,

--
Pierre

[Davis Mendoza Paco]

was my mistake

rm /usr/share/tomcat7/lib/gdal.jar

ln -s /usr/share/java/gdal.jar /var/lib/tomcat7/shared/

and it works correctly importing files "GPX" 

mapfishapp.log

2014-06-02 11:30:49 DefaultListableBeanFactory [DEBUG] Creating instance of bean 'org.georchestra.mapfishapp.ws.UpLoadGeoFileController#0'

2014-06-02 11:30:49 upload [INFO] try to load driver: ESRI Shapefile

2014-06-02 11:30:49 upload [INFO] format:shp Driver:  OGRDriver [name=ESRI Shapefile, options=null] was loaded.

2014-06-02 11:30:49 upload [INFO] try to load driver: MapInfo File

2014-06-02 11:30:49 upload [INFO] format:tab Driver:  OGRDriver [name=MapInfo File, options=[]] was loaded.

2014-06-02 11:30:49 upload [INFO] format:mif Driver:  OGRDriver [name=MapInfo File, options=[FORMAT=MIF]] was loaded.

2014-06-02 11:30:49 upload [INFO] try to load driver: UK .NTF

2014-06-02 11:30:49 upload [INFO] try to load driver: SDTS

2014-06-02 11:30:49 upload [INFO] try to load driver: TIGER

2014-06-02 11:30:49 upload [INFO] try to load driver: S57

2014-06-02 11:30:49 upload [INFO] try to load driver: DGN

2014-06-02 11:30:49 upload [INFO] try to load driver: VRT

2014-06-02 11:30:49 upload [INFO] try to load driver: REC

2014-06-02 11:30:49 upload [INFO] try to load driver: Memory

2014-06-02 11:30:49 upload [INFO] try to load driver: BNA

2014-06-02 11:30:49 upload [INFO] try to load driver: CSV

2014-06-02 11:30:49 upload [INFO] try to load driver: NAS

2014-06-02 11:30:49 upload [INFO] try to load driver: GML

2014-06-02 11:30:49 upload [INFO] format:gml Driver:  OGRDriver [name=GML, options=null] was loaded.

2014-06-02 11:30:49 upload [INFO] try to load driver: GPX

2014-06-02 11:30:49 upload [INFO] format:gpx Driver:  OGRDriver [name=GPX, options=null] was loaded.

2014-06-02 11:30:49 upload [INFO] try to load driver: LIBKML

2014-06-02 11:30:49 upload [INFO] try to load driver: KML

2014-06-02 11:30:49 upload [INFO] format:kml Driver:  OGRDriver [name=KML, options=null] was loaded.

2014-06-02 11:30:49 upload [INFO] try to load driver: GeoJSON

2014-06-02 11:30:49 upload [INFO] try to load driver: Interlis 1

2014-06-02 11:30:49 upload [INFO] try to load driver: Interlis 2

2014-06-02 11:30:49 upload [INFO] try to load driver: GMT

2014-06-02 11:30:49 upload [INFO] try to load driver: SQLite

2014-06-02 11:30:49 upload [INFO] try to load driver: DODS

2014-06-02 11:30:49 upload [INFO] try to load driver: ODBC

2014-06-02 11:30:49 upload [INFO] try to load driver: PGeo

2014-06-02 11:30:49 upload [INFO] try to load driver: MSSQLSpatial

2014-06-02 11:30:49 upload [INFO] try to load driver: OGDI

2014-06-02 11:30:49 upload [INFO] try to load driver: PostgreSQL

2014-06-02 11:30:49 upload [INFO] try to load driver: MySQL

2014-06-02 11:30:49 upload [INFO] try to load driver: PCIDSK

2014-06-02 11:30:49 upload [INFO] try to load driver: XPlane

2014-06-02 11:30:49 upload [INFO] try to load driver: AVCBin

2014-06-02 11:30:49 upload [INFO] try to load driver: AVCE00

2014-06-02 11:30:49 upload [INFO] try to load driver: DXF

2014-06-02 11:30:49 upload [INFO] try to load driver: Geoconcept

2014-06-02 11:30:49 upload [INFO] try to load driver: GeoRSS

2014-06-02 11:30:49 upload [INFO] try to load driver: GPSTrackMaker

2014-06-02 11:30:49 upload [INFO] try to load driver: VFK

2014-06-02 11:30:49 upload [INFO] try to load driver: PGDump

2014-06-02 11:30:49 upload [INFO] try to load driver: OSM

2014-06-02 11:30:49 upload [INFO] try to load driver: GPSBabel

2014-06-02 11:30:49 upload [INFO] try to load driver: SUA

2014-06-02 11:30:49 upload [INFO] try to load driver: OpenAir

2014-06-02 11:30:49 upload [INFO] try to load driver: PDS

2014-06-02 11:30:49 upload [INFO] try to load driver: WFS

2014-06-02 11:30:49 upload [INFO] try to load driver: HTF

2014-06-02 11:30:49 upload [INFO] try to load driver: AeronavFAA

2014-06-02 11:30:49 upload [INFO] try to load driver: Geomedia

2014-06-02 11:30:49 upload [INFO] try to load driver: EDIGEO

2014-06-02 11:30:49 upload [INFO] try to load driver: GFT

2014-06-02 11:30:49 upload [INFO] try to load driver: SVG

2014-06-02 11:30:49 upload [INFO] try to load driver: CouchDB

2014-06-02 11:30:49 upload [INFO] try to load driver: Idrisi

2014-06-02 11:30:49 upload [INFO] try to load driver: ARCGEN

2014-06-02 11:30:49 upload [INFO] try to load driver: SEGUKOOA

2014-06-02 11:30:49 upload [INFO] try to load driver: SEGY

2014-06-02 11:30:49 upload [INFO] try to load driver: XLS

2014-06-02 11:30:49 upload [INFO] try to load driver: ODS

2014-06-02 11:30:49 upload [INFO] try to load driver: XLSX

2014-06-02 11:30:49 upload [INFO] try to load driver: ElasticSearch

2014-06-02 11:30:49 upload [INFO] try to load driver: PDF

2014-06-02 11:30:49 upload [INFO] gdal/ogr is available in the system