running geoserver in OSv. I need help

[ellis...@ccri.com]

I have been trying to accomplish what should be a simple thing in OSv but not getting anywhere.  I have no issues starting and modfying osv-apps that are already published, but porting new apps to OSv is proving to be difficult.
Here's my issue:
I have been trying to instantiate an opensource geolocation java app called geoserver.  It comes packaged with Jetty and is typically started with a bash script (see below).  I've been able to modify the usr.manifest file to get all the necessary files onto the OSv image under /usr/geoserver, but I can't seem to start jetty.  I have tried emulating the tomcat module.py and jetty module.py examples and no luck (see below). I have also tried numerous other configurations without a starting app. Any help would be appreciated.

#!/bin/sh
# -----------------------------------------------------------------------------
# Start Script for GEOSERVER
#
# $Id$
# -----------------------------------------------------------------------------

# Guard against misconfigured JAVA_HOME
if [ ! -z "$JAVA_HOME" -a ! -x "$JAVA_HOME"/bin/java ]; then
  echo "The JAVA_HOME environment variable is set but JAVA_HOME/bin/java"
  echo "is missing or not executable:"
  echo "    JAVA_HOME=$JAVA_HOME"
  echo "Please either set JAVA_HOME so that the Java runtime is JAVA_HOME/bin/java"
  echo "or unset JAVA_HOME to use the Java runtime on the PATH."
  exit 1
fi

# Find java from JAVA_HOME or PATH
if [ ! -z "$JAVA_HOME" ]; then
  _RUNJAVA="$JAVA_HOME"/bin/java
elif [ ! -z "$(which java)" ]; then
  _RUNJAVA=java
else
  echo "A Java runtime (java) was not found in JAVA_HOME/bin or on the PATH."
  echo "Please either set the JAVA_HOME environment variable so that the Java runtime"
  echo "is JAVA_HOME/bin/java or add the Java runtime to the PATH."
  exit 1
fi

if [ -z $GEOSERVER_HOME ]; then
  #If GEOSERVER_HOME not set then guess a few locations before giving
  # up and demanding user set it.
  if [ -r start.jar ]; then
     echo "GEOSERVER_HOME environment variable not found, using current "
     echo "directory.  If not set then running this script from other "
     echo "directories will not work in the future."
     export GEOSERVER_HOME=`pwd`
  else
    if [ -r ../start.jar ]; then
      echo "GEOSERVER_HOME environment variable not found, using current "
      echo "location.  If not set then running this script from other "
      echo "directories will not work in the future."
      export GEOSERVER_HOME=`pwd`/..
    fi
  fi

  if [ -z "$GEOSERVER_HOME" ]; then
    echo "The GEOSERVER_HOME environment variable is not defined"
    echo "This environment variable is needed to run this program"
    echo "Please set it to the directory where geoserver was installed"
    exit 1
  fi

fi

if [ ! -r "$GEOSERVER_HOME"/bin/startup.sh ]; then
  echo "The GEOSERVER_HOME environment variable is not defined correctly"
  echo "This environment variable is needed to run this program"
  exit 1
fi

#Find the configuration directory: GEOSERVER_DATA_DIR
if [ -z $GEOSERVER_DATA_DIR ]; then
    if [ -r "$GEOSERVER_HOME"/data_dir ]; then
        export GEOSERVER_DATA_DIR="$GEOSERVER_HOME"/data_dir
    else
        echo "No GEOSERVER_DATA_DIR found, using application defaults"
          GEOSERVER_DATA_DIR=""
    fi
fi

cd "$GEOSERVER_HOME"

echo "GEOSERVER DATA DIR is $GEOSERVER_DATA_DIR"
#added headless to true by default, if this messes anyone up let the list
#know and we can change it back, but it seems like it won't hurt -ch
exec "$_RUNJAVA" $JAVA_OPTS -DGEOSERVER_DATA_DIR="$GEOSERVER_DATA_DIR" -Djava.awt.headless=true -DSTOP.PORT=8079 -DSTOP.KEY=geoserver -jar start.jar


Current OSv geoserver module.py

from osv.modules import api


_GEOSERVER_HOME = "/usr/geoserver"
_GEOSERVER_DATA_DIR="/usr/geoserver/data_dir"
_classpath = [_GEOSERVER_HOME + "/start.jar"]

default = api.run_java(
        classpath=_classpath,
        args=["-DGEOSERVER_HOME=%s" % _GEOSERVER_HOME,
                "-DGEOSERVER_DATA_DIR=%s" % _GEOSERVER_DATA_DIR,
                "-DSTOP.PORT=8079",
                "-DSTOP.KEY=geoserver",
                "-jar /usr/geoserver/start.jar"
        ])

[Nadav Har'El]

On Thu, Jun 2, 2016 at 4:50 PM, <ellis...@ccri.com> wrote:

#added headless to true by default, if this messes anyone up let the list
#know and we can change it back, but it seems like it won't hurt -ch
exec "$_RUNJAVA" $JAVA_OPTS -DGEOSERVER_DATA_DIR="$GEOSERVER_DATA_DIR" -Djava.awt.headless=true -DSTOP.PORT=8079 -DSTOP.KEY=geoserver -jar start.jar

It seems this is the main command line you need to use, with the appropriate directories.

Current OSv geoserver module.py

from osv.modules import api


_GEOSERVER_HOME = "/usr/geoserver"
_GEOSERVER_DATA_DIR="/usr/geoserver/data_dir"
_classpath = [_GEOSERVER_HOME + "/start.jar"]

Unless I'm missing something,  you don't need the "classpath" here, the "-jar" is supposed to replace it.

default = api.run_java(
        classpath=_classpath,
        args=["-DGEOSERVER_HOME=%s" % _GEOSERVER_HOME,

I don't think you actually need to pass GEOSERVER_HOME to the Java - the bash script you showed appears not to do this.
 

                "-DGEOSERVER_DATA_DIR=%s" % _GEOSERVER_DATA_DIR,
                "-DSTOP.PORT=8079",
                "-DSTOP.KEY=geoserver",

What about the "headless" parameter?
 

                "-jar /usr/geoserver/start.jar"
        ])

What kind of error do you see when running it?

An alternative to using api.run_java() might be to use api.run() and just give the Java command line explicitly - for example something like this might work for your module.py:

from osv.modules import api
api.require('java')
default = api.run(cmdline='java.so -DGEOSERVER_DATA_DIR="/usr/geoserver/data_dir" -Djava.awt.headless=true -DSTOP.PORT=8079 -DSTOP.KEY=geoserver -jar /usr/geoserver/start.jar")

--
You received this message because you are subscribed to the Google Groups "OSv Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to osv-dev+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ellis...@ccri.com]

Nadav,

Thanks so much for your reply.  I had actually tried the exact command you posted but I got this warning below:

WARNING: Nothing to start, exiting ...

Usage: java -jar start.jar [options] [properties] [configs]
       java -jar start.jar --help  # for more information

What's interesting is that if I strip the command down to:

default = api.run(cmdline='java.so -jar /usr/geoserver/start.jar')

I get the same error.  It seems like the start jar doesn't have an appropriate main entry point, but I know this isn't the case.  On my local ubuntu machine I can run the associated (see above) geoserver bash startup script without issue.  I can run the osv jetty app without issue and also run the tomcat app without issue.  I'm definitely missing something.  Could it have something to do with version of java used by osv?  I know geoserver requires java 8.  Any other ideas?

Thanks again for your help,
Ellis

[ellis...@ccri.com]

Update.  So I figured out one of my issues.  The geoserver startup script changes to the jetty base directory (i.e. cd "$GEOSERVER_HOME" ) and jetty.base is assigned to this "base" directory.  By adding -Djetty.base=/usr/geoserver to the cmdline argument I got things running (sort of).  The entire cmdline argument ended up being:

default = api.run(cmdline='java.so -DGEOSERVER_DATA_DIR=/usr/geoserver/data_dir -Djava.awt.headless=true -DSTOP.PORT=8079 -DSTOP.KEY=geoserver -Djetty.base=/usr/geoserver -jar /usr/geoserver/start.jar')

I'm now running into Java version issues. I tried using the openjdk8-fedora app but no dice since I'm working on an ubuntu 14.04 machine.   I'm going to work on getting openjdk8 working on an ubuntu machine, but any help you can give would be appreciated.

Thanks again for your help,
Ellis

[Nadav Har'El]

On Mon, Jun 6, 2016 at 9:20 PM, <ellis...@ccri.com> wrote:

Update.  So I figured out one of my issues.  The geoserver startup script changes to the jetty base directory (i.e. cd "$GEOSERVER_HOME" ) and jetty.base is assigned to this "base" directory.  By adding -Djetty.base=/usr/geoserver to the cmdline argument I got things running (sort of).  The entire cmdline argument ended up being:

default = api.run(cmdline='java.so -DGEOSERVER_DATA_DIR=/usr/geoserver/data_dir -Djava.awt.headless=true -DSTOP.PORT=8079 -DSTOP.KEY=geoserver -Djetty.base=/usr/geoserver -jar /usr/geoserver/start.jar')

By the way, you can also set the current directory by adding "--cwd=..." in the beginning of the command line (before the java.so).
 

I'm now running into Java version issues. I tried using the openjdk8-fedora app but no dice since I'm working on an ubuntu 14.04 machine.   I'm going to work on getting openjdk8 working on an ubuntu machine, but any help you can give would be appreciated.

The "openjdk8-fedora" indeed takes the Java shared libraries from Fedora, but why does it not work on Ubuntu? What fails?

By the way, you can use the openjdk8-fedora module as an example of how to take whatever JRE you have installed in the build machine and upload it into OSv. If you happen to write such a module, it would be a useful addition ;-)

[ellis...@ccri.com]

Thanks

The --cwd=... option would have been a huge help had I not figured out the work around.  I may need it for another application.

The openjdk8-fedora module likely requires a rpm package manager (e.g. yum) and ubuntu uses apt as a package manager (see error below).  I'll look into creating a openjdk8-debian version using the fedora example.

rpm2cpio upstream/java-1.8.0-openjdk-headless-1.8.0.91-8.b14.fc25.x86_64.rpm | (cd install && cpio -id)
/bin/sh: 1: rpm2cpio: not found
cpio: premature end of archive

I think I'm almost there.

[Nadav Har'El]

On Mon, Jun 6, 2016 at 10:29 PM, <ellis...@ccri.com> wrote:

Thanks

The --cwd=... option would have been a huge help had I not figured out the work around.  I may need it for another application.

The openjdk8-fedora module likely requires a rpm package manager (e.g. yum)

No, it should only require "rpm2cpio", a simple tool which I hope you can get for Ubuntu.
 

and ubuntu uses apt as a package manager (see error below).  I'll look into creating a openjdk8-debian version using the fedora example.

rpm2cpio upstream/java-1.8.0-openjdk-headless-1.8.0.91-8.b14.fc25.x86_64.rpm | (cd install && cpio -id)
/bin/sh: 1: rpm2cpio: not found
cpio: premature end of archive

I think I'm almost there.

Try searching if rpm2cpio is available for Ubuntu....
 

[ellis...@ccri.com]

installed rpm2cpio on my unbuntu machine and was able to run/start up geoserver but then got a missing java symbot error while geoserver was initializing (see below):

usr/lib/jvm/java-1.8.0-openjdk-1.8.0.91-8.b14.fc25.x86_64/jre/lib/amd64/libsunec.so: failed looking up symbol PR_CallOnce

Tried the same thing on a fedora machine and got the same error.  I'm fairly certain that I will eventually exhaust every possible error and get things running :)  I'm thinking that once I get this thing running it might be worth adding it to osv's repo.

[ellis...@ccri.com]

Success.  I was able to make a java 8 port to osv (i.e. non-openjdk) and get the latest version of geoserver running on osv (2.9-RC1).   To get things working I needed to run geoserver on my local machine and copy the files located in my local machine's /tmp/Geotools directory to the osv image's /tmp/Geotools directory prior to osv image geoserver startup.  The Geotools parent directory and associated files (i.e. /tmp/Geotools/*) are normally copied on geoserver startup to the tmp directory but for some reason this doesn't happen when running geoserver on osv.  Any ideas?

1. Does osv have a max file size? One of the files ( EPSG.data is 16M)
2. Does osv limit the number of files that can be copied?
3. Would it help if I ran osv and attached the debugger?  Currently, I get the following error when trying to load osv's symbols:

(gdb) osv syms
Python Exception <class 'gdb.MemoryError'> Cannot access memory at address 0x5f78745f756d6400:
Error occurred in Python command: Cannot access memory at address 0x5f78745f756d6400

Thanks again for all your help,
Ellis 

P.S. for anyone else dealing with this issue, the missing Geotools database/files produce this recurrent stack trace:
Error decode epsg code: EPSG:4326 
org.opengis.referencing.NoSuchAuthorityCodeException: No code "EPSG:4326" from authority "European Petroleum Survey Group" found for object of type "IdentifiedObject". 
        at org.geotools.referencing.factory.AbstractAuthorityFactory.noSuchAuthorityCode(AbstractAuthorityFactory.java:952) 
        at org.geotools.referencing.factory.PropertyAuthorityFactory.getWKT(PropertyAuthorityFactory.java:316) 
        at

[Nadav Har'El]

On Tue, Jun 21, 2016 at 7:21 PM, <ellis...@ccri.com> wrote:

Success.  I was able to make a java 8 port to osv (i.e. non-openjdk)

If there's anything you'd like to contribute to the osv-apps.git repository, it might be useful to others too.

 

and get the latest version of geoserver running on osv (2.9-RC1).   To get things working I needed to run geoserver on my local machine and copy the files located in my local machine's /tmp/Geotools directory to the osv image's /tmp/Geotools directory prior to osv image geoserver startup.  The Geotools parent directory and associated files (i.e. /tmp/Geotools/*) are normally copied on geoserver startup to the tmp directory but for some reason this doesn't happen when running geoserver on osv.  Any ideas?

I'm not familiar with geoserver so I really have no idea. There's nothing particularly special about the "/tmp" directory in OSv - it's just a normal directory in the filesystem. There should be no problem copying files into there.

 

1. Does osv have a max file size? One of the files ( EPSG.data is 16M)

No, there's obviously a limit to the file size, but it's not as ridiculously low as 16 MB :-)

 

2. Does osv limit the number of files that can be copied?

There's also a limit in the number of inodes in the filesystem, but again, it's something very high (I don't know the exact number), not something you are likely to have encountered.

 

3. Would it help if I ran osv and attached the debugger?

gdb can be useful for diagnosing crashes in OSv, but I don't see how it can help you to understand why your Java code doesn't do what you think it does (like copy those files).

 

  Currently, I get the following error when trying to load osv's symbols:

(gdb) osv syms
Python Exception <class 'gdb.MemoryError'> Cannot access memory at address 0x5f78745f756d6400:
Error occurred in Python command: Cannot access memory at address 0x5f78745f756d6400

One of the possible reasons for this is that you're running one kernel, but trying to debug another...

For example, if you're running the "release" build, make sure you are debugging build/release/loader.elf - and not something else.

Also you need to make sure that OSv is still running while you're trying to connect to it and debug it.

[ellis...@ccri.com]

Thanks so much for the quick response. 

We're planning on contributing our java8 and geoserver modules/apps (not sure which term to use) to your repo.  We just need to polish things.  

I was hoping for a simple osv limitation answer to my geoserver deployment woes but I guess I can't blame everything on osv (especially when geoserver's code is far from bug free) :)

Thanks again and I'll ping you when the java8/geoserver app PR is ready

Ellis