Apollo Docker problem

69 views
Skip to first unread message

Michał T. Lorenc

unread,
Feb 7, 2021, 7:38:31 PM2/7/21
to apo...@lbl.gov
Hi all,
I modified Apollo's Dokckerfile as shown below to add BLAST.

$ diff -u Dockerfile Dockerfile.new
--- Dockerfile 2021-02-08 10:16:06.332869890 +1000
+++ Dockerfile.new 2021-02-08 10:15:49.109599939 +1000
@@ -15,7 +15,7 @@
  apt-get --no-install-recommends -y install \
  git locales locales-all build-essential libpq-dev wget python3-pip \
  lsb-release gnupg2 wget xmlstarlet netcat libpng-dev postgresql-common \
- zlib1g-dev libexpat1-dev curl ssl-cert zip unzip openjdk-8-jdk-headless
+ zlib1g-dev libexpat1-dev curl ssl-cert zip unzip openjdk-8-jdk-headless ncbi-blast+
 
 RUN sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt/ `lsb_release -cs`-pgdg main" >> /etc/apt/sources.list.d/pgdg.list' && \
     wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
@@ -54,6 +54,7 @@
 ADD build* /apollo/
 ADD settings.gradle /apollo
 ADD application.properties /apollo
+COPY apollo-config.groovy /apollo
 RUN ls /apollo
 
 COPY docker-files/build.sh /bin/build.sh
@@ -72,7 +73,7 @@
 
 RUN pip3 install setuptools
 RUN pip3 install wheel
-RUN pip3 install nose apollo==4.2.10
+RUN pip3 install nose apollo==4.2.7
 
 RUN curl -s get.sdkman.io | bash && \
      /bin/bash -c "source $HOME/.sdkman/bin/sdkman-init.sh && yes | sdk install grails 2.5.5" && \


Furthermore, I modified
  • apollo-config.groovy
  • /apollo/src/gwt/org/bbop/apollo/gwt/client/LoginDialog.ui.xml
  • /apollo/grails-app/views/auth/login.gsp
which can be found attached.

My `docker-compose.yml` is written as:

version: '3.3'
services:
  apollo:
    build: Apollo-2.6.2
    restart: always
    links:
      - apollo_db
    ports:
      - "8888:8080"
    environment:
      - WEBAPOLLO_DB_USERNAME=postgres
      - WEBAPOLLO_DB_PASSWORD=password
      - WEBAPOLLO_DB_DRIVER="org.postgresql.Driver"
      - WEBAPOLLO_DB_DIALECT="org.hibernate.dialect.PostgresPlusDialect"
      - WEBAPOLLO_DB_URI="jdbc:postgresql://db/postgres"
    volumes:
      - ./data/jbrowse:/data
      - ./data/postgres-data:/var/lib/postgresql

  apollo_db:
    image: postgres:latest
    environment:
      - POSTGRES_PASSWORD=postgres

Unfortunately, `docker-compose build` caused the following errors:

Install contributed software...
mv: cannot stat '/apollo/.sdkman/tmp/stage/contrib/*': No such file or directory
Set version to 5.10.0+617 ..

...

Formatting Volvox example data ...done.
To see the volvox example data, browse to http://your.jbrowse.root/index.html?data=sample_data/json/volvox.

Formatting Yeast example data ...done.
To see the yeast example data, browse to http://your.jbrowse.root/index.html?data=sample_data/json/yeast.
:javacNote: /apollo/src/gwt/org/bbop/apollo/gwt/client/TrackPanel.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

:gwtc FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':gwtc'.
> Java returned: 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 7 mins 40.58 secs
***************************************
NOTE: Please set the memory for your servlet container (tomcat, jetty, etc.) or Apollo may not start correctly: http://genomearchitect.readthedocs.io/en/latest/Troubleshooting.html#suggested-tomcat-memory-settings
***************************************
cp: cannot stat '/apollo/target/apollo*.war': No such file or directory
mv: cannot stat '/tmp/apollo.war': No such file or directory
ERROR: Service 'apollo' failed to build: The command '/bin/sh -c /bin/bash -c "source $HOME/.sdkman/bin/sdkman-init.sh && /bin/bash /bin/build.sh"' returned a non-zero code: 1
(base)

What did I miss?

Thank you in advance,
Michal

Dockerfile.new
LoginDialog.ui.xml
login.gsp.new
docker-compose.yml

Nathan Dunn

unread,
Feb 8, 2021, 12:10:02 PM2/8/21
to "Michał T. Lorenc", apo...@lbl.gov

Michal,

You don’t need to use docker compose as docker should already have PostgreSQL installed (but would need to see the remaining file.

If this is the entirety of the docker file (and you’ve removed it), then you might need to mount the local file and make sure that the apollo-config.groovy in there. 

Nathan


<Dockerfile.new><LoginDialog.ui.xml><login.gsp.new><docker-compose.yml>

Nathan Dunn

unread,
Feb 9, 2021, 8:37:54 PM2/9/21
to apollo, "Michał T. Lorenc"

I see the problem, you are trying to build it directly, which I wouldn’t recommend, but maybe someone else has better advice. 

If you want to make software changes, I would fork the changes into another repo and build use quay.io or docker hub and then pull that image directly. 

You might also want to build locally, first, as well, as you’ve changed some get files and you are getting gwtc errors.



You might check to see how your yaml file does against the current Apollo image and then build out your own image. 

Also note that if you use that image that the docker-files/docker-apollo-config.groovy is automagically copied in as apollo-config.groovy with most of the options built in (but not all).   

Nathan

On Feb 8, 2021, at 3:54 PM, Michał T. Lorenc <m.t.l...@gmail.com> wrote:

Hi Nathan,
I would like to use docker-compose.yml file for Apollo because I run other web services. However, I changed the file to:

version: '3.3'
services:
  apollo:
    build: Apollo-2.6.2
    restart: always
    ports:
      - "8888:8080"

    volumes:
      - ./data/jbrowse:/data
      - ./data/postgres-data:/var/lib/postgresql

Unfortunately, the error remained:

Formatting Yeast example data ...done.
To see the yeast example data, browse to http://your.jbrowse.root/index.html?data=sample_data/json/yeast.
:javacNote: /apollo/src/gwt/org/bbop/apollo/gwt/client/TrackPanel.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

:gwtc FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':gwtc'.
> Java returned: 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 7 mins 47.591 secs

***************************************
NOTE: Please set the memory for your servlet container (tomcat, jetty, etc.) or Apollo may not start correctly: http://genomearchitect.readthedocs.io/en/latest/Troubleshooting.html#suggested-tomcat-memory-settings
***************************************
cp: cannot stat '/apollo/target/apollo*.war': No such file or directory
mv: cannot stat '/tmp/apollo.war': No such file or directory


Please find attached all files which I modified.

Thank you in advance,

Best wishes,

Michal
<docker-compose.yml><Dockerfile.new><login.gsp><LoginDialog.ui.xml><apollo-config.groovy>

Michał T. Lorenc

unread,
Feb 10, 2021, 11:14:38 PM2/10/21
to Nathan Dunn, apollo
Hi Nathan,
Thank you for your email. I modified docker-files/docker-apollo-config.groovy. However, I still get the following error:

:javacNote: /apollo/src/gwt/org/bbop/apollo/gwt/client/TrackPanel.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

:gwtc FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':gwtc'.
> Java returned: 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 7 mins 27.588 secs

***************************************
NOTE: Please set the memory for your servlet container (tomcat, jetty, etc.) or Apollo may not start correctly: http://genomearchitect.readthedocs.io/en/latest/Troubleshooting.html#suggested-tomcat-memory-settings
***************************************
cp: cannot stat '/apollo/target/apollo*.war': No such file or directory
mv: cannot stat '/tmp/apollo.war': No such file or directory
ERROR: Service 'apollo' failed to build: The command '/bin/sh -c /bin/bash -c "source $HOME/.sdkman/bin/sdkman-init.sh && /bin/bash /bin/build.sh"' returned a non-zero code: 1

I can not figure out why I can't build it locally. Please find attached all modified files?

Thank you in advance,

Best wishes,

Michal
docker-compose.yml
docker-apollo-config.groovy
Dockerfile
login.gsp
LoginDialog.ui.xml

Nathan Dunn

unread,
Feb 10, 2021, 11:38:40 PM2/10/21
to "Michał T. Lorenc", apollo

Are you able to build this locally using ./apollo clean-all && ./apollo deploy ? 

Nathan


<docker-compose.yml><docker-apollo-config.groovy><Dockerfile><login.gsp><LoginDialog.ui.xml>

Michał T. Lorenc

unread,
Feb 11, 2021, 8:01:47 PM2/11/21
to Nathan Dunn, apollo
Hi Nathan,
I added all files to https://github.com/mictadlo/genome-portal . Hope it helps.

Thank you in advance,

Best wishes,

Michal

On Thu, Feb 11, 2021 at 2:39 PM Nathan Dunn <ndu...@gmail.com> wrote:

Also, could you point me to your fork in the follow-up email if there is nothing public?   It would be easier than patching the individual files.

Nathan

Begin forwarded message:

Nathan Dunn

unread,
Feb 12, 2021, 10:06:58 AM2/12/21
to "Michał T. Lorenc", apollo

Trying to build locally, I did the gwtc command like this in the Apollo directory:

 ./gradlew gwtc -—info

….
Executing task ':gwtc' (up-to-date check took 0.0 secs) due to:
  Task has not declared any outputs.
[ant:java] Compiling module org.bbop.apollo.gwt.Annotator
[ant:java]    Computing all possible rebind results for 'org.bbop.apollo.gwt.client.LoginDialog.Binder'
[ant:java]       Rebinding org.bbop.apollo.gwt.client.LoginDialog.Binder
[ant:java]          Invoking generator com.google.gwt.uibinder.rebind.UiBinderGenerator
[ant:java]             [ERROR] The prefix "gwt" for element "gwt:HTML" is not bound.
[ant:java]             [ERROR] Error parsing XML (line 39): The prefix "gwt" for element "gwt:HTML" is not bound.
[ant:java]    [ERROR] Errors in 'org/bbop/apollo/gwt/client/LoginDialog.java'
….


Looks like there was some bad XML in the LoginDialog.ui.xml

diff --git a/Apollo-2.6.2/src/gwt/org/bbop/apollo/gwt/client/LoginDialog.ui.xml b/Apollo-2.6.2/src/gwt/org/bbop/apollo/gwt/client/LoginDialog.ui.xml
index 5e4b3b2..b81659b 100644
--- a/Apollo-2.6.2/src/gwt/org/bbop/apollo/gwt/client/LoginDialog.ui.xml
+++ b/Apollo-2.6.2/src/gwt/org/bbop/apollo/gwt/client/LoginDialog.ui.xml
@@ -1,10 +1,14 @@
 <ui:UiBinder xmlns:ui='urn:ui:com.google.gwt.uibinder'
              xmlns:b="urn:import:org.gwtbootstrap3.client.ui"
-             xmlns:bhtml="urn:import:org.gwtbootstrap3.client.ui.html"
+             xmlns:gwt="urn:import:com.google.gwt.user.client.ui"
         >

 

 

     <ui:style>
+        .inline-html {
+            padding: 0px;
+            margin: 10px;
+        }
         .northPanel {
             padding: 0px;
             margin: 10px;





That being said, I would strongly advocate building your fork of Apollo on https://quay.io (or docker hub) and then download the image from there via the docker-compose step. 

Cheers,

Nathan

Michał T. Lorenc

unread,
Feb 14, 2021, 11:06:40 PM2/14/21
to Nathan Dunn, apollo
Hi Nathan,
Thank you, the GWTC error disappeared with your solution. Unfortunately, a new error appeared:

--> Working on Heap::Simple
Fetching http://www.cpan.org/authors/id/T/TH/THOSPEL/Heap-Simple-0.13.tar.gz ... OK
Configuring Heap-Simple-0.13 ... OK
Building and testing Heap-Simple-0.13 ... OK
Successfully installed Heap-Simple-0.13
--> Working on Heap::Simple::Perl
Fetching http://www.cpan.org/authors/id/T/TH/THOSPEL/Heap-Simple-Perl-0.14.tar.gz ... OK
Configuring Heap-Simple-Perl-0.14 ... OK
Building and testing Heap-Simple-Perl-0.14 ... ! Installing Heap::Simple::Perl failed. See /apollo/.cpanm/work/1613360428.14536/build.log for details. Retry with --force to force install it.
FAIL
35 distributions installed
--> Working on Heap::Simple::XS
Fetching http://www.cpan.org/authors/id/T/TH/THOSPEL/Heap-Simple-XS-0.10.tar.gz ... OK
Configuring Heap-Simple-XS-0.10 ... OK
Building Heap-Simple-XS-0.10 ... OK
Successfully installed Heap-Simple-XS-0.10
1 distribution installed
********************************************************************
JBrowse Tools installed in './bin'
Perl dependencies in /apollo/extlib; add to PERL5LIB thusly:
   export PERL5LIB=/apollo/extlib/lib/perl5:$PERL5LIB
********************************************************************

***************************************
NOTE: Please set the memory for your servlet container (tomcat, jetty, etc.) or Apollo may not start correctly: http://genomearchitect.readthedocs.io/en/latest/Troubleshooting.html#suggested-tomcat-memory-settings
***************************************
rm: cannot remove '/apollo/': Permission denied
Removing intermediate container 808a9641ec88
 ---> 910fe647619c
Step 44/48 : USER root
 ---> Running in 01ea573b2b72
Removing intermediate container 01ea573b2b72
 ---> 489255c7918a
Step 45/48 : RUN rm -rf ${CATALINA_BASE}/webapps/* && cp /apollo/apollo*.war ${CATALINA_BASE}/apollo.war
 ---> Running in 5cf024e65983
Removing intermediate container 5cf024e65983
 ---> edd9f963ff79
Step 46/48 : ADD docker-files/createenv.sh /createenv.sh
 ---> 265f4e5fd6dc
Step 47/48 : ADD docker-files/launch.sh /launch.sh
 ---> 6fa53bab2c39
Step 48/48 : CMD "/launch.sh"
 ---> Running in 36ae967cea69
Removing intermediate container 36ae967cea69
 ---> 3ee74b5eacb7

Successfully built 3ee74b5eacb7
Successfully tagged genomeportal_apollo:latest

I uploaded the update to https://github.com/mictadlo/genome-portal. In near future, I will look into quay.io.

What did cause the problem with:
  • Heap::Simple::Perl failed and 
  • rm: cannot remove '/apollo/': Permission denied ?
Thank you in advance,

Best wishes,

Michal

Nathan Dunn

unread,
Feb 14, 2021, 11:30:09 PM2/14/21
to "Michał T. Lorenc", apollo

It’s working for me on your forwarded port 8888 based on my changes and what I believe you have in master:




I’ll remove images and containers (docker system prune -a) and check back tomorrow if its not working. 

Nathan

Nathan Dunn

unread,
Feb 15, 2021, 10:34:06 AM2/15/21
to Nathan Dunn, "Michał T. Lorenc", apollo
I confirmed that rebuilding it after removing the old images and containers works.  

Is it not showing up on port 8888? 

Nathan

On Feb 14, 2021, at 8:30 PM, Nathan Dunn <ndu...@gmail.com> wrote:



It’s working for me on your forwarded port 8888 based on my changes and what I believe you have in master:



<PastedGraphic-1.tiff>

Michał T. Lorenc

unread,
Feb 16, 2021, 9:40:03 PM2/16/21
to Nathan Dunn, Nathan Dunn, apollo
Hi Nathan,
Thank you. Apollo works. However, I was concerned about those new errors :

Nathan Dunn

unread,
Feb 16, 2021, 10:23:03 PM2/16/21
to "Michał T. Lorenc", apollo
• Heap::Simple::Perl failed and

Not sure. It shouldn’t affect the running of Apollo. It ends up installing the prior version successfully so nothing to worry about.

• rm: cannot remove '/apollo/': Permission denied ?

I believe it is trying to delete an empty directory. As it is doing this with rm -rf it “fails” properly.

If it is running then it worked. Sorry the warnings are so verbose.

Nathan

Michał T. Lorenc

unread,
Feb 17, 2021, 8:46:41 PM2/17/21
to Nathan Dunn, apollo
Hi Nathan,
Thank you. Tools menu does not appear in Apollo:
image.png


Furthermore, I executed these commands:

> docker-compose up -d
> docker-compose exec apollo /bin/bash
> cd /data/temporary/apollo_data/37-LAB330/seq
> makeblastdb -in LAB330.fa -parse_seqids -dbtype nucl

What did I miss?

Thank you in advance,

Michal

Nathan Dunn

unread,
Feb 17, 2021, 10:10:55 PM2/17/21
to "Michał T. Lorenc", apollo

I moved it to the new search tab:


Nathan


On Feb 17, 2021, at 5:46 PM, Michał T. Lorenc <m.t.l...@gmail.com> wrote:

Hi Nathan,
Thank you. Tools menu does not appear in Apollo:
Reply all
Reply to author
Forward
0 new messages