My ERDDAP instance stopped working (error 404) even with deafult docker image

[Amedeo Fadini]

Hi everybody,

I was just waiting a bit to customize my ERDDAP instance at

https://erddap.ve.ismar.cnr.it

But in the meantime it stopped working...

It's based on docker image axiom/docker-erddap and deployed on kubernetes, after recovering a problem with persistent volumes it gives onli a 404 error

I deployed a new server based on docker-swarm with the same image and default configuration (no datasets) and it gives the same.

In catalina log the only suspcious line is

At least one JAR was scanned for TLDs yet contained no TLDs.

and in localhost log i see

SEVERE [main] org.apache.tomcat.util.descriptor.web.SecurityConstraint.findUncoveredHttpMethods

How can i fix this? Thenk-you very much for your support

Logs attached

--

Amedeo Fadini

National Research Council - institute of Marine Science venice
CNR-ISMAR www.ismar.cnr.it

[bobsimons2.00]

This is a hard problem for me to diagnose.  I've never seen this problem before.

If the problem is indeed the findUncoveredHttpMethods problem (which I am not convinced of), I would have guessed the problem was something like a mismatch between the version of Tomcat that you are using and the versions that ERDDAP can work with. But I gather ERDDAP was working with this Tomcat before you had the other error, so that is unlikely (unless ERDDAP wasn't working before). You may find a clue by googling the error message and seeing what other people say who had that problem.

Did ERDDAP create a new log.txt in the [bigParentDirectory]/logs directory? If so, the true fatal error will probably be there. Please verify that log.txt isn't being created.

And you're using Axiom's Docker version of ERDDAP, which I have never used. What version of ERDDAP is it? What version of Tomcat are you using?  There are all kinds of possible Docker-related problems that I can't help with. 

Hopefully some Docker users will be able to help diagnose this problem.

[Roy Mendelssohn - NOAA Federal]

Like what Bob said, we have no experience running Docker, plus have no idea how the various parts of the Docker image are set up, so it is hard to diagnose. Hopefully someone from Axiom or IOOS will chime in, or someone else running Docker. I would add that while I have never see the error in the log you mentioned, I have seen the error I see when I go to your site when there is some problem in the proxying from whatever is in front of the Tomcat (I assume either Apache http or Apache apache2 but I don't know) ssl config file, or it can be something in how the port connectors are defined, but this is a wild guess. My other wild guess is that security constraints in tomcat are defined in the $TOMCAT_HOME/conf/web.xml file, perhaps something has been added there. There are things you can set in web.xml that in our configurations we just let Apache handle,

I understand the attraction of Docker in that set up is done for you, the downside is that you don't know how it is set up and it is more difficult to edit and change files. Anyway wish I could be of more help, we just have to hope others with more Docker knowledge will chime in.

-Roy

> --
> You received this message because you are subscribed to the Google Groups "ERDDAP" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to erddap+un...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/erddap/b27f93ca-6513-4255-9086-e640bbb21f70n%40googlegroups.com.

**********************
"The contents of this message do not reflect any position of the U.S. Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
***Note new street address***
110 McAllister Way
Santa Cruz, CA 95060
Phone: (831)-420-3666
Fax: (831) 420-3980
e-mail: Roy.Men...@noaa.gov www: https://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected"
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.

[Amedeo Fadini]

Thank-you very much Bob and Roy, I understand that a bare installation gives more control, but giving the lack of sysadmins in this project I would go  a bit further in this way.

As roy suggests the problem in docker environment is often related to proxying: in fact i have a nginx load balancer that manages ssl certificates, I will start from there..  Maybe  additional parameters in setup.xml are needed to manage http requests that are coming from outside as https.

The  deployment  is based on axiom image with tag "latest" wich now  Tomcat  10.1.0-M14 and Erddap 2.23, but even reverting to Erddap 2.22 nothing changes

Thanks, I will keep you updated here

Amedeo

[Marco Alba]

Hi,

we are using the docker version of ERDDAP a lot, with no problems.

We are using the image "axiom/docker-erddap:2.23-jdk17-openjdk"

I've checked the localhost log and I've the same error, but ERDDAP is working fine, so I think that this error is not relevant.

I've checked the catalina log and is similar.

Maybe is something related to docker-swarm?

Marco

[Amedeo Fadini]

Thank-you Marco, this is very helpful.
I was pretty sure that the 404 error is related to dataset availability or  access rights but after 2 weeks struggling with cephs volumes and mounts (files are visible from inside the container) I started wondering about some other issue about the web app.

Is definitely a problem with paths and mount points (both in kubernetes/cephs and docker-swarm)  and I will find the way to solve.

Amedeo