Difficulties with docker app install

[Matt Covington]

I realize I'm on the bleeding edge of current development in the repo,
but I have been trying to get my app working using docker-compose and
a Dockerfile that inherits from the tethys-core image. The new
development branch docs on this gave me a good start, but I haven't
quite been able to figure out how to really make an app work. What
might be really helpful is a working example case.

After running into problems with getting my own app working this way,
I decided to take a step back and try to get the dams_inventory app
from the tutorial working, with the thought that it might be a start
on a useful example for the docs. Luckily, I'm running into exactly
the same errors in the dams example as in my own app, suggesting I'm
probably making the same mistake.

Specifically, I'm having trouble bringing up the database for the app
(using syncstores). When I try to do this in the salt script, I get an
error that the django module isn't known. When I run a bash session on
the container and try running tethys syncstores there, I get an error
that says this:

FATAL: database "tethys_default" does not exist

The output from docker-compose also has some errors early on, which
seem to relate to the tethys db. I'm guessing that somehow it hasn't
been initialized correctly. A clip from that error message is below.

I put my code into a fork of the dams tutorial (advanced-solution). It's here:

https://github.com/speleophysics/tethysapp-dam_inventory/tree/docker-example

Any suggestions on where I might be going wrong?

Thanks,
Matt

--------------------------
dams-docker | db:5432 - accepting connections
dams-docker | tput: No value for $TERM and no -T specified
dams-docker | tput: No value for $TERM and no -T specified
dams-docker | - Enforcing start state... (This might take a bit)
dams-docker | tput: No value for $TERM and no -T specified
dams-docker | [WARNING ]
/usr/lib/python3/dist-packages/salt/states/file.py:278:
DeprecationWarning: Using or importing the ABCs from 'collections'
instead of from 'collections.abc' is deprecated, and in 3.8 it will
stop working
dams-docker | from collections import Iterable, Mapping, defaultdict
dams-docker |
db_1 | 2020-06-29 18:40:32.538 UTC [95] ERROR: relation
"tethys_apps_tethysapp" does not exist at character 483
db_1 | 2020-06-29 18:40:32.538 UTC [95] STATEMENT: SELECT
"tethys_apps_tethysapp"."id", "tethys_apps_tethysapp"."package",
"tethys_apps_tethysapp"."name", "tethys_apps_tethysapp"."description",
"tethys_apps_tethysapp"."enable_feedback",
"tethys_apps_tethysapp"."feedback_emails",
"tethys_apps_tethysapp"."tags", "tethys_apps_tethysapp"."index",
"tethys_apps_tethysapp"."icon", "tethys_apps_tethysapp"."root_url",
"tethys_apps_tethysapp"."color", "tethys_apps_tethysapp"."enabled",
"tethys_apps_tethysapp"."show_in_apps_library" FROM
"tethys_apps_tethysapp"
db_1 | 2020-06-29 18:40:32.551 UTC [95] ERROR: relation
"tethys_apps_tethysapp" does not exist at character 483
db_1 | 2020-06-29 18:40:32.551 UTC [95] STATEMENT: SELECT
"tethys_apps_tethysapp"."id", "tethys_apps_tethysapp"."package",
"tethys_apps_tethysapp"."name", "tethys_apps_tethysapp"."description",
"tethys_apps_tethysapp"."enable_feedback",
"tethys_apps_tethysapp"."feedback_emails",
"tethys_apps_tethysapp"."tags", "tethys_apps_tethysapp"."index",
"tethys_apps_tethysapp"."icon", "tethys_apps_tethysapp"."root_url",
"tethys_apps_tethysapp"."color", "tethys_apps_tethysapp"."enabled",
"tethys_apps_tethysapp"."show_in_apps_library" FROM
"tethys_apps_tethysapp" WHERE "tethys_apps_tethysapp"."package" =
'dam_inventory'
db_1 | 2020-06-29 18:40:32.580 UTC [95] ERROR: relation
"django_content_type" does not exist at character 106
db_1 | 2020-06-29 18:40:32.580 UTC [95] STATEMENT: SELECT
"django_content_type"."id", "django_content_type"."app_label",
"django_content_type"."model" FROM "django_content_type" WHERE
("django_content_type"."app_label" = 'tethys_apps' AND
"django_content_type"."model" = 'tethysapp')
db_1 | 2020-06-29 18:40:32.583 UTC [95] ERROR: relation
"tethys_quotas_resourcequota" does not exist at character 24
db_1 | 2020-06-29 18:40:32.583 UTC [95] STATEMENT: SELECT (1) AS
"a" FROM "tethys_quotas_resourcequota" WHERE
"tethys_quotas_resourcequota"."codename" = 'user_workspace_quota'
LIMIT 1


-------------------------
Matt Covington
Associate Professor
Department of Geosciences
340 N. Campus Dr.
216 Gearhart Hall
University of Arkansas
Fayetteville, AR 72701

webpage - http://www.speleophysics.com/
-------------------------

[Rohit Khattar]

HI Matt, 

I was able to reproduce your issue by using your branch. I was able to spend some time last night trying to successfully run the container but it seems like the main issue is that for some reason, the tethys db create step doesn't complete properly, and hence the tethys_default database is missing. However, pulling the latest master branch of tethys and running a docker container from that works successfully as I have recently been working on that. I feel there might be a gap between the versions of code in the latest master image of Tethys and github? 

Maybe @Nathan or others here will have more information about this and can point us in the right direction. 

Thanks

Rohit Khattar

Research Assistant - Hydroinformatics Lab
Brigham Young University - Provo
Utah

[speleo...@gmail.com]

Hi Rohit,

  Thanks for looking into it. I tried creating a new tagged docker image (docker build --no-cache) from the latest commit of the master branch. I still got similar behavior (same errors). I went into the running db container and looked at the tables. There were two tables created by tethys: tethys_platform (owner tethys_default) and tethys_super (owner tethys_super). I wondered whether this mismatch in the tethys_default db and user name might somehow be causing problems (e.g. if the db name tethys_default was hard-coded somewhere or somehow overridden after being set by the compose file). The name tethys_platform was coming from my compose file (I saw it somewhere as a default). I also see that the production docs show using this name for the tethys portal db.  

I tried changing my compose file so that the database is named tethys_default instead (same as the owner). When I run "docker-compose up" the error messages are essentially the same (syncstores fails complaining about no django module). However, now if I run an interactive shell in the dams-docker container and try to run syncstores from there, I get a slightly different error than before. Now it says:

tethys_apps.exceptions.PersistentStorePermissionError: Database user "tethys_default" has insufficient permissions to create the persistent store database "primary_db": must have CREATE DATABASES permission at a minimum.

If I run psql and add permissions to the tethys_default user, then syncstores succeeds and the app runs! This seems like progress, but I'm still not sure where things are going wrong in the automated setup occurring in the docker files and salt scripts. 

Thanks,

Matt

[speleo...@gmail.com]

Ok. I think I am zeroing in on the problem. I was creating my app database in the salt script using the tethys_db_user rather than tethys_db_superuser. When I create the db using the db superuser, then I can run syncstores in a bash session in the tethys container and everything works (without monkeying with db user permissions). 

Now...the lingering issue is that syncstores does not work from the salt script. I'm really scratching my head as to why. It still throws the error that there is no django module.  Seems almost like the tethys env isn't activating correctly, but I activate it in exactly the same way as in the other commands (e.g. the salt cmds for creating and linking the app db). What's different about the syncstores command? The relevant code is near the bottom of tethyscore.sls. I've pushed out today's changes into my dockerized dams app branch on github. 

[speleo...@gmail.com]

I think I have it working now. If I specify a bash shell for the syncstores salt command, then it works. I don't really understand why...but it works. 

This working version is pushed out to my branch on github. I'll continue to clean it up and add some more comments about how it might be customized or about things that were confusing to me along the way. I'd be happy to help get this dockerized advanced-solution dams_inventory app somehow linked into the docs. I think it would help others trying to figure out how to run their app in a docker container, particularly using docker-compose to bring up tethys and a db within linked containers. 

[Rohit Khattar]

Hello Matt, 

Glad to hear that you got it working. It makes sense how the database user might be the issue so that the DB is setup properly. As far as the bash shell requirement for the synctore command is concerned, my guess is that since the docker image is based of a miniconda image which activates conda in the shell environment for bash automatically, we need to use the bash shell if we wan't access to conda and its commands. That's probably why even though you might have tried to activate the environment it might not have actually done it and hence never found Django. 

I like this idea of having it as a tutorial within the tethys docs. What do you guys think @Tethys Committee? 

Thanks

Rohit

[Michael S]

I've suggested Tethys Docker as a future tutorial in the pinged post from Nathan from April.