Tethys 2.0 Production Installation

[Sarva Pulla]

Hi Everyone,

I am trying to setup a production instance of the stable version (2.0). I ran the commands as shown in the Production installation docs, but I don't see any settings.py file in src/tethys_apps/. I do see a settings.py file in src/tethys_portal. 

<The following is the command that I used>

 bash install_tethys.sh --skip-tethys-install -allowed-host 35.166.72.161 --db-username tethys_default --db-password pass --db-port 5436 --superuser admin --super-email pu...@byu.edu --super-pass pass --production

(This is an example only, the actual password is different)

I tried running the --production flag from the get go and tried it after installing tethys as well. In both cases it didn't generate a new settings.py file in tethys_apps. I know that in 1.4 the production installation had some extra steps on installing and transferring stuff over to apache. I am assuming that task was automated, but it keeps saying permission is denied. I am attaching it in a file as it's pretty long. Or do I just need wait as the instructions still need to be updated?

(tethys) ubuntu@ip-172-31-45-47:~$ bash install_tethys.sh --skip-tethys-install -allowed-host 35.166.72.161  --db-username tethys_default --db-password pass --db-port 5436 --superuser admin --super-email pu...@byu.edu --super-pass pass --production
Ignoring unrecognized option: -allowed-host
Ignoring unrecognized option: 35.166.72.161
Ignoring unrecognized option: --super-email
Ignoring unrecognized option: pu...@byu.edu
Ignoring unrecognized option: --super-pass
Ignoring unrecognized option: pass
Production installation requires some commands to be run with sudo. Please enter password:
Installing Tethys Production Server...
Hit:1 http://us-west-2.ec2.archive.ubuntu.com/ubuntu xenial InRelease
Hit:2 http://us-west-2.ec2.archive.ubuntu.com/ubuntu xenial-updates InRelease
Get:3 http://us-west-2.ec2.archive.ubuntu.com/ubuntu xenial-backports InRelease [102 kB]
Get:4 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
Hit:5 https://download.docker.com/linux/ubuntu xenial InRelease
Fetched 204 kB in 0s (412 kB/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done
20 packages can be upgraded. Run 'apt list --upgradable' to see them.
Reading package lists... Done
Building dependency tree
Reading state information... Done
nginx is already the newest version (1.10.0-0ubuntu0.16.04.4).
0 upgraded, 0 newly installed, 0 to remove and 20 not upgraded.
rm: cannot remove '/etc/nginx/sites-enabled/default': No such file or directory
/home/ubuntu/tethys/miniconda/envs/tethys/etc/conda/deactivate.d/tethys-deactivate.sh: line 6: unalias: tethys_start_db: not found
/home/ubuntu/tethys/miniconda/envs/tethys/etc/conda/deactivate.d/tethys-deactivate.sh: line 7: unalias: tstartdb: not found
/home/ubuntu/tethys/miniconda/envs/tethys/etc/conda/deactivate.d/tethys-deactivate.sh: line 8: unalias: tethys_stop_db: not found
/home/ubuntu/tethys/miniconda/envs/tethys/etc/conda/deactivate.d/tethys-deactivate.sh: line 9: unalias: tstopdb: not found
/home/ubuntu/tethys/miniconda/envs/tethys/etc/conda/deactivate.d/tethys-deactivate.sh: line 10: unalias: tms: not found
/home/ubuntu/tethys/miniconda/envs/tethys/etc/conda/deactivate.d/tethys-deactivate.sh: line 11: unalias: tstart: not found
/home/ubuntu/tethys/miniconda/envs/tethys/etc/conda/deactivate.d/tethys-deactivate.sh: line 14: unalias: tethys_user_own: not found
/home/ubuntu/tethys/miniconda/envs/tethys/etc/conda/deactivate.d/tethys-deactivate.sh: line 15: unalias: tuo: not found
/home/ubuntu/tethys/miniconda/envs/tethys/etc/conda/deactivate.d/tethys-deactivate.sh: line 16: unalias: tethys_server_own: not found
/home/ubuntu/tethys/miniconda/envs/tethys/etc/conda/deactivate.d/tethys-deactivate.sh: line 17: unalias: tso: not found
/home/ubuntu/tethys/miniconda/envs/tethys/etc/conda/deactivate.d/tethys-deactivate.sh: line 18: unalias: tethys_server_restart: not found
/home/ubuntu/tethys/miniconda/envs/tethys/etc/conda/deactivate.d/tethys-deactivate.sh: line 19: unalias: trs: not found
/home/ubuntu/tethys/miniconda/envs/tethys/etc/conda/deactivate.d/tethys-deactivate.sh: line 22: unalias: tethys_user_own: not found
/home/ubuntu/tethys/miniconda/envs/tethys/etc/conda/deactivate.d/tethys-deactivate.sh: line 23: unalias: tuo: not found
/home/ubuntu/tethys/miniconda/envs/tethys/etc/conda/deactivate.d/tethys-deactivate.sh: line 24: unalias: tethys_server_own: not found
/home/ubuntu/tethys/miniconda/envs/tethys/etc/conda/deactivate.d/tethys-deactivate.sh: line 25: unalias: tso: not found
/home/ubuntu/tethys/miniconda/envs/tethys/etc/conda/deactivate.d/tethys-deactivate.sh: line 26: unalias: tethys_server_restart: not found
/home/ubuntu/tethys/miniconda/envs/tethys/etc/conda/deactivate.d/tethys-deactivate.sh: line 27: unalias: trs: not found
pg_ctl: another server might be running; trying to start server anyway
server starting
Waiting for databases to startup...
Fetching package metadata ...........
Solving package specifications: .
 
# All requested packages already installed.
# packages in environment at /home/ubuntu/tethys/miniconda/envs/tethys:
#
uwsgi                     2.0.12                   py27_0    conda-forge
Loading Tethys Apps...
Tethys Apps Loaded:
Traceback (most recent call last):
  File "/home/ubuntu/tethys/miniconda/envs/tethys/bin/tethys", line 11, in <module>
    load_entry_point('tethys-platform', 'console_scripts', 'tethys')()
  File "/home/ubuntu/tethys/src/tethys_apps/cli/__init__.py", line 294, in tethys_command
    args.func(args)
  File "/home/ubuntu/tethys/src/tethys_apps/cli/gen_commands.py", line 192, in generate_command
    with open(destination_path, 'w') as f:
IOError: [Errno 13] Permission denied: '/home/ubuntu/tethys/src/tethys_portal/settings.py'
Loading Tethys Apps...
Tethys Apps Loaded:
Traceback (most recent call last):
  File "/home/ubuntu/tethys/miniconda/envs/tethys/bin/tethys", line 11, in <module>
    load_entry_point('tethys-platform', 'console_scripts', 'tethys')()
  File "/home/ubuntu/tethys/src/tethys_apps/cli/__init__.py", line 294, in tethys_command
    args.func(args)
  File "/home/ubuntu/tethys/src/tethys_apps/cli/gen_commands.py", line 192, in generate_command
    with open(destination_path, 'w') as f:
IOError: [Errno 13] Permission denied: '/home/ubuntu/tethys/src/tethys_portal/tethys_nginx.conf'
Loading Tethys Apps...
Tethys Apps Loaded:
Traceback (most recent call last):
  File "/home/ubuntu/tethys/miniconda/envs/tethys/bin/tethys", line 11, in <module>
    load_entry_point('tethys-platform', 'console_scripts', 'tethys')()
  File "/home/ubuntu/tethys/src/tethys_apps/cli/__init__.py", line 294, in tethys_command
    args.func(args)
  File "/home/ubuntu/tethys/src/tethys_apps/cli/gen_commands.py", line 192, in generate_command
    with open(destination_path, 'w') as f:
IOError: [Errno 13] Permission denied: '/home/ubuntu/tethys/src/tethys_portal/tethys_uwsgi.yml'
Loading Tethys Apps...
Tethys Apps Loaded:
Traceback (most recent call last):
  File "/home/ubuntu/tethys/miniconda/envs/tethys/bin/tethys", line 11, in <module>
    load_entry_point('tethys-platform', 'console_scripts', 'tethys')()
  File "/home/ubuntu/tethys/src/tethys_apps/cli/__init__.py", line 294, in tethys_command
    args.func(args)
  File "/home/ubuntu/tethys/src/tethys_apps/cli/gen_commands.py", line 192, in generate_command
    with open(destination_path, 'w') as f:
IOError: [Errno 13] Permission denied: '/home/ubuntu/tethys/src/tethys_portal/tethys.uwsgi.service'
Loading Tethys Apps...
Tethys Apps Loaded:
Loading Tethys Apps...
Tethys Apps Loaded:
INFO: Linking static and public directories of apps to "/home/ubuntu/tethys/static".
Loading Tethys Apps...
Tethys Apps Loaded:
 
0 static files copied to '/home/ubuntu/tethys/static', 1451 unmodified.
Loading Tethys Apps...
Tethys Apps Loaded:
INFO: Moving workspace directories of apps to "/home/ubuntu/tethys/workspaces" and linking back.
mkdir: cannot create directory ‘/var/log/uwsgi’: File exists
ln: failed to create symbolic link '/etc/nginx/sites-enabled/tethys_nginx.conf': File exists
The unit files have no [Install] section. They are not meant to be enabled
using systemctl.
Possible reasons for having this kind of units are:
1) A unit may be statically enabled by being symlinked from another unit's
   .wants/ or .requires/ directory.
2) A unit's purpose may be to act as a helper for some other unit which has
   a requirement dependency on it.
3) A unit may be started when needed via activation (socket, path, timer,
   D-Bus, udev, scripted systemctl call, ...).
/home/ubuntu/tethys/miniconda/envs/tethys/etc/conda/deactivate.d/tethys-deactivate.sh: line 19: unalias: trs: not found
/home/ubuntu/tethys/miniconda/envs/tethys/etc/conda/deactivate.d/tethys-deactivate.sh: line 22: unalias: tethys_user_own: not found
/home/ubuntu/tethys/miniconda/envs/tethys/etc/conda/deactivate.d/tethys-deactivate.sh: line 23: unalias: tuo: not found
/home/ubuntu/tethys/miniconda/envs/tethys/etc/conda/deactivate.d/tethys-deactivate.sh: line 24: unalias: tethys_server_own: not found
/home/ubuntu/tethys/miniconda/envs/tethys/etc/conda/deactivate.d/tethys-deactivate.sh: line 25: unalias: tso: not found
/home/ubuntu/tethys/miniconda/envs/tethys/etc/conda/deactivate.d/tethys-deactivate.sh: line 26: unalias: tethys_server_restart: not found
/home/ubuntu/tethys/miniconda/envs/tethys/etc/conda/deactivate.d/tethys-deactivate.sh: line 27: unalias: trs: not found

 

[sdc50]

Sarva,

I have a couple of comments that will hopefully help.

• You'll notice at the beginning of the output it says that a bunch of options aren't recognized. That could possibly be throwing a bunch of things off. The command options that you put were:

bash install_tethys.sh --skip-tethys-install -allowed-host 35.166.72.161 --db-username tethys_default --db-password pass --db-port 5436 --superuser admin --super-email pulla@byu.edu --super-pass pass --production

The correct options would be:

bash install_tethys.sh --skip-tethys-install --allowed-host 35.166.72.161 --db-username tethys_default --db-password pass --db-port 5436 --superuser admin --superuser-email pulla@byu.edu --superuser-pass pass --production

(notice the two dashes in the --allowed-host option and the user in the --superuser-email and --superuser-pass options)

•  I thought that ignoring unrecognized options would be handy, but on second thought maybe it causes more harm than good. Maybe I need to make the script exit if it encounters and unrecognized option to give the user a chance to correct it.

• The first time you ran the script with the --production option it would have changed the owner of a bunch of files in TETHYS_HOME to the Nginx user. Before running the script again you'll need to change them back with the following command:
  sudo chown -R ${USER} ${TETHYS_HOME}
  
  NOTE: this assumes that you have the tethys environment activated

Try changing the ownership and rerunning the script with the correct options and let me know how that goes.

A few other notes:

• The settings.py file has now been moved from tethys_apps to tethys_portal
• Nginx and uWSGI are now used to serve Tethys instead of Apache

[Sarva Pulla]

Hi Scott,

I tried what you suggested and now I end up with this error. Is there anything else that needs to happen such as restarting the nginx and uwsgi before expecting the portal to show up at the ip? Seems like its failing to create a symbolic link. I did change the ownership. This time I just installed tethys without the --production tag and then changed the ownership followed by running the production commands as mentioned earlier in this thread. 

This is a bit off topic, but this time I didn't install docker. Does installing geoserver and postgis docker containers throw off the production instance? Also, when I did install geoserver through docker it comes up with a new port every time. This is happening in development instance and the production instance.

There are a few things that need to be changed in the documentation: The documentation still says that the settings.py file is in tethys_apps (http://docs.tethysplatform.org/en/latest/installation/production/installation.html). It's definitely a user error on my part but for allowed-host parameter it is as follows "-allowed-host" instead of "--allowed-host"

[Scott Christensen]

I'm not sure what is going on in your case. What happens when you go to the IP or hostname that you set as the allowed-host? Do you just get the Nginx default site, or do you get a 504 gateway error?

I don't think I can help much with the geoserver error. It seems like it might be related to issue # 277 (https://github.com/tethysplatform/tethys/issues/277) 

Thanks for finding the errors in the docs. I've made the changes and they should get pushed to the live version of the docs soon.

--
You received this message because you are subscribed to the Google Groups "Tethys Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tethysplatfor...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tethysplatform/3a6f623c-651a-4dd6-be35-569709860225%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Scott D. Christensen, PhD

Research Civil Engineer

Information Technology Laboratory  

US Army Corps of Engineers

 

[Sarva Pulla]

It just doesn't connect. Nothing shows up. I've tried several ways to get it going but it keeps ending at this error. I even started from scratch on a new instance and it still kept showing this error.

--

Sarva Pulla

[Sarva Pulla]

Scott,

I am hoping that this will be my last installation question. I am trying to install a new app in production, every thing went well until when I tried to syncstores. I get an error saying that the tethys_default user has insufficient permissions to create a persistent store. Do I need to change anything the postgresql.conf file or what would you suggest that I do to change permissions? Thanks!

--

Sarva Pulla

[Sarva Pulla]

Oops, forgot to attach a screenshot.

--

Sarva Pulla

[sdc50]

Sarva, 

The error you are getting is referring to the database user that you specify in the app settings (which is now done through the portal). When you specify the persistent store database setting you have to give it the credentials to use. Assuming you are using the postgis docker, these would be the credentials that you specified when you ran `tethys docker init`.  If you just used the default options for the docker then there should be two users tethys_default and tethys_super. The tethys_super user has the permissions to create a new database, so that's what I've always put. I don't know if it would be possible/more secure to put tethys_super and then run syncstores to create the database and then change it to tethys_default. @swainn would be able to provide better advise as far as best practices for the DB settings.