Questions About Installing Mayan 2.6

[Mark Phillips]

I read the Docker install instructions for Mayan 2.6. Seems easy, but I have a couple of questions - 

1. After reading some about Docker, as I understand it, we don't need to create a virtualenv for the Docker image. Is that correct? If not, I am not sure how to combine the docker image with a virtualenv.

2. I see the environment variables make it easier to install with a non-SQLite backend. Are these environment variables only for the install process, or do then need to be a permanent feature? In other words, do I set the environment variables in the current shell, or do they need to go in my .profile so every time I log in they are present?

Thanks!

 Mark

[Roberto Rosario]

Yes, it is a less involved process and all dependencies are taken care of.

1. Correct, there is no need to use Python's virtualenv since the Docker container itself is already isolated from the OS.

2. Environment variables are only needed when the container is first created, afterwards it can be stopped and restarted and their values will persist internally in the container. 

[Mark Phillips]

Regarding environment variables, I tried to install mayan 2.6.1 two ways, and cannot get it to work. I am trying to use docker to create mayan 2.6.1 with mysql as per the instructions - https://gitlab.com/mayan-edms/mayan-edms-docker/tree/2.6.1.

1. I created a root shell (Ubuntu 14.01) and created the following shell environment variables using the following script - 

#!/bin/sh

MAYAN_DATABASE_DRIVER=\'django.db.backends.mysql\'

MAYAN_DATABASE_NAME=\'memorabilia\'

MAYAN_DATABASE_USER=\'mayan\'

MAYAN_DATABASE_PASSWORD=\'ooTh7ietieT0aKea\'

MAYAN_DATABASE_HOST=\'localhost\'

MAYAN_DATABASE_PORT=\'3306\'

export MAYAN_DATABASE_DRIVER

export MAYAN_DATABASE_NAME

export MAYAN_DATABASE_USER

export MAYAN_DATABASE_PASSWORD

export MAYAN_DATABASE_HOST

export MAYAN_DATABASE_PORT

echo $MAYAN_DATABASE_DRIVER

echo $MAYAN_DATABASE_NAME

echo $MAYAN_DATABASE_USER

echo $MAYAN_DATABASE_PASSWORD

echo $MAYAN_DATABASE_HOST

echo $MAYAN_DATABASE_PORT

The mayan instructions seem to indicate that the shell environment variables need to single quotes around them. Is this correct, or are then unquoted values? It the single quotes are not needed, I did not get an error message about this.

I then followed the instruction referenced above to pull mayan 2.6.1 and then run mayan 2.6.1. I was able to access mayan from http://localhost, but after looking into the settings.py file for this container, I found that it was using sqlite and not mysql as the db backend.

2. I removed the above container and image and started again. I pulled mayan 2.6.1, and created an env.list file as shown in the docker instructions for passing environemnt variables to docker:

MAYAN_DATABASE_DRIVER=django.db.backends.mysql

MAYAN_DATABASE_NAME=memorabilia

MAYAN_DATABASE_USER=mayan

MAYAN_DATABASE_PASSWORD=ooTh7ietieT0aKea

MAYAN_DATABASE_HOST=localhost

MAYAN_DATABASE_PORT=3306

I then ran mayan 2.6.1 with this command, passing the environment variables as shown in the docker user guide

docker run -d --name mayan-edms --restart=always -p 9005:80 --env-file ./env.list -v mayan_data:/var/lib/mayan mayanedms/mayanedms:2.6.1

The container never started, or more specifically, it would start and stop continuously. I removed the container and image as it was broken.

How do I create a mayan 2.6.1 instance using docker and using mysql as the backend database? It does not appear the environment variables are working Or I don't understand how to use them!), although the terminology is rather confusing. Are they docker environment variables (as in #2 above), or environment variables from the shell where docker is run (as in #1 above)?

Thanks!

Mark

--

---
You received this message because you are subscribed to the Google Groups "Mayan EDMS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mayan-edms+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Roberto Rosario]

Made a blog post with the full set of steps to use a MySQL container with the Mayan container: http://www.mayan-edms.org/post/deploy-mayan-docker-mysql/

In summary you are configuring the Mayan container but are not launching a separate MySQL container.

I tried to explain as much as possible in the post starting with some Docker basics. Hope you enjoy it and helps you.

[Mark Phillips]

Thanks!!!!!! A great write-up!!! As I was reading through the docker manual I was beginning to suspect that something was missing, but I found a partial mysq installation in the container (accessing it through the shell). But it was only a partial installation, so I tried to repair as was going to modify the settings.py to use mysql, but then your blog post arrived.

A couple of warnings from the mysql installation:

* The root password for mysql was not set

* Gtid table is not ready to be used. Table 'mysql.gtid_executed' cannot be opened.

root@orca:/home/mark/mayan2-docker# docker run -d --name mayan-mysql --restart=always --env-file env.list -v mayan_mysql:/var/lib/mysql --net=mayan mysql:latest

Unable to find image 'mysql:latest' locally

latest: Pulling from library/mysql

ad74af05f5a2: Pull complete 

0639788facc8: Pull complete 

de70fa77eb2b: Pull complete 

724179e94999: Pull complete 

50c77fb16ba6: Pull complete 

d51f459239fb: Pull complete 

937bbdd4305a: Pull complete 

35369f9634e1: Pull complete 

f6016aab25f1: Pull complete 

5f1901e920da: Pull complete 

fdf808213c5b: Pull complete 

Digest: sha256:96edf37370df96d2a4ee1715cc5c7820a0ec6286551a927981ed50f0273d9b43

Status: Downloaded newer image for mysql:latest

537501f942aad43f956343327ac3853021b62d44b044b92dbc3206e6db229156

root@orca:/home/mark/mayan2-docker# docker logs mayan-mysql

Initializing database

2017-07-27T15:13:35.448969Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).

2017-07-27T15:13:36.931315Z 0 [Warning] InnoDB: New log files created, LSN=45790

2017-07-27T15:13:37.257390Z 0 [Warning] InnoDB: Creating foreign key constraint system tables.

2017-07-27T15:13:37.366858Z 0 [Warning] No existing UUID has been found, so we assume that this is the first time that this server has been started. Generating a new UUID: 2a28206b-72de-11e7-ad79-0242ac120002.

2017-07-27T15:13:37.399268Z 0 [Warning] Gtid table is not ready to be used. Table 'mysql.gtid_executed' cannot be opened.

2017-07-27T15:13:37.399779Z 1 [Warning] root@localhost is created with an empty password ! Please consider switching off the --initialize-insecure option.

The mayan installation went well - no errors.

root@orca:/home/mark/mayan2-docker# docker run -d --name mayan-edms --restart=always --env-file env.list -v mayan_data:/var/lib/mayan --net=mayan -p 9005:80 mayanedms/mayanedms:2.6.4-3

Unable to find image 'mayanedms/mayanedms:2.6.4-3' locally

2.6.4-3: Pulling from mayanedms/mayanedms

e0a742c2abfd: Pull complete 

486cb8339a27: Pull complete 

dc6f0d824617: Pull complete 

4f7a5649a30e: Pull complete 

672363445ad2: Pull complete 

efb1cec07b5e: Pull complete 

a801f5e9adaa: Pull complete 

8eb23d7c826c: Pull complete 

92fbf659e773: Pull complete 

c0f0219c11c6: Pull complete 

e77aab94a7a9: Pull complete 

4741b155d5ce: Pull complete 

cdf3d5b0c8fe: Pull complete 

cf2f940f4cc5: Pull complete 

a22f7b8219aa: Pull complete 

b06c1d87e40c: Pull complete 

ba3f50217ec1: Pull complete 

427849b0be68: Pull complete 

4284e8fc0cb3: Pull complete 

99227ced2705: Pull complete 

60a3086c3d00: Pull complete 

5ce3b814e326: Pull complete 

7f69be0f8504: Pull complete 

d43efc48a70f: Pull complete 

6b060a72026e: Pull complete 

c42c5eba1b9b: Pull complete 

f195e5af15a9: Pull complete 

2b37d227a29d: Pull complete 

5113bad4a09d: Pull complete 

2ec6a0237cc8: Pull complete 

9fd5f74e7daf: Pull complete 

68b2e131502b: Pull complete 

5f64c354aef1: Pull complete 

7a1d9773d33e: Pull complete 

bc702ec47793: Pull complete 

Digest: sha256:de00429beb8736db87ad41ad1645787e5330beae11aed1479323a79ef40d1231

Status: Downloaded newer image for mayanedms/mayanedms:2.6.4-3

51cbdb670142c55aa764b4bf018ce9377e6d4a26229bcf60e04ba2c29bf7a376

Except I don't know the internally generated mayan admin password. The login screen did not present the admin password at the first login, so I have no way to login to the mayan installation. The forgot password email link does not work because there is no email set up for the docker container. How do I get the password for the mayan admin login?

Thanks!

Mark

[Roberto Rosario]

Thanks! I will try to do a blog post a month to complement the forum.

- The root password for the MySQL container is set using a separate variable called MYSQL_ROOT_PASSWORD. It should not be needed since the container created the database, user, and grants all the permissions. 

- I don't know that the 'mysql.gtid_executed' table is or does. Some containers spin up but are still not ready to be used. I add a 5 or 10 second sleep after services containers. This is not needed using Docker Compose since you can specify dependencies using the health check system. 

An automatic admin with a random password should have been created. I gave it a try and it works well for me, but yours is the second report of this issue I've seen so there is something being skipped during initialization. I will write a test for the container and find out.

In the meantime you can do:

docker exec -ti mayan-edms /bin/bash

to get inside the mayan container and the do:

mayan-edms.py changepassword admin

to set a new password until I get to the bottom of the autoadmin issue.

[Mark Phillips]

Roberto,

I used the MYSQL_ROOT_PASSWORD as you listed in your blog. Here is my env.list file

# MySQL container

MYSQL_ROOT_PASSWORD=goF3duufAa1aajea

MYSQL_PASSWORD=kuL3caipAeree3Xe

MYSQL_DATABASE=memorabilia

MYSQL_USER=mayan_user

# Mayan container

MAYAN_DATABASE_DRIVER=django.db.backends.mysql

MAYAN_DATABASE_NAME=memorabilia

MAYAN_DATABASE_USER=mayan_user

MAYAN_DATABASE_PASSWORD=kuL3caipAeree3Xe

MAYAN_DATABASE_HOST=mayan-mysql

MAYAN_DATABASE_PORT=3306

Got these error messages - 

root@orca:/home/mark/mayan2-docker# docker exec -ti mayan-edms /bin/bash

root@51cbdb670142:/# mayan-edms.py changepassword admin

mayan.settings <9519> [INFO] "<module>() Good local.py found. Using user settings."

CommandError: user 'admin' does not exist

root@51cbdb670142:/# find . -name gtid_executed

root@51cbdb670142:/# 

No user admin was created with the mayan install.

I googled the mysql.gtid_executed error, found this old bug page (https://bugs.mysql.com/bug.php?id=78495). It seems to have something to do with SSL connections. The two files gtid_executed.frm and gtid_executed.ibd do not seem to exist in the container, so there is something missing from the mysql docker recipe.

Mark

[Roberto Rosario]

The MySQL error doesn't seem to be critical if I understood the bug page explanation.

If no admin was created them use the command:

mayan-edms.py createautoadmin

I have not been able to recreate this issue. Can you post the output of the command:

docker logs mayan-edms

I'm interested in this segment:

  Applying sources.0016_auto_20170630_2040... OK

  Applying statistics.0001_initial... OK

  Applying tags.0001_initial... OK

  Applying tags.0002_tag_selection... OK

  Applying tags.0003_remove_tag_color... OK

  Applying tags.0004_auto_20150717_2336... OK

  Applying tags.0005_auto_20150718_0616... OK

  Applying tags.0006_documenttag... OK

  Applying tags.0007_auto_20170118_1758... OK

Superuser created successfully.    <=================== !!!!!

2017-07-28 04:46:17,250 CRIT Supervisor running as root (no user in config file)

2017-07-28 04:46:17,250 WARN Included extra file "/etc/supervisor/conf.d/beat.conf" during parsing

2017-07-28 04:46:17,250 WARN Included extra file "/etc/supervisor/conf.d/nginx.conf" during parsing

2017-07-28 04:46:17,250 WARN Included extra file "/etc/supervisor/conf.d/redis.conf" during parsing

2017-07-28 04:46:17,250 WARN Included extra file "/etc/supervisor/conf.d/uwsgi.conf" during parsing

2017-07-28 04:46:17,250 WARN Included extra file "/etc/supervisor/conf.d/workers.conf" during parsing

2017-07-28 04:46:17,261 INFO RPC interface 'supervisor' initialized

2017-07-28 04:46:17,261 CRIT Server 'unix_http_server' running without any HTTP authentication checking

2017-07-28 04:46:17,261 INFO supervisord started with pid 1

Normally it should show "Superuser created successfully." I need to see what error is being raised at the moment the 'createautoadmin' command is execute after the database has been migrated.

Thanks!

[Mark Phillips]

Roberto,

I have attached the log file you requested. There is no "Superuser created successfully" entry, and no error around the log segment you sent in your email

  Applying sources.0015_auto_20170206_0835... OK

  Applying sources.0016_auto_20170630_2040... OK

  Applying statistics.0001_initial... OK

  Applying tags.0001_initial... OK

  Applying tags.0002_tag_selection... OK

  Applying tags.0003_remove_tag_color... OK

  Applying tags.0004_auto_20150717_2336... OK

  Applying tags.0005_auto_20150718_0616... OK

  Applying tags.0006_documenttag... OK

  Applying tags.0007_auto_20170118_1758... OK

Deleting 'suit/css/suit.css'

Deleting 'suit/css/filer.css'

Deleting 'suit/css/djangocms.css'

Deleting 'suit/less/mixins.less'

Deleting 'suit/less/suit.less'

Deleting 'suit/less/left-column.less'

Deleting 'suit/less/footer.less'

Deleting 'suit/less/print.less'

Deleting 'suit/less/variables.less'

and then much later (still no errors...)

686 static files copied to '/usr/local/lib/python2.7/dist-packages/mayan/media/static'.

2017-07-27 16:05:49,339 CRIT Supervisor running as root (no user in config file)

2017-07-27 16:05:49,339 WARN Included extra file "/etc/supervisor/conf.d/beat.conf" during parsing

2017-07-27 16:05:49,339 WARN Included extra file "/etc/supervisor/conf.d/nginx.conf" during parsing

2017-07-27 16:05:49,339 WARN Included extra file "/etc/supervisor/conf.d/redis.conf" during parsing

2017-07-27 16:05:49,340 WARN Included extra file "/etc/supervisor/conf.d/uwsgi.conf" during parsing

2017-07-27 16:05:49,340 WARN Included extra file "/etc/supervisor/conf.d/workers.conf" during parsing

2017-07-27 16:05:49,423 INFO RPC interface 'supervisor' initialized

2017-07-27 16:05:49,423 CRIT Server 'unix_http_server' running without any HTTP authentication checking

2017-07-27 16:05:49,423 INFO supervisord started with pid 1

2017-07-27 16:05:50,426 INFO spawned: 'mayan-worker-medium' with pid 80

2017-07-27 16:05:50,430 INFO spawned: 'mayan-worker-slow' with pid 81

2017-07-27 16:05:50,433 INFO spawned: 'celery-beat' with pid 82

2017-07-27 16:05:50,437 INFO spawned: 'mayan-worker-fast' with pid 83

2017-07-27 16:05:50,442 INFO spawned: 'redis' with pid 84

2017-07-27 16:05:50,449 INFO spawned: 'nginx' with pid 85

2017-07-27 16:05:50,454 INFO spawned: 'uwsgi' with pid 86

2017-07-27 16:05:52,376 INFO success: redis entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

2017-07-27 16:05:52,376 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

2017-07-27 16:05:52,376 INFO success: uwsgi entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

Mark

[Roberto Rosario]

Thanks Mark,

Indeed no error at all...

Try this:

docker exec -ti test-mayan-edms sh -c "cat /usr/local/lib/python2.7/dist-packages/mayan/error.log"

Version 2.6 added a feature to do extra logging of errors in production for situations like this. Let's see if this log caught something.

[Mark Phillips]

Sorry, Roberto, but the error.log file is empty...size 0. I always strive to run an error-free environment....;)

root@orca:/home/mark/mayan2-docker# docker exec -ti mayan-edms /bin/bash

root@51cbdb670142:/# cat /usr/local/lib/python2.7/dist-packages/mayan/error.log

root@51cbdb670142:/#  

root@51cbdb670142:/# ls -alh /usr/local/lib/python2.7/dist-packages/mayan/error.log

-rw-r--r-- 1 www-data www-data 0 Jul 26 07:16 /usr/local/lib/python2.7/dist-packages/mayan/error.log

root@51cbdb670142:/# find . -name error.log

./usr/local/lib/python2.7/dist-packages/mayan/error.log

./var/log/nginx/error.log

root@51cbdb670142:/# cat ./var/log/nginx/error.log 

root@51cbdb670142:/#  

root@51cbdb670142:/# ls -alh ./var/log/nginx/error.log 

-rw-r----- 1 www-data adm 0 Jul 26 07:09 ./var/log/nginx/error.log

root@51cbdb670142:/# 

Mark

[Roberto Rosario]

Hahaha! Good one :D

Only thing I can think of is the a few changes to the 'initialsetup' signal, I think start there. Thanks for helping track this down.

On Friday, July 28, 2017 at 11:59:30 AM UTC-4, Mark Phillips wrote:

Sorry, Roberto, but the error.log file is empty...size 0. I always strive to run an error-free environment....;)

root@orca:/home/mark/mayan2-docker# docker exec -ti mayan-edms /bin/bash

root@51cbdb670142:/# cat /usr/local/lib/python2.7/dist-packages/mayan/error.log

root@51cbdb670142:/#  

root@51cbdb670142:/# ls -alh /usr/local/lib/python2.7/dist-packages/mayan/error.log

-rw-r--r-- 1 www-data www-data 0 Jul 26 07:16 /usr/local/lib/python2.7/dist-packages/mayan/error.log

root@51cbdb670142:/# find . -name error.log

./usr/local/lib/python2.7/dist-packages/mayan/error.log

./var/log/nginx/error.log

root@51cbdb670142:/# cat ./var/log/nginx/error.log 

root@51cbdb670142:/#  

root@51cbdb670142:/# ls -alh ./var/log/nginx/error.log 

-rw-r----- 1 www-data adm 0 Jul 26 07:09 ./var/log/nginx/error.log

root@51cbdb670142:/# 

Mark

[Roberto Rosario]

Please pull the latest version of the container, version 2.6.4-4 and try again with that one.

[Mark Phillips]

Roberto,

I will try in a week or so. Headed to Puerto Vallarta for some sun and scuba diving with the family. Be back in a week!

Mark

[Roberto Rosario]

Hi Mark,

Enjoy your trip! We'll talk later.

On Monday, July 31, 2017 at 1:24:38 AM UTC-4, Mark Phillips wrote:

Roberto,

I will try in a week or so. Headed to Puerto Vallarta for some sun and scuba diving with the family. Be back in a week!

Mark