development setup gcd-django-docker

[Jochen G.]

I made a docker container setup for development purposes.

The repo with the needed files and a read.me on the steps for setting
this up is here:
https://github.com/GrandComicsDatabase/gcd-django-docker

If someone can try if it works for you ? You need to install docker and
docker-compose and have a docker service running, but there are enough
infos on that findable.

I guess one could also make the containers available, but I am not sure
how useful that is. For dev purposes the web-container will be updated
anyhow.

One can see if this is the best to handle the two containers. Need to be
careful not to do a docker-compose down, because that resets the
db-container, needed a full db import gain. So I guess it is
docker-compose stop. Or we use a volume for storing the data for the
db-container.

Not the following:
If doing development work on the code for editing, note that right now
you cannot edit existing data, since we do not export the change history
in the dump. But, you can add new data (with dexter_1234), approve it
(with editor), and then edit the newly added data. We might can add a
changesets for the existing data to allow their editing in this
development setup.

Jochen

[Adam Knights]

Firstly this is awesome, and a game changer. As i'm primarily a .net dev, setting everything up for a dev env in python and mysql world was a barrier to get started.

At step 3 I got a long list of errors the first couple tries (will paste at bottom incase useful), but then on third attempt the migrations ran all ok. Migrations take a long time to run, feels like maybe 45 minutes, so if we update the readme at all could mention that's normal.

On the next bit to import the dump, theres a typo with _'s rather than dashes, and missing a space after -p:

After unzipping the dump, run the following with the name of the 'current_dump':
docker exec -i gcd-django-docker_db_1 mysql -u gcd-django my-gcd-db -p'db-gcd' < 'current_dump'

becomes

After unzipping the dump, run the following with the name of the 'current_dump':
docker exec -i gcd-django-docker-db-1 mysql -u gcd-django my-gcd-db -p 'db-gcd' < 'current_dump'

That doesnt seem to do anything for me (with 2022-02-15.sql), it just shows a list of mysql options, so might need tweaking.

My biggest issue is that the web container exits immediately with:

C:\git_repos\gcd-django-docker>docker logs a5c23007c913
standard_init_linux.go:228: exec user process caused: no such file or directory
standard_init_linux.go:228: exec user process caused: no such file or directory
standard_init_linux.go:228: exec user process caused: no such file or directory

I tried running again after db migrations, still same issue, and also tried adding the users with docker-compose run web python gcd-django/manage.py loaddata gcd-django/apps/indexer/fixtures/users.yaml, but same issue.

So in summary, looking ok so far, just the web container issue and potentially need a tweak to the docker exec command to import the dump.

Cheers

Adam

Exceptions at step 3:

[+] Running 1/0d-django-docker>docker-compose run web /usr/local/bin/python gcd-django/manage.py migrate

 - Container gcd-django-docker-db-1  Running                                                                       0.0s

Traceback (most recent call last):

  File "/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py", line 219, in ensure_connection

    self.connect()

  File "/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py", line 33, in inner

    return func(*args, **kwargs)

  File "/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py", line 200, in connect

    self.connection = self.get_new_connection(conn_params)

  File "/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py", line 33, in inner

    return func(*args, **kwargs)

  File "/usr/local/lib/python3.10/site-packages/django/db/backends/mysql/base.py", line 234, in get_new_connection

    connection = Database.connect(**conn_params)

  File "/usr/local/lib/python3.10/site-packages/MySQLdb/__init__.py", line 130, in Connect

    return Connection(*args, **kwargs)

  File "/usr/local/lib/python3.10/site-packages/MySQLdb/connections.py", line 185, in __init__

    super().__init__(*args, **kwargs2)

MySQLdb._exceptions.OperationalError: (2002, "Can't connect to MySQL server on 'db' (115)")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

  File "/code/gcd-django/manage.py", line 9, in <module>

    execute_from_command_line(sys.argv)

  File "/usr/local/lib/python3.10/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line

    utility.execute()

  File "/usr/local/lib/python3.10/site-packages/django/core/management/__init__.py", line 413, in execute

    self.fetch_command(subcommand).run_from_argv(self.argv)

  File "/usr/local/lib/python3.10/site-packages/django/core/management/base.py", line 354, in run_from_argv

    self.execute(*args, **cmd_options)

  File "/usr/local/lib/python3.10/site-packages/django/core/management/base.py", line 398, in execute

    output = self.handle(*args, **options)

  File "/usr/local/lib/python3.10/site-packages/django/core/management/base.py", line 89, in wrapped

    res = handle_func(*args, **kwargs)

  File "/usr/local/lib/python3.10/site-packages/django/core/management/commands/migrate.py", line 75, in handle

    self.check(databases=[database])

  File "/usr/local/lib/python3.10/site-packages/django/core/management/base.py", line 419, in check

    all_issues = checks.run_checks(

  File "/usr/local/lib/python3.10/site-packages/django/core/checks/registry.py", line 76, in run_checks

    new_errors = check(app_configs=app_configs, databases=databases)

  File "/usr/local/lib/python3.10/site-packages/django/core/checks/database.py", line 13, in check_database_backends

    issues.extend(conn.validation.check(**kwargs))

  File "/usr/local/lib/python3.10/site-packages/django/db/backends/mysql/validation.py", line 9, in check

    issues.extend(self._check_sql_mode(**kwargs))

  File "/usr/local/lib/python3.10/site-packages/django/db/backends/mysql/validation.py", line 13, in _check_sql_mode

    if not (self.connection.sql_mode & {'STRICT_TRANS_TABLES', 'STRICT_ALL_TABLES'}):

  File "/usr/local/lib/python3.10/site-packages/django/utils/functional.py", line 48, in __get__

    res = instance.__dict__[self.name] = self.func(instance)

  File "/usr/local/lib/python3.10/site-packages/django/db/backends/mysql/base.py", line 405, in sql_mode

    sql_mode = self.mysql_server_data['sql_mode']

  File "/usr/local/lib/python3.10/site-packages/django/utils/functional.py", line 48, in __get__

    res = instance.__dict__[self.name] = self.func(instance)

  File "/usr/local/lib/python3.10/site-packages/django/db/backends/mysql/base.py", line 366, in mysql_server_data

    with self.temporary_connection() as cursor:

  File "/usr/local/lib/python3.10/contextlib.py", line 135, in __enter__

    return next(self.gen)

  File "/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py", line 603, in temporary_connection

    with self.cursor() as cursor:

  File "/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py", line 33, in inner

    return func(*args, **kwargs)

  File "/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py", line 259, in cursor

    return self._cursor()

  File "/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py", line 235, in _cursor

    self.ensure_connection()

  File "/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py", line 33, in inner

    return func(*args, **kwargs)

  File "/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py", line 218, in ensure_connection

    with self.wrap_database_errors:

  File "/usr/local/lib/python3.10/site-packages/django/db/utils.py", line 90, in __exit__

    raise dj_exc_value.with_traceback(traceback) from exc_value

  File "/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py", line 219, in ensure_connection

    self.connect()

  File "/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py", line 33, in inner

    return func(*args, **kwargs)

  File "/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py", line 200, in connect

    self.connection = self.get_new_connection(conn_params)

  File "/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py", line 33, in inner

    return func(*args, **kwargs)

  File "/usr/local/lib/python3.10/site-packages/django/db/backends/mysql/base.py", line 234, in get_new_connection

    connection = Database.connect(**conn_params)

  File "/usr/local/lib/python3.10/site-packages/MySQLdb/__init__.py", line 130, in Connect

    return Connection(*args, **kwargs)

  File "/usr/local/lib/python3.10/site-packages/MySQLdb/connections.py", line 185, in __init__

    super().__init__(*args, **kwargs2)

django.db.utils.OperationalError: (2002, "Can't connect to MySQL server on 'db' (115)")

[Jochen G.]

Am 21.02.22 um 14:43 schrieb Adam Knights:

> Firstly this is awesome, and a game changer. As i'm primarily a .net
> dev, setting everything up for a dev env in python and mysql world was a
> barrier to get started.
>
> At step 3 I got a long list of errors the first couple tries (will paste
> at bottom incase useful), but then on third attempt the migrations ran
> all ok. Migrations take a long time to run, feels like maybe 45 minutes,
> so if we update the readme at all could mention that's normal.

Were the containers running ? I.e. was docker-compose up ? For testing,
it is best to not use the -d. The mysql container needs some time for
the first setup.

The migrations take some time, but since one sees something is going on
by the shell output I think this should be OK ?

> On the next bit to import the dump, theres a typo with _'s rather than
> dashes, and missing a space after -p:

This is not really a typo, this is how the containers are called on my
system. Seems like their naming is not consistent over the host systems.

This is the call on my system, no space needed, at least over here:

docker exec -i gcd-django-docker_db_1 mysql -u gcd-django my-gcd-db

-p'db-gcd' < /tmp/2022-02-15.sql

> My biggest issue is that the web container exits immediately with:
> C:\git_repos\gcd-django-docker>docker logs a5c23007c913
> standard_init_linux.go:228: exec user process caused: no such file or
> directory
> standard_init_linux.go:228: exec user process caused: no such file or
> directory
> standard_init_linux.go:228: exec user process caused: no such file or
> directory

https://stackoverflow.com/questions/51508150/standard-init-linux-go190-exec-user-process-caused-no-such-file-or-directory

Jochen

[Alexandros Diamantidis]

Thanks, Jochen! That was a very good idea, since Docker has much bigger
mindshare than our previous try with Vagrant (which needs updates anyway).

Vagrant has an advantage in that you can easily edit files in the VM
filesystem from the host - it doesn't seem seem to be so easy with
Docker, so how can it be used for development?

> If doing development work on the code for editing, note that right now
> you cannot edit existing data, since we do not export the change history
> in the dump. But, you can add new data (with dexter_1234), approve it

Perhaps we could distribute a minimal SQL dump with all migrations
applied already, some users and a couple of test series, creators and
issues...

And I agree with adding a volume for MySQL to store its data.

* Jochen G. [2022-02-21 15:08]:

> Were the containers running ? I.e. was docker-compose up ? For testing,
> it is best to not use the -d. The mysql container needs some time for
> the first setup.

I tried it and it worked for me, although one thing to note is that you
should have the needed ports free - in my case, I had both MySQL running
locally on the host as well as another web server at port 8000 so
neither container started at first. Maybe you could change the
Dockerfile and config to use two higher ports?

> > On the next bit to import the dump, theres a typo with _'s rather than
> > dashes, and missing a space after -p:
>
> This is not really a typo, this is how the containers are called on my
> system. Seems like their naming is not consistent over the host systems.

You can use "docker ps" to verify that the containers are running, and
what exactly their names are.

Alexandros

[Jochen G.]

Am 21.02.22 um 21:56 schrieb Alexandros Diamantidis:

> Thanks, Jochen! That was a very good idea, since Docker has much bigger
> mindshare than our previous try with Vagrant (which needs updates anyway).
>
> Vagrant has an advantage in that you can easily edit files in the VM
> filesystem from the host - it doesn't seem seem to be so easy with
> Docker, so how can it be used for development?

You can edit to some degree on the container ?

Or instead of using the git-clone from beta, clone your own dev-branch,
be it local or on github. Or copy over in the last step of the
Dockerfile the files which changed, or do a rsync ?

>> If doing development work on the code for editing, note that right now
>> you cannot edit existing data, since we do not export the change history
>> in the dump. But, you can add new data (with dexter_1234), approve it
>
> Perhaps we could distribute a minimal SQL dump with all migrations
> applied already, some users and a couple of test series, creators and
> issues...

Sure, if someone works on that. But it needs regularly maintenance with
every migration or update we do. The migration path makes sure that it
is always fresh.

> And I agree with adding a volume for MySQL to store its data.

Feel free to update the setup.

> * Jochen G. [2022-02-21 15:08]:
>> Were the containers running ? I.e. was docker-compose up ? For testing,
>> it is best to not use the -d. The mysql container needs some time for
>> the first setup.
>
> I tried it and it worked for me, although one thing to note is that you
> should have the needed ports free - in my case, I had both MySQL running
> locally on the host as well as another web server at port 8000 so
> neither container started at first. Maybe you could change the
> Dockerfile and config to use two higher ports?

I would prefer to keep it standard. You can just edit the
docker-compose.yml ? Is mysql affected by this ? I have it running
locally as well without an issue ?

Jochen

[Alexandros Diamantidis]

* Jochen G. [2022-02-21 22:45]:

> > Vagrant has an advantage in that you can easily edit files in the VM
> > filesystem from the host - it doesn't seem seem to be so easy with
> > Docker, so how can it be used for development?
>
> You can edit to some degree on the container ?
>
> Or instead of using the git-clone from beta, clone your own dev-branch,
> be it local or on github. Or copy over in the last step of the
> Dockerfile the files which changed, or do a rsync ?

Sure, all of that will work, but being able to view and edit the files
from outside the container would be better... But I see that the current
directory where I'm running docker-compose from is mounted in the
container using "volumes: - .:/gcd-django" - so would it be possible to
do the installation there and view the files? I'll have to read more
about Docker and do some testing...

> > Perhaps we could distribute a minimal SQL dump with all migrations
> > applied already, some users and a couple of test series, creators and
> > issues...
>
> Sure, if someone works on that. But it needs regularly maintenance with
> every migration or update we do. The migration path makes sure that it
> is always fresh.

You are right, no need for that overhead - and having all the real data
to experiment with is nicer.

> > And I agree with adding a volume for MySQL to store its data.
>
> Feel free to update the setup.

> > I tried it and it worked for me, although one thing to note is that you
> > should have the needed ports free - in my case, I had both MySQL running
> > locally on the host as well as another web server at port 8000 so
>

> I would prefer to keep it standard. You can just edit the
> docker-compose.yml ? Is mysql affected by this ? I have it running
> locally as well without an issue ?

I think you can run mysql listening only to a unix domain socket
(/var/run/mysqld/mysqld.sock) but in my case I had enabled listening to
the network as well, and this caused the conflict...

I'm attaching docker-compose.yml with a couple of changes: first, I
removed the "ports" directive from mysql - it's not needed for the
service to work, because the network is automatically connected between
the containers. It's only needed if you need to connect to MySQL from
outside. Second, I added a data volume for the mysql container to make it
persistent.

Thanks!
Alexandros

[Adam Knights]

The volume worked really well!

I eventually got the web project loading by using dos2unix on the wait.sh file to get LF file endings.

There was an issue with the current requirements.txt file in that github discontinued the git protocol, I've raised a PR to fix that, for now you can work around it by adding:

RUN sed -i "s|git+git|git+https| g" /code/gcd-django/requirements.txt

to your Dockerfile, after the clone.

I had some further issues with importing the sql dump into Mysql, again I think its a Windows thing "Access denied for user 'root'@'localhost' (using password: NO)", I tried the usual docker compose down -v and starting again, but got the same. I managed to work around this by copying the sql file onto the container with:

docker cp 2022-03-15.sql gcd-django-docker_db_1:/root

And then on the box, cd into root and mysql -u root -p my-gcd-db < 2022-03-15.sql

Finally, the site wouldnt load on 0.0.0.0, but I added localhost to settings_local.py and I've just loaded the site in my browser :D.

^ Documenting all of the above incase anyone else on windows hits similar issues.

I'm now going to try the users and getting my own change in locally etc.

Cheers,

Adam

[Adam Knights]

I logged in as the indexer user, but every time I try to 'add' something, it just takes me to my profile page. I tried adding an issue to an existing series, adding a series, and adding a publisher. All the same. I also tried the editor user and it does the same. I also tried resetting stats.

Any thoughts?

[Jochen G.]

Am 22.03.22 um 14:15 schrieb Adam Knights:

> I eventually got the web project loading by using dos2unix on the
> wait.sh file to get LF file endings.

I gather we can handle this by using .gitattributes ?
https://github.com/docker/for-win/issues/1340#issuecomment-716941493

# Ensure shell script uses LF.
Dockerfile eol=lf
*.sh eol=lf

> I had some further issues with importing the sql dump into Mysql, again
> I think its a Windows thing "Access denied for user 'root'@'localhost'
> (using password: NO)", I tried the usual docker compose down -v and
> starting again, but got the same. I managed to work around this by
> copying the sql file onto the container with:

This sounds like the password is not observed when calling mysql ?

> Finally, the site wouldnt load on 0.0.0.0, but I added localhost to
> settings_local.py and I've just loaded the site in my browser :D.

What did happen ?

Jochen

[Jochen G.]

Can you try the admin user and go the admin page ? Does for the users
show permissions, i.e. can_reserve ?

Jochen

Am 22.03.22 um 16:19 schrieb Adam Knights:

> --
> You received this message because you are subscribed to the Google
> Groups "gcd-tech" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to gcd-tech+u...@googlegroups.com
> <mailto:gcd-tech+u...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/gcd-tech/f34bc2b0-8326-419d-9bd6-a3faf907d437n%40googlegroups.com
> <https://groups.google.com/d/msgid/gcd-tech/f34bc2b0-8326-419d-9bd6-a3faf907d437n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Adam Knights]

The hidden menu :D -> So yes I think that's the issue, there are 0 user perms on the right hand side for dexter or editor - is that another script to run to add to the list or something I missed / or was unexpected?

Yes .gitattributes sounds right for that.

It was very strange with mysql, i tried various combos, -p -p= etc, but it never liked it all combined in one. But going onto the container and then accessing mysql worked fine there.

For 0.0.0.0:8000 I get 'this site cannot be reached' 'err_address_invalid'. Localhost works great now I added the allowed hosts for it.

Cheers,

Adam

[Jochen G.]

Hmm, you can try loading the fixtures again:

python manage.py loaddata apps/indexer/fixtures/users.yaml

Adding users and permissions should happen in the same step though.

Are there permissions assignable or not present at all ?

Jochen

Am 22.03.22 um 23:15 schrieb Adam Knights:

> <https://groups.google.com/d/msgid/gcd-tech/f34bc2b0-8326-419d-9bd6-a3faf907d437n%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/gcd-tech/f34bc2b0-8326-419d-9bd6-a3faf907d437n%40googlegroups.com?utm_medium=email&utm_source=footer>>.

>
>
> --
> You received this message because you are subscribed to the Google
> Groups "gcd-tech" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to gcd-tech+u...@googlegroups.com
> <mailto:gcd-tech+u...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/gcd-tech/233e4bb9-b258-4965-b16e-e2aa2cacbb1cn%40googlegroups.com
> <https://groups.google.com/d/msgid/gcd-tech/233e4bb9-b258-4965-b16e-e2aa2cacbb1cn%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Adam Knights]

Thats interesting, on the second run that script has errored with:

django.db.utils.IntegrityError: Problem installing fixture '/code/gcd-django/apps/indexer/fixtures/users.yaml': Could not load auth.Group_permissions(pk=1): (1048, "Column 'permission_id' cannot be null")

There are permissions assignable, on the left... i filtered on 'reserve' and see 1 for publisher? Maybe there should be more or is it a catch all? See below:

Cheers,

Adam

[Adam Knights]

These are the permissions that the indexer group has:

Cheers

Adam

[Jochen G.]

Strange.

It should be
indexer | indexer | Can reserve a record for ...
indexer | indexer | Can upload covers...

Jochen


Am 23.03.22 um 10:23 schrieb Adam Knights:

> These are the permissions that the indexer group has:
>

> indexer.png
> Cheers
> Adam
>
> On Wednesday, 23 March 2022 at 08:21:05 UTC Adam Knights wrote:
>
> Thats interesting, on the second run that script has errored with:
>
> django.db.utils.IntegrityError: Problem installing fixture
> '/code/gcd-django/apps/indexer/fixtures/users.yaml': Could not load
> auth.Group_permissions(pk=1): (1048, "Column 'permission_id' cannot
> be null")
>
> There are permissions assignable, on the left... i filtered on
> 'reserve' and see 1 for publisher? Maybe there should be more or is
> it a catch all? See below:
>

> reserve.png
> Cheers,
> Adam
> On Wednesday, 23 March 2022 at 07:20:31 UTC Jochen G. wrote:
>
> Hmm, you can try loading the fixtures again:
>
> python manage.py loaddata apps/indexer/fixtures/users.yaml
>
> Adding users and permissions should happen in the same step though.
>
> Are there permissions assignable or not present at all ?
>
> Jochen
>
> Am 22.03.22 um 23:15 schrieb Adam Knights:
> > The hidden menu :D -> So yes I think that's the issue, there
> are 0 user
> > perms on the right hand side for dexter or editor - is that
> another
> > script to run to add to the list or something I missed / or
> was unexpected?
> >
> > Yes .gitattributes sounds right for that.
> >
> > It was very strange with mysql, i tried various combos, -p
> -p= etc, but
> > it never liked it all combined in one. But going onto the
> container and
> > then accessing mysql worked fine there.
> >

> > For 0.0.0.0:8000 <http://0.0.0.0:8000> I get 'this site

> <https://groups.google.com/d/msgid/gcd-tech/233e4bb9-b258-4965-b16e-e2aa2cacbb1cn%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/gcd-tech/233e4bb9-b258-4965-b16e-e2aa2cacbb1cn%40googlegroups.com?utm_medium=email&utm_source=footer>>.

>
>
> --
> You received this message because you are subscribed to the Google
> Groups "gcd-tech" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to gcd-tech+u...@googlegroups.com
> <mailto:gcd-tech+u...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/gcd-tech/54c9d461-40f9-43cf-b9c4-877bef4262d9n%40googlegroups.com
> <https://groups.google.com/d/msgid/gcd-tech/54c9d461-40f9-43cf-b9c4-877bef4262d9n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Jochen G.]

hmm, don't know.

fixture should just be installed, don't know why there can be an error.

Jochen

Am 23.03.22 um 09:21 schrieb Adam Knights:

> Thats interesting, on the second run that script has errored with:
>
> django.db.utils.IntegrityError: Problem installing fixture
> '/code/gcd-django/apps/indexer/fixtures/users.yaml': Could not load
> auth.Group_permissions(pk=1): (1048, "Column 'permission_id' cannot be
> null")
>
> There are permissions assignable, on the left... i filtered on 'reserve'
> and see 1 for publisher? Maybe there should be more or is it a catch
> all? See below:
>

> reserve.png
> Cheers,
> Adam
> On Wednesday, 23 March 2022 at 07:20:31 UTC Jochen G. wrote:
>
> Hmm, you can try loading the fixtures again:
>
> python manage.py loaddata apps/indexer/fixtures/users.yaml
>
> Adding users and permissions should happen in the same step though.
>
> Are there permissions assignable or not present at all ?
>
> Jochen
>
> Am 22.03.22 um 23:15 schrieb Adam Knights:
> > The hidden menu :D -> So yes I think that's the issue, there are
> 0 user
> > perms on the right hand side for dexter or editor - is that another
> > script to run to add to the list or something I missed / or was
> unexpected?
> >
> > Yes .gitattributes sounds right for that.
> >
> > It was very strange with mysql, i tried various combos, -p -p=
> etc, but
> > it never liked it all combined in one. But going onto the
> container and
> > then accessing mysql worked fine there.
> >

> > For 0.0.0.0:8000 <http://0.0.0.0:8000> I get 'this site cannot be

> <https://groups.google.com/d/msgid/gcd-tech/233e4bb9-b258-4965-b16e-e2aa2cacbb1cn%40googlegroups.com>
>
> >
> <https://groups.google.com/d/msgid/gcd-tech/233e4bb9-b258-4965-b16e-e2aa2cacbb1cn%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/gcd-tech/233e4bb9-b258-4965-b16e-e2aa2cacbb1cn%40googlegroups.com?utm_medium=email&utm_source=footer>>.

>
>
> --
> You received this message because you are subscribed to the Google
> Groups "gcd-tech" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to gcd-tech+u...@googlegroups.com
> <mailto:gcd-tech+u...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/gcd-tech/ad355ad6-7d59-46f2-9ba4-4bc6d3758174n%40googlegroups.com
> <https://groups.google.com/d/msgid/gcd-tech/ad355ad6-7d59-46f2-9ba4-4bc6d3758174n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Adam Knights]

I'll try again from the start with a new volume, as I was hitting issues along the way, now I have a happy path I can make sure its all ran in the correct order.

As an FYI, the RUN git clone line will cache (see screenshot below), so if new commits are pushed, you won't get them in the current form. I found https://stackoverflow.com/questions/36996046/how-to-prevent-dockerfile-caching-git-clone and added ADD https://api.github.com/repos/GrandComicsDatabase/gcd-django/git/refs/heads/master version.json above the clone which should fix that.

Cheers,

Adam

[Jochen G.]

Am 22.02.22 um 17:13 schrieb Alexandros Diamantidis:

> I think you can run mysql listening only to a unix domain socket
> (/var/run/mysqld/mysqld.sock) but in my case I had enabled listening to
> the network as well, and this caused the conflict...
>
> I'm attaching docker-compose.yml with a couple of changes: first, I
> removed the "ports" directive from mysql - it's not needed for the
> service to work, because the network is automatically connected between
> the containers. It's only needed if you need to connect to MySQL from
> outside. Second, I added a data volume for the mysql container to make it
> persistent.

How is the nc call in wait.sh without a port number ?

Jochen

[Jochen G.]

Am 22.03.22 um 14:15 schrieb Adam Knights:

> I had some further issues with importing the sql dump into Mysql, again
> I think its a Windows thing "Access denied for user 'root'@'localhost'
> (using password: NO)", I tried the usual docker compose down -v and
> starting again, but got the same. I managed to work around this by
> copying the sql file onto the container with:
>
> docker cp 2022-03-15.sql gcd-django-docker_db_1:/root
>
> And then on the box, cd into root and mysql -u root -p my-gcd-db <
> 2022-03-15.sql

Why root ? The db should load/run under user gcd-django ?

Jochen

[Jochen G.]

I updated the repo for the docker setup.

https://github.com/GrandComicsDatabase/gcd-django-docker/commit/72cb772d5a35be884d6d6390fe36ad0516a51716

Jochen

Am 24.03.22 um 10:41 schrieb Adam Knights:

> I'll try again from the start with a new volume, as I was hitting issues
> along the way, now I have a happy path I can make sure its all ran in
> the correct order.
>
> As an FYI, the RUN git clone line will cache (see screenshot below), so
> if new commits are pushed, you won't get them in the current form. I
> found https://stackoverflow.com/questions/36996046/how-to-prevent-dockerfile-caching-git-clone
> and added ADD
> https://api.github.com/repos/GrandComicsDatabase/gcd-django/git/refs/heads/master
> version.json above the clone which should fix that.
>

> > > For 0.0.0.0:8000 <http://0.0.0.0:8000> <http://0.0.0.0:8000

> <https://groups.google.com/d/msgid/gcd-tech/ad355ad6-7d59-46f2-9ba4-4bc6d3758174n%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/gcd-tech/ad355ad6-7d59-46f2-9ba4-4bc6d3758174n%40googlegroups.com?utm_medium=email&utm_source=footer>>.

>
>
> --
> You received this message because you are subscribed to the Google
> Groups "gcd-tech" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to gcd-tech+u...@googlegroups.com
> <mailto:gcd-tech+u...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/gcd-tech/0637a562-3822-4718-93b4-9f1e8383cca8n%40googlegroups.com
> <https://groups.google.com/d/msgid/gcd-tech/0637a562-3822-4718-93b4-9f1e8383cca8n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Adam Knights]

Why root ? The db should load/run under user gcd-django ? - both work if I go into the container, i just so happened to be trying root at the time I got something working.

Thanks for pushing up all the updates.

Going back to the user migration error:

django.db.utils.IntegrityError: Problem installing fixture '/code/gcd-django/apps/indexer/fixtures/users.yaml': Could not load auth.Group_permissions(pk=1): (1048, "Column 'permission_id' cannot be null")

That must be this line: https://github.com/GrandComicsDatabase/gcd-django/blob/master/apps/indexer/fixtures/users.yaml#L286 ?

So that means permission: [can_reserve, indexer, indexer] isn't matching to something in my tables? If I query my auth_permission table I see 650 rows, if that's what is corresponds to?

Is there something I can check for in that table that should be there for that fixture line to work? What file has the info that populates auth_permission table?

Cheers

Adam

[Jochen G.]

The permissions should come in the db during the initial migration.

I never needed to look at the underlying tables...

Am 25.03.22 um 09:47 schrieb Adam Knights:

> <http://0.0.0.0:8000>> <http://0.0.0.0:8000 <http://0.0.0.0:8000>

> <https://groups.google.com/d/msgid/gcd-tech/0637a562-3822-4718-93b4-9f1e8383cca8n%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/gcd-tech/0637a562-3822-4718-93b4-9f1e8383cca8n%40googlegroups.com?utm_medium=email&utm_source=footer>>.

>
>
> --
> You received this message because you are subscribed to the Google
> Groups "gcd-tech" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to gcd-tech+u...@googlegroups.com
> <mailto:gcd-tech+u...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/gcd-tech/a603343d-3b75-4a0c-8edd-e0731ad2ee0dn%40googlegroups.com
> <https://groups.google.com/d/msgid/gcd-tech/a603343d-3b75-4a0c-8edd-e0731ad2ee0dn%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Adam Knights]

I wonder if its an ordering thing.. https://github.com/GrandComicsDatabase/gcd-django/blob/beta/docs/Getting_Started.txt says to run the users fixture and then import the data dump. Whereas the docker instructions have dump and then users, which is what I've been doing.

Would it be worth me trying the other way round? - users and then dump?

Cheers

Adam

[Jochen G.]

That ordering wouldn't make a difference.

Did you try a fresh db from scratch ? Does the output of the migration
look OK ? Could you start the website after the migration ?

Jochen

Am 25.03.22 um 11:52 schrieb Adam Knights:

> > <http://0.0.0.0:8000 <http://0.0.0.0:8000>>> <http://0.0.0.0:8000

> <https://groups.google.com/d/msgid/gcd-tech/a603343d-3b75-4a0c-8edd-e0731ad2ee0dn%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/gcd-tech/a603343d-3b75-4a0c-8edd-e0731ad2ee0dn%40googlegroups.com?utm_medium=email&utm_source=footer>>.

>
>
> --
> You received this message because you are subscribed to the Google
> Groups "gcd-tech" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to gcd-tech+u...@googlegroups.com
> <mailto:gcd-tech+u...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/gcd-tech/69a85df5-4bd9-4cdc-ab8c-d2f8d1ebd30fn%40googlegroups.com
> <https://groups.google.com/d/msgid/gcd-tech/69a85df5-4bd9-4cdc-ab8c-d2f8d1ebd30fn%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Adam Knights]

Yeah the migration itself is all green OK's. Yeah I did fresh overnight and it was the same today with the errors. I'll try everything from scratch over the weekend, including the new docker commits and triple check everything.

Cheers,

Adam

[Alexandros Diamantidis]

* Jochen G. [2022-03-24 20:26]:

> Am 22.02.22 um 17:13 schrieb Alexandros Diamantidis:

> > I'm attaching docker-compose.yml with a couple of changes: first, I
> > removed the "ports" directive from mysql - it's not needed for the
> > service to work, because the network is automatically connected between
> > the containers. It's only needed if you need to connect to MySQL from
> > outside.
>

> How is the nc call in wait.sh without a port number ?

It works fine, MySQL does listen to port 3306, but it's exposed only to
the internal docker network, not to the host. Both nc and Django connect
fine. Of course, without the ports you can't easily connect to the
database to run SQL queries, but for Django development it's not so
necessary...

I guess the "editing the ports" part now added to the README is warning
enough if anyone hits this snag.

Thanks!
Alexandros