Issue with Django migrate - fine with 2.7x python, but error with 3.6.4?

[Benjamin Schollnick]

I've an established Django application that I'm finishing up a new release for, and I'm working on solving any last python 3 issues.

I'm confused at what's happening here though...

Run manage.py migrations under v2.7x:

nerv:quickbbs Benjamin$ manage.py migrate --run-syncdb

Running on nerv.local

Operations to perform:

  Synchronize unmigrated apps: _humanize, allauth, bootstrap3, django_jinja, grappelli, humanize, messages, staticfiles

  Apply all migrations: account, admin, auth, contenttypes, quickbbs, sessions, sites, socialaccount

Synchronizing apps without migrations:

  Creating tables...

    Running deferred SQL...

Running migrations:

  No migrations to apply.

Run manage.py migrations under v3.65.

(quickbbs) nerv:quickbbs Benjamin$ python3 manage.py migrate

Running on nerv.local

CommandError: Conflicting migrations detected; multiple leaf nodes in the migration graph: (0019_auto_20171108_0155, 0044_auto_20180519_1914 in quickbbs).

To fix them run 'python manage.py makemigrations --merge'

(quickbbs) nerv:quickbbs Benjamin$ 

Okay?  

Manage.py makemigrations --merge

(quickbbs) nerv:quickbbs Benjamin$ python3 manage.py makemigrations --merge

Running on nerv.local

Traceback (most recent call last):

  File "manage.py", line 10, in <module>

    execute_from_command_line(sys.argv)

  File "/Users/Benjamin/django-py3/quickbbs/lib/python3.6/site-packages/django/core/management/__init__.py", line 371, in execute_from_command_line

    utility.execute()

  File "/Users/Benjamin/django-py3/quickbbs/lib/python3.6/site-packages/django/core/management/__init__.py", line 365, in execute

    self.fetch_command(subcommand).run_from_argv(self.argv)

  File "/Users/Benjamin/django-py3/quickbbs/lib/python3.6/site-packages/django/core/management/base.py", line 288, in run_from_argv

    self.execute(*args, **cmd_options)

  File "/Users/Benjamin/django-py3/quickbbs/lib/python3.6/site-packages/django/core/management/base.py", line 335, in execute

    output = self.handle(*args, **options)

  File "/Users/Benjamin/django-py3/quickbbs/lib/python3.6/site-packages/django/core/management/commands/makemigrations.py", line 124, in handle

    return self.handle_merge(loader, conflicts)

  File "/Users/Benjamin/django-py3/quickbbs/lib/python3.6/site-packages/django/core/management/commands/makemigrations.py", line 251, in handle_merge

    raise ValueError("Could not find common ancestor of %s" % migration_names)

ValueError: Could not find common ancestor of {'0019_auto_20171108_0155', '0044_auto_20180519_1914'}

I have found several other people reporting similar issues, but I don't see any best practices to prevent this, and no straight forward explanations on how to fix this?

But my big question is why is this happening with Py3, and not Py2?  

            - Benjamin

[Melvyn Sopacua]

On woensdag 23 mei 2018 12:55:58 CEST Benjamin Schollnick wrote:

> nerv:quickbbs Benjamin$ manage.py migrate --run-syncdb

<snip>

> (quickbbs) nerv:quickbbs Benjamin$ python3 manage.py migrate
>
> Running on nerv.local
>
> CommandError: Conflicting migrations detected; multiple leaf nodes in the
> migration graph: (0019_auto_20171108_0155, 0044_auto_20180519_1914 in
> quickbbs).

Not convinced yet this is a py2 versus py3 problem. First, you're running
different command flags (though, not sure it matters in this case). Second, the
Django version for py3 can be different from the py2 version.

From the backtrace it isn't clear though which app is causing this. If it
really is a py2 versus py3 issue, then I suspect that some migrations are done
only for py3, though that is really bad practice.

--
Melvyn Sopacua

[Daniel Germano Travieso]

As Melvyn statet, I do not believe this is a py2 vs py3 issue.

On your stacktrace it seems as the migrations of the quickbbs app are conflicting in some way.

Maybe a migration was created using py2 and another using py3 (as they were created 6 months appart - the name of the migration is a timestamp). Take a look at each migration file to analyse what it is doing. And if possible, you can try to merge manually what the migrations are doing and leave only one of them to be run.

Another issue could be VCS related, and some migration file was added/ignored on the filesystem making the migrations fail.

To test everything, you can backup you DB somewhere and make the migrations again from scratch.

Hope it helps!

[]'s

Daniel Germano Travieso

Engenharia da Computação Turma: 013

Unicamp

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/2403869.zZKC4CeI1l%40fritzbook.

For more options, visit https://groups.google.com/d/optout.

[Benjamin Schollnick]

Not convinced yet this is a py2 versus py3 problem. First, you're running
different command flags (though, not sure it matters in this case). Second, the
Django version for py3 can be different from the py2 version.

From the backtrace it isn't clear though which app is causing this. If it
really is a py2 versus py3 issue, then I suspect that some migrations are done
only for py3, though that is really bad practice.

Well, all migrations were made using makemigrations,  so if there are any specific PY2 vs PY3 differences, I'm unaware of any...

I've looked, and I don't have any clue on how to manually merge these because there are two 0018's, with different timestamps, and I don't want to break anything further.

Would it be simpler to reset the migrations using the current database state as the base?  Is there a way to do this?  After all, it's working fine... with both versions of django.

This seems to be more of a false positive then anything else?

Or maybe squash the migrations?  Is there a good guide to squashing?

           - Benjamin

[Benjamin Schollnick]

Okay, squashing seems to have resolved this issue...

Seems a bit voodoo-ish, but it worked.  Any suggestions for preventing this in the future would be nice!

       - Benjamin

[Melvyn Sopacua]

On donderdag 24 mei 2018 00:31:57 CEST Benjamin Schollnick wrote:
> Okay, squashing seems to have resolved this issue...

Did you find any issues with the documentation on squashing?

> Seems a bit voodoo-ish, but it worked. Any suggestions for preventing this
> in the future would be nice!

It's voodoo-ish cause you stopped analyzing the problem and went for the work-
around.

So my best guess is:
- The two 0018 migrations were caused by 2 different branches in the source
repository, adding migrations in parallel
- These were applied to the database also in parallel
- Further migrations were added, depending on the 0018 migrations
- One migration was removed that was the common ancenstor for one of the 0018
migrations and another migration.

Without reconstructing the dependency graph it will be difficult to see how you
could have solved it.
To prevent it, establish a protocol for your developers to communicate with
eachother when adding and removing migrations.

--
Melvyn Sopacua

[Umar Kambala]

please help me, this is the error mgs am been getting whenever i run my manage.py runserver

Not Found: /

[24/May/2018 14:16:40] "GET / HTTP/1.1" 404 2027

Internal Server Error: /polls/

Traceback (most recent call last):

  File "C:\Users\Admin\AppData\Local\Programs\Python\Python36-32\lib\site-packages\django\core\handlers\exception.py", line 35, in inner

    response = get_response(request)

  File "C:\Users\Admin\AppData\Local\Programs\Python\Python36-32\lib\site-packages\django\core\handlers\base.py", line 158, in _get_response

    response = self.process_exception_by_middleware(e, request)

  File "C:\Users\Admin\AppData\Local\Programs\Python\Python36-32\lib\site-packages\django\core\handlers\base.py", line 156, in _get_response

    response = response.render()

  File "C:\Users\Admin\AppData\Local\Programs\Python\Python36-32\lib\site-packages\django\template\response.py", line 106, in render

    self.content = self.rendered_content

  File "C:\Users\Admin\AppData\Local\Programs\Python\Python36-32\lib\site-packages\django\template\response.py", line 81, in rendered_content

    template = self.resolve_template(self.template_name)

  File "C:\Users\Admin\AppData\Local\Programs\Python\Python36-32\lib\site-packages\django\template\response.py", line 63, in resolve_template

    return select_template(template, using=self.using)

  File "C:\Users\Admin\AppData\Local\Programs\Python\Python36-32\lib\site-packages\django\template\loader.py", line 47, in select_template

    raise TemplateDoesNotExist(', '.join(template_name_list), chain=chain)

django.template.exceptions.TemplateDoesNotExist: polls/index.html, polls/question_list.html

[24/May/2018 14:16:52] "GET /polls/ HTTP/1.1" 500 79698

Not Found: /favicon.ico

[24/May/2018 14:17:54] "GET /favicon.ico HTTP/1.1" 404 2078

i dont know where exactly the problem is

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/1555096.CvhEgTB1t9%40fritzbook.

[Benjamin Schollnick]

Melvyn,

First, I'm sorry if this come off a bit snarky, it's been a long day...  

I want to say Thank you for your assistance, and I appreciate the help, even though I took potentially the easy way out...

But squashing the migrations, made me realize that the root cause, may have been an earlier squashed migration that wasn't cleaned up properly.

Was it?  I'm not sure, but after squashing and cleaning, it did clear up.

> Okay, squashing seems to have resolved this issue...

Did you find any issues with the documentation on squashing?

Sadly, yes.  

1) Finding it

       For example:

                    There are several commands which you will use to interact with migrations and Django’s handling of database schema:

• migrate, which is responsible for applying and unapplying migrations.
• makemigrations, which is responsible for creating new migrations based on the changes you have made to your models.
• sqlmigrate, which displays the SQL statements for a migration.
• showmigrations, which lists a project’s migrations and their status.

               Squashing isn't even listed in the Beginning discussion?  It's a command, isn't it?

               I had to search the web page to find it.  

        Overall, 

> Seems a bit voodoo-ish, but it worked.  Any suggestions for preventing this
> in the future would be nice!

It's voodoo-ish cause you stopped analyzing the problem and went for the work-
around.

Sure, I'll accept that.  I disagree that staring at two different 0018 files, which don't give me much to work from.  But the work around actually made me realize what happened to cause this in the first place...  Feel free to take a look at the migrations....

# -*- coding: utf-8 -*-

# Generated by Django 1.11.6 on 2017-11-08 01:49

from __future__ import unicode_literals

from django.db import migrations, models

class Migration(migrations.Migration):

    dependencies = [

        ('quickbbs', '0017_auto_20171107_1415'),

    ]

    operations = [

        migrations.CreateModel(

            name='IndexData',

            fields=[

                ('id', models.AutoField(primary_key=True, serialize=False)),

                ('LastScan', models.FloatField(db_index=True)),

                ('LastMod', models.FloatField(db_index=True)),

                ('Name', models.CharField(db_index=True, default=None, max_length=512)),

                ('SortName', models.CharField(db_index=True, default='', editable=False, max_length=512)),

                ('Size', models.IntegerField(default=0)),

                ('FQPNDirectory', models.CharField(db_index=True, default=0, max_length=512)),

                ('ParentDirID', models.IntegerField(default=0)),

                ('is_dir', models.BooleanField(db_index=True, default=False)),

                ('is_pdf', models.BooleanField(db_index=True, default=False)),

                ('is_archive', models.BooleanField(db_index=True, default=False)),

                ('Ignore', models.BooleanField(default=False)),

                ('DeletePending', models.BooleanField(default=False)),

            ],

        ),

        migrations.DeleteModel(

            name='DirData',

        ),

        migrations.DeleteModel(

            name='FileData',

        ),

    ]

vs 

#-*- coding: utf-8 -*-

#Generated by Django 1.11.9 on 2018-03-15 21:47

from __future__ import unicode_literals

from django.db import migrations

class Migration(migrations.Migration):

    dependencies = [

        ('quickbbs', '0017_auto_20180315_2138'),

    ]

    operations = [

        migrations.RemoveField(

            model_name='thumbnails_files',

            name='DeletePending',

        ),

        migrations.RemoveField(

            model_name='thumbnails_files',

            name='Flag_For_Regeneration',

        ),

    ]

 

So my best guess is:
- The two 0018 migrations were caused by 2 different branches in the source repository, adding migrations in parallel

You're guessing, but that's because you don't have the details.. (And that is my fault).

One, I'm the only one working on this  - http://github.com/bschollnick/quickbbs

And no, sadly there has only been one active tree in development.  

 

- These were applied to the database also in parallel
- Further migrations were added, depending on the 0018 migrations
- One migration was removed that was the common ancenstor for one of the 0018
migrations and another migration.

Actually, because of the workaround, I realized that an earlier squashed migration had caused Django to renumber the migrations, starting back at one. Yes, it's documented, but that doesn't mean that I remembered that..

Now this doesn't answer why it was only an issue under python 3 / Django v2.05. But I may just have to live with not understanding that for now.

| Without reconstructing the dependency graph it will be difficult to see how you 

could have solved it.
To prevent it, establish a protocol for your developers to communicate with
eachother when adding and removing migrations.

As I mentioned, there's only one developer... I'm happy to have other input, but for now at least, that's all there is...

(And I still have all the migration files, so if you'd like, I'll be happy to make them available...  But they should also be on GitHub also).

                - Benjamin