Django 1.11 and Python 3 SQLite Migration "table already exists"

[Alex Laird]

I've recently started receiving an error that I'm having trouble identifying with a Django 1.11 project using SQLite in development. The project linked below was/is previously Python 2/3 compatible, though I primarily ran it using Python 2. Recently, I switched over most of my computers and projects to default to using Python 3. Additionally, I upgraded the project from Django 1.7 to 1.11. After this, the project started receiving the table already exists error, but only when running the migrate command using python3.

I also only get the error when running python3 manage.py migrate. For instance, python3 manage.py test works just fine, which is a bit baffling given test first runs the migrations. Running python2 manage.py migrate works just fine, no errors.

I am running Python 3.6.4, installed via Homebrew on a Mac, and the error I received is:

File "/usr/local/lib/python3.6/site-packages/django/db/migrations/recorder.py", line 59, in ensure_schema
   raise MigrationSchemaMissing("Unable to create the django_migrations table (%s)" % exc)
django.db.migrations.exceptions.MigrationSchemaMissing: Unable to create the django_migrations table (table "django_migrations" already exists)

I run into this exact same issue—broken with Python 3 but working with Python 2—on multiple computers, so I'd imagine anyone else will see the same issue. You should be able to reproduce the issue with the following steps:

1. git clone g...@github.com:alexdlaird/django-bootstrap-authentication-template-project.git && cd django-bootstrap-authentication-template-project
2. make install
3. python2 manage.py migrate - note that it works just fine
4. rm db.sqlite for a fresh start
5. python3 manage.py migrate - note that it fails with the error shown above 5x. rm db.sqlitefor a fresh start 5x. python3 manage.py test - note that it works just fine

To see the migrations run against a MySQL instance (the project assumes you have a default Homebrew MySQL instance running locally, but this can be configured in .env if not), run python3 manage.py migrate and observe that this works just fine with Python 2 or 3, so the issue appears isolated to SQLite migrations.

I have done a bit of debugging with this issue myself, swapping back and forth between Python 2.7 and Python 3.6. What I'm seeing is that I believe the issue is related to Unicode strings and table names in Python 3, but I'm not sure where the best place to address this would be. For example, watching the returned get_tables_list in value of in python3.6/django/db/backends/sqlite3/introspection.py shows an empty list (I believe because of the b'' preceding the string, which is hijacking what otherwise should be an array index), whereas in the same file and function in python2.7 all the existing tables are properly return.

Am I doing something wrong? Is this a valid bug? I have also written about it on Stackoverflow here: https://stackoverflow.com/questions/49503235/table-django-migrations-already-exists-error-with-python-3-and-sqlite3/49505445?noredirect=1#comment86058087_49505445

[Alex Laird]

Finally found the solution. I have marked it as the answer on my Stackoverflow question here: https://stackoverflow.com/a/49641729/1128413