Why does sqlmigrate need a connection to an existing database?

203 views
Skip to first unread message

Shmuel Treiger

unread,
Dec 7, 2021, 7:46:42 PM12/7/21
to Django users
Wondering why sqlmigrate needs a connection to an existing database.

I understand that for certain commands, it needs the database to generate the migration. But for most (basic) commands, no connection is really needed.

For example, I created a test project and generated migration files. Without ever running the `migrate` command, I was able to use `sqlmigrate` perfectly well on all migrations, save the last, which drops a `unique_together`.

I'm sure there are other commands which actually need a database connection (though I haven't found them yet). Seems to me sqlmigrate could be rewritten along the lines of:
  • Try to generate sql from migration file
  • If it hits a command it needs the database, attempt to connect
Or else:
  • If no connection, check if migration file needs connection to database
Is there something I'm unaware of?


This email may contain confidential material; unintended recipients must not disseminate, use, or act upon any information in it. If you received this email in error, please contact the sender and permanently delete the email.

Jason

unread,
Dec 9, 2021, 8:01:31 AM12/9/21
to Django users
It uses db connection in two places:

building the migration graph in order to load previous applied migrations and transaction begin/end wrapping

This makes sense, because you can have N unapplied migrations between the db and your migrations history package.  

Shmuel Treiger

unread,
Dec 10, 2021, 12:06:21 PM12/10/21
to Django users
Build graph has a specific separate path for `if self.connection is None:`, so that shouldn't be an issue?
As far as atomicity, what if, in a case where no connection is found, it could just prompt: "No database connection found. Do you want your migration to be atomic? (y/N)"

Jason

unread,
Dec 11, 2021, 10:59:32 AM12/11/21
to Django users
https://github.com/django/django/blob/main/django/db/migrations/loader.py#L20-L40

the comment in the migration loader class probably explains your question.  also, look at `collect_sql` at the bottom, and how its used at the end of sqlmigrate. specifically, it uses schema editor (https://docs.djangoproject.com/en/3.2/ref/schema-editor/#module-django.db.backends.base.schema) as a context manager to have the connection turn migration nodes to SQL.

Shmuel Treiger

unread,
Dec 24, 2021, 6:39:42 PM12/24/21
to Django users
To any future readers, the real answer is that connections are deeply baked into how Django migrations operate. Things as seemingly simple as `can_migrate_model` require a connection. The other problems pointed out by Jason are easily patched, but the schema_editor (and all of the migration operations in migrations/operations which rely on the schema_editor) totally break without a connection.

I think it could be done, but it isn't an easy patch.
Reply all
Reply to author
Forward
0 new messages