[Django] #30431: Migration hangs when using UUIDField with unique default value & multiple databases
Django
databases
-------------------------------------+-------------------------------------
Reporter: mandm | Owner: nobody
Type: Bug | Status: new
Component: | Version: 1.11
Migrations | Keywords: migrations, uuid,
Severity: Normal | uuidfield, database router,
Triage Stage: | multiple database
Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
I'm trying to add a UUIDField in an existing user model following this
method (with unique=True & default=uuid.uuid4).
https://docs.djangoproject.com/en/1.11/howto/writing-migrations
/#migrations-that-add-unique-fields
This works with a single database but hangs in the following scenarios
with multiple databases :
- Migration without `--database=foobar` works
- Migration with `--database=foobar` option hangs
- Migration without `--database=foobar`, but with `using('foobar')` in the
query in gen_uuid function hangs
Here's a snippet of the code I've used. I would be glad to provide
additional information to reproduce the issue.
User model :
{{{
class CustomUser(AbstractUser):
address = models.TextField()
uniq_id = models.UUIDField(unique=True, default=uuid.uuid4)
class Meta:
db_table = 'customuser'
}}}
Migration file
{{{
def gen_uuid(apps, schema_editor):
UserModel = apps.get_model('myapp', 'customuser')
for row in UserModel.objects.all(): #
.objects.using('foobar').all(): -- HANGS !
row.uniq_id = uuid.uuid4()
row.save(update_fields=['uniq_id']) # .save(using='foobar', ...)
-- HANGS with database routers !
class Migration(migrations.Migration):
dependencies = [
('myapp', '003_previous_migration'),
]
operations = [
migrations.AddField(
model_name='customuser',
name='uniq_id',
field=models.UUIDField(default=uuid.uuid4, null=True),
),
migrations.RunPython(gen_uuid,
reverse_code=migrations.RunPython.noop),
migrations.AlterField(
model_name='customuser',
name='uniq_id',
field=models.UUIDField(default=uuid.uuid4, unique=True),
),
]
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/30431>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
databases
-------------------------------------+-------------------------------------
Reporter: mandm | Owner: nobody
Type: Bug | Status: new
Severity: Normal | Resolution:
Keywords: migrations, uuid, | Triage Stage:
uuidfield, database router, | Unreviewed
multiple database |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Old description:
> Migration file
>
New description:
I'm trying to add a UUIDField in an existing user model following this
method (with unique=True & default=uuid.uuid4).
https://docs.djangoproject.com/en/1.11/howto/writing-migrations
/#migrations-that-add-unique-fields
This works with a single database but hangs in the following scenarios
with multiple databases :
- Hangs ! Migration with option `--database=foobar`
- Hangs ! Migration without option `--database=foobar`, but with
`using('foobar')` in the query in gen_uuid function
- Works ! Migration without option `--database=foobar`
User model :
Migration file
--
--
Ticket URL: <https://code.djangoproject.com/ticket/30431#comment:1>
Django
-------------------------------------+-------------------------------------
Reporter: mandm | Owner: nobody
Type: Bug | Status: new
Severity: Normal | Resolution:
Keywords: migrations, uuid, | Triage Stage:
uuidfield, database router, | Unreviewed
multiple database |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Old description:
> I'm trying to add a UUIDField in an existing user model following this
> method (with unique=True & default=uuid.uuid4).
> https://docs.djangoproject.com/en/1.11/howto/writing-migrations
> /#migrations-that-add-unique-fields
>
> This works with a single database but hangs in the following scenarios
> with multiple databases :
>
> - Hangs ! Migration with option `--database=foobar`
> - Hangs ! Migration without option `--database=foobar`, but with
> `using('foobar')` in the query in gen_uuid function
> - Works ! Migration without option `--database=foobar`
>
> Here's a snippet of the code I've used. I would be glad to provide
> Migration file
>
New description:
I'm trying to add a UUIDField in an existing user model following this
method (with unique=True & default=uuid.uuid4).
https://docs.djangoproject.com/en/1.11/howto/writing-migrations
/#migrations-that-add-unique-fields
This works with a single database but hangs in the following scenarios
with multiple databases :
- Hangs
- python manage.py migrate --database=nondefault
- also hangs with `.using('nondefault')` in the query in gen_uuid
function
- Works
- python manage.py migrate
- python manage.py migrate --database=default
It seems to always hang at this statement `UserModel.objects.all()` when a
non-default database is used either via `--database` option in migrate or
via `.using('nondefault')` in query.
User model :
Migration file
--
--
Ticket URL: <https://code.djangoproject.com/ticket/30431#comment:2>
Django
Reporter: mandm | Owner: nobody
Type: Bug | Status: new
Severity: Normal | Resolution:
Keywords: migrations, uuid, | Triage Stage:
uuidfield, database router, | Unreviewed
multiple database |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Old description:
> I'm trying to add a UUIDField in an existing user model following this
> method (with unique=True & default=uuid.uuid4).
> https://docs.djangoproject.com/en/1.11/howto/writing-migrations
> /#migrations-that-add-unique-fields
>
> This works with a single database but hangs in the following scenarios
> with multiple databases :
>
> - Hangs
> - python manage.py migrate --database=nondefault
> - also hangs with `.using('nondefault')` in the query in gen_uuid
> function
>
> - Works
> - python manage.py migrate
> - python manage.py migrate --database=default
>
> It seems to always hang at this statement `UserModel.objects.all()` when
> a non-default database is used either via `--database` option in migrate
> or via `.using('nondefault')` in query.
>
> Migration file
>
New description:
I'm trying to add a UUIDField in an existing user model following this
method (with unique=True & default=uuid.uuid4).
https://docs.djangoproject.com/en/1.11/howto/writing-migrations
/#migrations-that-add-unique-fields
This works with a single database but hangs in the following scenarios
with multiple databases :
- Hangs
- python manage.py migrate --database=nondefault
- also hangs with `.using('nondefault')` in the query in gen_uuid
function
- Works
- python manage.py migrate
- python manage.py migrate --database=default
It seems to always hang at this statement `UserModel.objects.all()` when a
non-default database is used either via `--database` option in migrate or
via `.using('nondefault')` in query.
Here's a snippet of the code I've used. I would be glad to provide
additional information to reproduce the issue.
User model :
{{{
class CustomUser(AbstractUser):
address = models.TextField()
uniq_id = models.UUIDField(unique=True, default=uuid.uuid4)
class Meta:
db_table = 'customuser'
}}}
Migration file
{{{
def gen_uuid(apps, schema_editor):
UserModel = apps.get_model('myapp', 'customuser')
for row in UserModel.objects.all(): #
.objects.using('nondefault').all(): -- HANGS !
row.uniq_id = uuid.uuid4()
row.save(update_fields=['uniq_id'])
class Migration(migrations.Migration):
dependencies = [
('myapp', '003_previous_migration'),
]
operations = [
migrations.AddField(
model_name='customuser',
name='uniq_id',
field=models.UUIDField(default=uuid.uuid4, null=True),
),
migrations.RunPython(gen_uuid,
reverse_code=migrations.RunPython.noop),
migrations.AlterField(
model_name='customuser',
name='uniq_id',
field=models.UUIDField(default=uuid.uuid4, unique=True),
),
]
}}}
--
--
Ticket URL: <https://code.djangoproject.com/ticket/30431#comment:3>
Django
-------------------------------------+-------------------------------------
Reporter: mandm | Owner: nobody
Component: Migrations | Version: master
Severity: Normal | Resolution:
Keywords: migrations, uuid, | worksforme
uuidfield, database router, | Triage Stage:
multiple database | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => closed
* version: 1.11 => master
* resolution: => worksforme
Comment:
Thanks for the report, but described case works for me.
First of all you should use `schema_editor.connection.alias` in
`gen_uuid()` to fix it for multiple databases (see
[https://docs.djangoproject.com/en/stable/ref/migration-
operations/#runpython RunPython]), .e.g.
{{{#!python
def gen_uuid(apps, schema_editor):
UserModel = apps.get_model('myapp', 'customuser')
db_alias = schema_editor.connection.alias
for row in UserModel.objects.using(db_alias).all():
row.uniq_id = uuid.uuid4()
row.save(update_fields=['uniq_id'])
}}}
Secondly in mentioned documentation we have three separate migrations not
one. It depends on database but in e.g. PostgreSQL you will not be able to
alter column and update date in a single transaction. My advice is to
split migration files and use `schema_editor.connection.alias`.
Please use one of
[https://code.djangoproject.com/wiki/TicketClosingReasons/UseSupportChannels
support channels] if you need furher help.
--
Ticket URL: <https://code.djangoproject.com/ticket/30431#comment:4>
Django
Reporter: mandm | Owner: nobody
Component: Migrations | Version: master
Severity: Normal | Resolution:
Keywords: migrations, uuid, | worksforme
multiple database | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by mandm):
Thank you @felixxm for the suggestions. A quick checks shows that
`schema_editor.connection.alias` does fix the problem, but this solution
only works in conjunction with routers i.e. unless I use `allow_migrate`,
an accidental migration will affect the default database. I'll try the
support channels to see if there's a way to use the non-default database
directly in a migration without resorting to routers.
--
Ticket URL: <https://code.djangoproject.com/ticket/30431#comment:5>