[Django] #30526: migration to UUID id field doesn't properly convert integers (SQLite)
Django
----------------------------------------+------------------------
Reporter: Martin Baker | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 2.2
Severity: Normal | Keywords: UUID
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
----------------------------------------+------------------------
This may be present in other DBs, but I've noticed it with SQLite under
Django 2.2.1.
Steps to reproduce:
1. Start a model with an implicit integer id, and create and run the
migration:
{{{#!python
from django.db import models
class Something(models.Model):
name = models.CharField(max_length=32)
}}}
2. Then create one instance via the shell (`python manage.py shell`):
{{{#!python
from uuidbug.models import Something
s = Something(name="One")
s.save()
}}}
In the DB it looks like this (can use `python manage.py dbshell`):
{{{
sqlite> .headers on
sqlite> select * from uuiditem_something;
name|id
One|1
}}}
3. Edit the model to use UUIDs:
{{{#!python
import uuid
from django.db import models
class Something(models.Model):
name = models.CharField(max_length=32)
id = models.UUIDField(primary_key=True, default=uuid.uuid4,
editable=False)
}}}
4. After creating and running the migration, add another instance:
{{{#!python
from uuidbug.models import Something
s = Something(name="After UUID-ization")
s.save()
}}}
DB now looks like this:
{{{
sqlite> select * from uuiditem_something;
name|id
One|1
After UUID-ization|693e1942d7d142289bb9fb3664f2c11a
}}}
Note that the id for the first instance has not been converted to a UUID
properly. If you try to access the objects, you get an error:
{{{#!python
>>> Something.objects.all()
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/Users/martin/Connery/opencc-backend/venv/lib/python3.7/site-
packages/django/db/models/manager.py", line 82, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/Users/martin/Connery/opencc-backend/venv/lib/python3.7/site-
packages/django/db/models/query.py", line 653, in first
for obj in (self if self.ordered else self.order_by('pk'))[:1]:
File "/Users/martin/Connery/opencc-backend/venv/lib/python3.7/site-
packages/django/db/models/query.py", line 274, in __iter__
self._fetch_all()
File "/Users/martin/Connery/opencc-backend/venv/lib/python3.7/site-
packages/django/db/models/query.py", line 1242, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/Users/martin/Connery/opencc-backend/venv/lib/python3.7/site-
packages/django/db/models/query.py", line 72, in __iter__
for row in compiler.results_iter(results):
File "/Users/martin/Connery/opencc-backend/venv/lib/python3.7/site-
packages/django/db/models/sql/compiler.py", line 1044, in apply_converters
value = converter(value, expression, connection)
File "/Users/martin/Connery/opencc-backend/venv/lib/python3.7/site-
packages/django/db/backends/sqlite3/operations.py", line 294, in
convert_uuidfield_value
value = uuid.UUID(value)
File
"/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/uuid.py",
line 160, in __init__
raise ValueError('badly formed hexadecimal UUID string')
ValueError: badly formed hexadecimal UUID string
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/30526>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: Martin Baker | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 2.2
Keywords: UUID | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Description changed by Martin Baker:
Old description:
New description:
Steps to reproduce:
{{{#!python
import uuid
from django.db import models
{{{
sqlite> select * from uuidbug_something;
--
--
Ticket URL: <https://code.djangoproject.com/ticket/30526#comment:1>
Django
------------------------------+--------------------------------------
Reporter: Martin Baker | Owner: nobody
Component: Migrations | Version: master
Severity: Normal | Resolution: invalid
Keywords: UUID | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by felixxm):
* status: new => closed
* version: 2.2 => master
* resolution: => invalid
Comment:
Thanks for this report, however changing column data type is always a
sensitive operation and you need to be aware of database caveats. SQLite
doesn't have a native `UUID` data type, so Django handles it as a
`char(32)`. Altering data type from `integer` to `char` is possible with
auto-conversion that's why you didn't get any issue when running this
migration. I don't think that there is much that Django can do here.
To fix existing data you can add
[https://docs.djangoproject.com/en/stable/topics/migrations/#data-
migrations custom data migration] and use Python's
`uuid.UUID(int=existing_pk)` to convert data from `int` to `UUID`.
--
Ticket URL: <https://code.djangoproject.com/ticket/30526#comment:2>
Django
------------------------------+--------------------------------------
Reporter: Martin Baker | Owner: nobody
Component: Migrations | Version: master
Severity: Normal | Resolution: invalid
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Martin Baker):
I guess to me this just seems like something Django should be able to
automagically take care of. This seems like a semi-frequent use case: I
made some objects, oh oops I want them to be non-sequential / UUID-y.
--
Ticket URL: <https://code.djangoproject.com/ticket/30526#comment:3>