[Django] #31682: sqlite introspection does not work with constraints
Django
-------------------------------------+-------------------------------------
Reporter: sturmf | Owner: nobody
Type: Bug | Status: new
Component: Database | Version: 2.2
layer (models, ORM) |
Severity: Release | Keywords:
blocker |
Triage Stage: | Has patch: 1
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 1
UI/UX: 0 |
-------------------------------------+-------------------------------------
When I run a migration during testing with sqlite as database backend I
get a backtrace. The reason is a wrong constraint predicate in a list
comprehension in introspection.py in line 333:
tokens = (token for token in statement.flatten() if not
token.is_whitespace)
it works for me with this change:
tokens = (token for token in statement.flatten() if not
token.is_whitespace())
Without this change the tokens list is empty and hence the crash shown
below.
This all is on Django 2.2.13 on Ubuntu 18.04 and Python 3.6.9.
backtrace:
{{{
Applying main.0006_auto_20200128_1650...Traceback (most recent call
last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "env/lib/python3.6/site-
packages/django/core/management/__init__.py", line 381, in
execute_from_command_line
utility.execute()
File "env/lib/python3.6/site-
packages/django/core/management/__init__.py", line 375, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "env/lib/python3.6/site-
packages/django/core/management/commands/test.py", line 23, in
run_from_argv
super().run_from_argv(argv)
File "env/lib/python3.6/site-packages/django/core/management/base.py",
line 323, in run_from_argv
self.execute(*args, **cmd_options)
File "env/lib/python3.6/site-packages/django/core/management/base.py",
line 364, in execute
output = self.handle(*args, **options)
File "env/lib/python3.6/site-
packages/django/core/management/commands/test.py", line 53, in handle
failures = test_runner.run_tests(test_labels)
File "env/lib/python3.6/site-packages/django/test/runner.py", line 629,
in run_tests
old_config = self.setup_databases(aliases=databases)
File "env/lib/python3.6/site-packages/django/test/runner.py", line 554,
in setup_databases
self.parallel, **kwargs
File "env/lib/python3.6/site-packages/django/test/utils.py", line 174,
in setup_databases
serialize=connection.settings_dict.get('TEST', {}).get('SERIALIZE',
True),
File "env/lib/python3.6/site-
packages/django/db/backends/base/creation.py", line 72, in create_test_db
run_syncdb=True,
File "env/lib/python3.6/site-
packages/django/core/management/__init__.py", line 148, in call_command
return command.execute(*args, **defaults)
File "env/lib/python3.6/site-packages/django/core/management/base.py",
line 364, in execute
output = self.handle(*args, **options)
File "env/lib/python3.6/site-packages/django/core/management/base.py",
line 83, in wrapped
res = handle_func(*args, **kwargs)
File "env/lib/python3.6/site-
packages/django/core/management/commands/migrate.py", line 234, in handle
fake_initial=fake_initial,
File "env/lib/python3.6/site-packages/django/db/migrations/executor.py",
line 117, in migrate
state = self._migrate_all_forwards(state, plan, full_plan, fake=fake,
fake_initial=fake_initial)
File "env/lib/python3.6/site-packages/django/db/migrations/executor.py",
line 147, in _migrate_all_forwards
state = self.apply_migration(state, migration, fake=fake,
fake_initial=fake_initial)
File "env/lib/python3.6/site-packages/django/db/migrations/executor.py",
line 245, in apply_migration
state = migration.apply(state, schema_editor)
File "env/lib/python3.6/site-
packages/django/db/migrations/migration.py", line 124, in apply
operation.database_forwards(self.app_label, schema_editor, old_state,
project_state)
File "env/lib/python3.6/site-
packages/django/db/migrations/operations/models.py", line 530, in
database_forwards
getattr(new_model._meta, self.option_name, set()),
File "env/lib/python3.6/site-
packages/django/db/backends/base/schema.py", line 363, in
alter_unique_together
self._delete_composed_index(model, fields, {'unique': True},
self.sql_delete_unique)
File "env/lib/python3.6/site-
packages/django/db/backends/base/schema.py", line 391, in
_delete_composed_index
**constraint_kwargs
File "env/lib/python3.6/site-
packages/django/db/backends/base/schema.py", line 1115, in
_constraint_names
constraints = self.connection.introspection.get_constraints(cursor,
model._meta.db_table)
File "env/lib/python3.6/site-
packages/django/db/backends/sqlite3/introspection.py", line 375, in
get_constraints
constraints.update(self._parse_table_constraints(table_schema,
columns))
File "env/lib/python3.6/site-
packages/django/db/backends/sqlite3/introspection.py", line 353, in
_parse_table_constraints
if end_token.match(sqlparse.tokens.Punctuation, ')'):
AttributeError: 'NoneType' object has no attribute 'match'
Makefile:3: recipe for target 'test' failed
make: *** [test] Error 1
}}}
migration:
{{{
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import models, migrations
class Migration(migrations.Migration):
dependencies = [
('main', '0005_software_deleted'),
]
operations = [
migrations.AlterField(
model_name='module',
name='file_name',
field=models.CharField(help_text=b'filename under which the
module should be saved', max_length=255, editable=False),
preserve_default=True,
),
migrations.AlterField(
model_name='testcaseresultdata',
name='test_case_result',
field=models.ForeignKey(on_delete=models.CASCADE,
related_name='extra_data', to='main.TestCaseResult', unique=True),
preserve_default=True,
),
migrations.AlterUniqueTogether(
name='module',
unique_together=set([]),
),
]
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/31682>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Fabian Sturm):
the necessary change is now implemented in this pull request:
https://github.com/django/django/pull/13039
--
Ticket URL: <https://code.djangoproject.com/ticket/31682#comment:1>
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Simon Charette):
Mariusz in the light of this issue and
[https://github.com/django/django/pull/12025#issuecomment-550342889 this
comment] what do you think of backporting the `>=0.2.2` specifier to
2.2.x? It's more of a packaging issue IMO.
--
Ticket URL: <https://code.djangoproject.com/ticket/31682#comment:2>
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 2.2
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Fabian Sturm):
Yes please backport the version specifier, it cost me a whole day figuring
it out. Otherwise how can someone know if Django is running with
compatible dependencies?
--
Ticket URL: <https://code.djangoproject.com/ticket/31682#comment:3>
Django
-------------------------------+------------------------------------
Reporter: Fabian Sturm | Owner: felixxm
Type: Bug | Status: assigned
Component: Documentation | Version: 2.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
Changes (by felixxm):
* status: new => assigned
* severity: Release blocker => Normal
* component: Database layer (models, ORM) => Documentation
* owner: nobody => felixxm
* stage: Unreviewed => Accepted
Comment:
Agreed, I missed that we backported breaking change to the Django 2.2. I
backported the main change (cdad78e6ee562f8f3acce065e396fb90d04180a1) and
prepared a doc fix [https://github.com/django/django/pull/13042 PR].
--
Ticket URL: <https://code.djangoproject.com/ticket/31682#comment:4>
Django
-------------------------------+------------------------------------
Reporter: Fabian Sturm | Owner: felixxm
Type: Bug | Status: assigned
Component: Documentation | Version: 2.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
Comment (by GitHub <noreply@…>):
In [changeset:"4339f2aff272bceabd67e452c65bcfe0700b3f09" 4339f2af]:
{{{
#!CommitTicketReference repository=""
revision="4339f2aff272bceabd67e452c65bcfe0700b3f09"
Refs #31682 -- Doc'd minimal sqlparse version in Django 2.2.
Support for sqlparse < 0.2.2 was broken in
40b0a58f5ff949fba1072627e4ad11ef98aa7f36 because is_whitespace property
was added in sqlparse 0.2.2.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/31682#comment:5>
Django
-------------------------------+------------------------------------
Reporter: Fabian Sturm | Owner: felixxm
Type: Bug | Status: assigned
Component: Documentation | Version: 2.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"714d7cb41521d5da03f437814112dfb6ce873f34" 714d7cb]:
{{{
#!CommitTicketReference repository=""
revision="714d7cb41521d5da03f437814112dfb6ce873f34"
[3.1.x] Refs #31682 -- Doc'd minimal sqlparse version in Django 2.2.
Support for sqlparse < 0.2.2 was broken in
40b0a58f5ff949fba1072627e4ad11ef98aa7f36 because is_whitespace property
was added in sqlparse 0.2.2.
Backport of 4339f2aff272bceabd67e452c65bcfe0700b3f09 from master
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/31682#comment:6>
Django
-------------------------------+------------------------------------
Reporter: Fabian Sturm | Owner: felixxm
Type: Bug | Status: assigned
Component: Documentation | Version: 2.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"9ec6eca136cbc906f55e5844732cde6ea55b2152" 9ec6eca1]:
{{{
#!CommitTicketReference repository=""
revision="9ec6eca136cbc906f55e5844732cde6ea55b2152"
[3.0.x] Refs #31682 -- Doc'd minimal sqlparse version in Django 2.2.
Support for sqlparse < 0.2.2 was broken in
40b0a58f5ff949fba1072627e4ad11ef98aa7f36 because is_whitespace property
was added in sqlparse 0.2.2.
Backport of 4339f2aff272bceabd67e452c65bcfe0700b3f09 from master
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/31682#comment:7>
Django
-------------------------------+------------------------------------
Reporter: Fabian Sturm | Owner: felixxm
Type: Bug | Status: assigned
Component: Documentation | Version: 2.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"9ecce34afe7ca2932bec0dfadd4c68b3f5e1fb4b" 9ecce34]:
{{{
#!CommitTicketReference repository=""
revision="9ecce34afe7ca2932bec0dfadd4c68b3f5e1fb4b"
[2.2.x] Refs #31682 -- Doc'd minimal sqlparse version in Django 2.2.
Support for sqlparse < 0.2.2 was broken in
40b0a58f5ff949fba1072627e4ad11ef98aa7f36 because is_whitespace property
was added in sqlparse 0.2.2.
Backport of 4339f2aff272bceabd67e452c65bcfe0700b3f09 from master.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/31682#comment:8>
Django
-------------------------------+------------------------------------
Reporter: Fabian Sturm | Owner: felixxm
Component: Documentation | Version: 2.2
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
Changes (by felixxm):
* status: assigned => closed
* resolution: => fixed
--
Ticket URL: <https://code.djangoproject.com/ticket/31682#comment:9>