[Django] #31398: multiple_database.tests.AuthTestCase doesn't flush the default database if transactions aren't supported
Django
transactions aren't supported
----------------------------------------+------------------------
Reporter: Tim Graham | Owner: nobody
Type: Bug | Status: new
Component: Core (Other) | Version: master
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
----------------------------------------+------------------------
On databases that don't support transactions, the database is flushed
between test methods.
In Django's test suite,
[https://github.com/django/django/blob/8fe2447a01232d9598bf4692d1face12fa074288/tests/multiple_database/tests.py#L1527-L1577
multiple_database.tests.AuthTestCase.test_auth_manager()] is the first
test to run in that class. It creates two Users: alice in the 'other'
database and bob in the 'default' database.
The test class uses `AuthRouter` which has an
[https://github.com/django/django/blob/8fe2447a01232d9598bf4692d1face12fa074288/tests/multiple_database/routers.py#L48-L52
allow_migrate()] method that makes sure the auth app only appears in the
'other' db, however, the test database creation doesn't respect that
router and so the table is created in both databases (thus no errors when
creating the users as described above).
The problem comes during `TransactionTestCase. _fixture_teardown()` after
the first test. It invokes the flush command which doesn't include any
tables from the auth app because of the `allow_migrate()` method on the
router. (The flush command calls
[https://github.com/django/django/blob/8fe2447a01232d9598bf4692d1face12fa074288/django/core/management/commands/flush.py#L47
django.core.management.sql.sql_flush] which calls
[https://github.com/django/django/blob/8fe2447a01232d9598bf4692d1face12fa074288/django/core/management/sql.py#L13
BaseDatabaseIntrospection.django_table_names()] which calls
[https://github.com/django/django/blob/8fe2447a01232d9598bf4692d1face12fa074288/django/db/backends/base/introspection.py#L85
router.get_migratable_models()] which eventually invokes
[https://github.com/django/django/blob/8fe2447a01232d9598bf4692d1face12fa074288/django/db/utils.py#L309-L320
allow_migrate()].) Since the auth_user table isn't flushed after the first
test, an `IntegrityError` (violating the unique username constraint) is
raised when
[https://github.com/django/django/blob/8fe2447a01232d9598bf4692d1face12fa074288/tests/multiple_database/tests.py#L1564
the second test creates bob in the 'default' database].
--
Ticket URL: <https://code.djangoproject.com/ticket/31398>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
------------------------------+------------------------------------
Reporter: Tim Graham | Owner: nobody
Type: Bug | Status: new
Component: Core (Other) | Version: master
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by felixxm):
* stage: Unreviewed => Accepted
--
Ticket URL: <https://code.djangoproject.com/ticket/31398#comment:1>
Django
-------------------------------------+-------------------------------------
Reporter: Tim Graham | Owner: Hassaan
| Ali Wattoo
Type: Bug | Status: assigned
Component: Core (Other) | Version: master
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Hassaan Ali Wattoo):
* owner: nobody => Hassaan Ali Wattoo
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/31398#comment:2>
Django
-------------------------------------+-------------------------------------
Reporter: Tim Graham | Owner: Hassaan
| Ali Wattoo
Type: Bug | Status: assigned
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Hassaan Ali Wattoo):
I'm having trouble duplicating this issue. Running the multiple_database
test suite for the latest code on master results in no failures.
{{{
#!div style="font-size: 80%"
{{{#!python
.............................................................................
----------------------------------------------------------------------
Ran 77 tests in 0.366s
}}}
}}}
Could you give us more information about how you reproduced this issue?
(version of Django, operating system, steps to reproduce the issue)?
Thanks!
--
Ticket URL: <https://code.djangoproject.com/ticket/31398#comment:3>
Django
-------------------------------------+-------------------------------------
Reporter: Tim Graham | Owner: Hassaan
| Ali Wattoo
Type: Bug | Status: assigned
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Tim Graham):
This issue only affects databases that don't support transactions. I used
[https://github.com/googleapis/python-spanner-django Cloud Spanner].
--
Ticket URL: <https://code.djangoproject.com/ticket/31398#comment:4>
Django
-------------------------------------+-------------------------------------
Reporter: Tim Graham | Owner: Hassaan
| Ali Wattoo
Type: Bug | Status: assigned
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Simon Charette):
Hassaan, you could try running the suite with MySQL's MyISAM to reproduce
locally.
--
Ticket URL: <https://code.djangoproject.com/ticket/31398#comment:5>
Django
-------------------------------------+-------------------------------------
Reporter: Tim Graham | Owner: Hassaan
| Ali Wattoo
Type: Bug | Status: assigned
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Hassaan Ali Wattoo):
Replying to [comment:5 Simon Charette]:
> Hassaan, you could try running the suite with MySQL's MyISAM to
reproduce locally.
Using MySQL 8.0.11 MyISAM, I ran into a separate issue. It seems that the
following line creates a record in the default database instead of 'other'
as expected.
{{{
1535. User.objects.create_user('alice', 'al...@example.com')
}}}
The behavior then is that the test case fails at the following assert
{{{
1546. with self.assertRaises(User.DoesNotExist):
User.objects.using('default').get(username='alice')
}}}
However, the second test case passes successfully, indicating that the
database flushes correctly. Do you have any idea why this might happen?
--
Ticket URL: <https://code.djangoproject.com/ticket/31398#comment:6>
Django
-------------------------------------+-------------------------------------
Reporter: Tim Graham | Owner: Hassaan
| Ali Wattoo
Type: Bug | Status: assigned
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Tim Graham):
Hi Hassaan, I can't reproduce the issue you described.
After adding:
{{{
'OPTIONS': {
'init_command': 'SET default_storage_engine=MYISAM',
}
}}}
to both databases in the test settings files, I can reproduce the issue I
reported:
{{{
$ ./tests/runtests.py --settings=test_mysql --parallel=1
multiple_database.tests.AuthTestCase
Testing against Django installed in '/home/tim/code/django/django'
Creating test database for alias 'default'...
Creating test database for alias 'other'...
System check identified no issues (0 silenced).
.E
======================================================================
ERROR: test_dumpdata (multiple_database.tests.AuthTestCase)
dumpdata honors allow_migrate restrictions on the router
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/tim/code/django/django/db/backends/utils.py", line 84, in
_execute
return self.cursor.execute(sql, params)
File "/home/tim/code/django/django/db/backends/mysql/base.py", line 73,
in execute
return self.cursor.execute(query, args)
File "/home/tim/.virtualenvs/django37/lib/python3.7/site-
packages/MySQLdb/cursors.py", line 209, in execute
res = self._query(query)
File "/home/tim/.virtualenvs/django37/lib/python3.7/site-
packages/MySQLdb/cursors.py", line 315, in _query
db.query(q)
File "/home/tim/.virtualenvs/django37/lib/python3.7/site-
packages/MySQLdb/connections.py", line 239, in query
_mysql.connection.query(self, query)
MySQLdb._exceptions.IntegrityError: (1062, "Duplicate entry 'bob' for key
'username'")
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/tim/code/django/tests/multiple_database/tests.py", line
1564, in test_dumpdata
User.objects.db_manager('default').create_user('bob',
'b...@example.com')
File "/home/tim/code/django/django/contrib/auth/models.py", line 146, in
create_user
return self._create_user(username, email, password, **extra_fields)
File "/home/tim/code/django/django/contrib/auth/models.py", line 140, in
_create_user
user.save(using=self._db)
File "/home/tim/code/django/django/contrib/auth/base_user.py", line 66,
in save
super().save(*args, **kwargs)
File "/home/tim/code/django/django/db/models/base.py", line 750, in save
force_update=force_update, update_fields=update_fields)
File "/home/tim/code/django/django/db/models/base.py", line 788, in
save_base
force_update, using, update_fields,
File "/home/tim/code/django/django/db/models/base.py", line 891, in
_save_table
results = self._do_insert(cls._base_manager, using, fields,
returning_fields, raw)
File "/home/tim/code/django/django/db/models/base.py", line 931, in
_do_insert
using=using, raw=raw,
File "/home/tim/code/django/django/db/models/manager.py", line 85, in
manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/home/tim/code/django/django/db/models/query.py", line 1248, in
_insert
return query.get_compiler(using=using).execute_sql(returning_fields)
File "/home/tim/code/django/django/db/models/sql/compiler.py", line
1386, in execute_sql
cursor.execute(sql, params)
File "/home/tim/code/django/django/db/backends/utils.py", line 66, in
execute
return self._execute_with_wrappers(sql, params, many=False,
executor=self._execute)
File "/home/tim/code/django/django/db/backends/utils.py", line 75, in
_execute_with_wrappers
return executor(sql, params, many, context)
File "/home/tim/code/django/django/db/backends/utils.py", line 84, in
_execute
return self.cursor.execute(sql, params)
File "/home/tim/code/django/django/db/utils.py", line 90, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/home/tim/code/django/django/db/backends/utils.py", line 84, in
_execute
return self.cursor.execute(sql, params)
File "/home/tim/code/django/django/db/backends/mysql/base.py", line 73,
in execute
return self.cursor.execute(query, args)
File "/home/tim/.virtualenvs/django37/lib/python3.7/site-
packages/MySQLdb/cursors.py", line 209, in execute
res = self._query(query)
File "/home/tim/.virtualenvs/django37/lib/python3.7/site-
packages/MySQLdb/cursors.py", line 315, in _query
db.query(q)
File "/home/tim/.virtualenvs/django37/lib/python3.7/site-
packages/MySQLdb/connections.py", line 239, in query
_mysql.connection.query(self, query)
django.db.utils.IntegrityError: (1062, "Duplicate entry 'bob' for key
'username'")
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/31398#comment:7>
Django
------------------------------+------------------------------------
Type: Bug | Status: new
Component: Core (Other) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* owner: Hassaan Ali Wattoo => (none)
* status: assigned => new
--
Ticket URL: <https://code.djangoproject.com/ticket/31398#comment:8>
Django
------------------------------+------------------------------------
Reporter: Tim Graham | Owner: (none)
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Ülgen Sarıkavak):
* cc: Ülgen Sarıkavak (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/31398#comment:9>
Django
------------------------------+------------------------------------
Reporter: Tim Graham | Owner: (none)
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Calvin Vu):
* cc: Calvin Vu (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/31398#comment:10>
Django
------------------------------+------------------------------------
Reporter: Tim Graham | Owner: (none)
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Calvin Vu):
--
Ticket URL: <https://code.djangoproject.com/ticket/31398#comment:11>