[Django] #33906: TestCase inconsistent behavior on Db integrity error
Django
-------------------------------------+-------------------------------------
Reporter: an-mile | Owner: nobody
Type: Bug | Status: new
Component: Testing | Version: 4.0
framework | Keywords: testsuite
Severity: Normal | integrityError bug inconsistent
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
Hello! I'm filing my first Django bug and hopefully it is meaningful:
When a db integrity error occurs in some test, the exception is captured
by the test suite and following code in the test is executed. This is
inconsistent with any other exception that might arise from test code
execution.
It might be similar to the closed ticket
https://code.djangoproject.com/ticket/14223
models.py:
{{{
from django.db import models
# inspired by https://code.djangoproject.com/ticket/14223
class Reporter(models.Model):
first_name = models.CharField(max_length=30)
last_name = models.CharField(max_length=30)
email = models.EmailField(null=True)
def __unicode__(self):
return u"%s %s" % (self.first_name, self.last_name)
class Article(models.Model):
reporter = models.ForeignKey(Reporter, on_delete=models.CASCADE,)
headline = models.CharField(max_length=100)
}}}
tests.py:
{{{
from django.test import TestCase
from mod.models import Reporter, Article
class DatabasePsycopgTest(TestCase):
def test_db_integrity_error_handling(self):
objs = [Article(reporter_id=123, headline='Hello DB integrity')]
# NammeErrorWouldRaiseHere
Article.objects.bulk_create(objs) # DB Integrity Error
captured...
print('Hello Dear Django Coders!')
self.assertEqual(1, 2)
}}}
Here we get following traceback:
{{{
Traceback (most recent call last):
File "/..../python3.9/site-packages/django/test/testcases.py", line 299,
in _setup_and_call
self._post_teardown()
File "/..../python3.9/site-packages/django/test/testcases.py", line
1199, in _post_teardown
self._fixture_teardown()
File "/..../python3.9/site-packages/django/test/testcases.py", line
1461, in _fixture_teardown
connections[db_name].check_constraints()
File "/..../python3.9/site-packages/django/db/backends/sqlite3/base.py",
line 383, in check_constraints
raise IntegrityError(
django.db.utils.IntegrityError: The row in table 'mod_article' with
primary key '1' has an invalid foreign key: mod_article.reporter_id
contains a value '123' that does not have a corresponding value in
mod_reporter.id.
File "/..../codelab/dabase/mod/tests.py", line 12, in
test_db_integrity_error_handling
self.assertEqual(1, 2)
AssertionError: 1 != 2
}}}
But when uncommenting the "NammeErrorWouldRaiseHere" , "Hello Dear Django
Coders!'" is not printed, nor the last "AssertionError" is not executed.
This seams confusing and inconsistent behavior.
It appears on SQLite as well as with PsycoPg / PG. Furthermore, the
exception can not be captured within the Code with a regular try-except.
Following modification of the test will not print "Hello" :
{{{
try:
Article.objects.bulk_create(objs)
except Exception:
print('Hello')
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/33906>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Reporter: an-mile | Owner: nobody
Type: Bug | Status: new
Severity: Normal | Resolution:
Keywords: testsuite | Triage Stage:
integrityError bug inconsistent | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Old description:
New description:
Hello! I'm filing my first Django bug and hopefully it is meaningful:
When a db integrity error occurs in some test, the exception is captured
by the test suite and following code in the test is executed. This seems
models.py:
{{{
from django.db import models
}}}
tests.py:
{{{
from django.test import TestCase
class DatabasePsycopgTest(TestCase):
Coders!'" is not printed, nor is the last "AssertionError" executed. This
seems confusing and inconsistent behavior.
It appears on SQLite as well as with PsycoPg / PG. Furthermore, the
exception can not be captured within the Code with a regular try-except.
Following modification of the test will not print "Hello" :
{{{
try:
Article.objects.bulk_create(objs)
except Exception:
print('Hello')
}}}
--
--
Ticket URL: <https://code.djangoproject.com/ticket/33906#comment:1>
Django
-------------------------------------+-------------------------------------
Reporter: an-mile | Owner: nobody
Type: Bug | Status: new
Component: Testing framework | Version: 4.0
Severity: Normal | Resolution:
Keywords: testsuite | Triage Stage:
integrityError bug inconsistent | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Description changed by an-mile:
Old description:
> Hello! I'm filing my first Django bug and hopefully it is meaningful:
>
> When a db integrity error occurs in some test, the exception is captured
> File "/..../python3.9/site-
> packages/django/db/backends/sqlite3/base.py", line 383, in
> check_constraints
> raise IntegrityError(
> django.db.utils.IntegrityError: The row in table 'mod_article' with
> primary key '1' has an invalid foreign key: mod_article.reporter_id
> contains a value '123' that does not have a corresponding value in
> mod_reporter.id.
>
> File "/..../codelab/dabase/mod/tests.py", line 12, in
> test_db_integrity_error_handling
> self.assertEqual(1, 2)
> AssertionError: 1 != 2
> }}}
> But when uncommenting the "NammeErrorWouldRaiseHere" , "Hello Dear
> Django Coders!'" is not printed, nor is the last "AssertionError"
> executed. This seems confusing and inconsistent behavior.
>
> It appears on SQLite as well as with PsycoPg / PG. Furthermore, the
> exception can not be captured within the Code with a regular try-except.
> Following modification of the test will not print "Hello" :
> {{{
> try:
> Article.objects.bulk_create(objs)
> except Exception:
> print('Hello')
> }}}
New description:
models.py:
{{{
from django.db import models
}}}
tests.py:
{{{
from django.test import TestCase
class DatabasePsycopgTest(TestCase):
[Edit: It will also print "Hello Dear Django Coders!"]. But when
uncommenting the "NammeErrorWouldRaiseHere" , "Hello Dear Django
Coders!'" is not printed, nor is the last "AssertionError" executed. This
seems confusing and inconsistent behavior.
It appears on SQLite as well as with PsycoPg / PG. Furthermore, the
exception can not be captured within the Code with a regular try-except.
Following modification of the test will not print "Hello" :
{{{
try:
Article.objects.bulk_create(objs)
except Exception:
print('Hello')
}}}
--
--
Ticket URL: <https://code.djangoproject.com/ticket/33906#comment:2>
Django
-------------------------------------+-------------------------------------
Reporter: an-mile | Owner: nobody
Component: Testing framework | Version: 4.0
Keywords: testsuite | Triage Stage:
integrityError bug inconsistent | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => invalid
Comment:
Thanks for this report, however I don't see any issue here. You raised
`NameError` before trying to insert a new row to the database
(`bulk_create()` call is where we hit the database), that's why
`IntegrityError` is not raised.
If you're having trouble understanding how Django works, see
TicketClosingReasons/UseSupportChannels for ways to get help.
--
Ticket URL: <https://code.djangoproject.com/ticket/33906#comment:3>
Django
-------------------------------------+-------------------------------------
Reporter: an-mile | Owner: nobody
Type: Bug | Status: closed
Component: Testing framework | Version: 4.0
Severity: Normal | Resolution: invalid
Keywords: testsuite | Triage Stage:
integrityError bug inconsistent | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by an-mile):
Here another detail: it does not merrily excessively capturing an
**django.db.utils.IntegrityError** since the following will just fail with
an **IntegrityError** without executing further code :
{{{
def test_db_integrity_error_handling(self):
rera = Reporter.objects.create(id=124)
reru = Reporter.objects.create(id=124)
# test execution interrupted here
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/33906#comment:4>
Django
-------------------------------------+-------------------------------------
Reporter: an-mile | Owner: nobody
Component: Testing framework | Version: 4.0
Severity: Normal | Resolution:
integrityError bug inconsistent | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: closed => new
* resolution: invalid =>
Comment:
Replying to [comment:3 Mariusz Felisiak]:
> Thanks for this report, however I don't see any issue here. You raised
`NameError` before trying to insert a new row to the database
(`bulk_create()` call is where we hit the database), that's why
`IntegrityError` is not raised.
>
hi,
Unfortunately you did not understand the issue: the NameError is not the
issue at all, it was just added to illustrate different behavior between
the two different situations.
The `bulk_create` is executed, raises an exception, but is then somehow
captured by the test suite or so, and the the execution of the test goes
on... Just weird...
--
Ticket URL: <https://code.djangoproject.com/ticket/33906#comment:5>
Django
-------------------------------------+-------------------------------------
Reporter: an-mile | Owner: nobody
Type: Bug | Status: new
Component: Testing framework | Version: 4.0
Severity: Normal | Resolution:
Keywords: testsuite | Triage Stage:
integrityError bug inconsistent | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by an-mile):
I'm sorry for creating confusion with "NammeErrorWouldRaiseHere", but the
whole bug is just sooo weird to me that I just added some elementary
NameErrors, **print** and the like, just to make sure that I am not wrong
...
Replying to [comment:5 an-mile]:
> Replying to [comment:3 Mariusz Felisiak]:
> > Thanks for this report, however I don't see any issue here. You raised
`NameError` before trying to insert a new row to the database
(`bulk_create()` call is where we hit the database), that's why
`IntegrityError` is not raised.
> >
>
> hi,
>
> Unfortunately you did not understand the issue: the NameError is not the
issue at all, it was just added to illustrate different behavior between
the two different situations.
> The `bulk_create` is executed, raises an exception, but is then somehow
captured by the test suite or so, and the the execution of the test goes
on... Just weird...
--
Ticket URL: <https://code.djangoproject.com/ticket/33906#comment:6>
Django
-------------------------------------+-------------------------------------
Reporter: an-mile | Owner: nobody
Component: Testing framework | Version: 4.0
Keywords: testsuite | Triage Stage:
integrityError bug inconsistent | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => invalid
Comment:
> Unfortunately you did not understand the issue: the NameError is not the
issue at all, it was just added to illustrate different behavior between
the two different situations.
That's how you described the issue: ''"But when **uncommenting** the
"NammeErrorWouldRaiseHere" , "Hello Dear Django Coders!'" is not printed,
nor is the last "AssertionError" executed. This seems confusing and
inconsistent behavior. "''. Which is not an issue, it's the expected
behavior.
> The `bulk_create` is executed, raises an exception, but is then somehow
captured by the test suite or so, and the the execution of the test goes
on... Just weird...
`bulk_create()` doesn't raise an exception because you use `TestCase` see
[https://docs.djangoproject.com/en/dev/topics/testing/tools/#testcase
docs]: ''"Checks deferrable database constraints at the end of each
test."''. Use `TransactionTestCase` if you want to raise an exception
immediately.
--
Ticket URL: <https://code.djangoproject.com/ticket/33906#comment:7>
Django
-------------------------------------+-------------------------------------
Reporter: an-mile | Owner: nobody
Type: Bug | Status: closed
Component: Testing framework | Version: 4.0
Severity: Normal | Resolution: invalid
Keywords: testsuite | Triage Stage:
integrityError bug inconsistent | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by an-mile):
Ok I took a further look at the issue: the posted traceback shows that we
enter and execute `_post_teardown()` in "django/test/testcases.py", line
299, `_setup_and_call`
So it seems what is happening is the following :
* At least in the **SQLite** case, the `bulk_create` is executed
completely, hence creating inconsistent data in the DB.
* The `_tear_down` runs some DB integrity check and prints a corresponding
exception...
Does this make sense?
So I am not so sure anymore if it's a bug or an undocumented feature or
so, but it's still confusing...
What also probably created additional confusion is that in the PG case, it
did not insert data into DB creating a slightly different situation.
--
Ticket URL: <https://code.djangoproject.com/ticket/33906#comment:8>
Django
-------------------------------------+-------------------------------------
Reporter: an-mile | Owner: nobody
Type: Bug | Status: closed
Component: Testing framework | Version: 4.0
Severity: Normal | Resolution: invalid
Keywords: testsuite | Triage Stage:
integrityError bug inconsistent | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by an-mile):
Replying to [comment:7 Mariusz Felisiak]:
> That's how you described the issue: ''"But when **uncommenting** the
"NammeErrorWouldRaiseHere" , "Hello Dear Django Coders!'" is not printed,
nor is the last "AssertionError" executed. This seems confusing and
inconsistent behavior. "''. Which is not an issue, it's the expected
behavior.
Well that was surely not what I meant to be the issue but indeed the
regular behavior, meant as comparison. Again sorry for confusion.
--
Ticket URL: <https://code.djangoproject.com/ticket/33906#comment:9>