Should TransactionTestCase flush db in teardown rather than setup?
Sean Legassick
(I'm posting this here rather than filing a ticket, because I'm not
sure whether I'm missing something. If I'm not than I will file a
ticket with a patch).
It strikes me that TransactionTestCase should be flushing the database
in the teardown stage of the test case rather than the setup. My
thinking is that when using a mixture of TestCase and
TransactionTestCase (where the latter is only used when testing
transaction-related behaviour) what's important is that the test db is
left in the 'clean' state at the end of the test. Otherwise subsequent
TestCase tests will be operating on a test db with any modifications
made by the TransactionTestCase test still in place, and thus
potentially failing.
So am I missing something or should this be changed?
Sean
--
Sean Legassick
sean.le...@gmail.com
George Vilches
On Apr 15, 2009, at 2:28 PM, Sean Legassick wrote:
> It strikes me that TransactionTestCase should be flushing the database
> in the teardown stage of the test case rather than the setup. My
> thinking is that when using a mixture of TestCase and
> TransactionTestCase (where the latter is only used when testing
> transaction-related behaviour) what's important is that the test db is
> left in the 'clean' state at the end of the test. Otherwise subsequent
> TestCase tests will be operating on a test db with any modifications
> made by the TransactionTestCase test still in place, and thus
> potentially failing.
>
> So am I missing something or should this be changed?
>
> Sean
As someone who's managed to hard-crash the Python interpreter while
building tests for the test suite, they most definitely should be in
the setup phase. It's the only way to ensure that right before your
test starts, no matter what horrible things happened previously, your
test DB will stay intact.
If there's a need for end cleaning as well, I don't know. I'll let
someone smarter than I speak to that.
George
Sean Legassick
On 15 Apr 2009, at 19:44, George Vilches wrote:
> As someone who's managed to hard-crash the Python interpreter while
> building tests for the test suite, they most definitely should be in
> the setup phase. It's the only way to ensure that right before your
> test starts, no matter what horrible things happened previously, your
> test DB will stay intact.
I sort of see what you mean by that, but bear in mind that all the
flush stage does is truncate most of the tables, so you could still
theoretically do damage to your database prior to running tests that
this wouldn't undo (such as changing table structures or altering the
content types data). Obviously it will help if by 'horrible things'
you mean inserting broken or unwanted data into your model tables, but
it strikes me that the solution to that is to fix the code that's
doing it!
--
Sean Legassick
sean.le...@gmail.com
Karen Tracey
TestCase testcases should not be running after TransactionTestCase testcases. I had this concern prior to checking in the test changes (see http://groups.google.com/group/django-developers/browse_thread/thread/1e4f4c840b180895/) and (thought) I addressed it by adding code to reorder the test suite after all the tests are discovered so that all TestCase testcases (which assume a clean DB on entry and ensure one on exit) are run first. It's even documented (http://docs.djangoproject.com/en/dev/topics/testing/#django.test.TransactionTestCase) since the reordering revealed a few testcase-order dependent bugs in the Django test suite so might also cause hiccups for people with their own suites when upgrading to 1.1.
Are you actually seeing TestCases run after TransactionTestCases?
Karen
Sean Legassick
On 15 Apr 2009, at 22:25, Karen Tracey wrote:
>> So am I missing something or should this be changed?
>
> TestCase testcases should not be running after TransactionTestCase
> testcases. I had this concern prior to checking in the test changes
> (see http://groups.google.com/group/django-developers/browse_thread/thread/1e4f4c840b180895/)
> and (thought) I addressed it by adding code to reorder the test
> suite after all the tests are discovered so that all TestCase
> testcases (which assume a clean DB on entry and ensure one on exit)
> are run first. It's even documented (http://docs.djangoproject.com/en/dev/topics/testing/#django.test.TransactionTestCase
> ) since the reordering revealed a few testcase-order dependent bugs
> in the Django test suite so might also cause hiccups for people with
> their own suites when upgrading to 1.1.
Ah, so that's what I was missing. That does ring a bell now you
mention it, I was reading some of your discussion of this at the time.
> Are you actually seeing TestCases run after TransactionTestCases?
I am seeing this, however I am using the GIS test runner which doesn't
have the reordering code.
So the correct solution instead of what I suggested is for me to work
up a patch for the GIS test runner that uses your reordering code.
Thanks,
Sean
(PS I've hit another issue with the new-style TestCase to do with
using the transaction decorators, but I'm more confident that I know
what's going on with that so I'm preparing a ticket about it with a
patch right now).
--
Sean Legassick
sean.le...@gmail.com