[Django] #31612: Running tests with --keepdb messy to flush.
Django
---------------------------------------------+-----------------------------
Reporter: Jarek Glowacki | Owner: nobody
Type: Uncategorized | Status: new
Component: Testing framework | Version: 3.0
Severity: Normal | Keywords: test keepdb
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
---------------------------------------------+-----------------------------
We run tests with `--keepdb` flag by default. If there are changes that
require dropping the test database, we run the testsuite once without the
keepdb flag.
However this not only drops the db before tests, but also drops it after
tests too.
So next time we run the testsuite with the --keepdb flag, the database is
recreated and migrations are run once more (unnecessarily).
It feels very clunky.
It all works as initially intended in
https://code.djangoproject.com/ticket/20550:
> I'd like to implement a similar feature in Django. In the light of my
first experience, I suggest a simpler API. I would add a single flag
--keepdb. When this flag is set:
> - at the beginning of the tests, if the database already exists, Django
doesn't ask to delete it; it goes ahead and reuses it;
> - at the end of the tests, Django doesn't drop the database.
The original ticket author mentioned they'd implemented the `--nocreatedb`
and `--nodropdb` options in their own project, prior to `--keepdb`
existing, so they didn't have this issue.
I like the simplification that django offers and am not proposing to split
keepdb into two separate flags, however offering a flag that enables
either of the following would greatly unclunk `--keepdb`s use:
- "Don't drop db after running tests." (when `--keepdb` is off). Something
like `--preservedb`, `--nodropdb`, etc.
- "Drop db prior to running tests." (when `--keepdb` is on). Something
like `--refreshdb`, etc.
Yes, it's an extra flag, but it doesn't work the same way as
`--nocreatedb` and `--nodropdb`, because for the majority of runs users
just need to pass one flag (`--keepdb`), rather than multiple.
Another solution could be to offer a separate mcommand called `droptestdb`
or something similar -- it's sole role would be to clean up anything that
was left by the last `--keepdb`, so that the next one can run with a fresh
db.
Finally, changing behaviour on `--keepdb` so that it somehow _knows_ when
to prompt to start from scratch when it feels there is something not
lining up in the migrations could be another way to go. Solution would be
very magic, and maybe not great at guessing when to prompt, but would save
the user from having to learn any of the other proposed options.
In short, `--keepdb` is flawed in how it requires TWO rounds of
dropping/creating the db and running migrations (running without keepdb,
then with keepdb again) to refresh a database.
I've put forward a few ideas on how to address this, but none particularly
compelling.
Any thoughts?
--
Ticket URL: <https://code.djangoproject.com/ticket/31612>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: Jarek Glowacki | Owner: nobody
Component: Testing framework | Version: master
Severity: Normal | Resolution: wontfix
Keywords: test keepdb | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by felixxm):
* status: new => closed
* type: Uncategorized => New feature
* version: 3.0 => master
* resolution: => wontfix
Comment:
I don't agree that `--keepdb` is clunky it works as intended.
> If there are changes that require dropping the test database.
I think that's something unusual. Everything will work without an extra
roundtrip if you'll control these changes with migrations and without
dropping the test database. So I would say that an issue is in your
workflow rather than in Django itself. Running tests wasn't designed to
drop a test database when something terrible happens. You use mix of
`--keepdb` on/off calls to do this, but it's not a "supported" flow.
> Finally, changing behaviour on --keepdb so that it somehow _knows_ when
to prompt to start from scratch when it feels there is something not
lining up in the migrations could be another way to go. Solution would be
very magic, and maybe not great at guessing when to prompt, but would save
the user from having to learn any of the other proposed options.
I think you've already answered that it's not feasible by using words like
`it somehow _knows_ ` and `very magic`.
--
Ticket URL: <https://code.djangoproject.com/ticket/31612#comment:1>
Django
Reporter: Jarek Glowacki | Owner: nobody
Component: Testing framework | Version: master
Severity: Normal | Resolution: wontfix
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Jarek Glowacki):
Sorry, I should've provided some examples for when dropping the test db
might be necessary:
- Doctoring an existing migration on a local branch. eg. I run tests on a
local branch, applying a migration i've just added. I notice the migration
is wrong and wish to change it before pushing it upstream. Running the
tests again won't pick up my change. And to my knowledge I can't just
unapply the migration like i can with the `migrate` mcommand.
- Rebasing migrations. eg. I've got a stale PR with a migration on it. I
rebase my work and rename the migration and its dependencies to sit at the
end of the list. Local test db still thinks I've already applied said
changes and falls over trying to apply them a second time. (Yes I could
just add a merge migration here, but it's unnecessary clutter if the
migration exists only on my PR. And i don't think faking migrations is
possible here, like it is with `migrate`.)
Are such workflows out of the ordinary?
Anyway, if this still isn't sufficiently convincing, won't argue further.
Will just add some custom code to do it in our own product I guess..
Thanks for the quick triage!
--
Ticket URL: <https://code.djangoproject.com/ticket/31612#comment:2>