[Django] #21278: Using dumpdata to create unit test fixtures causes duplicate foreign keys for auth permissions. Excluding auth causes other referenced auth models to be missing.
Django
for auth permissions. Excluding auth causes other referenced auth models
to be missing.
-------------------------------+--------------------
Reporter: ellisd23@… | Owner: nobody
Type: Uncategorized | Status: new
Component: contrib.auth | Version: 1.5
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------
This seems to be a pretty fundamental issue, since models will frequently
reference User. There are many instances of users having frustrations
with contrib apps that are auto-generated upon migration/sync (e.g.
[http://stackoverflow.com/questions/853796/problems-with-contenttypes-
when-loading-a-fixture-in-django contenttypes]).
For contenttypes, this issue can be mitigated by excluding the app and
using --natural. But for contrib.auth, if the app is excluded, any
referenced User objects in fixture data will be missing. If the app is
not excluded, duplicate permissions will be created, since the post-
migration signal will have already created them. Since there is a unique
key restraint, the importing of the fixture data will fail. It's a catch
22.
Developers can manually remove the permissions from the fixture data, or
override fixture loading in their tests to remove the data, but it seems
to me that since this is such a basic piece of functionality, there should
be a simple interface to do such a thing. It'd be nice to have something
simpler for contenttypes as well, rather than "--exclude contenttypes
--natural".
--
Ticket URL: <https://code.djangoproject.com/ticket/21278>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
for auth permissions. Excluding auth causes other referenced auth models
to be missing.
Reporter: ellisd23@… | Owner: nobody
Type: Uncategorized | Status: new
Component: contrib.auth | Version: 1.5
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by akaariai):
* needs_better_patch: => 0
* needs_docs: => 0
* needs_tests: => 0
* stage: Unreviewed => Accepted
Comment:
I agree that fixture loading + contenttypes / permissions is a problem. I
am marking this as accepted, although I don't have any idea what could be
done here.
--
Ticket URL: <https://code.djangoproject.com/ticket/21278#comment:1>
Django
for auth permissions. Excluding auth causes other referenced auth models
to be missing.
Reporter: ellisd23@… | Owner: nobody
Type: New feature | Status: new
Component: contrib.auth | Version: 1.5
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by carljm):
* type: Uncategorized => New feature
--
Ticket URL: <https://code.djangoproject.com/ticket/21278#comment:2>
Django
for auth permissions. Excluding auth causes other referenced auth models
to be missing.
Reporter: ellisd23@… | Owner: nobody
Type: New feature | Status: new
Component: contrib.auth | Version: 1.5
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by russellm):
I'm confused by this report. This is the exact problem that led to the
introduction of natural foreign keys in the first place, and I was under
the impression that it was fixed. Permissions have natural keys defined,
so they should be able to be dumped with `--natural`.
Has something regressed in the 3-4 years since we committed natural keys,
or have I missed an important detail about this report?
--
Ticket URL: <https://code.djangoproject.com/ticket/21278#comment:3>
Django
for auth permissions. Excluding auth causes other referenced auth models
to be missing.
Reporter: ellisd23@… | Owner: nobody
Type: New feature | Status: new
Component: contrib.auth | Version: 1.5
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by ellisd23@…):
It's been a while since I made the ticket, but IIRC the issue is that even
with --natural passed, the problem is that an attempt is made to create
permissions twice: once during the loading of the fixtures, and once
automatically. It's that second time that causes duplicate key errors.
--
Ticket URL: <https://code.djangoproject.com/ticket/21278#comment:4>
Django
for auth permissions. Excluding auth causes other referenced auth models
to be missing.
Reporter: ellisd23@… | Owner: nobody
Type: New feature | Status: new
Component: contrib.auth | Version: 1.5
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by powderflask):
In case anyone finds this ticket, searching for an answer, this seems to
be fixed in django1.9
I was having the problem described here, but it was resolved when I did my
dump with:
dumpdata --natural-foreign --natural-primary
--
Ticket URL: <https://code.djangoproject.com/ticket/21278#comment:5>
Django
for auth permissions. Excluding auth causes other referenced auth models
to be missing.
Reporter: ellisd23@… | Owner: nobody
Type: New feature | Status: new
Component: contrib.auth | Version: 1.5
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by powderflask):
Here is a KEY insight that seems obvious now, but busted my *** for a
couple hours:
if your models contain an M2M relation (e.g., auth.Group,
auth.Permission), do NOT dump the M2M table (e.g., auth.group_permissions)
into your test fixture! Otherwise, the M2M table will be populated once
when the related objects are created, and then AGAIN when the M2M data is
loaded from the fixture, thus causing an integrity error like the one
reported here.
This mistake is SO easy to make and SUCH a bugger to track down, it might
be worth a word of warning in the docs about creating test fixtures?
--
Ticket URL: <https://code.djangoproject.com/ticket/21278#comment:6>