[Django] #22421: Loading fixtures with OneToOne relation and natural keys fails since 1.7 Beta
Django
Beta
--------------------------------------+--------------------
Reporter: stanislas.guerra@… | Owner: nobody
Type: Bug | Status: new
Component: Core (Serialization) | Version: master
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------------+--------------------
Hi,
I am testing a project with the 1.7Beta release / master branch and Django
can't load some fixtures when I am running my test-suite:
{{{
======================================================================
ERROR: test_validation_ajouter_double_page
(fab4.front.tests.tests.DossierTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/stan/src/venv_dj_1_7/lib/python2.7/site-
packages/Django-1.8.dev20140410233300-py2.7.egg/django/test/testcases.py",
line 175, in __call__
self._pre_setup()
File "/Users/stan/src/venv_dj_1_7/lib/python2.7/site-
packages/Django-1.8.dev20140410233300-py2.7.egg/django/test/testcases.py",
line 747, in _pre_setup
self._fixture_setup()
File "/Users/stan/src/venv_dj_1_7/lib/python2.7/site-
packages/Django-1.8.dev20140410233300-py2.7.egg/django/test/testcases.py",
line 869, in _fixture_setup
return super(TestCase, self)._fixture_setup()
File "/Users/stan/src/venv_dj_1_7/lib/python2.7/site-
packages/Django-1.8.dev20140410233300-py2.7.egg/django/test/testcases.py",
line 788, in _fixture_setup
**{'verbosity': 0, 'database': db_name, 'skip_checks': True})
File "/Users/stan/src/venv_dj_1_7/lib/python2.7/site-
packages/Django-1.8.dev20140410233300-py2.7.egg/django/core/management/__init__.py",
line 167, in call_command
return klass.execute(*args, **defaults)
File "/Users/stan/src/venv_dj_1_7/lib/python2.7/site-
packages/Django-1.8.dev20140410233300-py2.7.egg/django/core/management/base.py",
line 337, in execute
output = self.handle(*args, **options)
File "/Users/stan/src/venv_dj_1_7/lib/python2.7/site-
packages/Django-1.8.dev20140410233300-py2.7.egg/django/core/management/commands/loaddata.py",
line 60, in handle
self.loaddata(fixture_labels)
File "/Users/stan/src/venv_dj_1_7/lib/python2.7/site-
packages/Django-1.8.dev20140410233300-py2.7.egg/django/core/management/commands/loaddata.py",
line 89, in loaddata
self.load_label(fixture_label)
File "/Users/stan/src/venv_dj_1_7/lib/python2.7/site-
packages/Django-1.8.dev20140410233300-py2.7.egg/django/core/management/commands/loaddata.py",
line 146, in load_label
obj.save(using=self.using)
File "/Users/stan/src/venv_dj_1_7/lib/python2.7/site-
packages/Django-1.8.dev20140410233300-py2.7.egg/django/core/serializers/base.py",
line 176, in save
setattr(self.object, accessor_name, object_list)
File "/Users/stan/src/venv_dj_1_7/lib/python2.7/site-
packages/Django-1.8.dev20140410233300-py2.7.egg/django/db/models/fields/related.py",
line 1183, in __set__
manager = self.__get__(instance)
File "/Users/stan/src/venv_dj_1_7/lib/python2.7/site-
packages/Django-1.8.dev20140410233300-py2.7.egg/django/db/models/fields/related.py",
line 1169, in __get__
through=self.field.rel.through,
File "/Users/stan/src/venv_dj_1_7/lib/python2.7/site-
packages/Django-1.8.dev20140410233300-py2.7.egg/django/db/models/fields/related.py",
line 821, in __init__
(instance, source_field_name))
ValueError: Problem installing fixture
'/Users/stan/Dropbox/Projets/Aden/Publish/repos/publish/fab4/fabrication/fixtures/tests/fabrication/fabrication2.json':
"<Assistant: >" needs to have a value for field "user" before this many-
to-many relationship can be used.
}}}
My models are :
{{{
from django.contrib.sites.models import Site
from django.contrib.auth.models import User
class SupportManager(models.Manager):
def get_current(self):
return self.get(pk=settings.SITE_ID)
def get_by_natural_key(self, short_name):
return self.get(short_name=short_name)
class Support(Site):
short_name = models.CharField('nom court', max_length=10, unique=True)
logo = models.ImageField('logo', upload_to='images/supports',
blank=True)
site_ptr = models.OneToOneField(Site, primary_key=True) # allow to do
select related without an extra query to Site.
objects = SupportManager()
def __unicode__(self):
return self.name
def natural_key(self):
return (self.short_name,)
class Organisation(models.Model):
name = models.CharField(max_length=100)
short_name = models.CharField('nom court', max_length=16)
class Employe(User):
supports = models.ManyToManyField(Support)
organisation = models.ForeignKey('Organisation', null=True,
blank=True)
filtre_organisation = models.BooleanField(default=False)
telephone = models.CharField(u'téléphone', max_length=100, blank=True)
class Meta:
abstract = True
def __unicode__(self):
return self.get_full_name()
class Assistant(Employe):
class Meta:
verbose_name = 'chargé de clientèle'
}}}
The fixture causing the crash is a concat from the following dumps (master
version of dumpdata):
{{{
python manage.py dumpdata sites.Site --indent=2 --natural-foreign --pk=1
python manage.py dumpdata parametrage.Support --indent=2 --natural-foreign
--pk=1
python manage.py dumpdata auth.Group --indent=2 --natural-foreign --pk=2
python manage.py dumpdata auth.User --indent=2 --natural-foreign --pk=8
python manage.py dumpdata parametrage.Assistant --indent=2 --natural-
foreign --pk=8
}}}
Which is :
{{{
[
{
"fields": {
"domain": "proprietesdefrance.com",
"name": "Propri\u00e9t\u00e9s de France"
},
"model": "sites.site",
"pk": 1
},
{
"fields": {
"logo": "images/supports/logo_PDF.jpg",
"short_name": "PDF"
},
"model": "parametrage.support",
"pk": 1
},
{
"fields": {
"name": "Relation Client",
"permissions": [
[
"add_logentry",
"admin",
"logentry"
],
[
"change_statut_refus_bat_to_service_fab",
"fabrication",
"changementstatut"
],
[
"see_all_status",
"fabrication",
"changementstatut"
],
[
"see_chemindefer",
"fabrication",
"chemindefer"
],
[
"add_dossier",
"fabrication",
"dossier"
],
[
"change_dossier",
"fabrication",
"dossier"
],
[
"change_prix_dossier",
"fabrication",
"dossier"
],
[
"see_bat_pdf_dossier",
"fabrication",
"dossier"
],
[
"change_logoconstructeur",
"saisie",
"logoconstructeur"
],
[
"delete_logoconstructeur",
"saisie",
"logoconstructeur"
]
]
},
"model": "auth.group",
"pk": 2
},
{
"fields": {
"username": "igixxxxx",
"first_name": "Isxxxxx",
"last_name": "Gxxxxxx",
"is_active": true,
"is_superuser": false,
"is_staff": true,
"last_login": "2014-04-04T10:34:28",
"groups": [
[
"Relation Client"
]
],
"user_permissions": [],
"password": "sha1$17783$7152b7f25xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"email": "igib...@xxxxxxxxx.fr",
"date_joined": "2009-10-07T16:08:28"
},
"model": "auth.user",
"pk": 8
},
{
"fields": {
"organisation": 1,
"telephone": "01 xx xx xx xx",
"groups": [
[
"Relation Client"
]
],
"user_permissions": [],
"filtre_organisation": false,
"supports": [
[
"PDF"
]
]
},
"model": "parametrage.assistant",
"pk": 8
}
]
}}}
Any idea ?
Thanks.
--
Ticket URL: <https://code.djangoproject.com/ticket/22421>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
fails since 1.7 Beta
-------------------------------------+-------------------------------------
Reporter: stanislas.guerra@… | Owner: nobody
Type: Bug | Status: new
(Serialization) | Resolution:
Severity: Normal | Triage Stage:
Keywords: | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by stanislas.guerra@…):
* needs_better_patch: => 0
* needs_docs: => 0
* needs_tests: => 0
Comment:
Looks like it came from this commit in
''django/db/models/fields/related.py#ForeignObject.get_instance_value_for_fields(instance,
fields)'' :
https://github.com/django/django/commit/244e2b71f512605f3d0a8e1ba4c9d6b538acf69d#diff-301
Which is related to the ticket #20820
{{{
- ret.append(instance.pk)
- else:
- ret.append(getattr(instance, field.attname))
+ possible_parent_link =
opts.get_ancestor_link(field.model)
+ if not possible_parent_link or
possible_parent_link.primary_key:
+ ret.append(instance.pk)
+ continue
+ ret.append(getattr(instance, field.attname))
return tuple(ret)
}}}
And especially the last line: `ret.append(getattr(instance,
field.attname))` (`<assistant obj>.id` in that case) which add a `None` to
the returned Tuple raising an Exception in `create_many_related_manager()`
:
{{{
self.related_val =
source_field.get_foreign_related_value(instance)
if None in self.related_val:
raise ValueError('"%r" needs to have a value for field
"%s" before '
'this many-to-many relationship can be
used.' %
(instance, source_field_name))
}}}
Because the Assistant object has not been saved yet.
FYI, the fixture loading fails when trying to save the ''Assistant''
object (which is a one-to-one to ''User'') because of the M2M
''user_permissions'' relation.
Hope this helps.
--
Ticket URL: <https://code.djangoproject.com/ticket/22421#comment:1>
Django
fails since 1.7 Beta
-------------------------------------+-------------------------------------
Reporter: stanislas.guerra@… | Owner: nobody
Type: Bug | Status: new
(Serialization) | Resolution:
Severity: Normal | Triage Stage:
Keywords: | Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by stanislas.guerra@…):
Here a patch on the tests that cause a failure.
I just added a proxy model.
{{{
$ git diff
diff --git a/tests/fixtures_regress/fixtures/special-article.json
b/tests/fixtures_regress/fixtures/special-article.json
index a36244a..75f2d65 100644
--- a/tests/fixtures_regress/fixtures/special-article.json
+++ b/tests/fixtures_regress/fixtures/special-article.json
@@ -2,14 +2,14 @@
{
"pk": 1,
"model": "fixtures_regress.article",
- "fields": {"title": "foof"
+ "fields": {
+ "title": "foof"
}
},
{
"pk": 1,
"model": "fixtures_regress.specialarticle",
"fields": {
- "title": "Article Title 1",
"channels": []
}
}
diff --git a/tests/fixtures_regress/models.py
b/tests/fixtures_regress/models.py
index 95f9488..e440184 100644
--- a/tests/fixtures_regress/models.py
+++ b/tests/fixtures_regress/models.py
@@ -66,7 +66,14 @@ class Article(models.Model):
# Subclass of a model with a ManyToManyField for test_ticket_20820
-class SpecialArticle(Article):
+class CommonSpecialArticle(Article):
+ subtitle = models.CharField(max_length=255)
+
+ class Meta:
+ abstract = True
+
+
+class SpecialArticle(CommonSpecialArticle):
pass
}}}
And the failure (tested with sqlite and MySQL):
{{{
PYTHONPATH=..:$PYTHONPATH ./runtests.py --settings=test_sqlite
fixtures_regress.tests.TestFixtures.test_ticket_20820
Testing against Django installed in
'/Users/stan/src/Django/repos/django/django/django'
Creating test database for alias 'default'...
Creating test database for alias 'other'...
E
======================================================================
ERROR: test_ticket_20820 (fixtures_regress.tests.TestFixtures)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/Users/stan/src/Django/repos/django/django/tests/fixtures_regress/tests.py",
line 480, in test_ticket_20820
verbosity=0,
File
"/Users/stan/src/Django/repos/django/django/django/core/management/__init__.py",
line 167, in call_command
return klass.execute(*args, **defaults)
File
"/Users/stan/src/Django/repos/django/django/django/core/management/base.py",
line 337, in execute
output = self.handle(*args, **options)
File
"/Users/stan/src/Django/repos/django/django/django/core/management/commands/loaddata.py",
line 60, in handle
self.loaddata(fixture_labels)
File
"/Users/stan/src/Django/repos/django/django/django/core/management/commands/loaddata.py",
line 90, in loaddata
self.load_label(fixture_label)
File
"/Users/stan/src/Django/repos/django/django/django/core/management/commands/loaddata.py",
line 147, in load_label
obj.save(using=self.using)
File
"/Users/stan/src/Django/repos/django/django/django/core/serializers/base.py",
line 176, in save
setattr(self.object, accessor_name, object_list)
File
"/Users/stan/src/Django/repos/django/django/django/db/models/fields/related.py",
line 1185, in __set__
manager = self.__get__(instance)
File
"/Users/stan/src/Django/repos/django/django/django/db/models/fields/related.py",
line 1171, in __get__
through=self.field.rel.through,
File
"/Users/stan/src/Django/repos/django/django/django/db/models/fields/related.py",
line 823, in __init__
(instance, source_field_name))
ValueError: Problem installing fixture
'/Users/stan/src/Django/repos/django/django/tests/fixtures_regress/fixtures
/special-article.json': "<SpecialArticle: SpecialArticle object>" needs to
have a value for field "article" before this many-to-many relationship can
be used.
----------------------------------------------------------------------
Ran 1 test in 0.021s
FAILED (errors=1)
Destroying test database for alias 'default'...
Destroying test database for alias 'other'...
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/22421#comment:2>
Django
natural keys fails since 1.7 Beta
Reporter: stanislas.guerra@… | Owner: nobody
Type: Bug | Status: new
(Serialization) | Resolution:
Severity: Normal | Triage Stage:
Keywords: | Unreviewed
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
Changes (by stanislas.guerra@…):
* needs_better_patch: 0 => 1
* has_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/22421#comment:3>
Django
Reporter: stanislas.guerra@… | Owner: nobody
Type: Bug | Status: new
Component: Core (Serialization) | Version: master
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Changes (by timo):
* severity: Normal => Release blocker
* stage: Unreviewed => Accepted
--
Ticket URL: <https://code.djangoproject.com/ticket/22421#comment:4>
Django
fails since 1.7 Beta
--------------------------------------+------------------------------------
Type: Bug | Status: new
Component: Core (Serialization) | Version: master
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Comment (by stanislas.guerra@…):
Edit:
This is not a proxy model but an abstract one.
And it does not looks like it is related to the natural keys since the
regression test fails without them.
--
Ticket URL: <https://code.djangoproject.com/ticket/22421#comment:5>
Django
fails since 1.7 Beta
--------------------------------------+------------------------------------
Type: Bug | Status: assigned
Component: Core (Serialization) | Version: master
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Changes (by ramiro):
* owner: nobody => ramiro
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/22421#comment:6>
Django
fails since 1.7 Beta
--------------------------------------+------------------------------------
Reporter: stanislas.guerra@… | Owner: ramiro
Type: Bug | Status: assigned
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Comment (by ramiro):
PR is ready for review at https://github.com/django/django/pull/2679
--
Ticket URL: <https://code.djangoproject.com/ticket/22421#comment:7>
Django
fails since 1.7 Beta
Reporter: stanislas.guerra@… | Owner: ramiro
Type: Bug | Status: assigned
(Serialization) | Resolution:
Keywords: | checkin
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timo):
* needs_better_patch: 1 => 0
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/22421#comment:8>
Django
fails since 1.7 Beta
-------------------------------------+-------------------------------------
Reporter: stanislas.guerra@… | Owner: ramiro
Component: Core | Version: master
(Serialization) | Resolution: fixed
Severity: Release blocker | Triage Stage: Ready for
Keywords: | checkin
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Ramiro Morales <ramiro@…>):
In [changeset:"d731f48ecea6ceeabc2b015b48655a9f9428a730"]:
{{{
#!CommitTicketReference repository=""
revision="d731f48ecea6ceeabc2b015b48655a9f9428a730"
Merge pull request #2679 from ramiro/t22421
Fixed #22421 -- Regression in fixtures loading.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/22421#comment:10>
Django
fails since 1.7 Beta
-------------------------------------+-------------------------------------
Reporter: stanislas.guerra@… | Owner: ramiro
Type: Bug | Status: closed
Component: Core | Version: master
(Serialization) | Resolution: fixed
Severity: Release blocker | Triage Stage: Ready for
Keywords: | checkin
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Ramiro Morales <cramm0@…>):
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"862e1ff2340a1e28a3e7c6904d2b0283085f34c8"]:
{{{
#!CommitTicketReference repository=""
revision="862e1ff2340a1e28a3e7c6904d2b0283085f34c8"
Fixed #22421 -- Regression in fixtures loading.
Loading fixtures were failing since the refactoring in 244e2b71f5 for
inheritance setups where the chain contains abstract models and the
root ancestor contains a M2M relation.
Thanks Stanislas Guerra for the report.
Refs #20946.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/22421#comment:9>
Django
fails since 1.7 Beta
-------------------------------------+-------------------------------------
Reporter: stanislas.guerra@… | Owner: ramiro
Type: Bug | Status: closed
Component: Core | Version: master
(Serialization) | Resolution: fixed
Severity: Release blocker | Triage Stage: Ready for
Keywords: | checkin
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Tim Graham <timograham@…>):
In [changeset:"fb45e666c204d7188c38469ca1f661a9b113b85b"]:
{{{
#!CommitTicketReference repository=""
revision="fb45e666c204d7188c38469ca1f661a9b113b85b"
[1.7.x] Fixed #22421 -- Regression in fixtures loading.
Loading fixtures were failing since the refactoring in 244e2b71f5 for
inheritance setups where the chain contains abstract models and the
root ancestor contains a M2M relation.
Thanks Stanislas Guerra for the report.
Refs #20946.
Backport of 862e1ff234 from master
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/22421#comment:11>