Possible Model Inheritance Issue? Error: [Model_Name] instance needs to have a primary key value before a many-to-many relationship can be used
Keyton Weissinger
OK. I'm totally stuck. I am trying to create an object (of type
Student -- see below) with the following:
new_student = Student.objects.create(**import_object_dict)
I'm using the following dictionary as "import_object_dict" above
(aside from the name of the school, this is fake data, so no privacy
worries):
import_object_dict={'address': u'142, Quilly Lane', 'city':
u'Columbus', 'dob': '1956-12-29', 'email':
u'Miles....@spambob.com', 'first_name': u'Yeung', 'phone_primary':
u'614-468-5940', 'school': <School: ARBOR MONTESSORI (DECATUR, GA)>,
'state': u'OH', 'title': u'Mr.', 'type': u'Student', 'zip': 43215.0}
Note that I am NOT yet trying to SAVE the Student object, just
instantiate it.
But every time I try to instantiate this new Student, I get the
following error:
'Student' instance needs to have a primary key value before a many-to-
many relationship can be used.
The Student object DOES have a ManyToMany relationship (with Parent --
see below) but I'm not in any way trying to access it here.
I am using inheritance which may be complicating matters (see models
below).
And here's a clue that may help you (but hasn't yet helped me):
When I try this on the command line (using "manage.py shell" and then
importing Student, creating a School object, etc) it works just fine.
No error. Using the exact same dictionary as above the exact same way.
Also, the code above isn't exactly accurate. I'm actually dynamically
getting the model object using get_model. I simplify the question here
because I've used the same exact method to create other types of
objects (without ManyToMany fields) with no problem whatsoever.
Any suggestions would be VERY much appreciated.
Thank you.
Keyton
class Person(models.Model):
title = models.CharField("Title", max_length=4,
choices=TITLE_CHOICES)
first_name = models.CharField("First name", max_length=100)
last_name = models.CharField("Last name", max_length=100)
address = models.CharField("Home street address", max_length=100)
city = models.CharField("City", max_length=100)
state = models.CharField("State", choices=STATE_CHOICES,
max_length=100)
zip = models.CharField("Zipcode", max_length=100)
email = models.EmailField("Preferred email address")
phone_primary = PhoneNumberField("Primary phone #")
phone_secondary = PhoneNumberField("Secondary phone #",
blank=True, null=True)
invite_key = models.CharField("Last invitation code",
max_length=50, blank=True, null=True)
invite_datetime = models.DateTimeField("Last invitation
date",blank=True, null=True)
type = models.CharField("User type", max_length=15,
choices=USER_TYPE_CHOICES, blank=True, null=True)
school = models.ForeignKey(School, blank=True, null=True)
user = models.ForeignKey(User, unique=True, blank=True, null=True)
def __unicode__(self):
return self.last_name + ', ' + self.first_name
def get_absolute_url(self):
return ('profiles_profile_detail', (), { 'username':
self.user.username })
get_absolute_url = models.permalink(get_absolute_url)
def save(self, force_insert=False, force_update=False):
if self.user:
self.user.email = self.email
super(Person, self).save(force_insert, force_update) # Call
the "real" save() method.
class Parent(Person):
pass
class Student(Person):
dob = models.DateField("Student's birth date", help_text='Used for
determining age.')
parents = models.ManyToManyField(Parent,
related_name="student_parent",blank=True, null=True, )
def __unicode__(self):
return self.last_name
Malcolm Tredinnick
Bonus points for a clear problem description, so thanks for doing that.
Nothing jumps out at me as obviously wrong, unfortunately.
You don't mention how you're using this when it fails -- is it part of a
script or a view or what? In any case, I would try removing the call to
get_model(). Even if you just test that the model name is what you think
should return the Student model and then explicitly set the class you're
creating to be Student, instead of whatever get_model() returns. I
*hope* that won't make a difference, because get_model() and friends
give me a nose-bleed (and James Bennett gets far too much enjoyment out
of watching me fix those bugs), but it would be nice to know one way or
the other.
Are you using get_model() at the interactive prompt? Does it work there
instead of failing?
I'll try to make some time later on today to experiment with the code
you've given here and see if I can repeat it (in the middle of something
right now and just rest my brain by answering email, so not able to do
it this minute, sorry).
Right now, though, it's not obvious what you're doing that is strange.
Which is a shame.
Regards,
Malcolm
Keyton Weissinger
Thanks very much for the quick reply. I'm using this code as part of a
view in which I'm processing an import of data (part of my reusable
django-batchimport app on google code). Here's a dpaste of the main
part of the code: http://dpaste.com/111306/
"model_import_info.model_for_import" (from the code) returns the
appropriate model. I've confirmed that it is returning a Student model
in this case (as the call to get at line #3 works just fine). I *am*
reusing this model over and over, btw. However, I have confirmed that
even if I create it anew each time, I get the same behavior. Also,
none of the data I am using actually has a Parent in it (the
ManyToMany field culprit).
To your point, I've tried it both with explicitly setting the model to
Student and using get_model. Using both methods, it works at
interactive prompt but not in the view.
Here's a dpaste of the exact stack trace I'm getting: http://dpaste.com/111304/
Not to confuse the issue, but looking at the django code from the
dpaste -- at the "setattr(self, field.attname, val)" call (line 38 on
dpaste link) -- the local variables tell me I'm dealing with a
ManyToMany object (<django.db.models.fields.related.ManyToManyField
object at 0x2e04c90>). I say that just to rule out some problem with
the School object (a OneToMany) that *IS* being passed in.
I appreciate any time you have to spend on it very much.
Best,
Keyton
On Jan 20, 11:02 pm, Malcolm Tredinnick <malc...@pointy-stick.com>
wrote:
> > Hello,
>
> > OK. I'm totally stuck. I am trying to create an object (of type
> > Student -- see below) with the following:
>
> > new_student = Student.objects.create(**import_object_dict)
>
> > I'm using the following dictionary as "import_object_dict" above
> > (aside from the name of the school, this is fake data, so no privacy
> > worries):
>
> > import_object_dict={'address': u'142, Quilly Lane', 'city':
> > u'Columbus', 'dob': '1956-12-29', 'email':
Keyton Weissinger
import of School or Parent data (neither of which have a ManyToMany
field).
Keyton
Martin Conte Mac Donell
>
> Oh and just to re-state, this exact same code works like a champ on
> import of School or Parent data (neither of which have a ManyToMany
> field).
>
> Keyton
I can't reproduce this exception. Try to print import_object_dict just
before "new_object =
model_import_info.model_for_import.objects.create(**import_object_dict)".
> Note that I am NOT yet trying to SAVE the Student object, just instantiate it.
Actually Model.objects.create() do try to save() object. If you don't
want it, call Model(**import_object_dict)
M.
Keyton Weissinger
Thanks for the suggestion. I've tried it both ways to no avail.
All,
Here's the code from django.db.models.base.py (in the Model.__init__)
where the trouble seems to start (around line 224):
# Now we're left with the unprocessed fields that *must* come
from
# keywords, or default.
print kwargs
for field in fields_iter:
print field.name
rel_obj = None
if kwargs:
if isinstance(field.rel, ManyToOneRel):
try:
# Assume object instance was passed in.
rel_obj = kwargs.pop(field.name)
except KeyError:
try:
# Object instance wasn't passed in -- must
be an ID.
val = kwargs.pop(field.attname)
except KeyError:
val = field.get_default()
else:
# Object instance was passed in. Special case:
You can
# pass in "None" for related objects if it's
allowed.
if rel_obj is None and field.null:
val = None
else:
val = kwargs.pop(field.attname, field.get_default
())
else:
val = field.get_default()
print field.name + ' default value ' + `val`
# If we got passed a related instance, set it using the
field.name
# instead of field.attname (e.g. "user" instead of
"user_id") so
# that the object gets properly cached (and type checked)
by the
# RelatedObjectDescriptor.
if rel_obj:
setattr(self, field.name, rel_obj)
else:
setattr(self, field.attname, val)
If I send in kwargs (and I've confirmed that it's getting what I think
it's getting) it goes merrily along until it gets to my ManyToMany
field at which time, it tries (using the last line above) to call
setattr(self, field.attname, val) with None (the default value for the
field). This triggers a call to the __set__ in related.py which in
turn tries to call the __get__ to find my related object (diligently)
and whammo.
The thing's doing exactly what it's coded to do.
My question is this:
If I'm in the init for this model and the field's default value is
None, why is the code ever trying to go get the related object? The
code should know it's not there. Shouldn't there be a check somewhere
in the stack to prevent the call to related.py in this case (in
__init__ when I don't actually have a related object)?
And why (the HECK) does this work just fine on the command line? That
points to my code, but all my code is doing is generating the kwargs
and feeding it into this process.
Very frustrating...
Thanks for whatever time anyone has on all of this...
Keyton
On Jan 21, 12:12 am, Martin Conte Mac Donell <refl...@gmail.com>
wrote:
Keyton Weissinger
think it should... Argh!
;-)
K
Keyton Weissinger
I stuck the following into my code at the same place the other code is
bombing. I did this just to make sure that I wasn't getting anything
weird in the dictionary here that I wasn't getting in the command
line.
mydict = {'city': u'Columbus', 'first_name': u'Miles', 'last_name':
u'Yeung', 'school': import_object_dict['school'], 'zip': 43215.0,
'title': u'Mr.', 'dob': '1956-12-29', 'phone_primary':
u'614-468-5940', 'state': u'OH', 'address': u'142, Quilly Lane',
'type': u'Student', 'email': u'Miles....@spambob.com'}
new_object = Student(**mydict)
Same exact error as before. It's almost as if the Student object is
behaving differently within the context that it does within manage.py
shell.
Just in case it was the school that was causing the issue above
(doubtful but good to rule it out) I tried the above WITHOUT the
school entry in the dictionary with the same results.
Still stuck.....
Keyton
Keyton Weissinger
this problem? I will try upgrading tonight.
Any other ideas?
K
On Jan 21, 8:33 am, Keyton Weissinger <key...@gmail.com> wrote:
> More information...
>
> I stuck the following into my code at the same place the other code is
> bombing. I did this just to make sure that I wasn't getting anything
> weird in the dictionary here that I wasn't getting in the command
> line.
>
> mydict = {'city': u'Columbus', 'first_name': u'Miles', 'last_name':
> u'Yeung', 'school': import_object_dict['school'], 'zip': 43215.0,
> 'title': u'Mr.', 'dob': '1956-12-29', 'phone_primary':
> u'614-468-5940', 'state': u'OH', 'address': u'142, Quilly Lane',
Martin Conte Mac Donell
It's working using SVN HEAD with your model and this view:
http://dpaste.com/111590/. Could you try to use that dict and
Student(**mydict) instead of
"model_import_info.model_for_import.objects.create" ?
Regards,
M
Keyton Weissinger
already tried the Student(**mydict) approach and get the exact same
issue. I will test tonight and respond. Thanks very much for taking
time...
Keyton
Keyton Weissinger
this can tell me WWWHHHYYY my solution works. For now, it's voodoo to
me...
Turns out that some of my code (in the view where I was having
trouble) called some code that looked sort of like this:
field_list = model._meta.fields
field_list.extend(model._meta.many_to_many)
for field in field_list:
related_model_name = None
related_model_app_name = None
if (not field.name[-4:] == '_ptr') and (not field.__class__ ==
AutoField):
if issubclass(field.__class__, related.RelatedField):
if not field.__class__ == related.ForeignKey:
related_model_app_name = field.rel.to.__module__.split
('.')[0]
# We'll ignore all django-specific models (such as
User, etc).
if not related_model_app_name == 'django':
related_model_name = field.rel.to.__name__
full_related_model_name = '.'.join
([field.rel.to.__module__, related_model_name])
relation_tuple_list.append((full_model_name
+'%relation'+field.name+'%' + full_related_model_name, \
'Mapping: ' +
model.__name__ + '-' + \
related_model_name))
else:
continue
It is just pulling together a list of relationships between two
models. I'm not using these models to actually DO anything. Just what
you see.
I could go to the command line and do the following:
>>> from batchimport.forms import ImportOptionsForm
>>> from sbase.models import Student
>>> mydict = {'city': u'Columbus', 'first_name': u'Jimmy', 'last_name': u'Jones', 'zip': 43215.0, 'title': u'Mr.', 'dob': '1956-12-29', 'phone_primary': u'614-468-5940', 'state': u'OH', 'address': u'142, Quilly Lane', 'type': u'Student', 'email': u'Miles....@spambob.com'}
>>> mystudent = Student(**mydict)
The import of ImportOptionsForms, it turns out, called the above code.
OK. The above series of commands would fail every time with the
following error.
many relationship can be used.
line. Yea.
So after a LONG time of messing with it, it turns out that django
didn't like how I was handling the many-to-many fields
(model._meta.many_to_many).
But I was able to fix my problem by just doing this:
for field_list in [model._meta.fields, model._meta.many_to_many]:
for field in field_list:
related_model_name = None
related_model_app_name = None
if (not field.name[-4:] == '_ptr') and (not
field.__class__ == AutoField):
if issubclass(field.__class__, related.RelatedField):
if not field.__class__ == related.ForeignKey:
related_model_app_name =
field.rel.to.__module__.split('.')[0]
# We'll ignore all django-specific models
(such as User, etc).
if not related_model_app_name == 'django':
related_model_name = field.rel.to.__name__
full_related_model_name = '.'.join
([field.rel.to.__module__, related_model_name])
relation_tuple_list.append((full_model_name
+'%relation'+field.name+'%' + full_related_model_name, \
'Mapping: ' +
model.__name__ + '-' + \
related_model_name))
else:
continue
YEAH! That's all I changed. Instead of extending my list of fields
with the many_to_many fields, I just iterated over one (to get the
ForeignKey relationships etc, and then I subsequently iterated over my
many-to-many fields.
Why (the HELL) does the first version trigger this error:
many relationship can be used.
If I'm an idiot, PLEASE tell me how so, because I'd really like to
avoid this kind of quagmire if I can....
Thank you VERY MUCH Martin and Malcolm. I appreciate your respective
looks at this problem very much. Your taking time to help with this
kind of thorny issue is what makes this community rock and, frankly,
inspires me to help out more.
Keyton
Malcolm Tredinnick
> OK. I figured it out and it was VERY STRANGE. Maybe some one reading
> this can tell me WWWHHHYYY my solution works. For now, it's voodoo to
> me...
You're going to kick yourself after this. The reason for the failure
turns out to be really easy to understand, now that your show the
code. :-)
>
> Turns out that some of my code (in the view where I was having
> trouble) called some code that looked sort of like this:
Hmm .. you didn't exactly simplify your failing case to the smallest
possible example, did you? All this extra stuff would have set off a lot
of alarm bells about other places to look (although my first thought
would have been whether you could do without it and retest to see if
that changed anything).
Before getting into the more-or-less two line explanation, some
background information. Firstly, model._meta is obviously internal API,
so there are some shortcuts taken for performance and maintainability
reasons. We don't send the black helicopters after people who access it,
but it's very much "user beware" country. It's one of the areas that is
currently intentionally undocumented, for example (we'll eventually drop
some documentation about internal datastructures into docs/internals/,
but have had other things to do so far).
Of relevance to this particular case, we use lists a lot internally and
return references to them from certain methods. We use lists, rather
than tuples, because they are updated a fair bit during creation and all
the extra copying isn't free (updating a tuple requires a copy). Lists
are just nicer to work with, generally.
> field_list = model._meta.fields
Have a look at the implementation here
(django.db.models.options.Options._fields(), since _meta.fields is a
property). It returns a reference to a list. A *reference*, not a copy!
Your code would have worked if you wrote
# Work with a copy of the fields list.
field_list = model._meta.fields[:]
> field_list.extend(model._meta.many_to_many)
Here you update that reference in place. So you've just corrupted one of
the internal data structures of self._meta.
That's pretty much going to guarantee that the wheels will fall off at
some point in the future. If not in the way you saw, in some other
really hard to debug fashion.
It's kind of a Python trap and if you're designing a public API, you
would generally return a copy of the list if the caller wasn't expected
to change the internal structure. In the Options class, we "know" (in
quotes, because there are people who call it from public code, but
that's their problem, not ours) that the only callers are functions that
are read-only on the data structure, so we save a bit of time and code
complexity. We could return a copy each time, but we access self._fields
a *lot*, particularly in queryset operations. Things might have changed
a bit in the last six months (although not dramatically), but when I was
working on the queryset-refactor branch, I spent a lot of time profiling
various common execution paths and a lot of the newer shared data
structures in Options are implemented based on that work (self.fields
used to be a simple list, for example, but when model inheritance came
in, it became, internally, more complicated).
Regards,
Malcolm
Keyton Weissinger
Thank you VERY much for taking the time to explain this to me. It
makes perfect sense and since yesterday I've delved a bit deeper (than
ever) into the django code itself to understand a little more. You are
ABSOLUTELY right to point out that I'm in user-beware country. I don't
apologize for that so much. I think the batchimport project has some
usefulness to folks and it's worth the risk -- **IF** I do it
right. ;-) (if you think otherwise, PLEASE let me know)
Even since yesterday I've begun a good scouring of the project to
reduce touching the internals as much as possible.
I apologize for not giving more details. I am always trying to balance
giving enough information while trying to avoid scaring people away
with too much detail. I didn't think it through enough I suppose. I
kept being stymied by the thing "working" on the command line. My
mistake.
Anyway. I understand the whole thing MUCH better now and have already
spent a little time reviewing the really "cutting edge" aspects of
this project like, you know, "how lists work in python." ;-)
Ugh.
Kicking Myself Diligently,
Keyton
On Jan 22, 3:10 am, Malcolm Tredinnick <malc...@pointy-stick.com>
wrote:
João Pedro Melo
>>> mydict = {'city': u'Columbus', 'first_name': u'Jimmy', 'last_name': u'Jones', 'zip': 43215.0, 'title': u'Mr.', 'dob': '1956-12-29', 'phone_primary': u'614-468-5940', 'state': u'OH', 'address': u'142, Quilly Lane', 'type': u'Student', 'email': u'Miles...@spambob.com'}