filtering on reverse foreignkey using isnull

[Eric Abrahamsen]

Hi,

I've got two models, Author and Entry, with a foreignkey from Entry to
Author. There are many Authors with no related entries, and until
recently I've been able to put this manager on the Author model to
only select Authors with Entries:

def contributors():
return self.exclude(entry__isnull=True)

Recently this started returning a FieldError, saying it could not
resolve "entry" into a field. I'm running the most recent version of
django. Before I go tearing up my app trying to figure out what I
changed, I'd like to make sure – is this the recommended way to check
for Authors with no Entries? I just found a couple of hints online
saying to expect errors from this kind of code, though it has worked
to date (and appears to still be working on some production code).
Should this work?

Thanks,
Eric

[Karen Tracey]

On Sun, May 31, 2009 at 10:13 AM, Eric Abrahamsen <gir...@gmail.com> wrote:

Hi,

I've got two models, Author and Entry, with a foreignkey from Entry to
Author. There are many Authors with no related entries, and until
recently I've been able to put this manager on the Author model to
only select Authors with Entries:

def contributors():
 return self.exclude(entry__isnull=True)

Recently this started returning a FieldError, saying it could not
resolve "entry" into a field.

What changed?  Did you update Django?  Change your code?  Something must have changed, and if you could identify it, that would be a clue.
 

I'm running the most recent version of
django. Before I go tearing up my app trying to figure out what I
changed, I'd like to make sure – is this the recommended way to check
for Authors with no Entries? I just found a couple of hints online
saying to expect errors from this kind of code, though it has worked
to date (and appears to still be working on some production code).
Should this work?

Where are these hints to expect errors from this sort of code?  In the Django docs would be a reliable source (but I'd hope it would be a lot clearer than a hint of possible errors), random blogs or old mailing list threads not so much.

FWIW, I cannot recreate any problem like this using this manager & models:

class AManager(models.Manager):
    def contributors(self):
        return self.exclude(entry__isnull=True)

class Author(models.Model):
    name = models.CharField(max_length=44)
    def __unicode__(self):
        return self.name
    objects = AManager()

class Entry(models.Model):
    value = models.CharField(max_length=23)
    author = models.ForeignKey(Author)
    def __unicode__(self):
        return u'%s (author: %s)' % (self.value, unicode(self.author))

in a Python shell using Django SVN r10865:

Python 2.5.2 (r252:60911, Oct  5 2008, 19:24:49)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from ttt.models import Author, Entry
>>> Author.objects.all()
[]
>>> Entry.objects.all()
[]
>>> Author.objects.contributors()
[]
>>> Author.objects.create(name="Jacinta")
<Author: Jacinta>
>>> Author.objects.create(name="Eamon")
<Author: Eamon>
>>> Author.objects.contributors()
[]
>>> Entry.objects.create(value="Icterus",author=Author.objects.all()[1])
<Entry: Icterus (author: Eamon)>
>>> Author.objects.contributors()
[<Author: Eamon>]
>>> quit()

I think the models/manager I created match what you describe, so how is you setup different?

Karen

[Eric Abrahamsen]

On May 31, 2009, at 11:45 PM, Karen Tracey wrote:

On Sun, May 31, 2009 at 10:13 AM, Eric Abrahamsen <gir...@gmail.com> wrote:

Hi,

I've got two models, Author and Entry, with a foreignkey from Entry to
Author. There are many Authors with no related entries, and until
recently I've been able to put this manager on the Author model to
only select Authors with Entries:

def contributors():
 return self.exclude(entry__isnull=True)

Recently this started returning a FieldError, saying it could not
resolve "entry" into a field.

What changed?  Did you update Django?  Change your code?  Something must have changed, and if you could identify it, that would be a clue.
 

I'm running the most recent version of
django. Before I go tearing up my app trying to figure out what I
changed, I'd like to make sure – is this the recommended way to check
for Authors with no Entries? I just found a couple of hints online
saying to expect errors from this kind of code, though it has worked
to date (and appears to still be working on some production code).
Should this work?

Where are these hints to expect errors from this sort of code?  In the Django docs would be a reliable source (but I'd hope it would be a lot clearer than a hint of possible errors), random blogs or old mailing list threads not so much.

Thanks for the detailed response, Karen. I was just worried at first that isnull was an improper way of checking for Authors with no Entry (the hints came from this message [http://groups.google.com/group/django-users/msg/47ba69359ceace29] which I may have misinterpreted, the docs don't seem to reference this situation directly). But now I see that all filters using Entry are returning the same error, not just isnull, so it's definitely a problem in my code.

I identified the commit in my own code that blew things up (thanks to my excellent version-tracking habits, it's a pretty large and tangled commit). This commit touches neither the Author nor the Entry model, it's got:

Two new models, one called Sample with a ForeignKey to Author, one called TranslatorProfile with a OneToOneField to Author (no related names on either).

ModelForms for both of these new models. I also moved two existing modelforms to a different place in the file.

I'm putting the diff file into dpaste here: http://dpaste.com/49912/, in case anyone's got the patience to look at it. That's a diff between -r301, which works, and working copy, which doesn't. The whole models.py file (broken working copy) is here: http://dpaste.com/49913/

The only other possibly relevant thing I can think of is that Entries also have a ManyToMany to Authors. An Entry is written by an Author (the ForeignKey), but can also be "about" some unspecified number of Authors. The ManyToMany has a related name on it, the ForeignKey doesn't, but changing that doesn't fix the code, and besides it has been working fine like that up until now.

Many thanks for any insight!

Eric

and, for the heck of it, the traceback:

Traceback: File "/Library/Python/2.5/site-packages/django-trunk/django/core/handlers/base.py" in get_response 92. response = callback(request, *callback_args, **callback_kwargs) File "/Library/Python/2.5/site-packages/django-trunk/django/utils/decorators.py" in _wrapped_view 48. response = view_func(request, *args, **kwargs) File "/Users/eric/stuff/dbase/views.py" in contributors 128. conts = list(Author.objects.contributors()) File "/Users/eric/stuff/dbase/managers.py" in contributors 34. return self.select_related("user").filter(is_contributor=True).exclude(entry__isnull=True) File "/Library/Python/2.5/site-packages/django-trunk/django/db/models/query.py" in exclude 474. return self._filter_or_exclude(True, *args, **kwargs) File "/Library/Python/2.5/site-packages/django-trunk/django/db/models/query.py" in _filter_or_exclude 483. clone.query.add_q(~Q(*args, **kwargs)) File "/Library/Python/2.5/site-packages/django-trunk/django/db/models/sql/query.py" in add_q 1610. self.add_q(child, used_aliases) File "/Library/Python/2.5/site-packages/django-trunk/django/db/models/sql/query.py" in add_q 1614. can_reuse=used_aliases) File "/Library/Python/2.5/site-packages/django-trunk/django/db/models/sql/query.py" in add_filter 1512. negate=negate, process_extras=process_extras) File "/Library/Python/2.5/site-packages/django-trunk/django/db/models/sql/query.py" in setup_joins 1677. "Choices are: %s" % (name, ", ".join(names))) Exception Type: FieldError at /contributors/ Exception Value: Cannot resolve keyword 'entry' into field. Choices are: book, categories, characters, contact, cv, desc, desc_HTML, edit_date, email, gender, given_name, id, is_author, is_chinese, is_contributor, is_living, is_translator, links, location, photo, post_date, profession, rel_entries, sample, slug, surname, translatorprofile, user, webpage

[Karen Tracey]

On Sun, May 31, 2009 at 11:40 PM, Eric Abrahamsen <gir...@gmail.com> wrote:

On May 31, 2009, at 11:45 PM, Karen Tracey wrote:

On Sun, May 31, 2009 at 10:13 AM, Eric Abrahamsen <gir...@gmail.com> wrote:

Hi,

I've got two models, Author and Entry, with a foreignkey from Entry to
Author. There are many Authors with no related entries, and until
recently I've been able to put this manager on the Author model to
only select Authors with Entries:

def contributors():
 return self.exclude(entry__isnull=True)

Recently this started returning a FieldError, saying it could not
resolve "entry" into a field.

What changed?  Did you update Django?  Change your code?  Something must have changed, and if you could identify it, that would be a clue.

[snip]

I identified the commit in my own code that blew things up (thanks to my excellent version-tracking habits, it's a pretty large and tangled commit). This commit touches neither the Author nor the Entry model, it's got:

Two new models, one called Sample with a ForeignKey to Author, one called TranslatorProfile with a OneToOneField to Author (no related names on either).

ModelForms for both of these new models. I also moved two existing modelforms to a different place in the file.

I'm putting the diff file into dpaste here: http://dpaste.com/49912/, in case anyone's got the patience to look at it. That's a diff between -r301, which works, and working copy, which doesn't. The whole models.py file (broken working copy) is here: http://dpaste.com/49913/

The only other possibly relevant thing I can think of is that Entries also have a ManyToMany to Authors. An Entry is written by an Author (the ForeignKey), but can also be "about" some unspecified number of Authors. The ManyToMany has a related name on it, the ForeignKey doesn't, but changing that doesn't fix the code, and besides it has been working fine like that up until now.

Identifying the commit is a good first step, next would be to identify which part of the commit caused the break.  (Even though it doesn't seem like any part of that commit should have caused the break.  But it did, so you need to narrow it down to what exact part of the commit in order to possibly figure out why.)

Playing around with your models file and reducing it to the essentials to recreate the problem reveals it's the new SampleForm.  Sample also has a ForeignKey to Author, one with a limit_choices_to depending on a field in Author.  Having that limit_choices_to AND defining the SampleForm before the Entry model apparently results in an incomplete Author model.  I've opened #11247 to track this.  In the meantime, moving all ModelForm definitions out of the models file (or at least after all model definitions) avoids the problem. 

Karen

[Eric Abrahamsen]

Holy crow, this is simultaneously genius on your part, and also something I should have been able to figure out (or at least pinpoint) on my own :)

Thanks enormously for the solution – moving all model forms to the bottom of the file did the trick, and I'll keep an eye on that ticket. Thanks for getting me past this.

Eric

Karen