Upgrade Confusion with ForeignKey

[Tim Sawyer]

I've just upgraded from a very old version of Django (1.4) up to 1.7.8, and some behaviour isn't how I expect.  I may have missed something - I've not been keeping an eye on the framework changes.

Two models, Venue and VenueAlias.  VenueAlias has a ForeignKey to Venue, one venue has many VenueAliases. (Full models pasted below.)

My problem is with accessing venuealias_set on my Venue, it doesn't seem to be limiting to Venue how it used to in the earlier version.

Here's an interactive session that shows the problem.  Venue 5854 has three aliases.

>>> from bbr.contests.models import Venue
>>> lVenue = Venue.objects.filter(id=5854)[0]
>>> lVenue
<Venue: Royal Northern College of Music (Haden Freeman Concert Hall), Manchester, Greater Manchester, England, UK>
>>> lVenue.exact
False
>>> lAliases = lVenue.venuealias_set.all()
>>> len(lAliases)
402
>>> print lAliases.query
SELECT "venues_venuealias"."id", "venues_venuealias"."last_modified", "venues_venuealias"."created", "venues_venuealias"."name", "venues_venuealias"."alias_start_date", "venues_venuealias"."alias_end_date", "venues_venuealias"."venue_id", "venues_venuealias"."lastChangedBy_id", "venues_venuealias"."owner_id" FROM "venues_venuealias" INNER JOIN "contests_venue" ON ( "venues_venuealias"."venue_id" = "contests_venue"."id" ) WHERE "contests_venue"."exact" = True ORDER BY "venues_venuealias"."name" ASC

Where's the SQL to limit by the foreign key to venue.id=5854?  Why is there a reference in here to contests_venue.exact?

Approaching the same problem from the other direction works fine, and I get the correct number of aliases returned.

>>> from bbr.venues.models import VenueAlias
>>> lVenueAliases = VenueAlias.objects.filter(venue_id=5854)
>>> len(lVenueAliases)
3
>>> print lVenueAliases.query
SELECT "venues_venuealias"."id", "venues_venuealias"."last_modified", "venues_venuealias"."created", "venues_venuealias"."name", "venues_venuealias"."alias_start_date", "venues_venuealias"."alias_end_date", "venues_venuealias"."venue_id", "venues_venuealias"."lastChangedBy_id", "venues_venuealias"."owner_id" FROM "venues_venuealias" WHERE "venues_venuealias"."venue_id" = 5854 ORDER BY "venues_venuealias"."name" ASC

On my old server, running the old django 1.4 code, both these approaches work as expected.  Venue and VenueAlias are actually in different models.py files, for historical reasons.

What have I missed?  Thanks for any input/education!

Tim.


class VenueAlias(models.Model):
    """
    An alias for a venue
    """
    last_modified = models.DateTimeField(default=datetime.now,editable=False)
    created = models.DateTimeField(default=datetime.now,editable=False)
    name = models.CharField(max_length=200, help_text='Name of Venue Alias')
    alias_start_date = models.DateField(blank=True, null=True, help_text="Start date for this alias (yyyy-mm-dd)")
    alias_end_date = models.DateField(blank=True, null=True, help_text="End date for this alias (yyyy-mm-dd)")
    venue = models.ForeignKey(Venue)
    lastChangedBy = models.ForeignKey(User, editable=False, related_name='VenueAliasLastChangedBy')
    owner = models.ForeignKey(User, editable=False, related_name='VenueAliasOwner')
   
    def save(self):
        self.last_modified = datetime.now()
        super(VenueAlias, self).save()
       
    def show_start_date(self):
        """
        Return true if start date is recent enough to make sense
        """
        lCutOffDate = date(year=1700, month=1, day=1)
        print lCutOffDate
        print self.alias_start_date
        if self.alias_start_date > lCutOffDate:
            return True
        return False
    
    @property
    def slug(self):
        return self.venue.slug
   
    def __unicode__(self):
        return "%s -> %s" % (self.name, self.venue.name)
   
    class Meta:
        ordering = ['name']
        verbose_name_plural = 'Venue aliases'


class Venue(models.Model):
    """
    A venue for a contest
    """
    last_modified = models.DateTimeField(default=datetime.now,editable=False)
    created = models.DateTimeField(default=datetime.now,editable=False)
    name = models.CharField(max_length=255)
    slug = models.SlugField()
    country = models.ForeignKey(Region, blank=True, null=True)
    latitude = models.CharField(max_length=15, blank=True, null=True)
    longitude = models.CharField(max_length=15, blank=True, null=True)
    point = geomodels.PointField(dim=3, geography=True, blank=True, null=True, editable=False)
    postcode = models.CharField(max_length=10, blank=True, null=True)
    exact = models.BooleanField(default=False, help_text="True if latitude and longitude is for a building, rather than a town")
    mapper = models.ForeignKey(User, editable=False, related_name='VenueMapper', blank=True, null=True)
    parent = models.ForeignKey("Venue", blank=True, null=True)
    notes = models.TextField(blank=True, null=True)
    lastChangedBy = models.ForeignKey(User, editable=False, related_name='VenueLastChangedBy')
    owner = models.ForeignKey(User, editable=False, related_name='VenueOwner')
       
    def __unicode__(self):
        return "%s" % (self.name)
   
    def asJson(self):
        lDict = {
                 'id' : self.id,
                 'name' : self.name,
                 }
        return json.dumps(lDict)
   
    def venue_name(self, pDateOfEvent):
        """
        Return venue name, or alias name if there is an alias for the date specified
        """
        try:
            lAliasMatch = self.venuealias_set.filter(alias_start_date__lte=pDateOfEvent, alias_end_date__gte=pDateOfEvent)[0]
            return lAliasMatch.name
        except IndexError:
            return self.name
   
    def save(self):
        if self.latitude != None and len(self.latitude) > 0 and self.longitude != None and len(self.longitude) > 0:
            lString = 'POINT(%s %s)' % (self.longitude.strip(), self.latitude.strip())
            self.point = fromstr(lString)
        self.last_modified = datetime.now()
        super(Venue, self).save()
   
    class Meta:
        ordering = ['name']  

[Carl Meyer]

Hi Tim,

On 06/05/2015 03:44 PM, Tim Sawyer wrote:
> I've just upgraded from a very old version of Django (1.4) up to 1.7.8,
> and some behaviour isn't how I expect. I may have missed something -
> I've not been keeping an eye on the framework changes.

I would strongly recommend that you upgrade version by version (that is,
first 1.4.x to 1.5.12, then 1.5.12 to 1.6.11, then 1.6.11 to 1.7.8),
each as a separate step, making sure that everything works as you expect
between each step, and that you carefully read the release notes (if you
haven't) for each version as you update to it.

Django tries to maintain backwards compatibility where we can, but we do
have to change and remove features sometimes. Those changes are
documented in the release notes for each version.

If you've upgraded all the way from 1.4 to 1.7 without reading the
release notes, which is what it sounds like, then I will be very
surprised if the below is the only problem you have.

That is very strange. I've never seen that, and I'm not aware of any
changes to Django that would cause that. Nor do I see anything in the
models code you pasted below that looks suspicious to me as a possible
cause.

I have to guess that there's some other code in your project causing
this problem, but I'm afraid I don't know what that might be.

I'm afraid the best I can advise is to go back to Django 1.4 and do the
upgrade again methodically, and see if you can track down at precisely
which version this problem first manifests.

Good luck!

Carl

[Tim Sawyer]

Thanks. :-)

Did that, read the release notes, made sure it worked after each upgrade stage.  This is an esoteric corner of my app, and I didn't notice it not working.  The other foreign key stuff I have seems to work ok.

Looks like I have more investigation to do...any more clues over what *could* be the cause appreciated!

Cheers,

Tim.

[Carl Meyer]

Hi Tim,

On 06/05/2015 04:26 PM, Tim Sawyer wrote:
> Thanks. :-)
>
> Did that, read the release notes, made sure it worked after each upgrade
> stage. This is an esoteric corner of my app, and I didn't notice it not
> working. The other foreign key stuff I have seems to work ok.

Ah, good! In that case, sorry for the unnecessary admonitions. :-)

> Looks like I have more investigation to do...any more clues over what
> *could* be the cause appreciated!

Well, I can give you a rundown of some things that crossed my mind as I
was thinking through what might cause that symptom. Some of them don't
make much sense, and none of them satisfy me as an explanation given the
data I have, but maybe they'll remind you of something in your codebase:

* Was your shell session pasted exactly as you ran it? No possibility
that queryset came from somewhere other than the reverse related manager?

* Any custom model managers in use on either of the relevant models?
(Didn't look like it from the code you pasted).

* Both the apps are in INSTALLED_APPS?

* No `to_field` on any relevant ForeignKeys?

Beyond that, if I were debugging this I would use PDB to step through
the entire process of accessing the reverse-relation attribute,
generating the related-manager, getting a queryset from it, etc, and
compare that to the same process for a reverse-related-set that's
working properly.

Good luck!

Carl

[Stephen J. Butler]

I don't know if this is the problem, but...

On Fri, Jun 5, 2015 at 4:44 PM, Tim Sawyer <list....@calidris.co.uk> wrote:
> class Venue(models.Model):
> """
> A venue for a contest
> """
> last_modified =
> models.DateTimeField(default=datetime.now,editable=False)
> created = models.DateTimeField(default=datetime.now,editable=False)
> name = models.CharField(max_length=255)
> slug = models.SlugField()
> country = models.ForeignKey(Region, blank=True, null=True)
> latitude = models.CharField(max_length=15, blank=True, null=True)
> longitude = models.CharField(max_length=15, blank=True, null=True)
> point = geomodels.PointField(dim=3, geography=True, blank=True,
> null=True, editable=False)
> postcode = models.CharField(max_length=10, blank=True, null=True)
> exact = models.BooleanField(default=False, help_text="True if latitude
> and longitude is for a building, rather than a town")
> mapper = models.ForeignKey(User, editable=False,
> related_name='VenueMapper', blank=True, null=True)
> parent = models.ForeignKey("Venue", blank=True, null=True)

... ever since 1.4 the only documented way for a Model to have a
ForeignKey to another instance of the same type is to use the special
value "self". You've put "Venue" in for the "parent" field.