Reverse relations, select_related, and the Queryset Refactor Branch

[robotika]

I was initially excited at the idea of specifying a list related
fields with a select_related call, but have since discovered it
doesn't seem to work on reverse relations. I don't know why I was
expecting it to work (stupid optimism!), but it was a little
disappointing. What I was curious about, and I guess this question is
mostly for Malcolm, is how difficult would it be to allow
select_related to follow reverse relations if the fields/models were
specified? If it isn't too difficult, would this be something that
could end up in Django at some point?

To explain what I'm trying to do, I'd like to return unrelated objects
via a shared Model (by using their coordinates) using a single query.
I do have a feeling I probably haven't really thought it through
properly, but at the moment at least, it seems to make sense to
attempt to do it this way.

Here are my models.

# The model that holds Longitude/Latitude coordinates
class GeoPoint(models.Model):
""" A GeoDjango model that only holds Geographic Coordinates """
coords = models.PointField(srid=4326)

# The model for a City
class City(models.Model):
name = models.CharField(max_length=99)
point = models.OneToOneField(GeoPoint, related_name='place')

class Landmark(models.Model):
""" Tourism related landmarks """
name = models.CharField(max_length=99)
point = models.OneToOneField(GeoPoint, related_name='landmark')

class Photo(models.Model):
""" User uploaded photos """
name = models.CharField(max_length=99)
point = models.OneToOneField(GeoPoint, related_name='photo')

class Cafe(models.Model):
name = models.CharField(max_length=99)
point = models.OneToOneField(GeoPoint, related_name='cafe')

class Event(models.Model):
name = models.CharField(max_length=99)
point = models.OneToOneField(GeoPoint, related_name='event')


With this structure, if we were to visit Los Angeles, we could do the
following to return a queryset of things to see or do when in LA.

# Only display 20 objects
limit = 20

# Get the city of Los Angeles
city = City.objects.get(slug='los-angeles')

# Get the Coordinates of Los Angeles via the
point = city.geopoint.coords

# Find things in Los Angeles to display on a map. This won't work :(
stuff_to_see = GeoPoint.objects.select_related('landmark', 'photo',
'cafe', 'event')

# Sort the found objects by their distance from Los Angeles
stuff_to_see = stuff_to_see.distance(point).order_by('distance')
[:limit]

Now the above won't work as selected_related won't return any objects
as the specified fields don't exist in the GeoPoint model. I guess my
question at this point, as mentioned above, would having
select_related follow reverse relations in this way be too difficult
to implement into the qs-rf branch. I guess my hope is it's simple,
but yeah, things are usually never that simple :)

If it is too difficult/painful to do, then it's probably better I
start thinking of another way to implement this without resorting to
individual queries for each model. Anyone have any suggestions at
other ways to accomplish something similar? Hopefully some of my
example made sense :)

Thanks :)
John

[Malcolm Tredinnick]

On Tue, 2008-04-15 at 07:54 -0700, robotika wrote:
> I was initially excited at the idea of specifying a list related
> fields with a select_related call, but have since discovered it
> doesn't seem to work on reverse relations. I don't know why I was
> expecting it to work (stupid optimism!), but it was a little
> disappointing. What I was curious about, and I guess this question is
> mostly for Malcolm, is how difficult would it be to allow
> select_related to follow reverse relations if the fields/models were
> specified? If it isn't too difficult, would this be something that
> could end up in Django at some point?

Anything that is multi-valued is problematic because you have to (a)
work out a way to represent the results (that bit's actually fairly easy
and (b) make sure that people don't shoot themselves in the foot by
trying to extract more than one multi-valued result at a time. Part (b)
is a little tricky, because it's surprisingly easy to do by accident.

The problem with multiple multi-valued results is that you cannot do it
efficiently in a single SQL query without using UNION (and constructing
the UNION to be efficient isn't completely trivial either in the general
case). So it's not too hard to construct the naïve query for a something
with multi-valued field #1 returning 1000 results and multi-valued field
#2 returning 1000 results that ends up retrieving 1000 * 1000 = 1 000
000 rows, rather than only 2000 rows (you must query for the #1 and #2
fields separately).

The same category of problem includes using many-to-many fields in
select_related. It also includes doing the same thing in values()
queries.

Not impossible to solve, but important to solve it efficiently and not
entirely trivial to do so. Therefore, something for The Future.

Regards,
Malcolm

--
Remember that you are unique. Just like everyone else.
http://www.pointy-stick.com/blog/

[robotika]

Awesome, thanks for the clarification Malcolm. It's certainly
something I think would be useful, especially when it comes to
geographically aware applications. I only wish I was smart enough to
help contribute towards it :) I'm probably going to try having a
GenericRelation model sit inbetween a GeoPoint model and the other
location based models in the hope that maybe it'll work without too
much stress on a server... especially in situations where there will
be the constant updating and plotting of objects when a user moves
around a map.

Thanks again :)

John

On Apr 16, 1:26 pm, Malcolm Tredinnick <malc...@pointy-stick.com>
wrote:

[Trey]

Are one to one fields going to allow reverse select_related style
caching? I thought that Queryset Refactor might have included this but
no such luck.