index view - returns *all* threads by default

[jaymzcd]

Hey all,

First off, many thanks for this, its been a great starting point for a
forum migration to django I'm working on. I loaded in some existing
data - approx 750,000 posts in around 90,000 threads and hit upon a
slight snap when I first went to the index view, it was loading /very/
slowly.

I left it and went home and when I got back in the next day the django-
debug-toolbar told me there had been just over 90,000 queries! :) The
category and thread views themselves don't suffer from this issue,
it's just the main index/recent discussions list. In the
views.thread_index there isn't any filtering done on the queryset so
it returns the entire 90,000 threads - its paginated but queries are
still made for *every* thread.

I've added a setting variable to my settings.py file and patched
views.py to limit it to a certain number. Seems to work ok. I couldnt
see any other way and seems to be resolved. Anyone had this before? (I
am using the most up to date trunk version).

jaymz

(heres the diff for views.py, it'll limit the index to 20 threads, its
pushed down some comments hence the extra). this sorts it out for me
for now. im guessing that the index isn't tied into the pagination for
some reason?

33a34
> INDEX_THREAD_DISPLAY = getattr(settings, 'SNAPBOARD_INDEX_THREAD_DISPLAY', 20)
173c174
<
---
>
189c190
<
---
>
305c306
< thread_list = filter(lambda t: t.category.can_view
(request.user), thread_list)
---
> thread_list = filter(lambda t: t.category.can_view(request.user), thread_list[:INDEX_THREAD_DISPLAY])
320c321
< # Count the number of visible posts before the one we are
looking for,
---
> # Count the number of visible posts before the one we are looking for,
441c442
< Although the Group model allows non-members to be admins, this
view won't
---
> Although the Group model allows non-members to be admins, this view won't
528c529
< Allows the UserBanMiddleware to limit the ban to SNAPboard in
larger
---
> Allows the UserBanMiddleware to limit the ban to SNAPboard in larger

[Bo]

Hi jaymz,

Thanks for the issue report and patch. If # of queries is unbounded
and linear with respect to # of items grabbed by the view, then it's
clearly broke as hell :P In addition to your patch, we should clearly
be using a join here (or using the db api more intelligently).

Thanks,
Bo

[Bo]

Also, in a slightly unrelated note, the list has been getting a ton of
spam of late and I've turned on moderation so there might be a longer
delay before your posts hit the web.