Performance of issue lists

[Andi Albrecht]

At the moment issue lists (mine, recent...) can easily reach the 10
seconds limit. The limit is reached when showing about 30 issue on a
page. This is caused by computing the comment count for each issue
during rendering (but it makes no significant difference if it happens
on the Python side).

I'm not sure what's the best solution for that. After thinking about
it for a while I came up with these ideas:

A) Never display more than ~20 issues on a page and implement
pagination
B) Calculate comment counts whenever the count changes and save it
along with the issue
C) Don't display comment counts on issue lists
D) Split up long running requests into smaller ones using some AJAX
features

But all of these solutions have some disadvantages. Here are some
random notes:

A) lists <~20 entries:
- pagination for recent issues isn't a problem, but the user's
dashboard can easily be longer than 20 issues and at least ongoing
issues should be visible on one page without pagination

B) pre-calculate counts:
- a counter for drafts needs to be on a per user basis, I'm not sure
if querying some kind of intermediate model increases performance
- more transactions need to be implemented

C) no counts at all:
- is obviously the worsest solution as it removes the most important
information on the user's dashboard
- hiding comment counts might be a fallback if a dashboard becomes
unusable (now that we can see other users dashboards, e.g. Guido's is
already quite extensive ;-). This should be relatively easy to
implement.

D) using AJAX:
- there should be some fallback if a user prefers not to enable
JavaScript for some reason.

For now it isn't a real life problem. The recent issues list has no UI
option to adjust the limit of displayed items per page and listing 10
issues could be more performant but it is ok for now. Only some
dashboards are close to raise a DeadlineExceeded...

In the long run I think we have to reconsider how issue lists are
generated.

Andi

[Jiayao Yu]

Good analysis. I am in the process of applying memcache all the data
query hotspots. So I will cover your solution B.

Jiayao

[Andi Albrecht]

I've applied your memcache patch to my local instance and it increases
performance a lot! Great!

On 7 Jun., 10:49, "Jiayao Yu" <jiayao...@gmail.com> wrote:
> Good analysis. I am in the process of applying memcache all the data
> query hotspots. So I will cover your solution B.
>
> Jiayao
>
> On Sat, Jun 7, 2008 at 9:27 AM, Andi Albrecht
>

[Andi Albrecht]

hm... after adding some more issues performance is better but not
comparable to my first test.

[Andi Albrecht]

Is this still an issue? I'm not sure which changes made it actually
faster, but at the moment I can view an issue list with 60 items
without any problems. Does anyone still have problems?