Conditional annotation super slow

[Mark]

I'm using django's (1.8.5) conditional annotation in order to add filtered data to each object dynamically. But the issue is that it's very slow on all machines I've tested it.

What it does is basically adds data to each user. How many messages they sent from provided date, how many messages have failed and how many messages they sent from yesterday. There is not much data. Just about 100 users and each of them may have about 50 messages daily.

objects = self.get_queryset().annotate(sent_count=
        Sum(
            Case(When(messages__date_sent__gte=date_from, then=1)),
            output_field=IntegerField()
        ),
        error_count=
        Sum(
            Case(When(messages__status__iexact='error', then=1)),
            output_field=IntegerField()
        ),
        sent_yesterday=
        Sum(
            Case(When(messages__date_sent__gte=yesterday, then=1)),
            output_field=IntegerField()
        )
    )

When checking a query, it takes up to 15 seconds to execute it. Am I doing something wrong or is conditional annotation not designed for things like this?

After looking at the generated query, it seems that Django is trying to GROUP BY by every available field in both models and their related models.. After adding prefetch_related to some related models, issue was fixed on one machine, but still persists on the second one. 

Is this a bug?

[Mark]

I've run query manually removing all fields from GROUP BY that are not related to aggregation and query runs fast. But how to do the same in Django? How to remove unrelated fields to the aggregation from GROUP BY clause?

[Simon Charette]

Hi Mark,

I suppose you opened #26045 about the same issue.

Are you using PostgreSQL? If it's the case the issue I suppose the issue is fixed in Django 1.9.

Simon