From Django 1.1 to Django 1.3.1 - Goes Very Slow

[ydjango]

 I have typical LAMP stack application. I use ORM very heavily. I recently migrated from django 1.1 to django 1.3.1.
Application has become very slow even though I have doubled the hardware.

Two major changes have been:
1) I have moved database (mysql) to different server
2) I have migrated django from 1.1 to 1.3.1

Any clues on what I should look at?
Any Django 1.3.1 specific settings I should look at?

[Paul Backhouse]

I think there are some browser tools that you could look at installing.
Firebug will tell you details about time to fetch files. I think there's
yslow too. Django debug toolbar will also tell you what queries are
being made and how long they take.

[ydjango]

Thank you Paul. I will use those tools.

I also plan to replace to some ORM queries by raw sql to avoid performance impact of cloning and multi db enhancements.

[akaariai]

On 21 elo, 19:04, ydjango <neerash...@gmail.com> wrote:
> Thank you Paul. I will use those tools.
>
> I also plan to replace to some ORM queries by raw sql to avoid performance
> impact of cloning and multi db enhancements.

If you can, please try to pinpoint the reason for regressions, and if
it seems to be in Django's ORM, then post to django-developers or
create a ticket in Django's trac. If there is some major regressions
in the ORM it would be very valuable to know about them.

Some possible explanations:
- The DB configuration isn't exactly as it was before.
- The ORM generates different query strings leading to slow query
plans.
- Using F() expressions and long filter chains can lead to slow
queryset .clone() speed (see https://code.djangoproject.com/ticket/16759)

You can try to pinpoint these by:
- Checking if the generated SQL is the same between 1.1 and 1.3 by
using qs.query.as_str() in 1.1 and str(qs.query) in 1.3.
- Check if the generation of the queryset takes too much time.
Easiest way is to generate a dummy script which does something like
this:
from datetime import datetime
start = datetime.now()
for i in range(0, 100):
# Do not execute the query, just construct the queryset!
qs = SomeModel.objects.filter(cond1).filter(cond2)... # Replace
with problematic queryset from production.
print datetime.now() - start

And see if there are any meaningful differences.

If the answer to the above is "no meaningful differences, the same SQL
generated" then it is reasonable to suspect the DB configuration.

- Anssi