Strange issue with django

[Andréas Kühne]

Hi all,

We have a django backend for our application - running with DRF. This has been working perfectly since November last year. Now we have upgraded to django 3.0 and also DRF to the latest version. We also added some DB indexes. After this upgrade we now have certain endpoints that went up to 25 s loading time (from about 5 s). We have reverted most of these changes and gone through our code but still can't find where this issue is.

We even reverted back to django 2.2 and the previous version of DRF which gave us reasonable speed, but it still doesn't work correctly. 

Does anyone have any ideas what the issue might be?

Regards,

Andréas

[Integr@te System]

Hi Andreas,

Let try:

# Perf tip 

https://gawron.sdsu.edu/compling/course_core/python_intro/intro_lecture_files/fastpython.html

# Release note Django 3.0, DRF vs. best version

Hope this helpful.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAK4qSCe1Nr3c41OOG2zKz3yt2z0pEE4G-629wWsRkLeHjqaNRQ%40mail.gmail.com.

[Andréas Kühne]

Yeah - we have gone through the release notes - and performance wise the environment is the same but we got an extreme performance dip - we have been trying to check where we have taken performance hits - but it seems to be random and also in places where we didn't have any issues previously....

We have checked the places where we are taking hits - but my main thing is I would like to find out WHY this has happened - because it just happened spontaneously....

Regards,

Andréas

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAP5HUWp2gXjRCKeqB5i3EZeSdp-tJUkHpkiUQUEJETCWp05ErQ%40mail.gmail.com.

[Amitesh Sahay]

Could you please let us know which part of the DRF performance dipped? We can try to troubleshoot accordingly.

Your statements are very generic.

Regards,

Amitesh

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAK4qSCdbL1h1b1dqhoLQh%3DTQ127XU0cx2Kp7KbCNKJzz_nVwYA%40mail.gmail.com.

[Integr@te System]

Hi Andreas,

some tools to check performance in Perf tip on mail above, to inspect deeper.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAK4qSCdbL1h1b1dqhoLQh%3DTQ127XU0cx2Kp7KbCNKJzz_nVwYA%40mail.gmail.com.

[Jason]

Do you have any performance monitoring in place, with profiling add-ons, like new relic's x-ray trace?

That was really helpful in figuring out a bottleneck in a django service last year.

[Jason]

Also, you haven't stated where the bottlenecks are happening.  Is it app code, internal in django, your stack, database, networking?  There's alot of variables in play for diagnosing a major slowdown, and you've provided no information about profiling/monitoring

[Amitesh Sahay]

Also let us know the procedure that you adopted to upgrade to the new version?

Regards,

Amitesh

On Thursday, 26 December, 2019, 10:19:31 pm IST, Jason <jjohn...@gmail.com> wrote:

Also, you haven't stated where the bottlenecks are happening.  Is it app code, internal in django, your stack, database, networking?  There's alot of variables in play for diagnosing a major slowdown, and you've provided no information about profiling/monitoring

--

You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To view this discussion on the web visit

https://groups.google.com/d/msgid/django-users/37b9526a-756f-4ac7-835a-af49f0dada02%40googlegroups.com

.

[Andréas Kühne]

Ok - let's take this one piece at a time.

1. As far as we have been able to tell is that we have nested serializers for certain aspects of the application - those went from loading in 3-4 s to over 40 s in different places. It's hard to exactly show, however the number of database requests skyrocketed as well. But our theory is currently that the related models (foreign keys) are loaded strangely.

2. We aren't exactly sure WHERE the bottlenecks are happening - that is also why the question is wide and generic.

3. We don't have any profiling tools running in production, however the figures we do have are on development.

4. We upgraded from the latest version of 2.2 to 3.0 in django and upgraded one minor version of DRF at the same time.

The main issue is that we got performance issues - major performance issues - while upgrading. This shouldn't be possible - not in the way that it happened. What worries us is if this happens again and we don't see this BEFORE a production release. That was the main problem we had.

We have now done a lot of workarounds in place where the code was slow - but we would still like to understand where the problems occur so that we don't get this surprise again.

My question was more generic - because we wanted to see if anyone else had experienced the same problems when upgrading.

The strangest thing now is that even if we downgrade the database (reverse migrations), downgrade the code and so on - the requests are still slow - where they haven't been previously.

Regards,

Andréas

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/2119895222.3536368.1577439175073%40mail.yahoo.com.

[Integr@te System]

Hi Andreas,

# First at all we couldn't help anymore without specified problem as your necessary show no more clear details.

# Tools(for profiling each running functions, many metrics need), advices from some other to help you see out of box and show them out to our supports.

# Plan to upgrade, include procedure, backward version recover from unexpected result. Libraries, dependency and so on.

Example: DRF drop python2, paging, Q object, _url... many so on.

So I wonder your hidden work or just want to survey on community for using these software packages.

Thank for sharing.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAK4qSCeUAhcoqGFesXTHvWyNGfE1-HEHASwPB8Lq-PPDhh_6%3Dw%40mail.gmail.com.

[Andréas Kühne]

Hi,

Thanks,

I know that I haven't given any other specifics, the reason for this is that it wasn't specific and it's hard to describe. I have described as much as possible currently.

Regarding downgrading - we have tried that on everything - we have reverted migrations, downgraded code and versions of libraries - everything. Unfortunately we haven't been able to rule out any culprit. The reason I asked my first question was to hear if anyone else has experienced this - and if they have found any reason for it. Apparently this is not the case - so we will continue searching. But thanks for the suggestions regardless :-)

Regards,

Andréas

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAP5HUWomX%2BVN6hbrnEuW29GtaWZg_gXYdNnLAacQOoHLgM_C4A%40mail.gmail.com.