register_converter doesn't work with ASGI

115 views
Skip to first unread message

konstanti...@gmail.com

unread,
Jun 19, 2021, 2:31:15 AM6/19/21
to Django users

I'm trying to switch from WSGI to ASGI and I'm having issues with my model id to model converters.
I define my converters like this
def to_python(self, primary_key):
     try:
          return self.model.objects.get(pk=primary_key)
     except self.model.DoesNotExist:
          raise ValueError
When I go to a url where a converter is used I get a SynchronousOnlyOperation.
Seems like converters are executed in an async context.
If I use sync_to_async, I get a coroutine in my view instead of a model instance. Awaiting it here would defeat the purpose of having a converter.

Any ideas how to fix this?

Andrew Godwin

unread,
Jun 20, 2021, 7:16:58 PM6/20/21
to 'Helmy Sayed' via Django users
Ah yes, that's because the URL parsing/conversion all happens in the main (async, in ASGI mode) thread - and because to_python is a a synchronous function interface, there is literally nothing you can do to make it work.

The only real way of fixing this would be for Django to patch the URL converters and run them all in a synchronous mode unless they were decorated as "safe" somehow, much like we do for middleware. It's not a terribly hard patch to make, but it does mean you don't have an immediate solution. There are ways of trying to work around it, but they will all result in you blocking the async thread while the ORM query runs, which would be disastrous in production.

My apologies for not getting this in for the original async view patch - I had forgotten converters ran in the URL resolver.

Andrew
--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.

Konstantin Kuchkov

unread,
Jun 21, 2021, 3:21:38 AM6/21/21
to Django users
Thanks for the explanation!

To get it working, I will replace my model converters with basic converters and do corresponding lookups in the views.

Sunday Iyanu Ajayi

unread,
Jun 21, 2021, 8:49:55 AM6/21/21
to Django users
Dear Team,

Please I need help on how to optimize API requests calls for over 10,000 users simultaneously without any issues and my system spec is 8vCPUs, 16GB RAM.

Regards


Chetan Ganji

unread,
Jun 21, 2021, 9:34:08 AM6/21/21
to django...@googlegroups.com
What is the problem you are trying to solve? 
What do you want to optimize exactly? 🤔

I hope this helps you - 
https://betterprogramming.pub/how-to-ask-questions-about-programming-dcd948fcd2bd


Regards,
Chetan Ganji
+91-900-483-4183


--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAKYSAw2UTC6zvJoTWO%2BFQFqs9JQYaFbybC8ymp8kdt4cfwn3Nw%40mail.gmail.com.

Sunday Iyanu Ajayi

unread,
Jun 21, 2021, 9:58:46 AM6/21/21
to Django users
I want to be able to manage API calls from over 10000 users without getting network timeout errors
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAMKMUjuxKtKcF1Y%3Dw3o2XgcpKi2zdAhuGq6UmWzSe6ggDbPbFA%40mail.gmail.com.

Kasper Laudrup

unread,
Jun 21, 2021, 10:19:43 AM6/21/21
to django...@googlegroups.com
On 21/06/2021 15.55, Sunday Iyanu Ajayi wrote:
> I want to be able to manage API calls from over 10000 users without
> getting network timeout errors

Are you having issues doing so now?

What have you done to try and profile your code for bottlenecks?

What kind of help are you looking? Paid consultancy?

Etc.

You cannot really expect anyone to be able to help you if you don't take
some time trying to formulate a real question.

Kind regards,

Kasper Laudrup

OpenPGP_signature

carlos

unread,
Jun 21, 2021, 10:58:36 AM6/21/21
to django...@googlegroups.com
maybe try used cache when call many users

Cheers

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/7a436427-1069-d173-3e32-0339be9328dc%40stacktrace.dk.


--
att.
Carlos Rocha

Alejandro Garrido Gongora

unread,
Jun 21, 2021, 11:20:40 AM6/21/21
to django...@googlegroups.com
For big problems the answer is in 99% of cases split the problem, maybe your problem is your architecture, also you will need cache, also Async, maybe you need to en queue  some routines maybe you need more hardware, maybe you need to optimize your data structures, maybe the join of all those solutions is what you need.

Regards.
De: django...@googlegroups.com <django...@googlegroups.com> en nombre de carlos <croch...@gmail.com>
Enviado: Monday, June 21, 2021 4:57:02 PM
Para: django...@googlegroups.com <django...@googlegroups.com>
Asunto: Re: Best way to optimize API requests calls from over 10,000 users simultaneously
 

Chetan Ganji

unread,
Jun 26, 2021, 4:33:41 PM6/26/21
to django...@googlegroups.com

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/VI1PR0402MB3342DC9605EF444EA2B6CF35F60A9%40VI1PR0402MB3342.eurprd04.prod.outlook.com.

David Nugent

unread,
Jun 27, 2021, 10:06:38 PM6/27/21
to django...@googlegroups.com
Well, one workaround that may serve if your dataset isn't too large is to use a cache (or dict) and pre-load it.

Unlike the db, caches tend to be more async friendly, unless of course your cache is in your db....  :-)

This just needs to be pre-loaded on startup (<your app>.apps.:<your app AppConfig>.ready() is a good spot for this), and maybe add a signal on the relevant model to reload if it changes.

HTH,
David

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/c130ef88-4dbf-42b9-a7b3-f0ba0b174a0an%40googlegroups.com.

madhav

unread,
Jan 28, 2022, 10:35:04 AM1/28/22
to Django users
What is the problem you are trying to solve? 
What do you want to optimize exactly? 🤔

hope this course helps you : https://acciojob.com/

Reply all
Reply to author
Forward
0 new messages