[Django] #31958: Django ORM sync_to_async error FATAL: remaining connection slots are reserved for non-replication superuser connections
Django
reserved for non-replication superuser connections
-------------------------------------+-------------------------------------
Reporter: super-sam | Owner: nobody
Type: Bug | Status: new
Component: Database | Version: 3.1
layer (models, ORM) |
Severity: Normal | Keywords: ORM Async
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
I am trying to implement Asynchronous
support(https://docs.djangoproject.com/en/3.1/topics/async/) for Django
ORM to write many records in Database(Postgres).
I am getting the ERROR:root:**FATAL: remaining connection slots are
reserved for non-replication superuser connections**
''I am creating coroutines add adding them to running the asyincio loop''
{{{
# Helper Class
class ModelsHelper:
@staticmethod
@sync_to_async
def __handle_resource_data(data):
// Some calculation to create kwargs dict
return Resource.objects.get_or_create(**kwargs)
async def store_data(metric):
// some calculation to get data
return await ModelsHelper.__handle_resource_data(data)
# Main File
def get_event_loop():
loop = None
try:
loop = asyncio.get_event_loop()
except Exception as e:
print(" New Event Loop ".center(50, '*'))
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
return loop
loop = get_event_loop()
future = asyncio.ensure_future(
asyncio.gather(*[ModelsHelper.store_data(metric) for metric in
metrics]),
loop=loop
)
loop.run_until_complete(future)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/31958>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
reserved for non-replication superuser connections
-------------------------------------+-------------------------------------
Type: Bug | Status: closed
Component: Database layer | Version: 3.1
(models, ORM) |
Severity: Normal | Resolution: invalid
Keywords: ORM Async | Triage Stage:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => invalid
Comment:
This is a usage issue. Please see TicketClosingReasons/UseSupportChannels.
I'd post the forum in the internals async channel.
https://forum.djangoproject.com/
Essentially you're going to need to wait for async ORM support to do this
properly.
Even then you'll need to consider the number of DB connections you're
creating. You're trying to create a connection for each item in `metrics`,
which is too many (This kind of issue is perennial...)
--
Ticket URL: <https://code.djangoproject.com/ticket/31958#comment:1>
Django
reserved for non-replication superuser connections
-------------------------------------+-------------------------------------
Type: Bug | Status: closed
Component: Database layer | Version: 3.1
(models, ORM) |
Severity: Normal | Resolution: invalid
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Carlton Gibson):
Just a follow-up: ORM calls should use the `thread_sensitive` parameter to
`sync_to_async()` this would likely make your example run, but it would do
so serially, rather than concurrently, since all the operations would be
run in a single thread executor. (Maybe that helps anyway.)
--
Ticket URL: <https://code.djangoproject.com/ticket/31958#comment:2>