How many maximum connections ckan make with postgresql dbs(ckandb and datastore db)

53 views
Skip to first unread message

Arqam Farooqui

unread,
Jul 6, 2021, 3:31:11 AM7/6/21
to CKAN Development Discussions
Hi all,

We have set 4 ckan_Default processes in our environment.
SqlAlchemy's queuepool size is 15 ( pool_size=5 + max_overflow=10).
Postgresql's max_connection is 100.

When all 100 postgresql connections get consumed then we are getting an error :
  "Description" : "Fatal,  too many clients already", 
   "Status"          : "500 Internal server error"

To avoid this error we want to increase the psql's `max_connections` value from 100 to maximum connections that ckan can use.

As per my investigation, maximum connections that ckan can use is:
1) Sqlalchemy's pool size is 15:
So ckan can make maximum 15 connections per process per db per engine.

for example if we have 1 ckan_default process, and 2 dbs (ckan db and datastore db) then:
15 pool connection * 1 process * 1 ckan db * 1 engine == 15 (ckan db) connections for 1 process.
15 pool connection * 1 process * 1 datastore db * 1 engine == 15 (datastore db) connections for 1 process.

2) We have set 4 ckan default processes in our ckan setup, then:

for ckan db:           15*4 == 60 maximum possible connection for 1 engine
for datastore db:   15*4 == 60 maximum possible connection for 1 engine
maximum total possible connections for ckan Application is 120 for 1 engine.

Question 1:) Is my above understanding is correct? 
Question 2:) Does any other process is also making connections with databases or not? (other than ckan_default processes)
Question 3:) How can I check, "How many engines are being used in my ckan application"

Please provide any suggestions on this, It will help a lot.
Thank you:):):)


Reply all
Reply to author
Forward
0 new messages