Django 1.11 doesn't work well behind pgBouncer

[Sergey Fursov]

Hi!

In our application stack we connect from our web/background workers to postgres database through dedicated pgBouncer service. 

pgBouncer use transaction pooling 

Connections work in autocommit mode.

After upgrading to 1.11 version with new server-side cursors for iterator method, many of our DB requests started failing with "Cursor _django_curs_<id> does not exist'. 

According to pgBouncer docs, it doesn't support using WITH HOLD cursors with transaction pooling mode. But it doesn't seems reasonable to run pgBouncer in session pooling mode, we can just switch to direct persistent connections to postgres DB. On the other hand in this case we have to increase minimum number of DB connection to number of our workers (from 5 to ~30) to prevent blocking blocking requests. And this is violate recommendation to have (core_count * 2) + effective_spindle_count max connections to DB.

So is there some recommendations how efficiently work with postgres DB in django 1.11?

Thanks,

Sergey

[Sergey Fursov]

Accepted ticket - https://code.djangoproject.com/ticket/28062

[David Baum]

Yep, we are experiencing the same issue after the upgrade, downgraded back to the working version - 1.10.7