AWS RDS Proxy and session pinning

[dans...@gmail.com]

Hi Django developers,

At the National Library of Medicine we are doing a lot of Django and AWS.  For some of the applications with heavier traffic, we are using RDS Proxy within AWS.  RDS Proxy is sort of like a managed version of pgbouncer or pg-pool-II - it must be rather AWS customized because of the way it manages authentication.

Anyway, we found some problems for two reasons:

• Our DBAs make the default timezone of the database UTF-8 is AWS CloudFormation to create the databases.
• When "ensure_timezone" runs and sets the timezone, that database session is "pinned", so that it will not be shared.

Since we already have our own PostgreSQL backend for a couple of other reasons, I just wrote a version of ensure_timezone which fails loudly if the timezone is not UTF-8 rather than set the timezone.  I probably should have discussed the issue on this list, and I am remedying this.

The failure to manage database connections with RDS Proxy is a pretty severe error, and I am wondering what the community thinks about a connection specific setting about how the timezone should be handled?   For lower traffic applications, SET TIMEZONE is fine.  For higher traffic applications, raising ImproperlyConfigured is better.   

Does the group think that a Postgreql specific setting/option should be implemented to prevent session pinning?

[dans...@gmail.com]

Sorry - lest I miscommunicate - the DBAs make the default equivalent to EST5EDT, rather than UTC.   Django team leads (me and David), want USE_TZ = True to be on for all the applications, and because of this the postgresql backend will issue SET TIMEZONE UTC in django/db/backends/postgreql/base.py:209 (ensure_timezone).  At the same time, different applications and frameworks have different opinions and the DBAs try to satisfy us all, but can forget.  This issue of session pinning with connection pooling servers (such as RDS Proxy)  may be more general, and it may be good to not always set the timezone, but I want to discuss before filing an issue.

[Adam Johnson]

Hi!

Good to hear about the use of RDS Proxy. I have considered looking at it to help scaling.

You should be able to bypass the timezone check yourself with a little subclassing. You can implement this yourself with a subclassed database backend like so: https://docs.djangoproject.com/en/4.0/ref/databases/#subclassing-the-built-in-database-backends . Use it to replace the ensure_timezone() method with appropriate logic.

That may be all that's required. It would be good to know if that works before considering changing Django.

Thanks,

Adam

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/ecc0c527-465a-45c7-b6ff-3c35c56824bfn%40googlegroups.com.

[dans...@gmail.com]

We already do that - our backend has some other work to do since we use AWS Secrets Manager to store database passwords, and also the hostname, so that our settings.DATABASES does not contain the PASSWORD but must contain a SECRET_ID.  The question is whether this is worth promoting since AWS RDS Proxy (and other similar proxy managers) are quite common.