[Django] #35349: Transaction API does not respect the DATABASE_ROUTERS configuration

[Django]

#35349: Transaction API does not respect the DATABASE_ROUTERS configuration
-------------------------------------+-------------------------------------
Reporter: diachkow | Owner: nobody
Type: Bug | Status: new
Component: Database | Version:
layer (models, ORM) | Keywords: transaction, db,
Severity: Normal | database routers
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
Using the transaction API from `django.db.transaction` module directly
does not respect the DATABASE_ROUTERS configuration and it does not grab
the db_for_write() database alias by default. Instead, it always falls
down to django.db.utils.DEFAULT_DB_ALIAS. The expected behaviour for this
case would either use db_for_write() or introduce a new
db_for_transaction() method for database routers as was proposed in
[https://groups.google.com/g/django-developers/c/clzg6MiixFc this
discussion]. The discussion itself has no resolution in a two years, so I
have decided to create a dedicated issue for that.

What I mean, if that whenever in code you use:

{{{
from django.db import trasnaction

@transaction.atomic()
def foo(...):
# ... do stuff ...
transaction.commit()
}}}


The `atomic()`, `commit()` and other functions should get the
write/transaction database under the hood.
----
** Note **: I am developing an application that switches the database
connection on per-tenant bases. The database configurations are added to
settings.DATABASES at runtime in a middleware and then, using the
`contextvars.ContextVar` thread-local variable, I am passing the database
alias to use from a middleware to my custom database router. It works fine
for reading and writing data outside the transactions, but it fails when
it comes to transaction. I could potentially pass the value of
`ContextVar` as an argument to all Transaction API calls, but it still
fails for the third-party libraries that are mostly calling this functions
without arguments. I have patched globally
`django.db.transaction.DEFAULT_DB_ALIAS` to a stub string-like object that
resolves dynamically in a runtime to a value of `ContextVar`, but that
solution seems to be weird and I wish I could make it through configuring
`DATABASE_ROUTERS`.
--
Ticket URL: <https://code.djangoproject.com/ticket/35349>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#35349: Transaction API does not respect the DATABASE_ROUTERS configuration
-------------------------------------+-------------------------------------
Reporter: diachkow | Owner: nobody
Type: Bug | Status: new

Component: Database layer | Version:
(models, ORM) |
Severity: Normal | Resolution:
Keywords: transaction, db, | Triage Stage:
database routers | Unreviewed
Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Description changed by diachkow:

Old description:

New description:

Using the transaction API from `django.db.transaction` module directly
does not respect the DATABASE_ROUTERS configuration and it does not grab
the db_for_write() database alias by default. Instead, it always falls
down to django.db.utils.DEFAULT_DB_ALIAS. The expected behaviour for this
case would either use db_for_write() or introduce a new
db_for_transaction() method for database routers as was proposed in
[https://groups.google.com/g/django-developers/c/clzg6MiixFc this
discussion]. The discussion itself has no resolution in a two years, so I
have decided to create a dedicated issue for that.

What I mean, if that whenever in code you use:

{{{
from django.db import trasnaction

@transaction.atomic()
def foo(...):
# ... do stuff ...
transaction.commit()
}}}

The `atomic()`, `commit()` and other functions should access the

write/transaction database under the hood.
----
** Note **: I am developing an application that switches the database
connection on per-tenant bases. The database configurations are added to
settings.DATABASES at runtime in a middleware and then, using the
`contextvars.ContextVar` thread-local variable, I am passing the database
alias to use from a middleware to my custom database router. It works fine
for reading and writing data outside the transactions, but it fails when
it comes to transaction. I could potentially pass the value of
`ContextVar` as an argument to all Transaction API calls, but it still
fails for the third-party libraries that are mostly calling this functions
without arguments. I have patched globally
`django.db.transaction.DEFAULT_DB_ALIAS` to a stub string-like object that
resolves dynamically in a runtime to a value of `ContextVar`, but that
solution seems to be weird and I wish I could make it through configuring
`DATABASE_ROUTERS`.

--

--
Ticket URL: <https://code.djangoproject.com/ticket/35349#comment:1>

[Django]

#35349: Transaction API does not respect the DATABASE_ROUTERS configuration
-------------------------------------+-------------------------------------

Reporter: Vitaliy Diachkov | Owner: nobody
Type: New feature | Status: closed

Component: Database layer | Version:
(models, ORM) |

Severity: Normal | Resolution: fixed

Keywords: transaction, db, | Triage Stage:
database routers | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce):

* resolution: => fixed
* status: new => closed
* type: Bug => New feature

Old description:

> Using the transaction API from `django.db.transaction` module directly
> does not respect the DATABASE_ROUTERS configuration and it does not grab
> the db_for_write() database alias by default. Instead, it always falls
> down to django.db.utils.DEFAULT_DB_ALIAS. The expected behaviour for this
> case would either use db_for_write() or introduce a new
> db_for_transaction() method for database routers as was proposed in
> [https://groups.google.com/g/django-developers/c/clzg6MiixFc this
> discussion]. The discussion itself has no resolution in a two years, so I
> have decided to create a dedicated issue for that.
>
> What I mean, if that whenever in code you use:
>
> {{{
> from django.db import trasnaction
>
> @transaction.atomic()
> def foo(...):
> # ... do stuff ...
> transaction.commit()
> }}}
>

> The `atomic()`, `commit()` and other functions should access the
> write/transaction database under the hood.
> ----
> ** Note **: I am developing an application that switches the database
> connection on per-tenant bases. The database configurations are added to
> settings.DATABASES at runtime in a middleware and then, using the
> `contextvars.ContextVar` thread-local variable, I am passing the database
> alias to use from a middleware to my custom database router. It works
> fine for reading and writing data outside the transactions, but it fails
> when it comes to transaction. I could potentially pass the value of
> `ContextVar` as an argument to all Transaction API calls, but it still
> fails for the third-party libraries that are mostly calling this
> functions without arguments. I have patched globally
> `django.db.transaction.DEFAULT_DB_ALIAS` to a stub string-like object
> that resolves dynamically in a runtime to a value of `ContextVar`, but
> that solution seems to be weird and I wish I could make it through
> configuring `DATABASE_ROUTERS`.

New description:

Using the transaction API from `django.db.transaction` module directly
does not respect the DATABASE_ROUTERS configuration and it does not grab
the db_for_write() database alias by default. Instead, it always falls
down to django.db.utils.DEFAULT_DB_ALIAS. The expected behaviour for this
case would either use db_for_write() or introduce a new
db_for_transaction() method for database routers as was proposed in
[https://groups.google.com/g/django-developers/c/clzg6MiixFc this
discussion]. The discussion itself has no resolution in a two years, so I
have decided to create a dedicated issue for that.

What I mean, if that whenever in code you use:

{{{

from django.db import transaction

@transaction.atomic()
def foo(...):
# ... do stuff ...
transaction.commit()
}}}


The `atomic()`, `commit()` and other functions should access the
write/transaction database under the hood.
----
** Note **: I am developing an application that switches the database
connection on per-tenant bases. The database configurations are added to
settings.DATABASES at runtime in a middleware and then, using the
`contextvars.ContextVar` thread-local variable, I am passing the database
alias to use from a middleware to my custom database router. It works fine
for reading and writing data outside the transactions, but it fails when
it comes to transaction. I could potentially pass the value of
`ContextVar` as an argument to all Transaction API calls, but it still
fails for the third-party libraries that are mostly calling this functions
without arguments. I have patched globally
`django.db.transaction.DEFAULT_DB_ALIAS` to a stub string-like object that
resolves dynamically in a runtime to a value of `ContextVar`, but that
solution seems to be weird and I wish I could make it through configuring
`DATABASE_ROUTERS`.

--

Comment:

Hi Vitaliy, thank you for your report.

> Using the transaction API from django.db.transaction module directly
does not respect the DATABASE_ROUTERS configuration and it does not grab
the db_for_write() database alias by default. Instead, it always falls
down to django.db.utils.DEFAULT_DB_ALIAS.

Correct, this behaviour is also
[https://docs.djangoproject.com/en/5.0/topics/db/transactions/#django.db.transaction.atomic
documentented]

> **atomic** takes a **using** argument which should be the name of a
database. If this argument isn’t provided, Django uses the "default"
database.

I am changing this from a bug to a feature request as to update the
default to be db_for_write() would be a change of behaviour and so needs
some thought around deprecations etc.
For cases like this, the recommended path forward is to first propose and
discuss the idea/request with the community and gain consensus. To do
that, please consider starting a new conversation on the
[https://forum.djangoproject.com/c/internals/5 Django Forum], where you'll
reach a wider audience and likely get extra feedback.

I'll close the ticket for now, but if there is a community agreement to
change this behaviour, you are welcome to come back to the ticket and
point to the forum topic, so we can then re-open it. For more details,
please see
[https://docs.djangoproject.com/en/stable/internals/contributing/bugs-and-
features/#requesting-features the documented guidelines for requesting
features].
--
Ticket URL: <https://code.djangoproject.com/ticket/35349#comment:2>

[Django]

#35349: Transaction API does not respect the DATABASE_ROUTERS configuration
-------------------------------------+-------------------------------------
Reporter: Vitaliy Diachkov | Owner: nobody
Type: New feature | Status: closed
Component: Database layer | Version:
(models, ORM) |

Severity: Normal | Resolution: wontfix

Keywords: transaction, db, | Triage Stage:
database routers | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Sarah Boyce):

* resolution: fixed => wontfix

--
Ticket URL: <https://code.djangoproject.com/ticket/35349#comment:3>

[Django]

#35349: Transaction API does not respect the DATABASE_ROUTERS configuration
-------------------------------------+-------------------------------------
Reporter: Vitaliy Diachkov | Owner: nobody
Type: New feature | Status: closed
Component: Database layer | Version:
(models, ORM) |
Severity: Normal | Resolution: wontfix
Keywords: transaction, db, | Triage Stage:
database routers | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Adam Johnson):

Forum discussion: https://forum.djangoproject.com/t/use-database-routers-
to-pick-a-database-connection-for-transaction-api-by-default/29744
--
Ticket URL: <https://code.djangoproject.com/ticket/35349#comment:4>