[Django] #37175: Add database backend methods to get hardcoded or nonexistent primary key values for tests

[Django]

#37175: Add database backend methods to get hardcoded or nonexistent primary key
values for tests
-------------------------------------+-------------------------------------
Reporter: Tim | Owner: Tim Graham
Graham |
Type: New | Status: assigned
feature |
Component: Database | Version: dev
layer (models, ORM) |
Severity: Normal | Keywords:
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
While hardcoding primary key values in the Django test suite is avoided as
much as possible, sometimes there are legitimate uses. The existing
hardcoded integer values, however, don't work on databases that don't use
integer primary key values. For example, MongoDB uses `DEFAULT_AUTO_FIELD
= "django_mongodb_backend.fields.ObjectIdAutoField"`.

I propose adding new methods `DatabaseOperations.get_hardcoded_pk()` and
`get_nonexistentd_pk()` to allow a database backend to provide suitable
values, minimizing the size of the Django fork needed for running the test
suite on such databases.
--
Ticket URL: <https://code.djangoproject.com/ticket/37175>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#37175: Add database backend methods to get hardcoded or nonexistent primary key
values for tests
-------------------------------------+-------------------------------------

Reporter: Tim Graham | Owner: Tim
| Graham
Type: New feature | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Tim Graham):

* has_patch: 0 => 1

Comment:

[https://github.com/django/django/pull/21497 PR]
--
Ticket URL: <https://code.djangoproject.com/ticket/37175#comment:1>

[Django]

#37175: Add database backend methods to get hardcoded or nonexistent primary key
values for tests
-------------------------------------+-------------------------------------
Reporter: Tim Graham | Owner: Tim
| Graham
Type: New feature | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:

Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Natalia Bidart):

* cc: Simon Charette (added)
* stage: Unreviewed => Accepted

Comment:

Hello Tim! Thank you for taking the time to create the report. I
understand your concerns and the issue that this raises for other
backends. I agree we could improve this situation by providing.
potentially, some hooks at the database backend layer, but I also think
that many of the tests currently hardcoding the PKs could be re-worked to
not use a PK at all, or at least not assume is an int.

Right now we are prioritizing 6.1 beta work, but I'll commit to do a more
informed triage/review of this ticket after beta (likely next week). I'm
accepting it right now since the problem statement is real, but I'd like
to evaluate other options for the solution, ideally a solution that would
not require a `get_hardcoded_pk()`. Have you considered posting in the
forum to see what our creative community can suggest?

Thanks for your patience!
--
Ticket URL: <https://code.djangoproject.com/ticket/37175#comment:2>

[Django]

#37175: Add database backend methods to get hardcoded or nonexistent primary key
values for tests
-------------------------------------+-------------------------------------
Reporter: Tim Graham | Owner: Tim
| Graham
Type: New feature | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Old description:

> While hardcoding primary key values in the Django test suite is avoided
> as much as possible, sometimes there are legitimate uses. The existing
> hardcoded integer values, however, don't work on databases that don't use
> integer primary key values. For example, MongoDB uses
> `DEFAULT_AUTO_FIELD = "django_mongodb_backend.fields.ObjectIdAutoField"`.
>
> I propose adding new methods `DatabaseOperations.get_hardcoded_pk()` and
> `get_nonexistentd_pk()` to allow a database backend to provide suitable
> values, minimizing the size of the Django fork needed for running the
> test suite on such databases.

New description:

While hardcoding primary key values in the Django test suite is avoided as
much as possible, sometimes there are legitimate uses. The existing
hardcoded integer values, however, don't work on databases that don't use
integer primary key values. For example, MongoDB uses `DEFAULT_AUTO_FIELD
= "django_mongodb_backend.fields.ObjectIdAutoField"`.

I propose adding new methods `DatabaseOperations.get_hardcoded_pk()` and

`get_nonexistent_pk()` to allow a database backend to provide suitable

values, minimizing the size of the Django fork needed for running the test
suite on such databases.

--

Comment (by Tim Graham):

The first commit in my PR removes hardcoded pks where they are unneeded
(available separately in [https://github.com/django/django/pull/21364 PR
21364]).

Trying to rework the tests that require hardcoded pks may be possible in
some cases, however, I believe some form of `get_hardcoded_pk()` API will
be needed, so I feel the most expedient path forward is to merge the patch
and leave test refactoring to a later task.

Incidentally, I didn't update serializers and fixtures tests to use
`get_hardcoded_pk()` since it's a bit trickier, but this would also be a
useful follow up task. There's a commit on the Django fork used to run the
test suite on MongoDB that updates all fixtures to use ObjectIds instead
of integers (e.g. `"pk": 1,` because `"pk": "000000000000000000000001",`)
and this would remain, but the hardcoded pks in files such as
`tests/fixtures_regress/tests.py` could be replaced.''

At your request, I created forum topic [https://forum.djangoproject.com/t
/adding-database-backend-methods-to-get-hardcoded-or-nonexistent-primary-
key-values-for-tests/45359 Adding database backend methods to get
hardcoded or nonexistent primary key values for tests] to solicit
feedback.
--
Ticket URL: <https://code.djangoproject.com/ticket/37175#comment:3>