[Django] #36657: Make advanced ORM functionality more discoverable

[Django]

#36657: Make advanced ORM functionality more discoverable
----------------------+-----------------------------------------
Reporter: Lily | Type: Uncategorized
Status: new | Component: Documentation
Version: dev | Severity: Normal
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
----------------------+-----------------------------------------
At Django on the Med we were discussing how many Django users reach for
raw SQL because they don't realise that Django provides tools to allow
building their query with the ORM.

I think there are several things we can do to improve the documentation
here:

* Add a how-to page explaining how to create custom database functions
with `Func`, custom lookups with `Lookup` and custom transforms with
`Transform`.
* Refactor the lookups reference page
(https://docs.djangoproject.com/en/5.2/ref/models/lookups/) to include a
list of the builtin lookups or create a new page.
* Add a page listing the builtin transforms.
* Add crosslinks between the how-to page and the reference docs for lists
of builtin functions, lookups and transforms.
--
Ticket URL: <https://code.djangoproject.com/ticket/36657>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#36657: Make advanced ORM functionality more discoverable

-------------------------------+--------------------------------------
Reporter: Lily | Owner: (none)

Type: Uncategorized | Status: new
Component: Documentation | Version: dev

Severity: Normal | Resolution:

Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

-------------------------------+--------------------------------------
Comment (by Lily):

For the how-to page, I was considering using [https://github.com/django
/new-features/issues/38#issuecomment-3380232816 `LastDayOfMonth` as a
worked example].
--
Ticket URL: <https://code.djangoproject.com/ticket/36657#comment:1>

[Django]

#36657: Make advanced ORM functionality more discoverable
-------------------------------+--------------------------------------
Reporter: Lily | Owner: (none)
Type: Uncategorized | Status: new
Component: Documentation | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------------------------

Changes (by Adam Johnson):

* cc: Adam Johnson (added)

--
Ticket URL: <https://code.djangoproject.com/ticket/36657#comment:2>

[Django]

#36657: Make advanced ORM functionality more discoverable
-------------------------------+------------------------------------

Reporter: Lily | Owner: (none)
Type: New feature | Status: new

Component: Documentation | Version: dev
Severity: Normal | Resolution:

Keywords: | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+------------------------------------

Changes (by Jacob Walls):

* stage: Unreviewed => Accepted
* type: Uncategorized => New feature

Comment:

I would rather this not become a dumping ground of advanced examples, but
rather a carefully scaffolded journey going from lower to higher
complexity, so that the reader can see that the ORM will be ready to
support them if they ever *do* develop more complex needs.

I'm also envisioning that the ''gathering requirements'' phase here would
involve surfing Trac, Discord, and the forum to collect frequently-asked-
questions around raw SQL usage or ORM help. Maybe about ten or twelve
examples of increasing complexity could be built around the same sample
models.
--
Ticket URL: <https://code.djangoproject.com/ticket/36657#comment:3>

[Django]

#36657: Make advanced ORM functionality more discoverable

-------------------------------+------------------------------------------
Reporter: Lily | Owner: Segni Mekonnen
Type: New feature | Status: assigned

Component: Documentation | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

-------------------------------+------------------------------------------
Changes (by Segni Mekonnen):

* has_patch: 0 => 1
* owner: (none) => Segni Mekonnen
* status: new => assigned

Comment:

Working on this ticket. PR submitted:
https://github.com/django/django/pull/19961

Created comprehensive advanced ORM documentation including:

**How-to Guide (advanced-orm-techniques.txt):**
- 12 scaffolded examples from basic to advanced complexity
- Built around consistent sample models (Article, Author, Comment)
- Covers custom functions, lookups, transforms, subqueries, and window
functions
- Each example includes real-world use cases and performance
considerations

**Built-in Lookups Reference (lookups-builtin.txt):**
- Complete list of all Django built-in lookups
- Examples and SQL equivalents for each lookup
- Cross-referenced with transforms and functions

**Built-in Transforms Reference (transforms-builtin.txt):**
- Complete list of all Django built-in transforms
- Examples showing transform chaining
- Cross-referenced with lookups and functions

**Improved Discoverability:**
- Added cross-links between how-to guide and reference docs
- Updated index pages to include new references
- All documentation follows Django's style guide

The implementation addresses all requirements from the ticket description
and follows Jacob's guidance for a "carefully scaffolded journey" rather
than a dumping ground of examples.

Ready for review.
--
Ticket URL: <https://code.djangoproject.com/ticket/36657#comment:4>

[Django]

#36657: Make advanced ORM functionality more discoverable
-------------------------------+------------------------------------------
Reporter: Lily | Owner: Segni Mekonnen
Type: New feature | Status: assigned
Component: Documentation | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+------------------------------------------

Changes (by Jacob Walls):

* has_patch: 1 => 0

Comment:

Hi, thanks for your interest here.

> The implementation addresses all requirements from the ticket
description and follows Jacob's guidance for a "carefully scaffolded
journey" rather than a dumping ground of examples.

I closed the PR just now, because while there are elements in there that
have a resemblance to what we want, I suspect that it was not sourced by

"surfing Trac, Discord, and the forum to collect frequently-asked-

questions".

For instance, here is a helpful forum post that could have been adapted
into an example:
https://forum.djangoproject.com/t/advanced-postgresql-functions-like-json-
to-recordset-json-jsonb-to-recordset-jsonb/35542/4

I understand my guidance may have been a little vague--and that in the
face of vagueness, an LLM can provide a starting point--but I'm hoping
we'll develop something written in the voice of a Django user, not in the
voice of an LLM, since users already have access to that kind of
documentation themselves.

Also, the PR includes extraneous things like a list of lookups and
transforms, which we already have. It also redundantly instructs how to
chain lookups together.

If others who try to pick up this ticket find the guidance too vague to be
actionable, we can set it to `needsinfo` or `Someday` pending a proof of
concept, but I'm hoping we won't have to do that just yet.
--
Ticket URL: <https://code.djangoproject.com/ticket/36657#comment:5>

[Django]

#36657: Make advanced ORM functionality more discoverable
-------------------------------+------------------------------------------
Reporter: Lily | Owner: Segni Mekonnen
Type: New feature | Status: assigned
Component: Documentation | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+------------------------------------------

Comment (by Jacob Walls):

> Refactor the lookups reference page
(​https://docs.djangoproject.com/en/5.2/ref/models/lookups/) to include a
list of the builtin lookups or create a new page.

> Add a page listing the builtin transforms.

I think the [https://docs.djangoproject.com/en/5.2/ref/models/querysets
/#field-lookups lookups] are already listed, and same with
[https://docs.djangoproject.com/en/5.2/ref/models/database-functions
/#module-django.db.models.functions transforms/database functions], so I
don't think we should create new pages. More links, though, would be 🍦.
--
Ticket URL: <https://code.djangoproject.com/ticket/36657#comment:6>