[Django] #35406: Using Django models in function type annotations, without dependency to settings.configure()

[Django]

#35406: Using Django models in function type annotations, without dependency to
settings.configure()
-------------------------------------+-------------------------------------
Reporter: HTErik | Owner: nobody
Type: New | Status: new
feature |
Component: Database | Version: 4.2
layer (models, ORM) |
Severity: Normal | Keywords: models, typing
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
When using Django defined Models together with type annotation on top
level functions in a module, the imports to the Django models first
require calling the global `settings.configure()`.
When used in a single file this works ok, in larger applications where
multiple files refer to the same model, this dependency spreads like a
plague and every file must either itself call `settings.configure()` at
the very top, or the imports must be placed in very careful order all the
way from the top level entrypoint of the application, creating huge
difficulties in reuse, testing, auto sort of imports.
It also creates implicit dependencies, where some times importing a module
works, depending on if any of the earlier modules has configured django.

Best practice for imports is to behave more like declarative code, by not
having side effects and not cause side effects. Django models and
`settings.configure()` are the exact opposite of this, creating big
difficulties with type checkers.

Consider following basic scenario:

**models.py**
{{{
from django.db import models

class Dog(models.Model):
name = models.CharField()

class Car(models.Model):
color = models.CharField()
}}}

**helpers1.py**
{{{
from models import Dog, Car #
django.core.exceptions.ImproperlyConfigured: Requested settings, but
settings are not configured.

def get_color(foo: Car) -> str:
return foo.color
}}}

How one can solve this is by introducing `settings.configure()` at the
top, giving us next attempt:

------------------------------------------------------------
**helpers_2.py**
{{{
from django.conf import settings
settings.configure()
from models import Dog, Car

def get_color(foo: Car) -> str:
return foo.color
}}}

While this works, it is just pushing the problem forward, now anyone
importing helpers2.py will have the same problem.

**application1.py**
{{{
from models import Dog #
django.core.exceptions.ImproperlyConfigured: Requested settings, but
settings are not configured.
from helpers_2 import get_color # OK

def main():
get_color(Car.objects.first())
}}}


------------------------------------------------------------

What's more problematic about placing `settings.configure()` during
imports, is that it can not accept input taken when the application
starts, for example by reading config files or argument parsers.

**application2.py**
{{{
import argparse
from django.conf import settings
from models import Dog # django.core.exceptions.ImproperlyConfigured:
Requested settings, but settings are not configured.
from helpers_2 import get_color # OK


def main():
parser = argparse.ArgumentParser()
hostname = parser.add_argument("--hostname")
args = parser.parse_args()
# Configuration based on arguments, configuration files, credential-
fetchers, and so on.
settings.configure(args.hostname, etc....)
}}}

--------------------------------------

I have also considered placing `settings.configure()` at the very top
inside `models.py` itself and accept that any argument-parsers are done in
this global scope. But this creates difficulties in unit testing, where
one need to mock the db before even importing a model, which again
transitively spreads up the chain to any code that imports something that
imports models.


---------------------------
I'm not sure what the to actually request as a solution for this. Some
ideas:

* Provide an alternative lightweight settings.configure() that loads the
plugins but doesn't connect to the database. This allows partial
configuration in the top level models, then final configuration when
application starts.
* Better integration with typing.TYPE_CHECKING, not sure how, something
that allows using models in type-annotations without having the complete
django configured maybe?
* Example projects using models in type annotations in helper-functions,
with settings configurable from external sources. To showcase best
practice of how one should structure such a Django project.
--
Ticket URL: <https://code.djangoproject.com/ticket/35406>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#35406: Using Django models in function type annotations, without dependency to
settings.configure()
-------------------------------------+-------------------------------------
Reporter: HTErik | Owner: nobody

Type: New feature | Status: closed
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution: invalid

Keywords: models, typing | Triage Stage:

| Unreviewed
Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce):

* resolution: => invalid
* status: new => closed

Comment:

Hi HTErik 👋 I recommend you look into [https://github.com/typeddjango
/django-stubs/ django-stubs] for configuring type checking with your
Django project
If you have issues setting up type annotations on your project, I
recommend asking for help on the [https://forum.djangoproject.com/ Django
forum].
I'm closing the ticket as I believe it is already possible to do this and
any further discussion around typing and Django is covered by #29299.
--
Ticket URL: <https://code.djangoproject.com/ticket/35406#comment:1>

[Django]

#35406: Using Django models in function type annotations, without dependency to
settings.configure()
-------------------------------------+-------------------------------------
Reporter: HTErik | Owner: nobody
Type: New feature | Status: closed
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution: invalid
Keywords: models, typing | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by HTErik):

Hi. Thanks for the reply.
I'm already aware of django-stubs and we are already using it for long
time with success.
The difficulty here is not getting mypy to work.

This is a problem with how Django relies deeply on global state
initialization with the `settings.configure()`, and how the order of how
you `import` models vs `configure` Django have significant differences,
that are not only confusing and unexpected to the average Python
developer, but also impossible to untangle even for the experienced.

Whenever one type-annotates a function to take any Django-model as input,
that infects the entire code base, so all code that even just imports this
function must be **declared and imported** *after* `settings.configure`
is called. Otherwise you can not even *import* the function. (Not being
able to *call* it once the application is running is totally expected)

Because of this, using django models in an components that are shared
across multiple services, that all may not be pure `manage.py`
applications, is today more or less impossible, if you at the same time
want to support type hinting your application completely.
--
Ticket URL: <https://code.djangoproject.com/ticket/35406#comment:2>