[Django] #30849: Enabling hooking into the autodetector from a third-party app.

Django

unread,

Oct 7, 2019, 7:21:14 AM10/7/19

to django-...@googlegroups.com

#30849: Enabling hooking into the autodetector from a third-party app.
-------------------------------------+-------------------------------------
Reporter: Emil | Owner: nobody
'Skeen' Madsen |
Type: New | Status: new
feature |
Component: | Version: 2.2
Migrations |
Severity: Normal | Keywords: automigrator
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
Idea and motivation presented on the
[https://groups.google.com/forum/#!searchin/django-
developers/sov...@gmail.com|sort:date/django-
developers/kqWJ2WsMW6w/TRakYLNRBQAJ mailinglist]

Summed up, what I suggest is:
- Creating a new signal fired from within the automigrator to allow third-
party apps to generate migrationfiles with their own custom operations
within them.

One compelling use-case for this, is automatically generating database
constraints based upon querysets (see the mailinglist post for details).

--
Ticket URL: <https://code.djangoproject.com/ticket/30849>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

Django

unread,

Oct 7, 2019, 7:21:40 AM10/7/19

to django-...@googlegroups.com

#30849: Enabling hooking into the autodetector from a third-party app.
-------------------------------------+-------------------------------------

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Emil 'Skeen' Madsen):

* owner: nobody => Emil 'Skeen' Madsen
* status: new => assigned

--
Ticket URL: <https://code.djangoproject.com/ticket/30849#comment:1>

Django

unread,

Oct 7, 2019, 7:45:28 AM10/7/19

to django-...@googlegroups.com

#30849: Enabling hooking into the autodetector from a third-party app.
-------------------------------------+-------------------------------------

Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Description changed by Emil 'Skeen' Madsen:

Old description:

> Idea and motivation presented on the
> [https://groups.google.com/forum/#!searchin/django-
> developers/sov...@gmail.com|sort:date/django-
> developers/kqWJ2WsMW6w/TRakYLNRBQAJ mailinglist]
>
> Summed up, what I suggest is:
> - Creating a new signal fired from within the automigrator to allow

> third-party apps to generate migrationfiles with their own custom

> operations within them.
>
> One compelling use-case for this, is automatically generating database
> constraints based upon querysets (see the mailinglist post for details).

New description:

Idea and motivation presented on the
[https://groups.google.com/forum/#!searchin/django-
developers/sov...@gmail.com|sort:date/django-
developers/kqWJ2WsMW6w/TRakYLNRBQAJ mailinglist]

Summed up, what I suggest is:

- Creating a new signal fired from within the autodetector to allow third-

party apps to generate migrationfiles with their own custom operations
within them.

One compelling use-case for this, is automatically generating database
constraints based upon querysets (see the mailinglist post for details).

--

--
Ticket URL: <https://code.djangoproject.com/ticket/30849#comment:2>

Django

unread,

Oct 8, 2019, 7:22:47 AM10/8/19

to django-...@googlegroups.com

#30849: Enabling hooking into the autodetector from a third-party app.
-------------------------------------+-------------------------------------

Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by felixxm):

* cc: Simon Charette (added)

--
Ticket URL: <https://code.djangoproject.com/ticket/30849#comment:3>

Django

unread,

Oct 8, 2019, 7:52:27 AM10/8/19

to django-...@googlegroups.com

#30849: Enabling hooking into the autodetector from a third-party app.
-------------------------------------+-------------------------------------

Reporter: Emil 'Skeen' Madsen | Owner: Emil
| 'Skeen' Madsen

Type: New feature | Status: closed
Component: Migrations | Version: 2.2
Severity: Normal | Resolution: wontfix

Keywords: automigrator | Triage Stage:
| Unreviewed

Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Simon Charette):

* status: assigned => closed
* resolution: => wontfix

Comment:

Hello Emil, thanks for posting to the mailing list before submitting a new
feature request.

I can't say this feature wouldn't be useful for other purposes but your
`Meta.constraint_triggers` motivation could likely be implemented with the
[https://docs.djangoproject.com/en/2.2/ref/models/constraints/ new
constraints API] by subclassing `Constraint` to return custom SQL for
trigger/function creation and removal.

I think it should be preferable that you try using this interface and
report any missing feature first than allowing third party apps to hook
into the non-formalized and already complex auto-detector logic. I'll
close the ticket for the aforementioned reasons but feel free to gather
more feedback on your ML thread to demonstrate wide interest in this
feature.

--
Ticket URL: <https://code.djangoproject.com/ticket/30849#comment:4>

Django

unread,

Oct 10, 2019, 5:04:38 AM10/10/19

to django-...@googlegroups.com

#30849: Enabling hooking into the autodetector from a third-party app.
-------------------------------------+-------------------------------------

Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Emil 'Skeen' Madsen):

Hello Simon,

I have been able to implement my library via the new constraints API.
Thank you for the hint.
The library now lives at:
* https://pypi.org/project/django-queryset-constraint/
* https://github.com/magenta-aps/django_queryset_constraint

----

I could conceivably similarly utilize the new constraints API to achieve
the side goals I listed, such as:
* Automatically enabling database extensions, when fields which require
these extensions are utilized
* Add comments to database tables

However I believe this would be overloading the constraints API for
something it was never designed for.
So, do you have any ideas, as to how I would achieve these side goals
without hooking into the autodetector and without utilizing the constraint
API?
If not, that is perfectly okay, these are, as stated, merely side goals.

--
Ticket URL: <https://code.djangoproject.com/ticket/30849#comment:5>

Django

unread,

Oct 10, 2019, 8:59:10 AM10/10/19

to django-...@googlegroups.com

#30849: Enabling hooking into the autodetector from a third-party app.
-------------------------------------+-------------------------------------

Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Simon Charette):

> I have been able to implement my library via the new constraints API.
Thank you

You're welcome, thanks for giving it a try. It looks like this will likely
be useful for a few people.

If I can give you a suggestion regarding the implementation I'd avoid the
`self.queryset == other.queryset` in `__eq__` as this will likely result
in two queries during comparison which is quite frequent during auto-
detection and operation reducing. Instead I'd compare each of the string
representation of queries which is a documented API. That is
`str(self.queryset.query) == str(other.queryset.query)`. I submitted
[https://github.com/magenta-aps/django_queryset_constraint/pull/1 a PR]
for it.

> I could conceivably similarly utilize the new constraints API to
achieve the side goals I listed, such as:
>

> - Automatically enabling database extensions, when fields which require
these extensions are utilized
> - Add comments to database tables

There's [https://github.com/django/django/pull/10540 an ongoing effort] to
add `pre` and `post_operation` signals (#29843) for this purpose but in
the team you can rely on `pre_migrate` to inject `CreateExtension` into
the `plan` if a `CreateModel`, `AddField`, or `AlterField` has a field
requiring an extension. Ditto with table comments.

--
Ticket URL: <https://code.djangoproject.com/ticket/30849#comment:6>

Django

unread,

Oct 10, 2019, 10:09:10 AM10/10/19

to django-...@googlegroups.com

#30849: Enabling hooking into the autodetector from a third-party app.
-------------------------------------+-------------------------------------

Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Emil 'Skeen' Madsen):

Hi,

Thanks for the feedback, the library is still in a very early stage, and I
really dislike how the M object is implemented.
If you happen to have a good idea for serializing a queryset, that would
be lovely.