Re: [Django] #14891: use_for_related_fields=False is not honored by reverse FK or M2M related managers

[Django]

#14891: use_for_related_fields=False is not honored by reverse FK or M2M related
managers
-------------------------------------+-------------------------------------
Reporter: sdksho@… | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.2
(models, ORM) | Resolution:
Severity: Normal | Triage Stage: Design
Keywords: Manager | decision needed
ManyToManyField | Needs documentation: 0
use_for_related_fields | Patch needs improvement: 0
Has patch: 1 | UI/UX: 0
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Changes (by carljm):

* ui_ux: => 0

Comment:

#17038 was a duplicate.

--
Ticket URL: <https://code.djangoproject.com/ticket/14891#comment:7>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#14891: use_for_related_fields=False is not honored by reverse FK or M2M related
managers
-------------------------------------+-------------------------------------
Reporter: sdksho@… | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.2
(models, ORM) | Resolution:
Severity: Normal | Triage Stage: Design
Keywords: Manager | decision needed
ManyToManyField | Needs documentation: 0
use_for_related_fields | Patch needs improvement: 0
Has patch: 1 | UI/UX: 0
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------

Comment (by anonymous):

Is there any plan to fix this? This is still happening in 1.3.

--
Ticket URL: <https://code.djangoproject.com/ticket/14891#comment:8>

[Django]

#14891: use_for_related_fields=False is not honored by reverse FK or M2M related
managers
-------------------------------------+-------------------------------------
Reporter: sdksho@… | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.2
(models, ORM) | Resolution:
Severity: Normal | Triage Stage: Design
Keywords: Manager | decision needed
ManyToManyField | Needs documentation: 0
use_for_related_fields | Patch needs improvement: 0
Has patch: 1 | UI/UX: 0
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------

Comment (by ramiro):

#17746 was another duplicate.

--
Ticket URL: <https://code.djangoproject.com/ticket/14891#comment:9>

[Django]

#14891: use_for_related_fields=False is not honored by reverse FK or M2M related
managers
-------------------------------------+-------------------------------------
Reporter: sdksho@… | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.2
(models, ORM) | Resolution:
Severity: Normal | Triage Stage: Design
Keywords: Manager | decision needed
ManyToManyField | Needs documentation: 0
use_for_related_fields | Patch needs improvement: 0
Has patch: 1 | UI/UX: 0
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------

Comment (by anonymous):

Is there any plan to fix this? How do we get this bug fixed? It would be
great if someone could come up with a solution or a workaround. Thanks!

--
Ticket URL: <https://code.djangoproject.com/ticket/14891#comment:10>

[Django]

#14891: use_for_related_fields=False is not honored by reverse FK or M2M related
managers
-------------------------------------+-------------------------------------
Reporter: sdksho@… | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.2
(models, ORM) | Resolution:
Severity: Normal | Triage Stage: Design
Keywords: Manager | decision needed
ManyToManyField | Needs documentation: 0
use_for_related_fields | Patch needs improvement: 0
Has patch: 1 | UI/UX: 0
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------

Comment (by anonymous):

Please someone fix this

--
Ticket URL: <https://code.djangoproject.com/ticket/14891#comment:11>

[Django]

#14891: use_for_related_fields=False is not honored by reverse FK or M2M related
managers
-------------------------------------+-------------------------------------
Reporter: sdksho@… | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.2
(models, ORM) | Resolution:
Severity: Normal | Triage Stage: Design
Keywords: Manager | decision needed
ManyToManyField | Needs documentation: 0
use_for_related_fields | Patch needs improvement: 0
Has patch: 1 | UI/UX: 0
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------

Changes (by bmihelac):

* cc: bmihelac@… (added)

--
Ticket URL: <https://code.djangoproject.com/ticket/14891#comment:12>

[Django]

#14891: use_for_related_fields=False is not honored by reverse FK or M2M related
managers
-------------------------------------+-------------------------------------
Reporter: sdksho@… | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.2
(models, ORM) | Resolution:
Severity: Normal | Triage Stage: Design
Keywords: Manager | decision needed
ManyToManyField | Needs documentation: 0
use_for_related_fields | Patch needs improvement: 0
Has patch: 1 | UI/UX: 0
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------

Comment (by alan.meadows@…):

Please for the love of god fix this

--
Ticket URL: <https://code.djangoproject.com/ticket/14891#comment:13>

[Django]

#14891: use_for_related_fields=False is not honored by reverse FK or M2M related
managers
-------------------------------------+-------------------------------------
Reporter: sdksho@… | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.2
(models, ORM) | Resolution:
Severity: Normal | Triage Stage: Design
Keywords: Manager | decision needed
ManyToManyField | Needs documentation: 0
use_for_related_fields | Patch needs improvement: 0
Has patch: 1 | UI/UX: 0
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------

Comment (by anonymous):

Really nasty bug... reading the documentation about use_for_related_fields
'implied' that it would actually use for ManyToMany as well...

Any progress on this?

Thanks!

--
Ticket URL: <https://code.djangoproject.com/ticket/14891#comment:14>

[Django]

#14891: use_for_related_fields=False is not honored by reverse FK or M2M related
managers
-------------------------------------+-------------------------------------
Reporter: sdksho@… | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.2
(models, ORM) | Resolution:
Severity: Normal | Triage Stage: Design
Keywords: Manager | decision needed
ManyToManyField | Needs documentation: 0
use_for_related_fields | Patch needs improvement: 0
Has patch: 1 | UI/UX: 0
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------

Comment (by anonymous):

SOMEONE FIX THIS!!! PLEAAAAASEEEEE

--
Ticket URL: <https://code.djangoproject.com/ticket/14891#comment:15>

[Django]

#14891: use_for_related_fields=False is not honored by reverse FK or M2M related
managers
-------------------------------------+-------------------------------------
Reporter: sdksho@… | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.2
(models, ORM) | Resolution:
Severity: Normal | Triage Stage: Design
Keywords: Manager | decision needed
ManyToManyField | Needs documentation: 0
use_for_related_fields | Patch needs improvement: 0
Has patch: 1 | UI/UX: 0
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------

Changes (by charettes):

* cc: charette.s@… (added)

--
Ticket URL: <https://code.djangoproject.com/ticket/14891#comment:16>

[Django]

#14891: use_for_related_fields=False is not honored by reverse FK or M2M related
managers
-------------------------------------+-------------------------------------
Reporter: sdksho@… | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.2
(models, ORM) | Resolution:
Severity: Normal | Triage Stage: Design
Keywords: Manager | decision needed
ManyToManyField | Needs documentation: 0
use_for_related_fields | Patch needs improvement: 0
Has patch: 1 | UI/UX: 0
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------

Comment (by anonymous):

I refuse to eat until this is fixed!
DAY 1 of my hunger strike begins today.. hopefully there will be a
resolution soon. I am eating my last piece of bread now.

--
Ticket URL: <https://code.djangoproject.com/ticket/14891#comment:17>

[Django]

#14891: use_for_related_fields=False is not honored by reverse FK or M2M related
managers
-------------------------------------+-------------------------------------
Reporter: sdksho@… | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.2
(models, ORM) | Resolution:
Severity: Normal | Triage Stage: Design
Keywords: Manager | decision needed
ManyToManyField | Needs documentation: 0
use_for_related_fields | Patch needs improvement: 0
Has patch: 1 | UI/UX: 0
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------

Comment (by anonymous):

DAY 2 of hunger strike.

I'm beginning to feel it slowly, I was really craving for some breakfast,
but I still refuse to eat until this is resolved.

--
Ticket URL: <https://code.djangoproject.com/ticket/14891#comment:18>

[Django]

#14891: use_for_related_fields=False is not honored by reverse FK or M2M related
managers
-------------------------------------+-------------------------------------
Reporter: sdksho@… | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.2
(models, ORM) | Resolution:
Severity: Normal | Triage Stage: Design
Keywords: Manager | decision needed
ManyToManyField | Needs documentation: 0
use_for_related_fields | Patch needs improvement: 0
Has patch: 1 | UI/UX: 0
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------

Comment (by akaariai):

One suggestion to move this forward: always use _default_manager,
deprecate use_for_related_fields, add ability to use per-relation
managers. Maybe with:
{{{
class SomeModel(models.Model):
somemanager = MyManager()
m2m = models.ManyToManyField(RelatedObject,
manager=RelatedObject.somemanager, reversemanager=somemanager)
}}}

I must admit I haven't done enough research yet, so the above suggestion
might be missing something crucial...

It seems just fixing this issue to match our docs would cause problems at
least for some users. So, even if we want to do that, then we would likely
want to invent a new flag behaving as the currently documented
use_for_related_fields and deprecate the old one so that users have time
to adapt to the change in behaviour.

--
Ticket URL: <https://code.djangoproject.com/ticket/14891#comment:19>

[Django]

#14891: use_for_related_fields=False is not honored by reverse FK or M2M related
managers
-------------------------------------+-------------------------------------
Reporter: sdksho@… | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.2
(models, ORM) | Resolution:
Severity: Normal | Triage Stage: Design
Keywords: Manager | decision needed
ManyToManyField | Needs documentation: 0
use_for_related_fields | Patch needs improvement: 0
Has patch: 1 | UI/UX: 0
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------

Comment (by russellm):

This is an open source project. If you truly feel that this is the single
greatest bug facing Django, and can't imagine how we've let it linger for
so long, **YOU CAN FIX IT YOURSELF**. You can fork the repo, make the
necessary change, and use that fork in your own work. Using an anonymous
account to howl at the moon about how hard done by you are because nobody
else will volunteer to fix your problem for free is neither helpful, nor
endearing.

--
Ticket URL: <https://code.djangoproject.com/ticket/14891#comment:20>

[Django]

#14891: use_for_related_fields=False is not honored by reverse FK or M2M related
managers
-------------------------------------+-------------------------------------
Reporter: sdksho@… | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.2
(models, ORM) | Resolution:

Severity: Normal | Triage Stage: Accepted
Keywords: Manager | Needs documentation: 0
ManyToManyField | Patch needs improvement: 0
use_for_related_fields | UI/UX: 0
Has patch: 1 |

Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------

Changes (by Alex):

* stage: Design decision needed => Accepted

Comment:

Marking as accepted, I'm not sure what the fix will be: the proposed
change is fairly backwards incompatible (but maybe that's ok), at a
minimum the docs are wrong.

--
Ticket URL: <https://code.djangoproject.com/ticket/14891#comment:21>

[Django]

#14891: use_for_related_fields=False is not honored by reverse FK or M2M related
managers
-------------------------------------+-------------------------------------
Reporter: sdksho@… | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.2
(models, ORM) | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: Manager | Needs documentation: 0
ManyToManyField | Patch needs improvement: 0
use_for_related_fields | UI/UX: 0
Has patch: 1 |
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------

Comment (by HM):

So, is there a workaround? I very much need the backwards manager to have
the same extra methods as the forwards manager.

--
Ticket URL: <https://code.djangoproject.com/ticket/14891#comment:22>

[Django]

#14891: use_for_related_fields=False is not honored by reverse FK or M2M related
managers
-------------------------------------+-------------------------------------
Reporter: sdksho@… | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.2
(models, ORM) | Resolution:
Severity: Normal | Triage Stage: Accepted

Keywords: Manager | Needs documentation: 1
ManyToManyField | Patch needs improvement: 1

use_for_related_fields | UI/UX: 0
Has patch: 1 |
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------

Changes (by palkeo):

* needs_better_patch: 0 => 1
* needs_docs: 0 => 1

Comment:

Hi !

I'm trying to add the support for per-relation custom managers.

I just implemented it to the ForeignKey relations, and it seems to works
fine.
It is also backwards compatible, and I think it may be possible to
deprecate progressively the « use_for_related_fields » option.

Here is the branch I am working on :
https://github.com/makinacorpus/django/tree/custom-managers

I'm waiting for some feedback, and if you approve my current patch, I will
add that support for the one-to-one and many-to-many fields.

--
Ticket URL: <https://code.djangoproject.com/ticket/14891#comment:23>

[Django]

#14891: use_for_related_fields=False is not honored by reverse FK or M2M related
managers
-------------------------------------+-------------------------------------
Reporter: sdksho@… | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.2
(models, ORM) | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: Manager | Needs documentation: 1
ManyToManyField | Patch needs improvement: 1
use_for_related_fields | UI/UX: 0
Has patch: 1 |
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Changes (by palkeo):

* cc: palkeo (added)

--
Ticket URL: <https://code.djangoproject.com/ticket/14891#comment:24>

[Django]

#14891: use_for_related_fields=False is not honored by reverse FK or M2M related
managers
-------------------------------------+-------------------------------------
Reporter: sdksho@… | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.2
(models, ORM) | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: Manager | Needs documentation: 1
ManyToManyField | Patch needs improvement: 1
use_for_related_fields | UI/UX: 0
Has patch: 1 |
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------

Changes (by loic84):

* cc: loic@… (added)

--
Ticket URL: <https://code.djangoproject.com/ticket/14891#comment:25>

[Django]

#14891: use_for_related_fields=False is not honored by reverse FK or M2M related
managers
-------------------------------------+-------------------------------------
Reporter: sdksho@… | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.2
(models, ORM) | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: Manager | Needs documentation: 1
ManyToManyField | Patch needs improvement: 1
use_for_related_fields | UI/UX: 0
Has patch: 1 |
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------

Comment (by jordonwii):

At the very least, there should be an update to the docs (particularly
[https://docs.djangoproject.com/en/dev/topics/db/managers/#manager-types
this page]) indicating that use_for_related_fields does not work.

A decision should be made as to whether this is going to get fixed, if
not. Django in practice is currently in direct contradiction to its
documentation.

--
Ticket URL: <https://code.djangoproject.com/ticket/14891#comment:26>

[Django]

#14891: use_for_related_fields=False is not honored by reverse FK or M2M related
managers
-------------------------------------+-------------------------------------
Reporter: sdksho@… | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.2
(models, ORM) |

Severity: Normal | Resolution:
Keywords: Manager | Triage Stage: Accepted
ManyToManyField |
use_for_related_fields |
Has patch: 1 | Needs documentation: 1
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by jarshwah):

Here's an idea to close out this ticket. Free to anyone that's hurt by
this issue and would like to attempt to fix it.

1) deprecate use_for_related_fields flag on manager
2) remove the documentation for use_for_related_fields
3) implement 'manager="manager_name"' and 'reverse_manager="manager_name"'
on related fields (ware migrations)
4) document the new keyword arguments to ForeignKey/M2MField/OneToOneField

This will give you full control over which managers are used at all times,
while still allowing explicit override: `model.foo_set(manager='other')`

This will be more verbose, but it'll be correct.

{{{
class MyModel(models.Model):
objects = MyDefaultManager()
related = MyRelatedManager()
foo = models.ForeignKey(Foo, manager='objects',
related_manager='related')
}}}

I don't think putting a big warning in the docs "this does not work" is
sufficient, especially when there are definitely avenues for fixing the
underlying problem. It's the lazy way out. Doing nothing is also bad. But
until someone steps up to actually do the work, then it's still going to
be an issue.

Now, does my proposal above have any significant drawbacks? It's a
combination of previous talk and my discussions with Loic on IRC.

--
Ticket URL: <https://code.djangoproject.com/ticket/14891#comment:27>

[Django]

#14891: use_for_related_fields=False is not honored by reverse FK or M2M related
managers
-------------------------------------+-------------------------------------
Reporter: sdksho@… | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: Manager | Triage Stage: Accepted
ManyToManyField |
use_for_related_fields |
Has patch: 1 | Needs documentation: 1
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by loic):

I like the `(manager='objects', related_manager='related')` API since it
allows fine grained control over the relation manager as expected by the
relational field. It's in the same spirit as
https://docs.djangoproject.com/en/1.9/topics/db/queries/#using-a-custom-
reverse-manager which allows controlling the reverse manager per query per
relational field.

But I also see merit in allowing the model to say which manager should be
used by default by relations. My work on #20932 can easily be extended to
support a `related_manager_name` meta.

It would work as follows:
{{{
class Model(models.Model):
objects = models.Manager()
custom_manager = CustomManager()

class Meta:
related_manager_name = 'custom_manager'
}}}

PR https://github.com/django/django/pull/6175 shows a working
implementation + deprecation.

--
Ticket URL: <https://code.djangoproject.com/ticket/14891#comment:28>

[Django]

#14891: use_for_related_fields=False is not honored by reverse FK or M2M related
managers
-------------------------------------+-------------------------------------
Reporter: sdksho@… | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: Manager | Triage Stage: Accepted
ManyToManyField |
use_for_related_fields |
Has patch: 1 | Needs documentation: 1
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by loic):

[https://github.com/django/django/pull/6175#issuecomment-219451336 PR6175]
deprecates `use_for_related_fields`. "to-one" relations go through
`_base_manager` which can be specified with the `base_manager_name` model
option and "to-many" relations go through `_default_manager` which can be
specified with the `default_manager_name` model option.

We tried to add a blanket "use for every relations" manager in the form of
`related_manager_name` but
[https://github.com/django/django/pull/6175#issuecomment-210315119
eventually decided against]. For finer control over relation managers, I
think comment:27 is the way to go but this is a new feature which deserves
a ticket of its own.

--
Ticket URL: <https://code.djangoproject.com/ticket/14891#comment:29>

[Django]

#14891: use_for_related_fields=False is not honored by reverse FK or M2M related
managers
-------------------------------------+-------------------------------------
Reporter: sdksho@… | Owner: nobody

Type: Bug | Status: closed

Component: Database layer | Version: 1.2
(models, ORM) |

Severity: Normal | Resolution: fixed

Keywords: Manager | Triage Stage: Accepted
ManyToManyField |
use_for_related_fields |
Has patch: 1 | Needs documentation: 1
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Loïc Bistuer <loic.bistuer@…>):

* status: new => closed
* resolution: => fixed

Comment:

In [changeset:"ed0ff913c648b16c4471fc9a9441d1ee48cb5420" ed0ff91]:
{{{
#!CommitTicketReference repository=""
revision="ed0ff913c648b16c4471fc9a9441d1ee48cb5420"
Fixed #10506, #13793, #14891, #25201 -- Introduced new APIs to specify
models' default and base managers.

This deprecates use_for_related_fields.

Old API:

class CustomManager(models.Model):
use_for_related_fields = True

class Model(models.Model):
custom_manager = CustomManager()

New API:

class Model(models.Model):
custom_manager = CustomManager()

class Meta:
base_manager_name = 'custom_manager'

Refs #20932, #25897.

Thanks Carl Meyer for the guidance throughout this work.
Thanks Tim Graham for writing the docs.
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/14891#comment:30>