#22268 - values_list() on a ManyToManyField returns

[anubhav joshi]

There has been quite a discussion on the ticket regarding this
Refs. : https://code.djangoproject.com/ticket/22268
I think we must discuss and decide what should be the solution we should be working towards.
Until now there have been two strategies:

1.) The query generated is correct therefore,
We just add pk to the query and then when final result is obtained we can club the dicts/tuples having same pk and remove pk.
And also remove the ones where the pk is None (due to the LEFT OUTER JOIN).

2.) Instead of retrieving the M2M or reverse foreign key values in the same SQL query, use a clever combination of:

• prefetch_related()
• subclassing dict and tuple

So values() and values_list return these subclassed dicts and tuples, which lazily construct and return the relevant M2M or reverse foreign key elements when they are accessed.In the end, the subclassed dict and tuple would work somewhat like the Django model, except that you access things in the form of dictionaries and tuples, and you limit the elements that can appear in them.

[Russell Keith-Magee]

You're missing an important third option here: do nothing, and document the fact that "values" over an m2m is an inherently nonsense operation in an ORM context. 

values() and values_list() are both intended as optimisations for a specific use case - retrieval of subsets of data without the overhead of creating a model instance. This metaphor completely falls apart when dealing with m2m relations, because the the "one row, one object" metaphor that underpins most of the ORM falls apart. 

While we could search for a deeper meaning for queries of this type, and restructure the internal SQL to account for this, there's another issue to consider. We're also talking about a part of the query API that has existed since at least the query set refactor, and possibly as far back as magic removal. That means there's between 5 and 7 years of legacy here, in which time nobody else has raised a ticket about this issue. It's also reasonable to assume that there will be a non-trivial number of people depending on the API as currently implemented. 

So, I'm inclined to say this is a known wart in the ORM, born of the leaky abstraction of pushing relational data into an object-based representation. Document the limitation, and move on.

Yours,

Russ Magee %-)

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/4c1b12a4-822a-4582-8d57-3a8f04f62a58%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[anubhav joshi]

values() and values_list() are both intended as optimisations for a specific use case - retrieval of subsets of data without the overhead of creating a model instance. This metaphor completely falls apart when dealing with m2m relations, because the the "one row, one object" metaphor that underpins most of the ORM falls apart. 

I agree with you on this. "one-row,one-object" cannot be implemented here.
 

[Russell Keith-Magee]

On Fri, Mar 28, 2014 at 2:29 PM, anubhav joshi <anubh...@gmail.com> wrote:

values() and values_list() are both intended as optimisations for a specific use case - retrieval of subsets of data without the overhead of creating a model instance. This metaphor completely falls apart when dealing with m2m relations, because the the "one row, one object" metaphor that underpins most of the ORM falls apart. 

I agree with you on this. "one-row,one-object" cannot be implemented here.

Well, it *can* be implemented - just not in a way that is both computationally efficient *and* backwards compatible.

Yours,

Russ Magee %-)