Make `raw_id_fields` the default functionality in the admin

[Yo-Yo Ma]

I have yet to ever come across a situation where the default <select> field is more useful than the raw ID field, pertaining foreign key fields in the admin.

I have, however, personally witnessed a major publishing company bring their production app servers to a halt (out of memory) due to Django attempting to generate 2.5 million <option value="..."> tags for some dozen admins that were all refreshing an admin changeview, wondering why it was taking so long to load.

Another thing worth noting is that when the <select> is most useful (when there are very few records to select) also happens to be when the raw ID field is most easily used (since the selection changelist only contains the same very few records). IOW, the raw ID field's usefulness is universal, working well with just a few records, and also working well (due to search / sort) when there are many records.

Nary a transition would be required, since the `raw_id_fields` could simply be ignored.

A new `select_fields = []` could be added for those who wish to easily  use the old functionality.

Is there any reason why this couldn't or shouldn't be done?

[Yo-Yo Ma]

One last note: M2M could also benefit from this, and could continue to look for `filter_horizontal` and not use a raw IDs field in those cases.

[Collin Anderson]

raw_id_fields can be a bit confusing to people. Have you tried the new autocomplete_fields? It's almost as easy to use as a select field, and it should help solve the problem. However, it's not as easy to make the default (because it requires specifying search_fields on the related model)

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/e0262150-ec7e-487d-8fb1-b64f8df2e7a3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Aymeric Augustin]

Hello,

The default <select> widget is fine for configuration tables with no more than a few dozen values.

In my experience, I rarely encountered use cases for editing foreign keys to very large tables in the admin.

Readonly or autocomplete are good choices.

Best regards,

-- 

Aymeric.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/76e35c9d-b62f-498f-a382-6f12f74ebd98%40googlegroups.com.

[Ivan Anishchuk]

Well, I did personally encounter this issue more than a couple times on several projects. In most cases simply switching to raw id or read only where required works fine however it does add additional maintenance overhead for projects that started small and at some point database grows and some admin pages stop working. I even had to write some custom code once when a client wanted to keep the default select interface but there were too many of those selects with the same options for some objects (different number of inlines IIRC), I basically added some js code to make sure the generation and transmission of the list only takes place once. Ah, another part of the problem was that by default every single object is iterated and passed to force_text (I think) and that was costly and there wasn't an obvious way to use, like, a combination of model fields instead. Maybe some internals got changed recently and it got better though, I didn't review these parts of django code in a while.

That being said, I do see a value in using the default select sometimes (aforementioned small configuration tables are a good example) and I did see a lot of frustration when people see otherwise nice interface failing miserably just because some new user accounts or other data was added. I'm not sure forcing raw id is the right way to handle this but doing something that works by default (i.e. without you having to write a custom admin class or even form) regardless of table size would be a good idea. How about some field that intelligently guesses the table size and looks either like default select or like raw id field depending on that? A COUNT(*) query is usually much less costly than fetching the whole table and doing things with it. Additional (and simple enough) optimization could be done by making sure options for the same queryset on multiple fields or formset forms are only generated once, maybe we could cache them somewhere in the modelform/formset objects?

Also, although it might be obvious and even documented somewhere that __str__() method shouldn't perform additional db queries if you expect to use string coercion on a bunch of objects (actually I don't know if it is documented at all, I just hope so, correct me if I'm wrong), maybe some simple way to override string coercion method for the options and in similar places where a bunch of objects has to be converted to a list of strings could be a good idea (possibly on model/queryset level, some attribute or method that modelform would pick up and use to make lighter values() query instead of the default fetching, instantiation, coercion). I'm not sure what particular solution would be the best here though or I would've proposed one already, but it definitely shouldn't however be complicated or require much customization (if you need something complicated and customized you can always customize forms and fields and it stops being a problem), it should be something simple and effective for when you prefer to use the default classes wherever possible.

Ivan.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/BF6BF10C-BFCE-47FE-A492-EF5B671E1039%40polytechnique.org.

[Aymeric Augustin]

On 6 Nov 2018, at 02:47, 'Ivan Anishchuk' via Django developers (Contributions to Django itself) <django-d...@googlegroups.com> wrote:

How about some field that intelligently guesses the table size and looks either like default select or like raw id field depending on that?

Yes, that would be interesting.

I implemented a manual version of this in a project:

- I ran a script to count the number of instances of each model and put those with more than 100 or 1000 instances in a list

- I wrote a custom ModelAdmin class that automatically added ForeignKeys to models in that list to raw_id_fields

Best regards,

-- 

Aymeric.