Re: Improving ForeignKeyRawIdWidget (raw_id_fields in admin)

[Wim Feijen]

Hi Richard,

Thanks for your rationale. Sounds like a good feature to me. 

Probably the best way to proceed is to make a ticket at: https://code.djangoproject.com/newticket , attach a patch or pull request and then we go forward.

Best regards,

Wim

On Wednesday, 12 June 2013 19:33:31 UTC+2, ric...@richard.ward.name wrote:

I think that the usability of ForeignKeyRawIdWidget could be vastly improved if the representation part of the widget (the object name, in bold) were to be updated when a new object gets chosen. I think this could be done relatively easily with a small change introducing little extra complexity.

At present the original representation remains, which is confusing to users and could lead to mistakes being made.

I think this can't be done simply by a 3rd party app without some seriously dirty monkey patching. The only way I can think of I to do it without monkey patching would be to make ajax requests to get the new representation of the chosen object - substantially more awkward both for the developer and the person installing the app.

I appreciate some people will have used ForeignKeyRawIdWidget outside admin and used a custom dismissRelatedLookupPopup function so ideally any change would not break their code. This means that the function signature of dismissRelatedLookupPopup should remain the same (unfortunately, as this would be the most obvious and easiest place to include representation).

My proposal would be for the string representation of the object to be included somewhere in the popup - for example as an html attribute called data-object-representation on the anchor with the onclick handler. If this anchor was also given an id (say choose-object-X where X is the object's pk) then it would be easily accessible to the dismissRelatedLookupPopup, which could extract it and update the representation on the calling page. The <strong> elem containing the representation might also benefit from having a unique id as RelatedObjectLookups.js seems to be written so as not to require jQuery.

Hope that all makes sense. Naturally I'm prepared to write a patch for this assuming it doesn't meet with disapproval.

Thanks,

Richard

P.S. Hope I'm putting this in the right place, and I couldn't find anything existing on the django issues list, and i got the impression that this list was the place to start with such things.

[Simon Meers]

On 13 June 2013 03:33, <ric...@richard.ward.name> wrote:

I think that the usability of ForeignKeyRawIdWidget could be vastly improved if the representation part of the widget (the object name, in bold) were to be updated when a new object gets chosen. I think this could be done relatively easily with a small change introducing little extra complexity.

At present the original representation remains, which is confusing to users and could lead to mistakes being made.

I think this can't be done simply by a 3rd party app without some seriously dirty monkey patching. The only way I can think of I to do it without monkey patching would be to make ajax requests to get the new representation of the chosen object - substantially more awkward both for the developer and the person installing the app.

I appreciate some people will have used ForeignKeyRawIdWidget outside admin and used a custom dismissRelatedLookupPopup function so ideally any change would not break their code. This means that the function signature of dismissRelatedLookupPopup should remain the same (unfortunately, as this would be the most obvious and easiest place to include representation).

My proposal would be for the string representation of the object to be included somewhere in the popup - for example as an html attribute called data-object-representation on the anchor with the onclick handler. If this anchor was also given an id (say choose-object-X where X is the object's pk) then it would be easily accessible to the dismissRelatedLookupPopup, which could extract it and update the representation on the calling page. The <strong> elem containing the representation might also benefit from having a unique id as RelatedObjectLookups.js seems to be written so as not to require jQuery.

Hope that all makes sense. Naturally I'm prepared to write a patch for this assuming it doesn't meet with disapproval.

Thanks,

Richard

P.S. Hope I'm putting this in the right place, and I couldn't find anything existing on the django issues list, and i got the impression that this list was the place to start with such things.

 

Hi Richard,

I think this is a reasonable place to have this discussion, though a there is also a related ticket [1] that I've just found.

The handling of raw IDs, particularly in a comma-separated list for many-to-many relationships, is arguably the biggest user-facing "wart" in the current admin implementation. The sheer number of available related objects often makes listing the full set in a dropdown or other widget unfeasible, forcing developers to resort to the raw ID mechanism to a) improve efficiency in terms of database-querying and response/DOM-size, and b) make the selection interface manageable and allow searching/filtering/pagination/etc. Providing a textual representation of the object as per the standard widgets makes sense here.

Your proposal for the textual representations "stowing away" in data attributes sounds like a reasonable solution for maintaining backwards-compatibility and utilising the existing popup mechanism. I wonder however whether this could also be solved by passing an additional parameter in the popup-triggering request which indicates that the listening widget is expecting a response which includes the text as well as the ID? I haven't yet had time to review the proposed solutions for [1], but there may be some interesting alternatives there. Also, can additional parameters be added to javascript functions and be safely ignored by unsuspecting old code? Perhaps the signature compatibility is a non-issue.

I think we should retain the RawIdWidgets and raw_id_fields for both backwards-compatibility and also situations in which handling the raw IDs is actually preferable. It may also be nice to consider the situation for generic foreign keys [2] while developing the solution.

Simon

PS. In my personal "scratch-pad" package/laboratory for developing experimental generic utilities for Django, I've drafted a "Cooked ID" implementation [3,4]. It works with the existing Django admin mechanisms using an AJAX mechanism similar to what you describe above. It works for both ForeignKeyRawIdWidget and ManyToManyRawIdWidget, and looks something like this [5].

[1] https://code.djangoproject.com/ticket/7028

[2] https://code.djangoproject.com/ticket/9976

[3] https://bitbucket.org/drmeers/django-generic/commits/eb0d519

[4] https://bitbucket.org/drmeers/django-generic/commits/c087cae

[5] http://imgur.com/hvoJoEG

[Stan]

Hi Richard,

As the last maintainer of the patch of the ticket #7028, I am happy to see you bring that one on the table!

What the patch does is to add the unicode representation of the related object next to the input field (the unicode value is sent back by the pop up.)

It also works for the ManyToMany fields.

It has not been merged in the trunk so far, a lack of manpower and tests I presume (there are some, but nothing that guarantee the good Javascript behaviour on different web browsers).

I also have some code that do the Ajax autocompletion for in-house projects and started to think about how to improve #7028 with an auto-complete feature (Ajax or not, but a cached solution has a big performance penalty imo.)

There is another ticket on the subject, #14370 but I don't really like the solution, sorry :-)

The ideas:

- Admin should benefit of an autocomplete solution beside raw_id_fields without any external dependancies ;

- That autocomplete functionality should be officially reusable outside the admin ;

- This should works without having to register() stuff or declare settings.AUTOCOMP_MEDIA_DIR, at least for the default behaviour.

- The raw_id_fields we know should still works the same, especially for non-javascript users.

I know and understand that core-devs are going to scream but a contrib.autocomplete app seems to be a good option, isn't?

And admin should auto-magically (or not) turn raw_id_fields in autocomplete ones if the app is in INSTALLED_APPS.

Cheers,

--

Stan

[Simon Meers]

On 13 June 2013 07:41, Simon Meers <drm...@gmail.com> wrote:

On 13 June 2013 03:33, <ric...@richard.ward.name> wrote:

I think that the usability of ForeignKeyRawIdWidget could be vastly improved if the representation part of the widget (the object name, in bold) were to be updated when a new object gets chosen. I think this could be done relatively easily with a small change introducing little extra complexity.

The handling of raw IDs, particularly in a comma-separated list for many-to-many relationships, is arguably the biggest user-facing "wart" in the current admin implementation. The sheer number of available related objects often makes listing the full set in a dropdown or other widget unfeasible, forcing developers to resort to the raw ID mechanism to a) improve efficiency in terms of database-querying and response/DOM-size, and b) make the selection interface manageable and allow searching/filtering/pagination/etc. Providing a textual representation of the object as per the standard widgets makes sense here.

Julien Phalip has recently put a new patch together for this -- see https://code.djangoproject.com/ticket/7028#comment:74 -- reviews would be greatly appreciated.

[Stan]

Hi DrMeers,

As I commented in the ticket #7028, this new patch needs some work. I tried to figure out a solution for inlines but there are too much ramifications for me.
Anyway, I think a better approach to the problem is a reusable app (usable inside and outside the admin). Into contrib or not.

There are some apps around : https://www.djangopackages.com/grids/g/auto-complete/
I have tested django-autocomplete-light because it got a lot of contributions and claimed to be production ready. Unfortunately it was broken when I tested it against one of my project (infamous UnicodeEncodeDecodeError exception). I should have tried to contribute to that very project but I choose to not. The main reasons being:

1. I am in a hurry for a #7028 remplacement.
2. I need something reliable.
3. I want to pip install in production, not to clone a fork of a git repo and python setup.py install etc.

So I started a really light app - django-yaaac - you can find on Github and Pypi.
I can't say it is production-ready / stable, but it is ready for my productions !

Ticket #7028 should be closed in my opinion.

Cheers,

Stanislas.