FK Autocomplete Widget [GSoC '09 Admin UI Improvements]

[Zain Memon]

Since GSoC officially started a couple of days ago, I've implemented a rough draft of an autocomplete widget for foreign keys in the admin. 

You can see a screenshot and a short write-up of the functionality on my blog here: http://inzain.net/blog/2009/05/a-foreign-key-autocomplete-widget-for-django/

Also, I've set up a GitHub repo for my work until official svn branches are handed out. You can follow my progress here: http://github.com/zain/django/tree/master

Zain

-----------------------------------

My GSoC proposal: http://inzain.net/blog/2009/05/django-gsoc-09-admin-ui-improvements/

[Margie]

I think that foreign key selection in the admin interface is a great
area to be putting cycles into. I've played around with Jannis
Leidel's foreign key autocomlete widget and am familiar with that, I
think you are modeling your stuff after that, is that right? While I
think that is a cool widget, I find that the real problem with foreign
key selection in the admin is that there is no easy way to prune down
the set of choices. If one has a lot of choices, autocomplete is not
really a help, especially if you don't know the choice names. For
example, in the app I am developing (a task management system), I have
tasks and owners of tasks. The owner of the task can be user in the
system, and there are 6000+ users.

What I would really like to see is foreign key selection widget where,
when the end-user clicks on the foreign key selection dropdown, it
sends an ajax request back to the server to execute a app-developer-
specified method that returns a queryset, and the results of this
queryset would then get placed into the select options for the foreign
key selection drop down.

I have proof-of-concept code that I have implemented and would be
happy to provide the code I have. While developing this code, I
wrestled extensively with a couple problems. One is described in a
bug I posted on Friday: http://code.djangoproject.com/ticket/11183.
The render function for my widget initially renders just a single
choice for the selections - the choice that is the currently selected
owner. The reason I override the default queryset from inside render
() is that the default queryset in my case would the 6000+ users. It
seemed silly to send 6000+ selection options when they would get
replaced on the user's click. Keep in mind this widget also gets used
in the changelist view, so the 6000 option selections would be sent
for each object in the changelist view (and there can be many).
Anyway, when overriding the selection in render(), I encountered
problems in the core django datastructurse that made this not work -
this is the bub described in 11183.

The second problem I wrestled with was how to pass the id of the
currently displayed object to the my get_owner_queryset() callback
method. IE, from the changelist view, when the user tries to change
the owner of task 'n', how to get my new widget's jquery/javascript
to send 'n' as an argument to the callback method. I did manage to do
this in javascript by finding the id of my owner field (ie, id_form-3-
owner) and replacing "owner" with "id", and then sending the value of
that field. This of course depends on there being a field that is
id_form-3-id whose value is the id of the object being displayed. I
would like to find a better way to do this, but I don't know if there
is one.

Anyway, perhaps some will say that I am trying to use the admin
interface for more than it was intended for. But I have been
impressed with how much the admin can do for me, and I would really
like to use it because as it gets extended, I feel I can really
benefit from those extensions. However, I find the inability to give
the user a viable way to choose foreign keys to be a real show
stopper. In the same vein, I think that there should be a way to
configure the behavior of the little green '+', (showAddAnother
popup). To use my task/owner example again - when the user is
choosing a task owner, I wouldn't ever want the user to be able to
add another user. I want them to be able to choose different owners
without being able to add new users.

Of course I can do all of this myself - and I have been working
(slowly) towards that end. But as a relatively experienced user, my .
02 regarding the admin interface and foreign keys is that it would be
a huge benefit to:
1) provide an interface that allows the app-developer to prune down
the choices for a foreign key via an ajax callback to an app-developer
specified method.
2) give the app-developer more control of the showAddAnotherPopup.


Is this the right place for this discussion? If not, what is?

Margie

On May 25, 3:11 am, Zain Memon <z...@inzain.net> wrote:
> Since GSoC officially started a couple of days ago, I've implemented a rough
> draft of an autocomplete widget for foreign keys in the admin.
>
> You can see a screenshot and a short write-up of the functionality on my

> blog here:http://inzain.net/blog/2009/05/a-foreign-key-autocomplete-widget-for-...

[Justin Lilly]

Not entirely sure if this is what you were referring to, Margie, but
you can limit a the queryset used for an admin dropdown (FK). Code
for this looks something like:

# defined in the ModelAdmin for this class.
def formfield_for_foreignkey(self, db_field, request=None, **kwargs):
field = super(HomepageAdmin,
self).formfield_for_foreignkey(db_field, request=None, **kwargs)
if db_field.name == 'promo_story':
field.queryset =
Article.objects.filter(id__in=Article.objects.all()[:100])
return field

Also, if you want this specifically in relation to an FK autocomplete,
you should be able to implement that in the view that returns your
results.

-justin

--
Justin Lilly
Python/Django Developer
http://justinlilly.com

[Margie]

Yes, I use that. Inside my formfield_for_foreignkey() method I
instantiate my special widget. My widget contains a render() method
that generates javascript that makes an ajax request that requests the
options for the select. There is no way to prune down the selections
with formfield_for_foreignkey(), in such a way that the selections are
different for different objects.

For example, for my Task object, I have

class TaskAdmin(admin.ModelAdmin:

def formfield_for_foreignkey(self, db_field, request, **kwargs):
if db_field.name == "owner":
kwargs["widget"] = OwnerSelectWidget(db_field.rel)
return db_field.formfield(**kwargs)
return super(TaskAdmin, self).formfield_for_foreignkey
(db_field, request, **kwargs)

However, in the changelist view, different tasks have different sets
of potential owners. IE, a task that is for the foo project can have
bob, joe, or mark as an owner. But a task that is for the bar project
can have jane, mary, or ann as an owner. There is no way to implement
this with just formfield_for_foreignkey(), because that method is for
*all* Task objects and is executed at a point in the django source
where there is no knowledge of what particular task is being
rendered. What I am trying to do is make it so that when the user
clicks on the owner field for the foo task, they have the choice of
bob, joe, or mark, and when they clock on the owner field for the bar
task, they have the choice of jane, mary, or ann. The Task has a
resources field, and that identifies the possible resources (which can
be the owner) for the task. So when the user clicks on the owner
field, the ajax request gets the resources for the particular task and
fills the drop down selections with them.

I have pasted a copy of my OwnerSelectWidget, as well as the above
code at http://dpaste.com/47694/. Maybe that will make it more
clear.

I don't know - maybe my needs are unusual. It's definitely not your
standard blog app ... so if I am too much on the outskirts for this to
be something that we consider in the development community, I can
understand that. But it seems to me that having a dynamic (ie, object
based) way to pair down options in a drop down would be something that
would be useful to others. And while I have a solution here, it was
pretty tough to get to and required a lot of studying and stepping
through the source code.

Anyway, hope the problem I am trying to solve and the solution I have
posted are understandable. Would love to hear from those in the
development community as to whether you think this is a wortwhile
addition.


Margie

[Yuri Baburov]

Hey guys,

Yes, I must say, in any advanced form there are fields that depends
one from another.
Country/Region/City, Project/Issue, Project/Owner, User/Account,
Group/User... when two selects appear, there's not zero probability
they will depend on each other. Also often a checkbox can
enable/disable select box, and so on.

Without javascript... these forms actually suck.
Not that I encouraging you to drop non-js forms support.

I just want to tell that fields data dependencies and field enable
dependencies are must-haves for usable, competitive and visually
appealing admin.

--
Best regards, Yuri V. Baburov, ICQ# 99934676, Skype: yuri.baburov,
MSN: bu...@live.com

[Ulrich Petri]

>
> I don't know - maybe my needs are unusual. It's definitely not your
> standard blog app ... so if I am too much on the outskirts for this to
> be something that we consider in the development community, I can
> understand that. But it seems to me that having a dynamic (ie, object
> based) way to pair down options in a drop down would be something that
> would be useful to others. And while I have a solution here, it was
> pretty tough to get to and required a lot of studying and stepping
> through the source code.
>
> Anyway, hope the problem I am trying to solve and the solution I have
> posted are understandable. Would love to hear from those in the
> development community as to whether you think this is a wortwhile
> addition.
>

It is quite possible to refer to the "parent" object without any ajax
magic (at least in the changeview, but I guess it should be doable for
the changelist also).

I put a short example here:
http://dpaste.com/hold/48040/

HTH
Ulrich

[Alex Gaynor]

Sure, but that assumes the current value on the model.  What Yuri is saying is if I have a continent and country drop down fields and I select North America as my continent the country values should be automatically populated.  To me all this says is that there should be a clear callback on the ModelAdmin for people to override, as well as making sure we POST all the current form values so we can do the appropriate logic in this function.

I also have some thoughts about the implementation (JS toolkits, etc...) but I'll hold onto those since this is just the first prototype.

Alex

--
"I disapprove of what you say, but I will defend to the death your right to say it." --Voltaire
"The people's good is the highest law."--Cicero

[Margie]

That is definitely a very interesting and useful example, appreciate
you posting it. I just took a quick look at the source for class
ModelAdmin and it seems to me that the get_changelist_formset() and
get_changelist_form() methods do not have the obj argument, so I don't
see how to do the same thing in the changelist. This is one of the
things I struggled with, and why I had to extract the id field out of
the form and send it back with my ajax request. Am interested in if
you see a workaround for that - I would definitely prefer to not be
using ajax calls.

Margie

[Margie]

Yes, I like the idea of a callback that can be overridden. Alex - are
you saying this is an ajax solution or a solution that takes place at
formset creation time?

Also, in the rendered output for a field, think it would be nice for
non-editable fields to have ids associated with them the same way that
editable fields have ids. In the ajax usage case - imagine a case
where you have a continent field rendered that is *not* editable, and
based on some other event (ie, a check box), you then want to send an
ajax request to populate the country dropdown to contain all countries
in the continent. It is not easy to send the country value as part of
an ajax request because its field in the form has no id at all.

Margie

On May 26, 5:10 pm, Alex Gaynor <alex.gay...@gmail.com> wrote:

[Yuri Baburov]

My usual workaround to this is to store object, request and action
("add", "change", "changelist") in thread local storage when add_view,
changelist_view or change_view are called, and use it later.

I will be happy if a ModelAdmin instance was created for each request,
then I'll able to use it to store these variables (object, action and
request), and make functions much more clean, and allow dynamic
properties instead of, say, list_display property and all others.

For me, it's design mistake of current django ModelAdmin, that devs
were consistently trying to fix by adding more parameters to functions
:) Isn't it fun? :)

[Margie]

Yuri, Thanks for suggesting thread local. I didn't know about that.

In the end, a think a concise description of my problem is that when I
create my own widget for use in the admin change list, it seems that I
am unable to identify the instance for which this widget is being
rendered. Thus, I am severly limited in my ability to do interesting
stuff like change the queryset choices for a foreign key based on
other data associated with the instance.

By using thread local, I was able to stash away the instance (I had to
modify the source code of items_for_result() in admin_list.py to stash
it), and then grab it in my render function. Then in my render code I
can simply figure out the right queryset for the field based on info
in the instance. Very simple.

Of course, using thread local is a bit of a hack. Ok, a big hack. So
if any developers are aware of another way for me to get the instance
from inside a widget's render() functin, please post! I'm assuming
silence means I'm in uncharted territory. I know the developers are
all busy with 1.1, but I do think this is an important issue.

Thanks again Yuri, that one little comment about thread local really
helped a lot.

Margie

[Yuri Baburov]

Hi Margie,

Not that much hacky.
http://code.djangoproject.com/wiki/CookBookThreadlocalsAndUser

Store request, user and ModelAdmin dynamic stuff in local storage and
you are happy.