[Django] #28717: Admin always queries default database when rendering a list_filter on a ForeignKey

4 views
Skip to first unread message

Django

unread,
Oct 17, 2017, 12:29:44 AM10/17/17
to django-...@googlegroups.com
#28717: Admin always queries default database when rendering a list_filter on a
ForeignKey
-----------------------------------------+------------------------
Reporter: Adam Brenecki | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 1.11
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-----------------------------------------+------------------------
I'm using Django's admin on a model on one of a number of secondary
databases. The model has {{{managed = False}}} and doesn't exist on the
{{{default}}} database. I'm not using a custom database router; I've
overridden methods on my {{{ModelAdmin}}} as per here:
https://docs.djangoproject.com/en/1.11/topics/db/multi-db/#exposing-
multiple-databases-in-django-s-admin-interface. This mostly works, except
when I add a {{{ForeignKey}}} field to {{{list_filter}}}, when it appears
to try to query the primary database for the values to display in the
filter, and I can't find a way to override this behaviour in
{{{ModelAdmin}}}.

This happens both on Django 1.11.5 and on master (as of commit 1b73ccc);
other models that have {{{list_filter}}}s that aren't foreign keys work
fine, as does this model if I comment out the line with {{{list_filter}}}.

Below is an excerpt from my {{{admin.py}}} as well as the full stack
trace:

{{{#!python
from django.contrib import admin

from . import models


class ApplyOLModelAdmin(admin.ModelAdmin):
def save_model(self, request, obj, form, change):
obj.save(using=request.applyol)

def delete_model(self, request, obj):
obj.delete(using=request.applyol)

def get_queryset(self, request):
return super().get_queryset(request).using(request.applyol)

def formfield_for_foreignkey(self, db_field, request, **kwargs):
return super().formfield_for_foreignkey(
db_field, request, using=request.applyol, **kwargs
)

def formfield_for_manytomany(self, db_field, request, **kwargs):
return super().formfield_for_manytomany(
db_field, request, using=request.applyol, **kwargs
)

def has_module_permission(self, request):
return True

def has_add_permission(self, request, obj=None):
return True

def has_change_permission(self, request, obj=None):
return True

def has_delete_permission(self, request, obj=None):
return True


@admin.register(models.CountryCity)
class CountryCityAdmin(ApplyOLModelAdmin):
list_display = ('country', 'code', 'name')
list_display_links = ('name',)
list_filter = ('country',)
ordering = ('country__name', 'name')
}}}

{{{
Environment:


Request Method: GET
Request URL: http://localhost:8000/dev/admin/applyol_editor/countrycity/

Django Version: 2.1.dev20171016175638
Python Version: 3.6.2
Installed Applications:
('sl_admin.apps.applyol_editor',
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'form_utils',
'webpack_loader',
'widget_tweaks',
'sl_admin.lib',
'sl_admin.apps.accounts',
'sl_admin.apps.landing',
'sl_admin.apps.form_builder')
Installed Middleware:
('django.middleware.security.SecurityMiddleware',
'whitenoise.middleware.WhiteNoiseMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'csp.middleware.CSPMiddleware',
'sl_admin.lib.middleware.XUACompatibleMiddleware',
'sl_admin.apps.accounts.middleware.SocialAuthErrorMiddleware',
'sl_admin.lib.environment.EnvironmentMiddleware',
'rollbar.contrib.django.middleware.RollbarNotifierMiddleware')

Traceback:

File "/Users/adambrenecki/Projects/django/django/db/backends/utils.py" in
_execute
86. return self.cursor.execute(sql, params)

The above exception (relation "APPLYOL.TBLCOUNTRY" does not exist
LINE 1: ...ODE", "APPLYOL"."TBLCOUNTRY"."ISNATIONALITY" FROM "APPLYOL"....
^
) was the direct cause of the following exception:

File
"/Users/adambrenecki/Projects/django/django/core/handlers/exception.py" in
inner
35. response = get_response(request)

File "/Users/adambrenecki/Projects/django/django/core/handlers/base.py" in
_get_response
128. response = self.process_exception_by_middleware(e,
request)

File "/Users/adambrenecki/Projects/django/django/core/handlers/base.py" in
_get_response
126. response = wrapped_callback(request,
*callback_args, **callback_kwargs)

File "/Users/adambrenecki/Projects/django/django/contrib/admin/options.py"
in wrapper
574. return self.admin_site.admin_view(view)(*args,
**kwargs)

File "/Users/adambrenecki/Projects/django/django/utils/decorators.py" in
_wrapped_view
142. response = view_func(request, *args, **kwargs)

File
"/Users/adambrenecki/Projects/django/django/views/decorators/cache.py" in
_wrapped_view_func
44. response = view_func(request, *args, **kwargs)

File "/Users/adambrenecki/Projects/django/django/contrib/admin/sites.py"
in inner
223. return view(request, *args, **kwargs)

File "/Users/adambrenecki/Projects/django/django/utils/decorators.py" in
_wrapper
62. return bound_func(*args, **kwargs)

File "/Users/adambrenecki/Projects/django/django/utils/decorators.py" in
_wrapped_view
142. response = view_func(request, *args, **kwargs)

File "/Users/adambrenecki/Projects/django/django/utils/decorators.py" in
bound_func
58. return func.__get__(self, type(self))(*args2,
**kwargs2)

File "/Users/adambrenecki/Projects/django/django/contrib/admin/options.py"
in changelist_view
1570. cl = self.get_changelist_instance(request)

File "/Users/adambrenecki/Projects/django/django/contrib/admin/options.py"
in get_changelist_instance
705. self,

File
"/Users/adambrenecki/Projects/django/django/contrib/admin/views/main.py"
in __init__
75. self.queryset = self.get_queryset(request)

File
"/Users/adambrenecki/Projects/django/django/contrib/admin/views/main.py"
in get_queryset
313. filters_use_distinct) = self.get_filters(request)

File
"/Users/adambrenecki/Projects/django/django/contrib/admin/views/main.py"
in get_filters
129. self.model, self.model_admin,
field_path=field_path

File "/Users/adambrenecki/Projects/django/django/contrib/admin/filters.py"
in create
157. return list_filter_class(field, request, params, model,
model_admin, field_path=field_path)

File "/Users/adambrenecki/Projects/django/django/contrib/admin/filters.py"
in __init__
168. self.lookup_choices = self.field_choices(field, request,
model_admin)

File "/Users/adambrenecki/Projects/django/django/contrib/admin/filters.py"
in field_choices
195. return field.get_choices(include_blank=False)

File
"/Users/adambrenecki/Projects/django/django/db/models/fields/__init__.py"
in get_choices
812. limit_choices_to)]

File "/Users/adambrenecki/Projects/django/django/db/models/query.py" in
__iter__
270. self._fetch_all()

File "/Users/adambrenecki/Projects/django/django/db/models/query.py" in
_fetch_all
1174. self._result_cache = list(self._iterable_class(self))

File "/Users/adambrenecki/Projects/django/django/db/models/query.py" in
__iter__
55. results =
compiler.execute_sql(chunked_fetch=self.chunked_fetch,
chunk_size=self.chunk_size)

File
"/Users/adambrenecki/Projects/django/django/db/models/sql/compiler.py" in
execute_sql
1043. cursor.execute(sql, params)

File "/Users/adambrenecki/Projects/django/django/db/backends/utils.py" in
execute
101. return super().execute(sql, params)

File "/Users/adambrenecki/Projects/django/django/db/backends/utils.py" in
execute
69. return self._execute_with_wrappers(sql, params, many=False,
executor=self._execute)

File "/Users/adambrenecki/Projects/django/django/db/backends/utils.py" in
_execute_with_wrappers
78. return executor(sql, params, many, context)

File "/Users/adambrenecki/Projects/django/django/db/backends/utils.py" in
_execute
86. return self.cursor.execute(sql, params)

File "/Users/adambrenecki/Projects/django/django/db/utils.py" in __exit__
89. raise dj_exc_value.with_traceback(traceback) from
exc_value

File "/Users/adambrenecki/Projects/django/django/db/backends/utils.py" in
_execute
86. return self.cursor.execute(sql, params)

Exception Type: ProgrammingError at /dev/admin/applyol_editor/countrycity/
Exception Value: relation "APPLYOL.TBLCOUNTRY" does not exist
LINE 1: ...ODE", "APPLYOL"."TBLCOUNTRY"."ISNATIONALITY" FROM "APPLYOL"....
^
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/28717>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

Django

unread,
Oct 21, 2017, 5:54:29 PM10/21/17
to django-...@googlegroups.com
#28717: Admin always queries default database when rendering a list_filter on a
ForeignKey
-------------------------------+--------------------------------------

Reporter: Adam Brenecki | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 1.11
Severity: Normal | Resolution:

Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------------------------

Comment (by Tomer Chachamu):

Yep, that's an oversight in the documentation. However, you can remove the
admin customisation and write a database router instead. That will work
not just for the admin but throughout your website too. If you are setting
`request.applyol` in middleware, you can in the same middleware tell your
database router to change the database that it should give for
`db_for_read(model=ApplyOl)`. Here's an example you can adapt:
https://github.com/yandex/django_replicated/blob/master/django_replicated/router.py

--
Ticket URL: <https://code.djangoproject.com/ticket/28717#comment:1>

Django

unread,
Nov 1, 2017, 3:56:44 PM11/1/17
to django-...@googlegroups.com
#28717: Document that using ModelAdmin.list_filter with foreign keys may require a
database router
--------------------------------------+------------------------------------

Reporter: Adam Brenecki | Owner: nobody
Type: Cleanup/optimization | Status: new
Component: Documentation | Version: 1.11
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------------+------------------------------------
Changes (by Tim Graham):

* component: Uncategorized => Documentation
* stage: Unreviewed => Accepted
* type: Uncategorized => Cleanup/optimization


Comment:

Absent another proposal, it sounds like documenting that a database router
is required for this case could be a solution.

--
Ticket URL: <https://code.djangoproject.com/ticket/28717#comment:2>

Reply all
Reply to author
Forward
0 new messages