[Django] #28462: list_editable unusable with large datasets
Django
-----------------------------------------+------------------------
Reporter: Ben Cole | Owner: nobody
Type: Bug | Status: new
Component: contrib.admin | Version: 1.10
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-----------------------------------------+------------------------
Since 1.10 `list_editable` on `ModelAdmin` is unusable for Models with a
large-ish dataset.
The problem is caused by a recent change to how the FormSet is generated
for the admin. Previously it was generated from the ChangeList result
list, but it has been changed to use the admin's `get_queryset` which will
return more than the current "pageful" of results (potentially the entire
dataset) causing Django to generate a form for each instance. This results
in Django consuming all available RAM and in some cases the python
instance crashing. My personal laptop became unresponsive and I had to
force power off.
--
Ticket URL: <https://code.djangoproject.com/ticket/28462>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
datasets
-------------------------------+------------------------------------
Reporter: Ben Cole | Owner: nobody
Type: Bug | Status: new
Component: contrib.admin | Version: 1.10
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham):
* stage: Unreviewed => Accepted
--
Ticket URL: <https://code.djangoproject.com/ticket/28462#comment:1>
Django
datasets
Reporter: Ben Cole | Owner: Srinivas
| Reddy Thatiparthy
Type: Bug | Status: assigned
Component: contrib.admin | Version: 1.10
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Srinivas Reddy Thatiparthy):
* owner: nobody => Srinivas Reddy Thatiparthy
* status: new => assigned
Comment:
I am taking this ticket
--
Ticket URL: <https://code.djangoproject.com/ticket/28462#comment:2>
Django
datasets
Reporter: Ben Cole | Owner: (none)
Type: Bug | Status: new
Component: contrib.admin | Version: 1.10
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Srinivas Reddy Thatiparthy):
* owner: Srinivas Reddy Thatiparthy => (none)
* status: assigned => new
--
Ticket URL: <https://code.djangoproject.com/ticket/28462#comment:3>
Django
datasets
-------------------------------+------------------------------------
Type: Bug | Status: new
Component: contrib.admin | Version: 1.10
Keywords: | Triage Stage: Accepted
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham):
* has_patch: 0 => 1
* needs_tests: 0 => 1
Comment:
[https://github.com/django/django/pull/8837 PR] (without a test)
--
Ticket URL: <https://code.djangoproject.com/ticket/28462#comment:2>
Django
datasets
-------------------------------+------------------------------------
Type: Bug | Status: new
Component: contrib.admin | Version: 1.10
Keywords: | Triage Stage: Accepted
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Pablo Martín):
+1 Same error. We have overwritten the ModelAdmin for this problem. I
think solve a strange problem and getting a memory problem is not a good
solution. I prefer a 500 error one time every three month that a serious
memory problem every day.
This commit
https://github.com/django/django/commit/917cc288a38f3c114a5440f0749b7e5e1086eb36#commitcomment-23412084
does not make any sense.
--
Ticket URL: <https://code.djangoproject.com/ticket/28462#comment:3>
Django
datasets
-------------------------------+------------------------------------
Type: Bug | Status: new
Component: contrib.admin | Version: 1.10
Keywords: | Triage Stage: Accepted
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Pablo Martín):
* Attachment "memory-problem.png" added.
Django
datasets
-------------------------------+------------------------------------
Type: Bug | Status: new
Component: contrib.admin | Version: 1.10
Keywords: | Triage Stage: Accepted
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------+------------------------------------
Changes (by Pablo Martín):
* easy: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/28462#comment:4>
Django
datasets
-------------------------------+------------------------------------
Type: Bug | Status: new
Component: contrib.admin | Version: 1.10
Keywords: | Triage Stage: Accepted
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham):
* easy: 1 => 0
Comment:
I don't think reverting is a good idea as that reintroduces possible data
loss.
--
Ticket URL: <https://code.djangoproject.com/ticket/28462#comment:5>
Django
datasets
-------------------------------+------------------------------------
Type: Bug | Status: new
Component: contrib.admin | Version: 1.10
Keywords: | Triage Stage: Accepted
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Rubén Gómez):
I just made a PR to solve this problem:
[https://github.com/django/django/pull/9820]
--
Ticket URL: <https://code.djangoproject.com/ticket/28462#comment:6>
Django
datasets
-------------------------------+------------------------------------
Type: Bug | Status: new
Component: contrib.admin | Version: 1.10
Keywords: | Triage Stage: Accepted
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Josh Smeaton):
* cc: josh.smeaton@… (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/28462#comment:7>
Django
datasets
Reporter: Ben Cole | Owner: nobody
Type: Bug | Status: new
Component: contrib.admin | Version: 1.10
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by AdamDonna):
* needs_tests: 1 => 0
* stage: Accepted => Ready for checkin
Comment:
Has tests
https://github.com/django/django/pull/9920
I'm proposing a backport to 1.11
--
Ticket URL: <https://code.djangoproject.com/ticket/28462#comment:8>
Django
datasets
Reporter: Ben Cole | Owner: nobody
Type: Bug | Status: new
Component: contrib.admin | Version: 1.10
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham):
* stage: Ready for checkin => Accepted
Comment:
"Ready for checkin" is set by the patch reviewer, not the author.
--
Ticket URL: <https://code.djangoproject.com/ticket/28462#comment:9>
Django
datasets
-------------------------------+------------------------------------
Type: Bug | Status: new
Component: contrib.admin | Version: 1.10
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
Changes (by Carlton Gibson):
* needs_better_patch: 0 => 1
Comment:
Hi AdamDonna. I'm going to mark this "Patch needs improvement" for now,
pending the extra testing outlined on the ticket.
* Please uncheck when that's ready
* @carltongibson me on GitHub if you want/need me to input/help.
Thanks for your effort!
--
Ticket URL: <https://code.djangoproject.com/ticket/28462#comment:10>
Django
datasets
Reporter: Ben Cole | Owner: nobody
Type: Bug | Status: new
Component: contrib.admin | Version: 1.10
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Carlton Gibson):
* needs_better_patch: 1 => 0
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/28462#comment:11>
Django
datasets
-------------------------------------+-------------------------------------
Component: contrib.admin | Version: 1.10
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham <timograham@…>):
* status: new => closed
* resolution: => fixed
Comment:
In [changeset:"b18650a2634890aa758abae2f33875daa13a9ba3" b18650a2]:
{{{
#!CommitTicketReference repository=""
revision="b18650a2634890aa758abae2f33875daa13a9ba3"
Fixed #28462 -- Decreased memory usage with ModelAdmin.list_editable.
Regression in 917cc288a38f3c114a5440f0749b7e5e1086eb36.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28462#comment:12>
Django
datasets
-------------------------------------+-------------------------------------
Component: contrib.admin | Version: 1.10
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Tim Graham <timograham@…>):
In [changeset:"10a6d444af48007f8e645517705d08d2d6ba0359" 10a6d444]:
{{{
#!CommitTicketReference repository=""
revision="10a6d444af48007f8e645517705d08d2d6ba0359"
[2.1.x] Fixed #28462 -- Decreased memory usage with
ModelAdmin.list_editable.
Regression in 917cc288a38f3c114a5440f0749b7e5e1086eb36.
Backport of b18650a2634890aa758abae2f33875daa13a9ba3 from master
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28462#comment:13>
Django
datasets
-------------------------------------+-------------------------------------
Component: contrib.admin | Version: 1.10
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Tim Graham <timograham@…>):
In [changeset:"22c7c2db1dcc3d0ba2799441e7de025d502b3a4c" 22c7c2d]:
{{{
#!CommitTicketReference repository=""
revision="22c7c2db1dcc3d0ba2799441e7de025d502b3a4c"
[2.0.x] Fixed #28462 -- Decreased memory usage with
ModelAdmin.list_editable.
Regression in 917cc288a38f3c114a5440f0749b7e5e1086eb36.
Backport of b18650a2634890aa758abae2f33875daa13a9ba3 from master
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28462#comment:14>
Django
datasets
-------------------------------------+-------------------------------------
Component: contrib.admin | Version: 1.10
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Tim Graham <timograham@…>):
In [changeset:"e9bd1a3e12df132527a8d8bea95858e856ac7be4" e9bd1a3]:
{{{
#!CommitTicketReference repository=""
revision="e9bd1a3e12df132527a8d8bea95858e856ac7be4"
Refs #28462 -- Fixed 'invalid escape sequence' warning on Python 3.6+.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28462#comment:15>
Django
datasets
-------------------------------------+-------------------------------------
Component: contrib.admin | Version: 1.10
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Tim Graham <timograham@…>):
In [changeset:"38bdac4752cd8b27b786fc4aec5516d785f4b6ba" 38bdac47]:
{{{
#!CommitTicketReference repository=""
revision="38bdac4752cd8b27b786fc4aec5516d785f4b6ba"
[2.1.x] Refs #28462 -- Fixed 'invalid escape sequence' warning on Python
3.6+.
Backport of e9bd1a3e12df132527a8d8bea95858e856ac7be4 from master
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28462#comment:16>
Django
datasets
-------------------------------------+-------------------------------------
Component: contrib.admin | Version: 1.10
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Tim Graham <timograham@…>):
In [changeset:"744a44dfa1df21d10e19c6928066eb1408993b29" 744a44df]:
{{{
#!CommitTicketReference repository=""
revision="744a44dfa1df21d10e19c6928066eb1408993b29"
[2.0.x] Refs #28462 -- Fixed 'invalid escape sequence' warning on Python
3.6+.
Backport of e9bd1a3e12df132527a8d8bea95858e856ac7be4 from master
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28462#comment:17>
Django
datasets
-------------------------------------+-------------------------------------
Component: contrib.admin | Version: 1.10
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Tim Graham <timograham@…>):
In [changeset:"56c5c1599a884f6d985c68c54d106db50381e02e" 56c5c159]:
{{{
#!CommitTicketReference repository=""
revision="56c5c1599a884f6d985c68c54d106db50381e02e"
[1.11.x] Fixed #28462 -- Decreased memory usage with
ModelAdmin.list_editable.
Regression in 917cc288a38f3c114a5440f0749b7e5e1086eb36.
Backport of b18650a2634890aa758abae2f33875daa13a9ba3 from master
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28462#comment:18>