[Django] #22330: Model.__reduce__() includes cached lookups
Django
-------------------------------+--------------------
Reporter: patrys | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 1.6
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------
If you try to pickle an object after accessing any of its related
instances, its `__dict__` will contain references to those instances and
will cause them to be pickled as part of your object.
Pickling them can cause additional instances to be traversed and can
result in an unexpectedly large number of objects being included. These
will be pickled/unpickled as a single chunk and I assume the unpickled
instance will continue to use its cached instances which may lead to
surprising results (values not reflecting current DB state).
Pickling additional instances can cause relation traversal to cache
additional reverse lookups thus modifying an instance's `__dict__` while
it is being pickled. This results in:
{{{
RuntimeError: dictionary changed size during iteration
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/22330>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: patrys | Owner: nobody
Type: | Status: new
Cleanup/optimization | Version: 1.6
Component: Database layer | Resolution:
(models, ORM) | Triage Stage: Accepted
Severity: Normal | Needs documentation: 0
Keywords: | Patch needs improvement: 0
Has patch: 0 | UI/UX: 0
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Changes (by timo):
* needs_better_patch: => 0
* component: Uncategorized => Database layer (models, ORM)
* needs_tests: => 0
* needs_docs: => 0
* type: Uncategorized => Cleanup/optimization
* stage: Unreviewed => Accepted
Comment:
I am not sure of the expected behavior. If we don't make a change here, we
should at least document like we've done for
[https://docs.djangoproject.com/en/dev/ref/models/querysets/#pickling-
querysets QuerySet pickling].
--
Ticket URL: <https://code.djangoproject.com/ticket/22330#comment:1>
Django
Reporter: patrys | Owner: nobody
Type: | Status: new
Cleanup/optimization |
Component: Database layer | Version: 1.6
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by akaariai):
If nobody offers a good reason why this must be changed, I think we should
opt for keeping the current behavior. Sometimes you might actually want to
pickle the related instances, too. And, changes here might result in
performance changes for deployed applications, which are nasty to detect
when updating Django. One example is that caching results of
select_related queryset would not cache the related selections after the
change.
Above, the RuntimeError thrown by pickling looks like a good argument for
fixing. But as long as we don't have a test to reproduce this issue, it is
hard to say if the problem is in Django or how to fix the problem if it is
in Django.
--
Ticket URL: <https://code.djangoproject.com/ticket/22330#comment:2>
Django
Reporter: patrys | Owner: nobody
Type: | Status: closed
Cleanup/optimization |
Component: Database layer | Version: 1.6
(models, ORM) |
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* status: new => closed
* resolution: => needsinfo
Comment:
It might be related to issues like #24381, but I agree we can close this
until we get a test case to reproduce the problem.
--
Ticket URL: <https://code.djangoproject.com/ticket/22330#comment:3>