[Django] #34986: Fix up support for PyPy 3.10
Django
-----------------------------------------+---------------------------
Reporter: Nick Pope | Owner: Nick Pope
Type: Bug | Status: assigned
Component: Uncategorized | Version: dev
Severity: Normal | Keywords: pypy
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-----------------------------------------+---------------------------
Python 3.10 support was released in PyPy v7.3.12 in June 2023.
It was highlighted [https://github.com/django/django/pull/17496 here] that
a number of tests are failing, and
[https://code.djangoproject.com/ticket/34983#comment:2 here] and
[https://code.djangoproject.com/ticket/34983#comment:4 here] that the
changes in #31262 have introduced some large "regressions". It turns out
that there is a [https://foss.heptapod.net/pypy/pypy/-/issues/4036 bug] in
PyPy related to abstract base classes being used for `isinstance()` checks
with structural pattern matching. This can be resolved by either waiting
for a fix in PyPy or implementing a workaround by removing the pattern
matching approach. Aside from that, however, there are also a number of
other failures and crashes that still need to be addressed.
[https://pypy.org/ PyPy] claims to be highly compatible with CPython and
that it supports Django. Django
[https://docs.djangoproject.com/en/4.2/topics/performance/#id1 claims] to
be compatible with PyPy. We need to determine whether it is possible to
fix these issues or whether we need to remove this claim.
One of the issues is that currently there is no CI to ensure that PyPy
support remains in a working state. An initial
[https://github.com/django/django/pull/17499 PR] has been created to run a
daily scheduled test run on SQLite only. Later it may make sense to enable
triggering testing via assigning a label to a PR.
--
Ticket URL: <https://code.djangoproject.com/ticket/34986>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: Nick Pope | Owner: Nick Pope
Type: Bug | Status: assigned
Severity: Normal | Resolution:
Keywords: pypy | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* component: Uncategorized => Core (Other)
* stage: Unreviewed => Accepted
--
Ticket URL: <https://code.djangoproject.com/ticket/34986#comment:1>
Django
------------------------------+-------------------------------------
Reporter: Nick Pope | Owner: Nick Pope
Type: Bug | Status: assigned
Component: Core (Other) | Version: dev
Severity: Normal | Resolution:
Keywords: pypy | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
------------------------------+-------------------------------------
* has_patch: 0 => 1
Comment:
[https://github.com/django/django/pull/17500 PR]
--
Ticket URL: <https://code.djangoproject.com/ticket/34986#comment:2>
Django
------------------------------+-------------------------------------
Reporter: Nick Pope | Owner: Nick Pope
Type: Bug | Status: assigned
Component: Core (Other) | Version: dev
Severity: Normal | Resolution:
Keywords: pypy | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
------------------------------+-------------------------------------
Comment (by GitHub <noreply@…>):
In [changeset:"0d407bcce5449d839fab138baeb9220f6df6bd6d" 0d407bc]:
{{{
#!CommitTicketReference repository=""
revision="0d407bcce5449d839fab138baeb9220f6df6bd6d"
Refs #34986 -- Installed libmemcached-dev in PyPy 3.10 daily builds.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34986#comment:3>
Django
------------------------------+-------------------------------------
Reporter: Nick Pope | Owner: Nick Pope
Type: Bug | Status: assigned
Component: Core (Other) | Version: dev
Severity: Normal | Resolution:
Keywords: pypy | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
------------------------------+-------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"9baaf89eed42f9e747c774bc79543f47ea9697f8" 9baaf89]:
{{{
#!CommitTicketReference repository=""
revision="9baaf89eed42f9e747c774bc79543f47ea9697f8"
Refs #34986 -- Added django.utils.version.PYPY.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34986#comment:5>
Django
------------------------------+-------------------------------------
Reporter: Nick Pope | Owner: Nick Pope
Type: Bug | Status: assigned
Component: Core (Other) | Version: dev
Severity: Normal | Resolution:
Keywords: pypy | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
------------------------------+-------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"174369a990c00d75cf40e0a06ab82bec9a89a7ef" 174369a9]:
{{{
#!CommitTicketReference repository=""
revision="174369a990c00d75cf40e0a06ab82bec9a89a7ef"
Refs #34986 -- Avoided pickling error in DjangoUnicodeDecodeError.
By using the existing object reference instead of a custom one, pickling
related issues when running the test suite in parallel can be avoided.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34986#comment:4>
Django
------------------------------+-------------------------------------
Reporter: Nick Pope | Owner: Nick Pope
Type: Bug | Status: assigned
Component: Core (Other) | Version: dev
Severity: Normal | Resolution:
Keywords: pypy | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
------------------------------+-------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"6089230d3ec580f2f44e85f23962c14905fefcfd" 6089230]:
{{{
#!CommitTicketReference repository=""
revision="6089230d3ec580f2f44e85f23962c14905fefcfd"
Refs #34986 -- Fixed mocking in
utils_tests.test_http.HttpDateProcessingTests.test_parsing_rfc850.
Mocking in the `datetime` module can be tricky. In CPython the datetime
C module is used, but PyPy uses a pure Python implementation. This
caused issues with the prior approach to mocking `datetime.datetime`.
See https://docs.python.org/3/library/unittest.mock-examples.html#partial-
mocking
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34986#comment:6>
Django
------------------------------+-------------------------------------
Reporter: Nick Pope | Owner: Nick Pope
Type: Bug | Status: assigned
Component: Core (Other) | Version: dev
Severity: Normal | Resolution:
Keywords: pypy | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
------------------------------+-------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"051dbb53884eb202131c27dfdeac7c3ddd7b1072" 051dbb53]:
{{{
#!CommitTicketReference repository=""
revision="051dbb53884eb202131c27dfdeac7c3ddd7b1072"
Refs #34986 -- Added PyPy support to
django.utils.autoreload.get_child_arguments().
It seems that `sys._xoptions` is also supported on PyPy.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34986#comment:7>
Django
------------------------------+-------------------------------------
Reporter: Nick Pope | Owner: Nick Pope
Type: Bug | Status: assigned
Component: Core (Other) | Version: dev
Severity: Normal | Resolution:
Keywords: pypy | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
------------------------------+-------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"baf705f34a8c8977d042ce43c71f508f9ca4f8ce" baf705f3]:
{{{
#!CommitTicketReference repository=""
revision="baf705f34a8c8977d042ce43c71f508f9ca4f8ce"
Refs #34986 -- Fixed some test assertions for PyPy.
These failures were due to minor inconsistencies or implementation
differences between CPython and PyPy.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34986#comment:8>
Django
------------------------------+-------------------------------------
Reporter: Nick Pope | Owner: Nick Pope
Type: Bug | Status: assigned
Component: Core (Other) | Version: dev
Severity: Normal | Resolution:
Keywords: pypy | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
------------------------------+-------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"5f9e5c1b0d5680f793ba7646d52ffab9e2c3623f" 5f9e5c1]:
{{{
#!CommitTicketReference repository=""
revision="5f9e5c1b0d5680f793ba7646d52ffab9e2c3623f"
Refs #34822, Refs #34986 -- Fixed migrations serializer support for
functools.lru_cache().
It turns out that `functools._lru_cache_wrapper` is only a class when
CPython's _functools C module provides it, otherwise it is a function.
PyPy also provides it as a function.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34986#comment:9>
Django
------------------------------+-------------------------------------
Reporter: Nick Pope | Owner: Nick Pope
Type: Bug | Status: assigned
Component: Core (Other) | Version: dev
Severity: Normal | Resolution:
Keywords: pypy | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
------------------------------+-------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"06c5cb1284557edc05f963afb6f6bcdd55b00911" 06c5cb12]:
{{{
#!CommitTicketReference repository=""
revision="06c5cb1284557edc05f963afb6f6bcdd55b00911"
Refs #34986 -- Removed redundant
CustomChoicesTests.test_timezone_unsupported().
This test relied on the behavior of subclassing `datetime.timezone`
which is not permitted by the C-extension version of CPython's
`datetime` module. This restriction isn't enforced by the pure
Python version, nor by PyPy.
See https://github.com/python/cpython/issues/112451
It's not critical, and doesn't test any Django behavior, so just
remove it.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34986#comment:10>
Django
------------------------------+-------------------------------------
Reporter: Nick Pope | Owner: Nick Pope
Type: Bug | Status: assigned
Component: Core (Other) | Version: dev
Severity: Normal | Resolution:
Keywords: pypy | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
------------------------------+-------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"1c3614e306e80d5a57c52d36c2a1868e1e5f9ab3" 1c3614e3]:
{{{
#!CommitTicketReference repository=""
revision="1c3614e306e80d5a57c52d36c2a1868e1e5f9ab3"
Refs #34986 -- Avoided implementation-specific unpickleable types.
The implementation of some core types differ between CPython and PyPy
and this may affect the way that pickling works such that errors are
raised in differing locations in the interpreter or not at all.
Use our own custom non-pickleable type instead to avoid these quirks.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34986#comment:11>
Django
------------------------------+-------------------------------------
Reporter: Nick Pope | Owner: Nick Pope
Type: Bug | Status: assigned
Component: Core (Other) | Version: dev
Severity: Normal | Resolution:
Keywords: pypy | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
------------------------------+-------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"fb9216382a387685e1082ac97caa3feaf3d5fe80" fb921638]:
{{{
#!CommitTicketReference repository=""
revision="fb9216382a387685e1082ac97caa3feaf3d5fe80"
Refs #34986 -- Moved garbage_collect() helper to django.test.utils.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34986#comment:12>
Django
------------------------------+-------------------------------------
Reporter: Nick Pope | Owner: Nick Pope
Type: Bug | Status: assigned
Component: Core (Other) | Version: dev
Severity: Normal | Resolution:
Keywords: pypy | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
------------------------------+-------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"66d58e77de3196404e0820a6fef0a6144186015a" 66d58e77]:
{{{
#!CommitTicketReference repository=""
revision="66d58e77de3196404e0820a6fef0a6144186015a"
Refs #34986 -- Added PyPy to tox.ini.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34986#comment:13>
Django
Reporter: Nick Pope | Owner: Nick Pope
Type: Bug | Status: assigned
Component: Core (Other) | Version: dev
Severity: Normal | Resolution:
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/34986#comment:14>
Django
-------------------------------------+-------------------------------------
Reporter: Nick Pope | Owner: Nick Pope
Type: Bug | Status: assigned
Component: Core (Other) | Version: dev
Severity: Normal | Resolution:
Keywords: pypy | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"af6e7e3de8c6167ec1848c72915469442eb8a965" af6e7e3d]:
{{{
#!CommitTicketReference repository=""
revision="af6e7e3de8c6167ec1848c72915469442eb8a965"
Refs #34986 -- Fixed
backends.postgresql.test_server_side_cursors.ServerSideCursorsPostgres
tests for PyPy.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34986#comment:16>
Django
-------------------------------------+-------------------------------------
Reporter: Nick Pope | Owner: Nick Pope
Type: Bug | Status: assigned
Component: Core (Other) | Version: dev
Severity: Normal | Resolution:
Keywords: pypy | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"2dca98f4f76e1296b842b183a9287086ec416b58" 2dca98f]:
{{{
#!CommitTicketReference repository=""
revision="2dca98f4f76e1296b842b183a9287086ec416b58"
Refs #34986 -- Added PyPy 3.10 with PostgreSQL to daily builds.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34986#comment:17>
Django
-------------------------------------+-------------------------------------
Reporter: Nick Pope | Owner: Nick Pope
Type: Bug | Status: assigned
Component: Core (Other) | Version: dev
Severity: Normal | Resolution:
Keywords: pypy | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"958a590e539feb1cc3feb7c87635dea906eae191" 958a590e]:
{{{
#!CommitTicketReference repository=""
revision="958a590e539feb1cc3feb7c87635dea906eae191"
Refs #34986 -- Used non-binary build of psycopg for PyPy.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34986#comment:15>
Django
-------------------------------------+-------------------------------------
Reporter: Nick Pope | Owner: Nick Pope
Component: Core (Other) | Version: dev
Keywords: pypy | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => fixed
--
Ticket URL: <https://code.djangoproject.com/ticket/34986#comment:18>