Re: [Django] #37106: pylibmc install fails from tests requirements on Python 3.12+ without libmemcached installed

[Django]

#37106: pylibmc install fails from tests requirements on Python 3.12+ without
libmemcached installed
-------------------------------------+-------------------------------------
Reporter: Kiko Barr | Owner: Jason
Type: | Judkins
Cleanup/optimization | Status: assigned
Component: Documentation | Version: 6.0
Severity: Normal | Resolution:
Keywords: test, pylibmc | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Malte Gerth):

The same issue was discussed and the PyCon Italy 2026 again.

There are already multiple issues and some discussion around this issue.
* https://code.djangoproject.com/ticket/27876
* https://code.djangoproject.com/ticket/29405
* https://forum.djangoproject.com/t/error-when-installing-requirements-
for-django-tests-fatal-error-libmemcached-memcached-h-file-not-found-
include-libmemcached-memcached-h/1685

Maybe the already existing MR could be reused and be accepted?
* https://github.com/django/django/pull/12660
--
Ticket URL: <https://code.djangoproject.com/ticket/37106#comment:3>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#37106: pylibmc install fails from tests requirements on Python 3.12+ without
libmemcached installed
-------------------------------------+-------------------------------------
Reporter: Kiko Barr | Owner: Jason
Type: | Judkins
Cleanup/optimization | Status: assigned
Component: Documentation | Version: 6.0
Severity: Normal | Resolution:
Keywords: test, pylibmc | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Tim Schilling):

Would an admonition that includes the error message and then links to a
Django wiki page that has various platform solutions be amenable? That
allows searching the docs to find the solution, but avoids flooding our
docs with the various platforms' solutions.

I'm not sure the upstream problem will be solved soon. The project is
hopefully transitioning maintainers, but it's in limbo:
https://github.com/lericson/pylibmc/issues/297
--
Ticket URL: <https://code.djangoproject.com/ticket/37106#comment:4>

[Django]

#37106: pylibmc install fails from tests requirements on Python 3.12+ without
libmemcached installed
-------------------------------------+-------------------------------------
Reporter: Kiko Barr | Owner: Jason
Type: | Judkins
Cleanup/optimization | Status: assigned
Component: Documentation | Version: 6.0
Severity: Normal | Resolution:
Keywords: test, pylibmc | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Tim Schilling):

The python-memcached library is also in a similar state of not being
updated in 8 months with the last release in 2024:
https://github.com/linsomniac/python-memcached
--
Ticket URL: <https://code.djangoproject.com/ticket/37106#comment:5>

[Django]

#37106: pylibmc install fails from tests requirements on Python 3.12+ without
libmemcached installed
-------------------------------------+-------------------------------------
Reporter: Kiko Barr | Owner: Jason
Type: | Judkins
Cleanup/optimization | Status: assigned
Component: Documentation | Version: 6.0
Severity: Normal | Resolution:
Keywords: test, pylibmc | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Tim Graham):

I think the simplest option would be to instruct contributors to comment
out the dependency if it doesn't install for them as they're unlikely to
need it.
--
Ticket URL: <https://code.djangoproject.com/ticket/37106#comment:6>

[Django]

#37106: pylibmc install fails from tests requirements on Python 3.12+ without
libmemcached installed
-------------------------------------+-------------------------------------
Reporter: Kiko Barr | Owner: Jason
Type: | Judkins
Cleanup/optimization | Status: assigned
Component: Documentation | Version: 6.0
Severity: Normal | Resolution:
Keywords: test, pylibmc | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Johannes Maron):

Ref #37098

We just added support to pass `LIBMEMCACHED` to tox.

The following works on a fresh macOS with Homebrew:

{{{
brew install libmemcached
echo 'export LIBMEMCACHED="${HOMEBREW_PREFIX}"' >> ~/.profile
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/37106#comment:7>

[Django]

#37106: pylibmc install fails from tests requirements on Python 3.12+ without
libmemcached installed
-------------------------------------+-------------------------------------
Reporter: Kiko Barr | Owner: Jason
Type: | Judkins
Cleanup/optimization | Status: assigned
Component: Documentation | Version: 6.0
Severity: Normal | Resolution:
Keywords: test, pylibmc | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Johannes Maron):

BTW, if we update the docs, maybe we point people towards **Tox** instead
of manually fiddling with `./tests/runtests.py`

{{{
tox -e py3 -- path.to.test
# and
tox -e docs
}}}

works for OK most patches unless you have to run Selenium or, worse,
Oracle…
--
Ticket URL: <https://code.djangoproject.com/ticket/37106#comment:8>

[Django]

#37106: pylibmc install fails from tests requirements on Python 3.12+ without
libmemcached installed
-------------------------------------+-------------------------------------
Reporter: Kiko Barr | Owner: Jason
Type: | Judkins
Cleanup/optimization | Status: assigned
Component: Documentation | Version: 6.0
Severity: Normal | Resolution:
Keywords: test, pylibmc | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Tim Schilling):

The person who had offered to help maintain pylibmc has started
maintaining a fork until activity resumes:
https://pypi.org/project/pylibmc2/
--
Ticket URL: <https://code.djangoproject.com/ticket/37106#comment:9>

[Django]

#37106: pylibmc install fails from tests requirements on Python 3.12+ without
libmemcached installed
-------------------------------------+-------------------------------------
Reporter: Kiko Barr | Owner: Jason
Type: | Judkins
Cleanup/optimization | Status: assigned
Component: Documentation | Version: 6.0
Severity: Normal | Resolution:
Keywords: test, pylibmc | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Jacob Walls):

Replying to [comment:9 Tim Schilling]:

> The person who had offered to help maintain pylibmc has started
maintaining a fork until activity resumes:
https://pypi.org/project/pylibmc2/

Let's keep the discussion about what to do about pylibmc in general to the
[https://forum.djangoproject.com/t/should-we-stop-recommending-
pylibmc/42993 forum thread], but I take the point that with over a dozen
people finding it a nontrivial research project to figure out how to
install it, it's not a good sign.
----
Tim G's suggestion in comment:6 is a good compromise given our general
stance against vendoring install instructions for packages (#29405).
Contributors are unlikely to need the cache libraries, as the tests are
skipped anyway unless you add pylibmc in your settings file, and most
sprint contributors are using test_sqlite.py, which does not configure any
caches.

Kiko, are you up for a tiny docs tweak to suggest commenting out
requirements pertaining only to tests that do not run under the default
test_sqlite.py configuration, or something like that?

Or even supplementing the advice to search the web with "or the Django
forum"? (Here is the [https://forum.djangoproject.com/t/error-when-
installing-requirements-for-django-tests-fatal-error-libmemcached-
memcached-h-file-not-found-include-libmemcached-memcached-h/1685/8
solution] that ought to be marked as the "solution" IMO.) Sounds obvious,
but since this was not a Django issue, my first search did not include
"Django Forum", so I found mostly garbage. (Apparently I still Google like
it's 2015...)
--
Ticket URL: <https://code.djangoproject.com/ticket/37106#comment:10>

[Django]

#37106: pylibmc install fails from tests requirements on Python 3.12+ without
libmemcached installed
-------------------------------------+-------------------------------------
Reporter: Kiko Barr | Owner: Jason
Type: | Judkins
Cleanup/optimization | Status: assigned
Component: Documentation | Version: 6.0
Severity: Normal | Resolution:
Keywords: test, pylibmc | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Kiko Barr):

Definitely! I can make a PR for the tiny docs tweak this weekend. I will
try to link it here by Sunday night.
--
Ticket URL: <https://code.djangoproject.com/ticket/37106#comment:11>

[Django]

#37106: pylibmc install fails from tests requirements on Python 3.12+ without
libmemcached installed
-------------------------------------+-------------------------------------
Reporter: Kiko Barr | Owner: Jason
Type: | Judkins
Cleanup/optimization | Status: assigned
Component: Documentation | Version: 6.0
Severity: Normal | Resolution:
Keywords: test, pylibmc | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Tim Schilling):

> Kiko, are you up for a tiny docs tweak to suggest commenting out
requirements pertaining only to tests that do not run under the default
test_sqlite.py configuration, or something like that?

Shouldn't we comment it out in the requirements file to start and leave it
to the folks who need it to figure out what to install instead? I'm not
sure we should plan on this resolving soon.
--
Ticket URL: <https://code.djangoproject.com/ticket/37106#comment:12>

[Django]

#37106: pylibmc install fails from tests requirements on Python 3.12+ without
libmemcached installed
-------------------------------------+-------------------------------------
Reporter: Kiko Barr | Owner: Jason
Type: | Judkins
Cleanup/optimization | Status: assigned
Component: Documentation | Version: 6.0
Severity: Normal | Resolution:
Keywords: test, pylibmc | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Tim Graham):

Replying to [comment:12 Tim Schilling]:

> Shouldn't we comment it out in the requirements file to start and leave
it to the folks who need it to figure out what to install instead? I'm not
sure we should plan on this resolving soon.

CI uses that file to install dependencies, so that approach would create
some complication.
--
Ticket URL: <https://code.djangoproject.com/ticket/37106#comment:13>

[Django]

#37106: pylibmc install fails from tests requirements on Python 3.12+ without
libmemcached installed
-------------------------------------+-------------------------------------
Reporter: Kiko Barr | Owner: Jason
Type: | Judkins
Cleanup/optimization | Status: assigned
Component: Documentation | Version: 6.0
Severity: Normal | Resolution:
Keywords: test, pylibmc | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Tim Schilling):

Replying to [comment:13 Tim Graham]:

>
> CI uses that file to install dependencies, so that approach would create
some complication.

Good point. Would this be a case to introduce a `requirements-ci.txt`
file? If not, a docs change to account for it seems reasonable.
--
Ticket URL: <https://code.djangoproject.com/ticket/37106#comment:14>

[Django]

#37106: pylibmc install fails from tests requirements on Python 3.12+ without
libmemcached installed
-------------------------------------+-------------------------------------

Reporter: Kiko Barr | Owner: Kiko Barr
Type: | Status: assigned
Cleanup/optimization |

Component: Documentation | Version: 6.0
Severity: Normal | Resolution:
Keywords: test, pylibmc | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Kiko Barr):

* has_patch: 0 => 1
* owner: Jason Judkins => Kiko Barr

Comment:

Here is a PR I submitted with the proposed doc fix:
https://github.com/django/django/pull/21433

I’m happy to make edits based on feedback if we'd still like to go the
docs route.
--
Ticket URL: <https://code.djangoproject.com/ticket/37106#comment:15>

[Django]

#37106: pylibmc install fails from tests requirements on Python 3.12+ without
libmemcached installed
-------------------------------------+-------------------------------------
Reporter: Kiko Barr | Owner: Kiko Barr
Type: | Status: assigned
Cleanup/optimization |
Component: Documentation | Version: 6.0
Severity: Normal | Resolution:

Keywords: test, pylibmc | Triage Stage: Ready for
| checkin

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Jacob Walls):

* stage: Accepted => Ready for checkin

--
Ticket URL: <https://code.djangoproject.com/ticket/37106#comment:16>

[Django]

#37106: pylibmc install fails from tests requirements on Python 3.12+ without
libmemcached installed
-------------------------------------+-------------------------------------
Reporter: Kiko Barr | Owner: Kiko Barr

Type: | Status: closed

Cleanup/optimization |
Component: Documentation | Version: 6.0

Severity: Normal | Resolution: fixed

Keywords: test, pylibmc | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Jacob Walls <jacobtylerwalls@…>):

* resolution: => fixed
* status: assigned => closed

Comment:

In [changeset:"beb40ed1d2b0503814591a37b08ecbd71f2ff729" beb40ed]:
{{{#!CommitTicketReference repository=""
revision="beb40ed1d2b0503814591a37b08ecbd71f2ff729"
Fixed #37106 -- Clarified pylibmc workaround in unit test docs.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/37106#comment:17>

[Django]

#37106: pylibmc install fails from tests requirements on Python 3.12+ without
libmemcached installed
-------------------------------------+-------------------------------------
Reporter: Kiko Barr | Owner: Kiko Barr
Type: | Status: closed
Cleanup/optimization |
Component: Documentation | Version: 6.0
Severity: Normal | Resolution: fixed
Keywords: test, pylibmc | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Jacob Walls <jacobtylerwalls@…>):

In [changeset:"cbf06b3c1d92f03ef1450eaf6b970251ee985da6" cbf06b3c]:
{{{#!CommitTicketReference repository=""
revision="cbf06b3c1d92f03ef1450eaf6b970251ee985da6"
[6.1.x] Fixed #37106 -- Clarified pylibmc workaround in unit test docs.

Backport of beb40ed1d2b0503814591a37b08ecbd71f2ff729 from main.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/37106#comment:18>

[Django]

#37106: pylibmc install fails from tests requirements on Python 3.12+ without
libmemcached installed
-------------------------------------+-------------------------------------
Reporter: Kiko Barr | Owner: Kiko Barr
Type: | Status: closed
Cleanup/optimization |
Component: Documentation | Version: 6.0
Severity: Normal | Resolution: fixed
Keywords: test, pylibmc | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Jacob Walls <jacobtylerwalls@…>):

In [changeset:"5f102bb803d49c504adf94a738892c1b9d4c5c6a" 5f102bb]:
{{{#!CommitTicketReference repository=""
revision="5f102bb803d49c504adf94a738892c1b9d4c5c6a"
[6.0.x] Fixed #37106 -- Clarified pylibmc workaround in unit test docs.

Backport of beb40ed1d2b0503814591a37b08ecbd71f2ff729 from main.
}}}
--

Ticket URL: <https://code.djangoproject.com/ticket/37106#comment:19>