Bug#1010284: python3-pip: runs into infinite loop when installing package with pyproject.toml file

Julian Gilbey

unread,

Apr 27, 2022, 3:30:04 PM4/27/22

to

Package: python3-pip
Version: 22.0.2+dfsg-1
Severity: important

I found this issue with my local Python project since I just upgraded
my testing system to python3 3.10.4-1, which now makes python3.10 the
default Python.

I then tested with this simple example repository:
https://github.com/pypa/sampleproject

Steps to reproduce:
1. Clone the above repository
2. Run `pip3 install --user .`

This is the result I observe:

Processing /<...>/sampleproject
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: peppercorn in /home/jdg/.local/lib/python3.10/site-packages (from sampleproject==2.0.0) (0.6)
Building wheels for collected packages: sampleproject
Building wheel for sampleproject (pyproject.toml) ... error
error: subprocess-exited-with-error

× Building wheel for sampleproject (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [2011 lines of output]
running bdist_wheel
running build
running build_py
running install
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/install.py", line 94, in _load_sysconfig_schemes
for scheme in sysconfig.get_scheme_names()
File "/usr/lib/python3.10/sysconfig.py", line 583, in get_scheme_names
return tuple(sorted(_INSTALL_SCHEMES))
RecursionError: maximum recursion depth exceeded while calling a Python object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
main()
File "/usr/lib/python3/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/usr/lib/python3/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 261, in build_wheel
return _build_backend().build_wheel(wheel_directory, config_settings,
File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 230, in build_wheel
return self._build_with_temp_dir(['bdist_wheel'], '.whl',
File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 215, in _build_with_temp_dir
self.run_setup()
File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 158, in run_setup
exec(compile(code, __file__, 'exec'), locals())
File "setup.py", line 20, in <module>
setup(
File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line 148, in setup
return run_commands(dist)
File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line 163, in run_commands
dist.run_commands()
File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 967, in run_commands
self.run_command(cmd)
File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 986, in run_command
cmd_obj.run()
File "/usr/lib/python3/dist-packages/wheel/bdist_wheel.py", line 335, in run
self.run_command('install')
File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 985, in run_command
cmd_obj.ensure_finalized()
File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 107, in ensure_finalized
self.finalize_options()
File "/usr/lib/python3/dist-packages/setuptools/command/install.py", line 45, in finalize_options
orig.install.finalize_options(self)
File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/install.py", line 320, in finalize_options
self.finalize_unix()
File "/usr/lib/python3.10/_distutils_system_mod.py", line 59, in finalize_unix
super().finalize_unix()
File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/install.py", line 488, in finalize_unix
self.select_scheme("posix_prefix")
File "/usr/lib/python3.10/_distutils_system_mod.py", line 55, in select_scheme
super().select_scheme(name)
File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/install.py", line 522, in select_scheme
scheme = _load_schemes()[name]
File "/usr/lib/python3.10/_distutils_system_mod.py", line 137, in wrapped_load_schemes
_inject_headers(name, scheme)
File "/usr/lib/python3.10/_distutils_system_mod.py", line 125, in _inject_headers
scheme['headers'] = orig_install._load_schemes()['posix_prefix']['headers']
File "/usr/lib/python3.10/_distutils_system_mod.py", line 137, in wrapped_load_schemes
_inject_headers(name, scheme)

and then the last four lines repeat hundreds of times until:

File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/install.py", line 103, in _load_schemes
sysconfig_schemes = _load_sysconfig_schemes() or {}
File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/install.py", line 91, in _load_sysconfig_schemes
with contextlib.suppress(AttributeError):
File "/usr/lib/python3.10/contextlib.py", line 436, in __exit__
return exctype is not None and issubclass(exctype, self._exceptions)
RecursionError: maximum recursion depth exceeded in __subclasscheck__
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for sampleproject
Failed to build sampleproject
ERROR: Could not build wheels for sampleproject, which is required to install pyproject.toml-based projects

I get the same behaviour (obviously) with `python3.10 -m pip install
--user .` but `python3.9 -m pip install --user .` works fine:

$ python3.9 -m pip install --user .
Processing /<...>/sampleproject
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting peppercorn
Using cached peppercorn-0.6-py3-none-any.whl (4.8 kB)
Building wheels for collected packages: sampleproject
Building wheel for sampleproject (pyproject.toml) ... done
Created wheel for sampleproject: filename=sampleproject-2.0.0-py3-none-any.whl size=4225 sha256=2bc606f6e112f9a897f877416f8b471b9fdd9986e53fad88dafc2a11bbfdf6c1
Stored in directory: /tmp/pip-ephem-wheel-cache-xmhzpetx/wheels/81/80/ff/23c143a1a2de938d596153b57b2a71f1fb269f3e1007dab618
Successfully built sampleproject
Installing collected packages: peppercorn, sampleproject
Successfully installed peppercorn-0.6 sampleproject-2.0.0

So there is definitely some interaction of pip with python3.10 that is
not working well.

Deleting the pyproject.toml file allows pip to work with 3.10; this
file is minimal:

[build-system]
# These are the assumed default build requirements from pip:
# https://pip.pypa.io/en/stable/reference/pip/#pep-517-and-518-support
requires = ["setuptools>=43.0.0", "wheel"]
build-backend = "setuptools.build_meta"

I've not got further than that, unfortunately.

Best wishes,

Julian

-- System Information:
Debian Release: bookworm/sid
APT prefers stable-security
APT policy: (500, 'stable-security'), (500, 'testing')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 5.17.0-1-amd64 (SMP w/4 CPU threads; PREEMPT)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8) (ignored: LC_ALL set to en_GB.UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages python3-pip depends on:
ii ca-certificates 20211016
ii python3 3.10.4-1
ii python3-distutils 3.9.12-1
ii python3-setuptools 59.6.0-1.2
ii python3-wheel 0.37.1-2

Versions of packages python3-pip recommends:
ii build-essential 12.9
ii python3-dev 3.10.4-1

python3-pip suggests no packages.

-- no debconf information

John Kyle Cronan

unread,

May 3, 2022, 2:30:03 PM5/3/22

to

I've run into this same problem, attempting to build the "pikepdf" package in a Docker image derived from debian:bookworm.

There weren't any changes to the package code, when it started happening.

The output looks like this:

#10 81.21
#10 81.21 EnvCommandError
#10 81.21
#10 81.21 Command ['/usr/bin/python3', '-m', 'pip', 'install', '--no-deps', '/root/.cache/pypoetry/artifacts/b7/d3/d1/a...7/pikepdf-5.1.2.tar.gz'] errored with the following return code 1, and output:
#10 81.21 Processing /root/.cache/pypoetry/artifacts/b7/d3/d1/a...7/pikepdf-5.1.2.tar.gz
#10 81.21 Installing build dependencies: started
#10 81.21 Installing build dependencies: finished with status 'done'
#10 81.21 Getting requirements to build wheel: started
#10 81.21 Getting requirements to build wheel: finished with status 'done'
#10 81.21 Preparing metadata (pyproject.toml): started
#10 81.21 Preparing metadata (pyproject.toml): finished with status 'done'
#10 81.21 Building wheels for collected packages: pikepdf
#10 81.21 Building wheel for pikepdf (pyproject.toml): started
#10 81.21 Building wheel for pikepdf (pyproject.toml): finished with status 'error'
#10 81.21 error: subprocess-exited-with-error
#10 81.21
#10 81.21 × Building wheel for pikepdf (pyproject.toml) did not run successfully.
#10 81.21 │ exit code: 1
#10 81.21 ╰─> [2059 lines of output]

...

[compiler output w/ no errors]

#10 81.21 running install
#10 81.21 Traceback (most recent call last):
#10 81.21 File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/install.py", line 94, in _load_sysconfig_schemes
#10 81.21 for scheme in sysconfig.get_scheme_names()
#10 81.21 File "/usr/lib/python3.10/sysconfig.py", line 583, in get_scheme_names
#10 81.21 return tuple(sorted(_INSTALL_SCHEMES))
#10 81.21 RecursionError: maximum recursion depth exceeded while calling a Python object
#10 81.21
#10 81.21 During handling of the above exception, another exception occurred:

#10 81.21
#10 81.21 Traceback (most recent call last):
#10 81.21 File "/usr/lib/python3/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
#10 81.21 main()
#10 81.21 File "/usr/lib/python3/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
#10 81.21 json_out['return_val'] = hook(**hook_input['kwargs'])
#10 81.21 File "/usr/lib/python3/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 261, in build_wheel
#10 81.21 return _build_backend().build_wheel(wheel_directory, config_settings,
#10 81.21 File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 230, in build_wheel
#10 81.21 return self._build_with_temp_dir(['bdist_wheel'], '.whl',
#10 81.21 File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 215, in _build_with_temp_dir
#10 81.21 self.run_setup()
#10 81.21 File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 158, in run_setup
#10 81.21 exec(compile(code, __file__, 'exec'), locals())
#10 81.21 File "setup.py", line 77, in <module>
#10 81.21 setup(
#10 81.21 File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 153, in setup
#10 81.21 return distutils.core.setup(**attrs)
#10 81.21 File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line 148, in setup
#10 81.21 return run_commands(dist)
#10 81.21 File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line 163, in run_commands
#10 81.21 dist.run_commands()

#10 81.21 File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 967, in run_commands
#10 81.21 self.run_command(cmd)
#10 81.21 File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 986, in run_command
#10 81.21 cmd_obj.run()
#10 81.21 File "/usr/lib/python3/dist-packages/wheel/bdist_wheel.py", line 335, in run
#10 81.21 self.run_command('install')
#10 81.21 File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 313, in run_command
#10 81.21 self.distribution.run_command(command)
#10 81.21 File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 985, in run_command
#10 81.21 cmd_obj.ensure_finalized()
#10 81.21 File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 107, in ensure_finalized
#10 81.21 self.finalize_options()
#10 81.21 File "/usr/lib/python3/dist-packages/setuptools/command/install.py", line 45, in finalize_options
#10 81.21 orig.install.finalize_options(self)
#10 81.21 File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/install.py", line 320, in finalize_options
#10 81.21 self.finalize_unix()
#10 81.21 File "/usr/lib/python3.10/_distutils_system_mod.py", line 59, in finalize_unix
#10 81.21 super().finalize_unix()
#10 81.21 File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/install.py", line 488, in finalize_unix
#10 81.21 self.select_scheme("posix_prefix")
#10 81.21 File "/usr/lib/python3.10/_distutils_system_mod.py", line 55, in select_scheme
#10 81.21 super().select_scheme(name)
#10 81.21 File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/install.py", line 522, in select_scheme
#10 81.21 scheme = _load_schemes()[name]

#10 81.21 File "/usr/lib/python3.10/_distutils_system_mod.py", line 137, in wrapped_load_schemes
#10 81.21 _inject_headers(name, scheme)
#10 81.21 File "/usr/lib/python3.10/_distutils_system_mod.py", line 125, in _inject_headers
#10 81.21 scheme['headers'] = orig_install._load_schemes()['posix_prefix']['headers']

... [it's the same lines that repeat] ...

#10 81.21 File "/usr/lib/python3.10/_distutils_system_mod.py", line 135, in wrapped_load_schemes
#10 81.21 schemes = _load_schemes()
#10 81.21 File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/install.py", line 103, in _load_schemes
#10 81.21 sysconfig_schemes = _load_sysconfig_schemes() or {}
#10 81.21 File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/install.py", line 91, in _load_sysconfig_schemes
#10 81.21 with contextlib.suppress(AttributeError):
#10 81.21 File "/usr/lib/python3.10/contextlib.py", line 436, in __exit__
#10 81.21 return exctype is not None and issubclass(exctype, self._exceptions)
#10 81.21 RecursionError: maximum recursion depth exceeded in __subclasscheck__
#10 81.21 [end of output]
#10 81.21
#10 81.21 note: This error originates from a subprocess, and is likely not a problem with pip.
#10 81.21 ERROR: Failed building wheel for pikepdf
#10 81.21 Failed to build pikepdf
#10 81.21 ERROR: Could not build wheels for pikepdf, which is required to install pyproject.toml-based projects

#10 81.21
#10 81.21
#10 81.21 at /usr/local/lib/python3.10/dist-packages/poetry/utils/env.py:1195 in _run

Initially, a couple weeks ago, I searched for the error message and found nothing. I didn't have time to open an issue, and wasn't sure where the responsibility lies, so I waited. Now there's this report, so I'm here! Happy to help how I can. Pikepdf is a dependency that I use for a project built for amd64 and arm64 architectures. I'm on 'testing' so that everything is recent enough to build for both.

Neil Tallim

unread,

May 9, 2022, 1:00:03 AM5/9/22

to

I've encountered the same issue when attempting to install the aiohttp package or anything that depends on it, which seems to be a lot.

File "/usr/lib/python3.10/_distutils_system_mod.py", line 125, in _inject_headers
scheme['headers'] = orig_install._load_schemes()['posix_prefix']['headers']
File "/usr/lib/python3.10/_distutils_system_mod.py", line 137, in wrapped_load_schemes
_inject_headers(name, scheme)
File "/usr/lib/python3.10/_distutils_system_mod.py", line 125, in _inject_headers
scheme['headers'] = orig_install._load_schemes()['posix_prefix']['headers']

File "/usr/lib/python3.10/_distutils_system_mod.py", line 135, in wrapped_load_schemes
schemes = _load_schemes()

File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/install.py", line 103, in _load_schemes
sysconfig_schemes = _load_sysconfig_schemes() or {}
File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/install.py", line 91, in _load_sysconfig_
schemes
with contextlib.suppress(AttributeError):

RecursionError: maximum recursion depth exceeded

[end of output]

Failed to build aiohttp

ERROR: Could not build wheels for aiohttp, which is required to install pyproject.toml-based projects

Julien Palard

unread,

May 13, 2022, 3:50:04 AM5/13/22

to

It can be reproduced with a mostly empty project:

$ ls ~/.local/lib/python3.10 # I have no locally installed packages:
ls: impossible d'accéder à '/home/mdk/.local/lib/python3.10': Aucun fichier ou dossier de ce type

$ which python # My python is the system one:
/usr/bin/python

$ python --version # Yes I do use python-is-python3:
Python 3.10.4

$ ls # A very simple "project":
pyproject.toml youpi.py

$ cat pyproject.toml
[build-system]
requires = ["setuptools", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "test"
description = "Just a test"
requires-python = ">= 3.7"

[tool.setuptools]
py-modules = ["youpi"]
include-package-data = false

$ cat youpi.py
"coucou"

$ pip install .
[...]

RecursionError: maximum recursion depth exceeded while calling a Python object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

[...]

File "/usr/lib/python3.10/_distutils_system_mod.py", line 125, in _inject_headers
scheme['headers'] = orig_install._load_schemes()['posix_prefix']['headers']
File "/usr/lib/python3.10/_distutils_system_mod.py", line 137, in wrapped_load_schemes
_inject_headers(name, scheme)
File "/usr/lib/python3.10/_distutils_system_mod.py", line 125, in _inject_headers
scheme['headers'] = orig_install._load_schemes()['posix_prefix']['headers']
File "/usr/lib/python3.10/_distutils_system_mod.py", line 137, in wrapped_load_schemes
_inject_headers(name, scheme)
File "/usr/lib/python3.10/_distutils_system_mod.py", line 125, in _inject_headers
scheme['headers'] = orig_install._load_schemes()['posix_prefix']['headers']
File "/usr/lib/python3.10/_distutils_system_mod.py", line 137, in wrapped_load_schemes
_inject_headers(name, scheme)
File "/usr/lib/python3.10/_distutils_system_mod.py", line 125, in _inject_headers
scheme['headers'] = orig_install._load_schemes()['posix_prefix']['headers']

File "/usr/lib/python3.10/_distutils_system_mod.py", line 135, in wrapped_load_schemes
schemes = _load_schemes()

File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/install.py", line 103, in _load_schemes
sysconfig_schemes = _load_sysconfig_schemes() or {}
File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/install.py", line 91, in _load_sysconfig_schemes
with contextlib.suppress(AttributeError):
File "/usr/lib/python3.10/contextlib.py", line 436, in __exit__
return exctype is not None and issubclass(exctype, self._exceptions)
RecursionError: maximum recursion depth exceeded in __subclasscheck__
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.

ERROR: Failed building wheel for UNKNOWN
Failed to build UNKNOWN
ERROR: Could not build wheels for UNKNOWN, which is required to install pyproject.toml-based projects

--
Julien Palard
https://mdk.fr

Stefano Rivera

unread,

May 15, 2022, 6:10:04 PM5/15/22

to

Control: reassign -1 libpython3.10-stdlib
Control: found -1 libpython3.10-stdlib/3.10.4-1
Control: fixed -1 libpython3.10-stdlib/3.10.4-2

Hi Zachary (2022.05.15_15:35:09_-0400)
> Should the bug be filed against libpython3.10-stdlib since the infinite loop
> seems to occur within the logic of
> /usr/lib/python3.10/_distutils_system_mod.py?

Yes. And it is already resolved there, in 3.10.4-2, which hasn't
migrated to testing, yet.

SR

--
Stefano Rivera
http://tumbleweed.org.za/
+1 415 683 3272