Bug#1003252: AttributeError: install_layout

99 views
Skip to first unread message

Tomas Janousek

unread,
Jan 6, 2022, 7:40:03 PMJan 6
to

Package: python3-setuptools
Version: 58.2.0-1
Severity: normal
X-Debbugs-CC: debian...@lists.debian.org

Since setuptools 60+ is out with SETUPTOOLS_USE_DISTUTILS defaulting to "local", pip install --editable in --system-site-packages venvs fails:

$ docker run --rm -it debian:sid  
# apt update  
# apt install git python3-setuptools python3-pip python3-venv  
# cd /tmp  
# git clone https://github.com/platformdirs/platformdirs  
# cd platformdirs  
# python3 -m venv --system-site-packages --without-pip .venv  
# .venv/bin/python -m pip install -e .  
Obtaining file:///tmp/platformdirs  
  Installing build dependencies ... done  
  Getting requirements to build wheel ... done  
    Preparing wheel metadata ... done  
Installing collected packages: platformdirs  
  Running setup.py develop for platformdirs  
    ERROR: Command errored out with exit status 1:  
     command: /tmp/platformdirs/.venv/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/platformdirs/setup.py'"'"'; __file__='"'"'/tmp/platformdirs/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps  
         cwd: /tmp/platformdirs/  
    Complete output (30 lines):  
    running develop  
    Traceback (most recent call last):  
      File "<string>", line 1, in <module>  
      File "/tmp/platformdirs/setup.py", line 5, in <module>  
        setup()  
      File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 153, in setup  
        return distutils.core.setup(**attrs)  
      File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line 148, in setup  
        dist.run_commands()  
      File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 967, in run_commands  
        self.run_command(cmd)  
      File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 985, in run_command  
        cmd_obj.ensure_finalized()  
      File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 107, in ensure_finalized  
        self.finalize_options()  
      File "/usr/lib/python3/dist-packages/setuptools/command/develop.py", line 52, in finalize_options  
        easy_install.finalize_options(self)  
      File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 293, in finalize_options  
        self.set_undefined_options(  
      File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 287, in set_undefined_options  
        src_cmd_obj.ensure_finalized()  
      File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 107, in ensure_finalized  
        self.finalize_options()  
      File "/usr/lib/python3/dist-packages/setuptools/command/install_lib.py", line 17, in finalize_options  
        self.set_undefined_options('install',('install_layout','install_layout'))  
      File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 290, in set_undefined_options  
        setattr(self, dst_option, getattr(src_cmd_obj, src_option))  
      File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 103, in __getattr__  
        raise AttributeError(attr)  
    AttributeError: install_layout  
    ----------------------------------------  
ERROR: Command errored out with exit status 1: /tmp/platformdirs/.venv/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/platformdirs/setup.py'"'"'; __file__='"'"'/tmp/platformdirs/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.  

This happens even though setuptools 60 isn't in Debian yet, because pip downloads latest setuptools for pep517 installs that require setuptools in the build-system section of pyproject.yaml, but then fails to actually use that version fully (this is a bug in pip: https://github.com/pypa/pip/issues/6264).

I'll explain what's going on in detail further down, but first I'll present a simpler reproducer that illustrates why this might be a bug in Debian's setuptools packaging as well:

$ docker run --rm -it debian:sid  
# apt update  
# apt install git python3-setuptools python3-pip  
# cd /tmp  
# git clone https://github.com/platformdirs/platformdirs  
# cd platformdirs  
# SETUPTOOLS_USE_DISTUTILS=local python3 setup.py install  
running install  
Traceback (most recent call last):  
  …
  File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 103, in __getattr__  
    raise AttributeError(attr)  
AttributeError: install_layout  

Here we are explicitly using Debian's setuptools package, just telling it to use the embedded distutils instead of the one in stdlib. Turns out the distutils install-layout patch 1 is only applied to stdlib but not to this embedded distutils, triggering the issue. The setuptools install-layout patch 2 omits the distutils bits.

I believe that patching the setuptools copy of distutils as well should fix this issue, and might be a solution for Debian 11 (which is affected as well). Going forward (setuptools 60+), the install-layout patches will need to be adapted anyway (see https://github.com/pypa/distutils/issues/2
and https://github.com/pypa/setuptools/issues/2956), so this is a temporary measure.

...

Now for the gory details of why this exception occurs:

PEP 517 recommends that build frontends create an isolated environment with only the build-deps declared in pyproject.toml, and pip indeed attempts to do so. It creates /tmp/pip-build-env-XXX and installs the newest setuptools, setuptools_scm, wheel, … in there. It then sets up /tmp/pip-build-env-XXX/site/sitecustomize.py to drop system paths from sys.path and points PYTHONPATH to /tmp/pip-build-env-XXX/site.

Unfortunately due to https://github.com/pypa/pip/issues/6264, this doesn't work well with --system-site-packages venvs and leaves system paths visible, but /tmp/pip-build-env-XXX/overlay/lib/python3.9/site-packages/distutils-precedence.pth is still evaluated, causing the distutils bundled in setuptools to be used instead of stdlib. But these don't have the install-layout patch, so it fails.

Ideally https://github.com/pypa/pip/issues/6264 would be fixed and then Debian's patching of distutils/setuptools shouldn't affect this any more, but until that happens (and it's been open for almost 3 years) …

...

Oh and if anyone else stumbles upon this, a temporary workaround is to force SETUPTOOLS_USE_DISTUTILS=stdlib.

-- System Information:
Debian Release: bookworm/sid
APT prefers stable-security
APT policy: (990, 'stable-security'), (990, 'testing'), (500, 'unstable-debug'), (500, 'testing-debug'), (500, 'stable-debug'), (500, 'unstable'), (500, 'stable'), (1, 'experimental-debug'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 5.15.0-2-amd64 (SMP w/4 CPU threads)
Kernel taint flags: TAINT_CPU_OUT_OF_SPEC, TAINT_USER
Locale: LANG=cs_CZ.UTF-8, LC_CTYPE=cs_CZ.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages python3-setuptools depends on:
ii python3 3.9.7-1
ii python3-distutils 3.9.9-3
ii python3-pkg-resources 58.2.0-1

python3-setuptools recommends no packages.

Versions of packages python3-setuptools suggests:
pn python-setuptools-doc <none>

-- no debconf information

--

Tomáš "liskin" ("Pivník") Janoušek, https://lisk.in/

Stefano Rivera

unread,
Mar 20, 2022, 8:40:03 PMMar 20
to
Control: reassign -1 python3.10
Control: fixed -1 python3.10/3.10.2-6
Control: found -1 python3.10/3.10.2-3
Control: close -1

This is resolved by the addition of _distutils_system_mod in Python
3.10.

python3.9 hasn't been fixed yet, but it'll be replaced by 3.10, soon.

SR

--
Stefano Rivera
http://tumbleweed.org.za/
+1 415 683 3272

de...@sumpfralle.de

unread,
Mar 31, 2022, 10:40:03 AMMar 31
to
Hello,

thank you for fixing this issue for Python 3.10!

Do you think, that this issue is relevant enough for being fixed in Bullseye?
(as hinted by the bug submitter)

Thank you for your time!

Cheers,
Lars
Reply all
Reply to author
Forward
0 new messages