Figuring out library holes in the package installation process

91 views
Skip to first unread message

Brett Cannon

unread,
Mar 19, 2018, 10:39:43 PM3/19/18
to pypa-dev
With the big push we are all working towards standards, I thought it would be interesting to outline the steps it takes to go from requesting a package to be installed to it ending up on disk (see my first email on such an outline). After getting feedback on that outline, I then decided to try and pair each complicated step of that outline with the PEP and a library that essentially implements that step. I'm doing this to see where there might be a missing PEP and where that are missing library implementations of a specific PEP.

I've ended up with some holes on both the PEP and library side, so I'm hoping that either I'm overlooking something or that I've at least identified a place where some help might be appreciated by people.

  1. Specify package requirement (PEP 508/packaging)
    1. Translate name to PyPI-compatible name
    2. Tease out requirement details (e.g. version, markers, etc.)
  2. Check if package is already installed (PEP 376/XXX)
  3. Check wheel cache (XXX/XXX)
  4. Check PyPI for package
  5. Choose appropriate file
    1. Get list of files (PEP 503/_)
    2. Calculate best-fitting wheel (PEPs 425 & 427/XXX)
    3. Fallback to .tar.gz sdist if necessary & available
  6. Download file
  7. If sdist:
    1. Extract
    2. Read pyproject.toml (PEP 518/XXX)
    3. Create venv
    4. Install build dependencies
      1. Static (PEP 518/_)
      2. Dynamic (PEP 517/XXX)
    5. Build wheel (PEP 517/XXX)
  8. Cache wheel (XXX/XXX)
  9. Get dependency requirements of the wheel (PEP 566/XXX)
  10. Check if dependencies are already installed (like step 2)
  11. Install missing dependencies (like step 12 below)
  12. Install wheel (PEP 427/XXX)
    1. To appropriate location (e.g. user, venv, etc.)
    2. Library, scripts, etc. to appropriate locations
  13. Record the installation (PEP 427/XXX)

Brett Cannon

unread,
Apr 6, 2018, 8:35:08 PM4/6/18
to pypa-dev
Based on no one speaking up, I'm assuming I wasn't wrong with my outline. ;) That looks like there may be one PEP to write and a few things to add to 'packaging' (or other projects as appropriate).
Reply all
Reply to author
Forward
0 new messages