How to modularize for fun and profit, I: Namespace vs. distribution name

[Matthias Koeppe]

As we are making progress on preparing the Sage library for modularization (https://trac.sagemath.org/ticket/29705), here's a discussion of three things that need to be decided whenever a portion of the Sage library is modularized:

(1) Namespace: Will it be imported as (+) "from sage.hermeneutics.quantum import QuantumGravity" or (-) "from transformative_hermeneutics.quantum import QuantumGravity"?

(2) Distribution name (= project name): Will it be installed using "pip install transformative-hermeneutics" or "pip install sagemath-hermeneutics"?

(3) Source repository: Is the source maintained as part of the Sage repository using Trac tickets (MONOREPO); or in a separate repository on GitHub, GitLab, ... (MULTIREPO)?

This post is dedicated to the relation between (1) and (2).

There is no technical restriction that links (1) and (2). However, https://packaging.python.org/tutorials/packaging-projects/ provides some guidance. The informational PEP https://www.python.org/dev/peps/pep-0423/ ("Naming conventions and recipes related to packaging") is outdated, but the section https://www.python.org/dev/peps/pep-0423/#id87 still provides good additional guidance.

Note that the distribution name is not required to be a Python identifier. In fact, using dashes (-) is preferred to underscores in distribution names; setuptools and other parts of Python's packaging infrastructure normalize underscores to dashes. (Using dots in distribution names, to indicate ownership by organizations, still mentioned in https://www.python.org/dev/peps/pep-0423/, appears to have largely fallen out of favor.) 

Recommendation: A distribution that provides functionality that does not need to import anything from the sage namespace (from sage.PAC.KAGE.MODULE import ...) should not use the sage namespace for its packages/modules.

Examples (existing): 

- The distribution https://pypi.org/project/pplpy/ provides the Python package ppl [according to PEP 423, this should have been "pplpy"]. pplpy is a much extended version of what used to be sage.libs.ppl (https://trac.sagemath.org/ticket/23024), a part of the Sage library. sage.libs.ppl had dependencies on sage.rings to convert to/from Sage number types. pplpy has no such dependencies and is therefore usable in a wider range of Python projects.

- The distribution https://pypi.org/project/memory-allocator/ provides the Python package memory_allocator. This used to be sage.ext.memory_allocator, a part of the Sage library; it was split out in https://trac.sagemath.org/ticket/31591

Recommendation: A distribution that provides Python modules in the sage namespace (sage.PAC.KAGE.MODULE) should be named sagemath-DISTRI-BUTION. Other distributions should not use the prefix sagemath- in the distribution name.

Examples (existing): 

- The distribution https://pypi.org/project/sage-conf/ provides the Python module sage_conf.

- The distribution https://pypi.org/project/sage-sws2rst/ provides the Python package sage_sws2rst.

- The distribution https://pypi.org/project/sagemath-standard/ currently provides the entirety of the Python package sage (i.e., sage.categories, sage.combinat, sage.rings, ...) - representing the status quo before modularization.

- The distribution https://pypi.org/project/sagemath-categories/ provides a small subset of the modules of the Sage library, mostly from the Python packages sage.structure, sage.categories, and sage.misc (from https://trac.sagemath.org/ticket/29865, needs review)

- The distribution sagemath-polyhedra provides a self-contained subset of the modules of the Sage library for linear algebra and polyhedral geometry over ℚ, mostly from sage.rings, sage.modules, sage.matrix, sage.geometry (https://trac.sagemath.org/ticket/32432, under development)

- The distribution https://github.com/mkoeppe/sage-numerical-interactive-mip is a user package, not part of the Sage library, providing the Python package sage_numerical_interactive_mip.