Python packaging with meson?

[Daniel Holth]

I'm curious about what it might take to build Python packages and extension modules with Meson. I've done it in SCons, and the Python-packaging-specific parts are easy. The process would seem to have a few requirements.

- 'Building' .py by copying

- Building shared modules in-place: during development, "import mylib._extension" expects the .so to exist in the mylib/ directory

- Generating metadata

- Creating .tar.gz and .zip files of the source and built artifacts. Archives with metadata and a specific directory structure are required for PyPI upload

Maybe a few archiving and metadata generation tools would need to be written to be invoked by ninja, and the 'install' functions could copy files into the right directory structure into a build directory that is later archived. How would you do it?

Thanks,

Daniel Holth

[Jussi Pakkanen]

On Wed, Dec 7, 2016 at 5:46 PM, Daniel Holth <dho...@gmail.com> wrote:

> I'm curious about what it might take to build Python packages and extension
> modules with Meson. I've done it in SCons, and the Python-packaging-specific
> parts are easy. The process would seem to have a few requirements.
>
> - 'Building' .py by copying
> - Building shared modules in-place: during development, "import
> mylib._extension" expects the .so to exist in the mylib/ directory
> - Generating metadata
> - Creating .tar.gz and .zip files of the source and built artifacts.
> Archives with metadata and a specific directory structure are required for
> PyPI upload

We already have a few test cases to build extensions and run them with
unit tests:

https://github.com/mesonbuild/meson/tree/master/test%20cases/python3

Built modules do not need to be in the current dir as long as
PYTHONPATH is set correctly (for example see how unit tests do it).

Install is missing but that is just a case of setting a few install
directives and install_subdirs. We have not looked into PyPI
integration because I have assumed that any packages uploaded there
would need to build with distutils/setuptools (there is at least one
SCons Python module thing that specifically generates setup.py files).
If this is not necessary but instead it is enough to produce a wheel
in a specific format then I guess we could look into providing that.

[Daniel Holth]

It is enough to generate a wheel to upload to PyPI. That is how https://pypi.python.org/pypi/flit works. https://pypi.python.org/pypi/enscons goes a little further by trying to emulate setup.py, but this is only needed if you are building from source rather than just running the wheel.

In the future setup.py emulation will not be necessary. There is a proposal in pip for pyproject.toml that would replace setup.py, allow dependencies to be installed before setup.py or an alternative build system runs, and provide a pluggable interface to build systems.

Thanks!