Setup.cfg Find

[Crisoforo Schuhmacher]

Ifcompatibility with legacy builds (i.e. those not using the PEP 517build API) is desired, a setup.py file containing a setup() functioncall is still required even if your configuration resides in setup.cfg.

In this example, the value for the package_dirconfiguration (i.e. =src) is parsed as "": "src".The "" key has a special meaning in this context, and indicates that all thepackages are contained inside the given directory.Also note that the value for [options.packages.find] where matches thevalue associated with "" in the package_dir dictionary.

It is advisable to use literal values together with attr: (e.g. str,tuple[str], see ast.literal_eval()). This is recommendin order to support the common case of a literal value assigned to a variablein a module containing (directly or indirectly) third-party imports.

If you are using an old version of setuptools, you might need to ensurethat all files referenced by the file: directive are included in the sdist(you can do that via MANIFEST.in or using plugins such as setuptools-scm,please have a look on Controlling files in the distribution for more information).

The find: and find_namespace: directive can be further configuredin a dedicated subsection options.packages.find. This subsection accepts thesame keys as the setuptools.find_packages and thesetuptools.find_namespace_packages function:where, include, and exclude.

The Python Packaging Tutorial recommends that "Static metadata (setup.cfg) should be preferred. Dynamic metadata (setup.py) should be used only as an escape hatch when absolutely necessary. setup.py used to be required, but can be omitted with newer versions of setuptools and pip."

The guide to packaging and distributing projects explains that "setup.cfg is an ini file that contains option defaults for setup.py commands. For an example, see the setup.cfg in the PyPA sample project."

That example is entirely useless, and there doesn't appear to be a lot of other helpful information. The examples in the tutorial suggest that some, or perhaps all, valid arguments to setuptools.setup() can be listed in setup.cfg, but there is no real explanation to this effect. In particular, it's not clear how to translate a list argument, like that for the very common and important install_requires parameter, into lines in setup.cfg.

so far, using the setuptools find_packages() function allowed me to add my main package as well as the subpackage to my build package. When using a setup.cfg file however, I am not able to include both in my package anymore. The documentation shows that subpackages can be included by adding the option

This actually works similarly to find_packages function inside a setup.py file. When called with no arguments, find_packages() in a project, it finds all of your top-level and sub-packages automatically.

This blog post is designed to clarify my thinking around installing Python packages and the use of setup.py, setup.cfg and pyproject.toml files. Hopefully it will be a useful reference for other people, and future me.

It is stimulated by my starting work on a new project where we have been discussing best practices in Python programming, and how to layout a git repository containing Python code. More broadly it is relevant to me as someone who programmes a lot in Python, mainly for my own local use, but increasingly for other people to consume my code. Prior to writing this the layout of my Python repositories was by a system of random inheritance dating back a number of years.

The minimal setup.py file simply contains an invocation of the setuptools setup function, if you do not intend to publish your project then no setup.py file is required at all, pip install -e . will work without it:

It is common and recommended practice to use virtual environments for work in Python. I use the Anaconda distribution of Python in which we setup and activate a virtual environment using the following, to be run before the pip install statement

The setup.py and setup.cfg files are artefacts of the setuptools module which is designed to help with the packaging process. It is used by pip whose purpose is to install a package either locally or remotely. If we do not configure setup.py/setup.cfg correctly then pip will not work. In the past we would have written a setup.py file which contained a bunch of configuration information but now we should put that configuration information into setup.cfg which is effectively an ini format file (i.e. does not need to be executed to be read). This is why we now have the minimal setup.py file.

The pyproject.toml file was introduced in PEP-518 (2016) as a way of separating configuration of the build system from a specific, optional library (setuptools) and also enabling setuptools to install itself without already being installed. Subsequently PEP-621 (2020) introduces the idea that the pyproject.toml file be used for wider project configuration and PEP-660 (2021) proposes finally doing away with the need for setup.py for editable installation using pip.

Tests go in a tests directory at the top-level of the project with an __init__.py file so they are discoverable by applications like pytest. The alternative of placing them inside the src/mypackage directory means they will get deployed into production which may not be desirable.

I've worked as a scientist for the last 30 years, at various universities, a large home and personal care company, a startup in Liverpool called The Sensible Code Company (formerly ScraperWiki Ltd), GBG and now as a consultant in data science.

I would like to add some datafiles to my PyPi package. This documentation for setup.cfg is aimed at someone more expert than me. It states the following without any explanation of what the elements are:

Static metadata ( setup.cfg ) should be preferred. Dynamic metadata ( setup.py ) should be used only as an escape hatch when absolutely necessary. setup.py used to be required, but can be omitted with newer versions of setuptools and pip.

The asterisk matches any package in your source tree that happens to contain a myapp.dat or myapp.html. Where are these files located in your repository? Package data only works if the files are actually located inside your Python package (the folder with the actual *.py files) to begin with.

My rationale, which is somewhat documented here, is that you currently only need setup.py (and depending on the project, a requirements.txt), pip and twine. However in the current packaging tutorial it is recommended that you use setup.py, setup.cfg and pyproject.toml, which is a lot of clutter, considering that you can achieve the same with just setup.py or apparently nowadays with only pyproject.toml. So my main goal is to simplify the recommendation to the minimum number of files required to get the job done.

Yes, I was talking specifically about Setuptools, not Flit. If the

suggestion is to switch the tutorial to recommend Flit with just a

pyproject.toml then that may work even for editable/develop

installs. If the idea is to stick with Setuptools in the tutorial,

then its not quite there yet (and not just because of editables, but

also because it clearly declares its pyproject.toml metadata API as

experimental right now).

I agree. Omitting the [build-system] table is only really for legacy purposes. PEP 517 makes it clear that if the table is missing, frontends should invoke setuptools using the setuptools.build_meta:__legacy__ backend, which is solely for backward compatibility (and I believe is subtly different to the real setuptools backend).

The reason is that the pytest team intends to fully utilize the rich TOML data formatfor configuration in the future, reserving the [tool.pytest] table for that.The ini_options table is being used, for now, as a bridge between the existing.ini configuration system and the future configuration format.

Usage of setup.cfg is not recommended unless for very simple use cases. .cfgfiles use a different parser than pytest.ini and tox.ini which might cause hard to trackdown problems.When possible, it is recommended to use the latter files, or pyproject.toml, to hold yourpytest configuration.

pytest determines a rootdir for each test run which depends onthe command line arguments (specified test files, paths) and onthe existence of configuration files. The determined rootdir and configfile areprinted as part of the pytest header during startup.

Is used by plugins as a stable location to store project/test run specific information;for example, the internal cache plugin creates a .pytest_cache subdirectoryin rootdir to store its cross-test run state.

The --rootdir=path command-line option can be used to force a specific directory.Note that contrary to other command-line options, --rootdir cannot be used withaddopts inside pytest.ini because the rootdir is used to find pytest.inialready.

Determine the common ancestor directory for the specified args that arerecognised as paths that exist in the file system. If no such paths arefound, the common ancestor directory is set to the current working directory.

If no setup.py was found, look for pytest.ini, pyproject.toml, tox.ini, andsetup.cfg in each of the specified args and upwards. If one ismatched, it becomes the configfile and its directory becomes the rootdir.

New in version 6.1: The config.rootpath and config.inipath properties. They are pathlib.Pathversions of the older config.rootdir and config.inifile, which have typepy.path.local, and still exist for backward compatibility.

Custom pytest plugin commandline arguments may include a path, as inpytest --log-output ../../test.log args. Then args is mandatory,otherwise pytest uses the folder of test.log for rootdir determination(see also issue #1435).A dot . for referencing to the current working directory is alsopossible.

3a8082e126