Improving packaging -> distutils backwards compatibility

[Erik Bray]

Greetings,

I know packaging has for a while been able to generate a setup.py file
using the pyset generate-setup action, but as it stands it's not
terribly useful for any but the simplest projects.

I've posted here before about how I wanted to start using packaging/
distutils2-style setup.cfgs with my projects without actually relying
on packaging being available yet. So what I came up with was a little
utility called 'd2to1' (http://pypi.python.org/pypi/d2to1).

The way it works (and yes, it requires distribute) is provided a very
simple boilerplate setup.py that has d2to1 as a setup_requires, it
knows how to read the setup.cfg and provide all the correct arguments
to the Distribution. It's a hack, but it works quite well and I've
been using it for a while for about 15 projects, a few of which are on
PyPI.

Since d2to1 relies heavily on distribute or setuptools I wouldn't
suggest trying to bring its exact functionality into packaging, but it
does have some features that might be able to work in the generate-
script action. These include support for:

* install_requires which, realistically speaking, is used by a lot of
projects
* C extensions
* setup_hooks
* pre/post-command hooks
* custom command classes
* custom compilers classes

All of these (with the exception of custom compilers) are features
used by some of the projects that I have using d2to1.

So my point is that it would be nice if the generate-setup action were
extended to support some of these features as well. I'd be happy to
work on a patch if this sounds like a good idea.

Erik

[Éric Araujo]

Hi Erik,

> [...]

> Since d2to1 relies heavily on distribute or setuptools I wouldn't
> suggest trying to bring its exact functionality into packaging, but it
> does have some features that might be able to work in the generate-
> script action. These include support for:
>
> * install_requires which, realistically speaking, is used by a lot of
> projects

We used to discuss these build/install dependencies every three months
without coming to an agreement. Before changing distutils2 we would
need to edit PEP 345. A temporary solution could be to translate
install_requires into Requires-Dist (i.e. runtime deps).

[new features for pysetup generate-setup]
> * C extensions
Sure, please open a feature request.

> * setup_hooks
> * pre/post-command hooks

You have setup_hooks in distutils/setuptools-style setup.py?

> * custom command classes
Please open a feature request. On the same subject, Tarek wants to
simplify command declaration and dependencies. At present we can set a
command class in setup.cfg, but I think the support for extending
commands (i.e. run this build_mo command during build) is not there yet
IIRC. Have a look at the list archives for a message by Tarek with
title “Static declaration for commands” or similar.

> * custom compilers classes
This should be easy to add to setup.cfg, thanks to
compiler.set_compiler. Please open a third report.

Thanks!

[Tarek Ziadé]

On Sat, Sep 3, 2011 at 12:12 AM, Éric Araujo <win...@netwok.org> wrote:
...

>
>> * custom compilers classes
> This should be easy to add to setup.cfg, thanks to
> compiler.set_compiler.  Please open a third report.

IIRC I've added a [compilers] section in setup.cfg for this

cheers

>
> Thanks!
>

--
Tarek Ziadé | http://ziade.org

[Erik Bray]

On Fri, Sep 2, 2011 at 11:12 AM, Éric Araujo <win...@netwok.org> wrote:
> Hi Erik,
>
>> [...]
>> Since d2to1 relies heavily on distribute or setuptools I wouldn't
>> suggest trying to bring its exact functionality into packaging, but it
>> does have some features that might be able to work in the generate-
>> script action.  These include support for:
>>
>> * install_requires which, realistically speaking, is used by a lot of
>> projects
> We used to discuss these build/install dependencies every three months
> without coming to an agreement.  Before changing distutils2 we would
> need to edit PEP 345.  A temporary solution could be to translate
> install_requires into Requires-Dist (i.e. runtime deps).

Remember the last discussion about that coming to something that
almost looked like an agreement. Nobody actually did the work of
offering an updated PEP though; something to think about--but sort of
a side issue.

As it is, my d2to1 tool treats Required-Dist as install_requires.

> [new features for pysetup generate-setup]
>> * C extensions
> Sure, please open a feature request.
>

Will do.

>> * setup_hooks
>> * pre/post-command hooks
> You have setup_hooks in distutils/setuptools-style setup.py?

Yup. And pre/post commands too. Here's an example of that working:
https://gist.github.com/1189618

It's a trivial example, but I also have projects using this with
non-trivial hooks.

>> * custom command classes
> Please open a feature request.  On the same subject, Tarek wants to
> simplify command declaration and dependencies.  At present we can set a
> command class in setup.cfg, but I think the support for extending
> commands (i.e. run this build_mo command during build) is not there yet
> IIRC.  Have a look at the list archives for a message by Tarek with
> title “Static declaration for commands” or similar.
>
>> * custom compilers classes
> This should be easy to add to setup.cfg, thanks to
> compiler.set_compiler.  Please open a third report.

Like Tarek said, this feature is already supported in packaging with
the [compilers] section. My point is that currently there's no way to
backport that feature to a setup.py-style install. d2to1 does it, but
admittedly it's an ugly hack (whereas the hook scripts implementation
is actually pretty straightforward).

I'll open a few feature requests and see if I can port some of the
existing features in d2to1 to work with pysetup generate-setup. If so
I'll post patches.

thanks,
Erik

[Éric Araujo]

Hi,

>>> * install_requires which, realistically speaking, is used by a lot of
>>> projects
>> We used to discuss these build/install dependencies every three months
>> without coming to an agreement. Before changing distutils2 we would
>> need to edit PEP 345. A temporary solution could be to translate
>> install_requires into Requires-Dist (i.e. runtime deps).
> Remember the last discussion about that coming to something that
> almost looked like an agreement.

I didn’t remember that, I’ll check my archive.

> Nobody actually did the work of offering an updated PEP though;
> something to think about--but sort of a side issue.

If we have consensus, we can work on a PEP patch and run it through
distutils-sig.

>>> * setup_hooks
>>> * pre/post-command hooks
>> You have setup_hooks in distutils/setuptools-style setup.py?
> Yup. And pre/post commands too. Here's an example of that working:
> https://gist.github.com/1189618

Ah, I was confused, I thought you were talking about
setup.py-to-setup.cfg conversion, not using-setup.cfg-in-setup.py.

Cheers