Thoughts about the implementation of pysetup
Éric Araujo
I have some ideas to improve the run module, a.k.a. pysetup.
As it is now, it suffers from a number of problems.
1) It has its own option parsing system which conflicts with the one
used by commands. As a result, “pysetup run build -h” gives the help of
run, not of build. This particular thing could be fixed, but a better
fix IMO would be to have only one command-line parsing system.
2) On a related note, run.Dispatcher duplicates a fair bit of code from
dist.Distribution. If we implement 1), we could rip off most of
Distribution. In d1, this class did everything; in d2, we have classes
like Dispatcher, Metadata, Manifest with more responsibility, and I
think it’s a good change. Maybe we could just replace Distribution with
Metadata + Files.
3) The worst difference between actions and commands is that command
modules are imported when needed, but functions implementing actions are
imported in the run module or in the action function wrapper. Some
actions functions are defined in run, other in another modules. I think
we should remove all definition of actions, options and help texts from
run and put them in other modules. This would be cleaner to read and
would allow lazy getting of actions, like what we do for commands and
compiler classes. To give pseudo-code:
# in create.py
@action([options..], 'help text')
def main(opts, args):
...
@action([options..], 'help text')
def generate_setup(opts, args):
...
# in run.py
actions = {'create': 'packaging.create.main',
'generate-setup': 'packaging.create.generate_setup'}
# + functions similar to set_compiler/get_command_class
3 can be implemented right now; Jeremy Kloth said on IRC that he’s
started something along these lines. 1 is trickier but needs to be
done. 2 follows 1.
Opinions?
michael mulich
> I have some ideas to improve the run module, a.k.a. pysetup.
>
Me too. I've been playing around with the run module for a few weeks a
few minutes per night before going to bed. I thought I'd speak up when
I had something significant to show. Attached is my patch thus far. I
won't be offended if it's declined since it was truly experimental.
> As it is now, it suffers from a number of problems.
>
> 1) It has its own option parsing system which conflicts with the one
> used by commands. As a result, “pysetup run build -h” gives the help of
> run, not of build. This particular thing could be fixed, but a better
> fix IMO would be to have only one command-line parsing system.
>
It was a few weeks ago that I noticed the double layer of deprecation
that we had in the docs first for fancyopts then for optparse.
If no one is opposed I'd like to suggest we use argparse for this
module, since (as far as I know) argparse is backwards compatible to
Python 2.4. Does anyone have strong opinions against it?
> 2) On a related note, run.Dispatcher duplicates a fair bit of code from
> dist.Distribution. If we implement 1), we could rip off most of
> Distribution. In d1, this class did everything; in d2, we have classes
> like Dispatcher, Metadata, Manifest with more responsibility, and I
> think it’s a good change. Maybe we could just replace Distribution with
> Metadata + Files.
>
IMO the Distribution class should know nothing and having nothing to
do with command line arguments. I never really understood why it was
command line oriented to begin with, but I'm sure there is something
I'm missing.
> 3) The worst difference between actions and commands is that command
> modules are imported when needed, but functions implementing actions are
> imported in the run module or in the action function wrapper. Some
> actions functions are defined in run, other in another modules. I think
> we should remove all definition of actions, options and help texts from
> run and put them in other modules. This would be cleaner to read and
> would allow lazy getting of actions, like what we do for commands and
> compiler classes.
In my changes I've made the action a class and the docstring the
action's help. This keeps the action and message together.
Additionally, I've put the argparser initialization in the action
class, so that the action could potentially be used without the
dispatcher, which is what is being done in the tests.
>
> 3 can be implemented right now; Jeremy Kloth said on IRC that he’s
> started something along these lines. 1 is trickier but needs to be
> done. 2 follows 1.
>
> Opinions?
>
I'm for anything, because the run module is a bit of a mess to read
(or write anything against it, in the case you wanted to extend
pysetup).
-Michael Mulich (pumazi)
Éric Araujo
Thanks for the reply.
> Me too. I've been playing around with the run module for a few weeks a
> few minutes per night before going to bed. I thought I'd speak up when
> I had something significant to show. Attached is my patch thus far. I
> won't be offended if it's declined since it was truly experimental.
Nice! For one thing, I’d really like your tests to be posted as a patch
for packaging on the bug tracker. It would be good to have them before
we change the implementation.
> It was a few weeks ago that I noticed the double layer of deprecation
> that we had in the docs first for fancyopts then for optparse.
>
> If no one is opposed I'd like to suggest we use argparse for this
> module, since (as far as I know) argparse is backwards compatible to
> Python 2.4. Does anyone have strong opinions against it?
Last summer, Tarek wanted to make fancy_getopt use argparse, but then
changed his mind in favor of optparse, which is included in 2.4+ and
will remove a bootstrapping issue (how to install argparse when the
installer needs argparse?). Another solution is of course to copy
argparse into distutils2/_backport, but I for one try to minimize the
amount of modules we put in that directory. optparse has a number of
problems that argparse addresses, but it’s still way better than getopt.
On the other hand, if we want lazy command-line parsing (i.e. to avoid
importing all actions and merging their options into one big option
parser), maybe we’ll have to write low-level code using shlex or getopt.
I don’t know yet.
> IMO the Distribution class should know nothing and having nothing to
> do with command line arguments. I never really understood why it was
> command line oriented to begin with, but I'm sure there is something
> I'm missing.
Probably just organic development: Distribution was the core of
distutils, with a few extra business in the setup function. Once most
of the command-line engine is in fancy_getopt and options are in
commands, one needs a place for global options, so why not Distribution?
> In my changes I've made the action a class and the docstring the
> action's help. This keeps the action and message together.
Hm. I’d like to see if we can keep the current use of functions; we can
move the help text from the decorator to the function docstring (and
having part of it generated from the options, for Guido’s sake!).
Distutils commands are classes for very good reasons: they have many
attributes and methods that are inherited or overridden to support the
multi-phase initialize/finalize/run process and to do their work. For
some reason, people far and wide seem to dislike having to write a class
to define a command; I don’t really understand why, given that a class
is nothing more than a name, a tuple of base classes and a dictionary of
attributes, but I can’t deny that people feel this way and that it has
created the false legend of “distutils is hard to extend”. If people
prefer functions and decorators and it does not make our life too hard,
then let’s use functions. What do you think?
Regards