PEP x: Static module/package inspection
anatoly techtonik
anatoly techtonik
Michael Foord
On a simple level, all of this is already "obtainable" by using the ast module that can parse Python code. I would love to see a "python-object" layer on top of this that will take an ast for a module (or other object) and return something that represents the same object as the ast.
So all module level objects will have corresponding objects - where they are Python objects (builtin-literals) then they will represented exactly. For classes and functions you'll get an object back that has the same attributes plus some metadata (e.g. for functions / methods what arguments they take etc).
That is certainly doable and would make introspecting-without-executing a lot simpler.
I think your specific use cases are better served by adding functionality to the packaging (distutils2) package however. I'd particularly like to see plugin support in packaging (a cutdown version of setuptools entry points).
All the best,
Michael
_______________________________________________
Python-ideas mailing list
Python...@python.org
http://mail.python.org/mailman/listinfo/python-ideas
--
http://www.voidspace.org.uk/
May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html
Nathan Rice
> user story PEPx.001:
> As a Python package maintainer, I find it bothersome to repeatedly write
> bolierplate code (e.g. setup.py) to package my single file module. The
> reason I should write setup.py is to provide version and description info.
> This info is already available in my module source code. So I need to either
> copy/paste the info from the module manually, or to import (and hence
> execute) my module during packaging and installation, which I don't want
> either, because modules are often installed with root privileges.
I agree this is a pain. I also agree with Micheal that this is moreso
a packager issue. Part of the problem is that I don't believe there
is a strong enough convention around writing modules with an eye to
being package tools accessible. If there was a PEP on module metadata
for packaging tools to use for introspection, that might motivate
package tool authors to support automated packaging :) *HINT HINT*
Sphinx could also take advantage of some of it too.
> With this PEP, packing tool will be able to extract meta information from my
> module without executing it or without me manually copying version fields
> into some 'package configuration file'.
>
> user story PEPx.002:
> As a Python Application developer, I find it really complicated to provide
> plugin extension subsystem for my users. Users need a mechanism to switch
> between different versions of the plugin, and this mechanism is usually
> provided by external tool such as setuptools to manage and install multiple
> versions of plugins in local Python package repository. It is rather hard to
> create an alternative approach, because you are forced to maintain external
> meta-data about your plugin modules even in case it is already available
> inside the module.
See above. Maintaining the same information twice is definitely a bad
thing, but we already have the ability to do everything required.
What is missing is good, strong conventions on module metadata
annotation that tool creators write to.
> With this PEP, Python Application will be able to inspect meta-data embedded
> inside of plugins before choosing which version to load. This will also
> provide a standard mechanism for applications to check modules returned by
> packaging tools without executing them. This will greatly simplify writing
> and debugging custom plugins loaders on different platforms.
Having more nuanced import behavior is something I can get behind.
Sure, I can wrap an import in a try except, and check the __version__
if is defined (after determining if it is a string/tuple/etc, and
possibly parsing it), but more nuanced behavior would certainly be
nice. Being able to specify version in the import line (and have
multiple versions installed), being able to get fine grained exception
beyond ImportError (ParseError, anyone?), not having to worry that the
same file is being imported twice, that sort of stuff.
I'm +1 getting a module level metadata conventions PEP draft started.
I'm also +1 on taking a look at import behavior (though that is
tangential here).
Nathan
anatoly techtonik
anatoly t.
Nick Coghlan
> On a simple level, all of this is already "obtainable" by using the ast
> module that can parse Python code. I would love to see a "python-object"
> layer on top of this that will take an ast for a module (or other object)
> and return something that represents the same object as the ast.
>
> So all module level objects will have corresponding objects - where they are
> Python objects (builtin-literals) then they will represented exactly. For
> classes and functions you'll get an object back that has the same attributes
> plus some metadata (e.g. for functions / methods what arguments they take
> etc).
>
> That is certainly doable and would make introspecting-without-executing a
> lot simpler.
The existing 'clbr' (class browser) module in the stdlib also attempts
to play in this same space. I wouldn't say it does it particularly
*well* (since it's easy to confuse with valid Python constructs), but
it tries.
Cheers,
Nick.
--
Nick Coghlan | ncog...@gmail.com | Brisbane, Australia
anatoly techtonik
> On Thu, Dec 29, 2011 at 1:28 AM, Michael Foord <fuzz...@gmail.com> wrote:
>> On a simple level, all of this is already "obtainable" by using the ast
>> module that can parse Python code. I would love to see a "python-object"
>> layer on top of this that will take an ast for a module (or other object)
>> and return something that represents the same object as the ast.
>>
>> So all module level objects will have corresponding objects - where they are
>> Python objects (builtin-literals) then they will represented exactly. For
>> classes and functions you'll get an object back that has the same attributes
>> plus some metadata (e.g. for functions / methods what arguments they take
>> etc).
>>
>> That is certainly doable and would make introspecting-without-executing a
>> lot simpler.
>
> The existing 'clbr' (class browser) module in the stdlib also attempts
> to play in this same space. I wouldn't say it does it particularly
> *well* (since it's easy to confuse with valid Python constructs), but
> it tries.
Unfortunately http://docs.python.org/library/pyclbr.html misses info
about variables.
In the meanwhile I've patches my `astdump` module even further:
- function to query top level variables changed from get_top_vars() to
top_level_vars(), which is now accepts filename as a parameter. Now it
will be even more convenient to use it for generating `setup.py` for
simple modules. Sample `setup.py` generator is included.
http://pypi.python.org/pypi/astdump/1.0
--
anatoly t.