Q on naming nested packages/modules
kj
I'm having a hard time getting the hang of Python's package/module
scheme. I'd like to find out what's considered best practice when
dealing with the scenario illustrated below.
The quick description of the problem is: how can I have two nested
modules, spam.ham and spam.ham.eggs?
Suppose I have a module (I'm not even sure this is the right word)
called spam.ham, so I start out with the following file structure:
spam/
|-- ham.py
`-- __init__.py
With this arrangement, the line
import spam.ham
...in client code works as expected.
But now suppose that I want to factor out some code in spam/ham.py
to a helper module. (The reason behind factoring out this new
module is to "declutter" spam/ham.py, and improve its readibility.)
My instinct (from my Perl past) is to put this factored-out code
in a file spam/ham/eggs.py, i.e. to create the "nested" module
spam.ham.eggs, which requires expanding the tree as follows
spam/
|-- ham/
| |-- eggs.py
| `-- __init__.py
|-- ham.py
`-- __init__.py
...and adding the following spam/ham.py to
# spam/ham.py
from . import eggs
This doesn't work so well, because now spam/ham.py is not read.
It seems that adding the spam/ham directory, or maybe adding the
file spam/ham/__init__.py, causes spam/ham.py to be overlooked.
Clearly, I'm not playing this game right...
What is considered "best practice" for the use case sketched above?
Should I, e.g. rename the directory spam/ham something like spam/ham_
and refer to the helper module as spam.ham_.eggs? Or is some other
convention preferred?
I consulted PEP 8, but besides recommending "short, all-lowercase
names" for modules, it gives little guidance on the situation
described above.
TIA!
kynn
Benjamin Kaplan
Take everything in ham.py and stick it in ham/__init__.py instead.
This will give you the behavior you're looking for (don't import
spam.ham.__init__. Everything in __init__.py is loaded into spam.ham)
> TIA!
>
> kynn
> --
> http://mail.python.org/mailman/listinfo/python-list
>
kj
>I'm having a hard time getting the hang of Python's package/module
>scheme. I'd like to find out what's considered best practice when
>dealing with the scenario illustrated below.
>The quick description of the problem is: how can I have two nested
>modules, spam.ham and spam.ham.eggs?
Following up my own post...
From inspecting the directory structure of some of the standard
Python modules I infer the following rules:
1. the source for "leaf" modules lives in files named after them
(e.g. if x.y.z is a "leaf" module, its source code is in x/y/z.py)
2. the source for "non-leaf" modules lives in files named __init__.py
(e.g. if x.y is a "non-leaf" module, its source code lives in
the file x/y/__init__.py)
In the examples above, the module x.y is a "non-leaf" module because
there is a module x.y.z.
I.e. the "leaf"-ness of a module depends solely on whether other
modules deeper in the hierarchy are present.
An implication of all this is that if now I wanted to create a new
module x.y.z.w, this means that the previously "leaf"-module x.y.z
would become "non-leaf". In other words, I'd have to:
1. create the new directory x/y/z
2. *rename* the file x/y/z.py to x/y/z/__init__.py
3. create the file x/y/z/w.py to hold the source for the new x.y.z.w
module
Is the above correct? (BTW, to my Perl-pickled brain, step 2 above
is the one that causes most distress... But I think I can cope.)
kynn
Carl Banks
There's a hint here.
You can move the contents of ham.py into ham/__init__.py, and it'll
work the way you want, maybe with only a minor change or two.
> Clearly, I'm not playing this game right...
>
> What is considered "best practice" for the use case sketched above?
> Should I, e.g. rename the directory spam/ham something like spam/ham_
> and refer to the helper module as spam.ham_.eggs? Or is some other
> convention preferred?
The way you were intending is often the approach people use. I doubt
there are any detailed consensus recommendations about this in the
Python community.
> I consulted PEP 8, but besides recommending "short, all-lowercase
> names" for modules, it gives little guidance on the situation
> described above.
Of course it doesn't, PEP 8 is a style guide, not a project
organization guide.
Carl Banks
Ethan Furman
Looking at the layout of the (most excellent!-) xlrd package, the bulk
of the code is in the __init__.py file, and other supporting code is the
same directory, accessed in __init__ with normal imports.
I also am unclear on when it's best to have supporting files in the same
directory versus a subdirectory... perhaps it is that flat is better
than nested?
~Ethan~
Terry Reedy
>
>
> But now suppose that I want to factor out some code in spam/ham.py
> to a helper module. (The reason behind factoring out this new
> module is to "declutter" spam/ham.py, and improve its readibility.)
> My instinct (from my Perl past) is to put this factored-out code
> in a file spam/ham/eggs.py, i.e. to create the "nested" module
> spam.ham.eggs, which requires expanding the tree as follows
>
> spam/
> |-- ham/
> | |-- eggs.py
> | `-- __init__.py
> |-- ham.py
> `-- __init__.py
>
> ...and adding the following spam/ham.py to
>
> # spam/ham.py
> from . import eggs
>
> This doesn't work so well, because now spam/ham.py is not read.
> It seems that adding the spam/ham directory, or maybe adding the
> file spam/ham/__init__.py, causes spam/ham.py to be overlooked.
>
>
> Clearly, I'm not playing this game right...
One way is the __init__.py answer you have already. Another is to put
eggs.py (or _eggs.py to indicate that it is private) in spam and skip
the ham directory.
spam/
__init__.py
ham.py
_eggs.py
tjr
Rami Chowdhury
> module x.y.z.w, this means that the previously "leaf"-module x.y.z
> would become "non-leaf". In other words, I'd have to:
>
> 1. create the new directory x/y/z
> 2. *rename* the file x/y/z.py to x/y/z/__init__.py
> 3. create the file x/y/z/w.py to hold the source for the new x.y.z.w
> module
With regard to point 2 -- would it be possible to just move z.py into
x/y/z, and put 'from z import *' into x/y/z/__init__.py, for the same
effect? Or is that not a good idea?
--
Rami Chowdhury
"Never attribute to malice that which can be attributed to stupidity" --
Hanlon's Razor
408-597-7068 (US) / 07875-841-046 (UK) / 0189-245544 (BD)
kj
>> An implication of all this is that if now I wanted to create a new
>> module x.y.z.w, this means that the previously "leaf"-module x.y.z
>> would become "non-leaf". In other words, I'd have to:
>>
>> 1. create the new directory x/y/z
>> 2. *rename* the file x/y/z.py to x/y/z/__init__.py
>> 3. create the file x/y/z/w.py to hold the source for the new x.y.z.w
>> module
>With regard to point 2 -- would it be possible to just move z.py into
>x/y/z, and put 'from z import *' into x/y/z/__init__.py, for the same
>effect? Or is that not a good idea?
I think in this case what you would need is something like 'from
..z import *', but this does not work either; one gets the error
"SyntaxError: 'import *' not allowed with 'from .'".
A third solution would be to change step 2 above to this:
2. create the *symlink* x/y/z/__init__.py --> ../z.py
This certainly would make things clearer-looking to me. Having
all these files called the same thing, __init__.py, but having
completely different meaning, makes my head hurt... In contrast,
the __init__.py *symlinks* would be there just to keep python happy;
the real content is where I'd expect it.
I ran a simple test of this idea and it works, but there may be
some subtle reasons for not doing it... Or maybe it's not a good
idea simply because it is not the Python way, and I'd agree that
this would be reason enough to avoid it.
kynn