I'm having a hard time getting the hang of Python's package/module scheme. I'd like to find out what's considered best practice when dealing with the scenario illustrated below.
The quick description of the problem is: how can I have two nested modules, spam.ham and spam.ham.eggs?
Suppose I have a module (I'm not even sure this is the right word) called spam.ham, so I start out with the following file structure:
spam/ |-- ham.py `-- __init__.py
With this arrangement, the line
import spam.ham
...in client code works as expected.
But now suppose that I want to factor out some code in spam/ham.py to a helper module. (The reason behind factoring out this new module is to "declutter" spam/ham.py, and improve its readibility.) My instinct (from my Perl past) is to put this factored-out code in a file spam/ham/eggs.py, i.e. to create the "nested" module spam.ham.eggs, which requires expanding the tree as follows
This doesn't work so well, because now spam/ham.py is not read. It seems that adding the spam/ham directory, or maybe adding the file spam/ham/__init__.py, causes spam/ham.py to be overlooked.
Clearly, I'm not playing this game right...
What is considered "best practice" for the use case sketched above? Should I, e.g. rename the directory spam/ham something like spam/ham_ and refer to the helper module as spam.ham_.eggs? Or is some other convention preferred?
I consulted PEP 8, but besides recommending "short, all-lowercase names" for modules, it gives little guidance on the situation described above.
On Tue, Sep 1, 2009 at 11:58 AM, kj<no.em...@please.post> wrote:
> I'm having a hard time getting the hang of Python's package/module > scheme. I'd like to find out what's considered best practice when > dealing with the scenario illustrated below.
> The quick description of the problem is: how can I have two nested > modules, spam.ham and spam.ham.eggs?
> Suppose I have a module (I'm not even sure this is the right word) > called spam.ham, so I start out with the following file structure:
> spam/ > |-- ham.py > `-- __init__.py
> With this arrangement, the line
> import spam.ham
> ...in client code works as expected.
> But now suppose that I want to factor out some code in spam/ham.py > to a helper module. (The reason behind factoring out this new > module is to "declutter" spam/ham.py, and improve its readibility.) > My instinct (from my Perl past) is to put this factored-out code > in a file spam/ham/eggs.py, i.e. to create the "nested" module > spam.ham.eggs, which requires expanding the tree as follows
> This doesn't work so well, because now spam/ham.py is not read. > It seems that adding the spam/ham directory, or maybe adding the > file spam/ham/__init__.py, causes spam/ham.py to be overlooked.
> Clearly, I'm not playing this game right...
> What is considered "best practice" for the use case sketched above? > Should I, e.g. rename the directory spam/ham something like spam/ham_ > and refer to the helper module as spam.ham_.eggs? Or is some other > convention preferred?
> I consulted PEP 8, but besides recommending "short, all-lowercase > names" for modules, it gives little guidance on the situation > described above.
Take everything in ham.py and stick it in ham/__init__.py instead. This will give you the behavior you're looking for (don't import spam.ham.__init__. Everything in __init__.py is loaded into spam.ham)
In <h7jga8$ij...@reader1.panix.com> kj <no.em...@please.post> writes:
>I'm having a hard time getting the hang of Python's package/module >scheme. I'd like to find out what's considered best practice when >dealing with the scenario illustrated below. >The quick description of the problem is: how can I have two nested >modules, spam.ham and spam.ham.eggs?
Following up my own post...
From inspecting the directory structure of some of the standard Python modules I infer the following rules:
1. the source for "leaf" modules lives in files named after them (e.g. if x.y.z is a "leaf" module, its source code is in x/y/z.py)
2. the source for "non-leaf" modules lives in files named __init__.py (e.g. if x.y is a "non-leaf" module, its source code lives in the file x/y/__init__.py)
In the examples above, the module x.y is a "non-leaf" module because there is a module x.y.z.
I.e. the "leaf"-ness of a module depends solely on whether other modules deeper in the hierarchy are present.
An implication of all this is that if now I wanted to create a new module x.y.z.w, this means that the previously "leaf"-module x.y.z would become "non-leaf". In other words, I'd have to:
1. create the new directory x/y/z 2. *rename* the file x/y/z.py to x/y/z/__init__.py 3. create the file x/y/z/w.py to hold the source for the new x.y.z.w module
Is the above correct? (BTW, to my Perl-pickled brain, step 2 above is the one that causes most distress... But I think I can cope.)
> I'm having a hard time getting the hang of Python's package/module > scheme. I'd like to find out what's considered best practice when > dealing with the scenario illustrated below.
> The quick description of the problem is: how can I have two nested > modules, spam.ham and spam.ham.eggs?
> Suppose I have a module (I'm not even sure this is the right word) > called spam.ham, so I start out with the following file structure:
> spam/ > |-- ham.py > `-- __init__.py
> With this arrangement, the line
> import spam.ham
> ...in client code works as expected.
> But now suppose that I want to factor out some code in spam/ham.py > to a helper module. (The reason behind factoring out this new > module is to "declutter" spam/ham.py, and improve its readibility.) > My instinct (from my Perl past) is to put this factored-out code > in a file spam/ham/eggs.py, i.e. to create the "nested" module > spam.ham.eggs, which requires expanding the tree as follows
> This doesn't work so well, because now spam/ham.py is not read. > It seems that adding the spam/ham directory, or maybe adding the > file spam/ham/__init__.py, causes spam/ham.py to be overlooked.
There's a hint here.
You can move the contents of ham.py into ham/__init__.py, and it'll work the way you want, maybe with only a minor change or two.
> Clearly, I'm not playing this game right...
> What is considered "best practice" for the use case sketched above? > Should I, e.g. rename the directory spam/ham something like spam/ham_ > and refer to the helper module as spam.ham_.eggs? Or is some other > convention preferred?
The way you were intending is often the approach people use. I doubt there are any detailed consensus recommendations about this in the Python community.
> I consulted PEP 8, but besides recommending "short, all-lowercase > names" for modules, it gives little guidance on the situation > described above.
Of course it doesn't, PEP 8 is a style guide, not a project organization guide.
kj wrote: > In <h7jga8$ij...@reader1.panix.com> kj <no.em...@please.post> writes:
>>I'm having a hard time getting the hang of Python's package/module >>scheme. I'd like to find out what's considered best practice when >>dealing with the scenario illustrated below.
>>The quick description of the problem is: how can I have two nested >>modules, spam.ham and spam.ham.eggs?
> Following up my own post...
>>From inspecting the directory structure of some of the standard > Python modules I infer the following rules:
> 1. the source for "leaf" modules lives in files named after them > (e.g. if x.y.z is a "leaf" module, its source code is in x/y/z.py)
> 2. the source for "non-leaf" modules lives in files named __init__.py > (e.g. if x.y is a "non-leaf" module, its source code lives in > the file x/y/__init__.py)
> In the examples above, the module x.y is a "non-leaf" module because > there is a module x.y.z.
> I.e. the "leaf"-ness of a module depends solely on whether other > modules deeper in the hierarchy are present.
> An implication of all this is that if now I wanted to create a new > module x.y.z.w, this means that the previously "leaf"-module x.y.z > would become "non-leaf". In other words, I'd have to:
> 1. create the new directory x/y/z > 2. *rename* the file x/y/z.py to x/y/z/__init__.py > 3. create the file x/y/z/w.py to hold the source for the new x.y.z.w > module
> Is the above correct? (BTW, to my Perl-pickled brain, step 2 above > is the one that causes most distress... But I think I can cope.)
> kynn
Looking at the layout of the (most excellent!-) xlrd package, the bulk of the code is in the __init__.py file, and other supporting code is the same directory, accessed in __init__ with normal imports.
I also am unclear on when it's best to have supporting files in the same directory versus a subdirectory... perhaps it is that flat is better than nested?
> But now suppose that I want to factor out some code in spam/ham.py > to a helper module. (The reason behind factoring out this new > module is to "declutter" spam/ham.py, and improve its readibility.) > My instinct (from my Perl past) is to put this factored-out code > in a file spam/ham/eggs.py, i.e. to create the "nested" module > spam.ham.eggs, which requires expanding the tree as follows
> This doesn't work so well, because now spam/ham.py is not read. > It seems that adding the spam/ham directory, or maybe adding the > file spam/ham/__init__.py, causes spam/ham.py to be overlooked.
> Clearly, I'm not playing this game right...
One way is the __init__.py answer you have already. Another is to put eggs.py (or _eggs.py to indicate that it is private) in spam and skip the ham directory.
> An implication of all this is that if now I wanted to create a new > module x.y.z.w, this means that the previously "leaf"-module x.y.z > would become "non-leaf". In other words, I'd have to:
> 1. create the new directory x/y/z > 2. *rename* the file x/y/z.py to x/y/z/__init__.py > 3. create the file x/y/z/w.py to hold the source for the new x.y.z.w > module
With regard to point 2 -- would it be possible to just move z.py into x/y/z, and put 'from z import *' into x/y/z/__init__.py, for the same effect? Or is that not a good idea?
On Tue, 01 Sep 2009 09:53:08 -0700, kj <no.em...@please.post> wrote: > In <h7jga8$ij...@reader1.panix.com> kj <no.em...@please.post> writes:
>> I'm having a hard time getting the hang of Python's package/module >> scheme. I'd like to find out what's considered best practice when >> dealing with the scenario illustrated below.
>> The quick description of the problem is: how can I have two nested >> modules, spam.ham and spam.ham.eggs?
> Following up my own post...
>> From inspecting the directory structure of some of the standard > Python modules I infer the following rules:
> 1. the source for "leaf" modules lives in files named after them > (e.g. if x.y.z is a "leaf" module, its source code is in x/y/z.py)
> 2. the source for "non-leaf" modules lives in files named __init__.py > (e.g. if x.y is a "non-leaf" module, its source code lives in > the file x/y/__init__.py)
> In the examples above, the module x.y is a "non-leaf" module because > there is a module x.y.z.
> I.e. the "leaf"-ness of a module depends solely on whether other > modules deeper in the hierarchy are present.
> An implication of all this is that if now I wanted to create a new > module x.y.z.w, this means that the previously "leaf"-module x.y.z > would become "non-leaf". In other words, I'd have to:
> 1. create the new directory x/y/z > 2. *rename* the file x/y/z.py to x/y/z/__init__.py > 3. create the file x/y/z/w.py to hold the source for the new x.y.z.w > module
> Is the above correct? (BTW, to my Perl-pickled brain, step 2 above > is the one that causes most distress... But I think I can cope.)
> kynn
-- Rami Chowdhury "Never attribute to malice that which can be attributed to stupidity" -- Hanlon's Razor 408-597-7068 (US) / 07875-841-046 (UK) / 0189-245544 (BD)
In <mailman.809.1251832870.2854.python-l...@python.org> "Rami Chowdhury" <rami.chowdh...@gmail.com> writes:
>> An implication of all this is that if now I wanted to create a new >> module x.y.z.w, this means that the previously "leaf"-module x.y.z >> would become "non-leaf". In other words, I'd have to:
>> 1. create the new directory x/y/z >> 2. *rename* the file x/y/z.py to x/y/z/__init__.py >> 3. create the file x/y/z/w.py to hold the source for the new x.y.z.w >> module >With regard to point 2 -- would it be possible to just move z.py into >x/y/z, and put 'from z import *' into x/y/z/__init__.py, for the same >effect? Or is that not a good idea?
I think in this case what you would need is something like 'from ..z import *', but this does not work either; one gets the error "SyntaxError: 'import *' not allowed with 'from .'".
A third solution would be to change step 2 above to this:
2. create the *symlink* x/y/z/__init__.py --> ../z.py
This certainly would make things clearer-looking to me. Having all these files called the same thing, __init__.py, but having completely different meaning, makes my head hurt... In contrast, the __init__.py *symlinks* would be there just to keep python happy; the real content is where I'd expect it.
I ran a simple test of this idea and it works, but there may be some subtle reasons for not doing it... Or maybe it's not a good idea simply because it is not the Python way, and I'd agree that this would be reason enough to avoid it.
