Coping with cyclic imports
Torsten Bronger
I have a rather fat module that represents a document parser --
inline elements, block elements, and the like. Now I want to split
it into many modules to make everything more manageable.
But at the moment I don't see how to avoid cyclic imports: A
document element A, which is represented my module parser.A, may
contain the element B, as defined in parser.B. And vice versa. So
both modules must import each other, as far as I can see.
I know that cyclic imports work in Python under certain
circumstances. Can anyone refer me to a page which explains *when*
this works? Because at least once, the imported module was not
"finished" and thus largely unusual.
Thank you!
Tschö,
Torsten.
--
Torsten Bronger, aquisgrana, europa vetus
Jabber ID: bro...@jabber.org
(See http://ime.webhop.org for further contact info.)
Jeffrey Froman
> I know that cyclic imports work in Python under certain
> circumstances. Can anyone refer me to a page which explains when
> this works?
I don't know of a specific URL offhand.
Cyclic imports are not a problem by themselves, but cyclic definitions are.
Thus:
# a.py
import b
x = 1
# b.py
import a
x = 2
works fine, but:
# a.py
import b
x = 1
# b.py
import a
x = a.x + 1 # circular definition
does not.
Jeffrey
Torsten Bronger
Jeffrey Froman writes:
> [...]
>
> Cyclic imports are not a problem by themselves, but cyclic
> definitions are. Thus:
>
> # a.py
> import b
> x = 1
>
> # b.py
> import a
> x = 2
>
> works fine, but:
>
> # a.py
> import b
> x = 1
>
> # b.py
> import a
> x = a.x + 1 # circular definition
>
> does not.
Okay, thanks, but after some own investigations, I think that the
clever bit is somewhere else.
The above works if you call a.py as the main program, because then
a.py is not yet loaded as such when "import a" is executed. So a.py
gets executed twice, once by "import a", and once after the whole
import thing as the main program.
Actually, there seems to be only one case that is dangerous: If you
import a module cyclicly, it may be that you only get its name
imported but this name points to a yet empty module. Then, you must
not refer to things in it while the current module itself is
executed. But you may well refer to things in it from functions or
methods, because they are called much later, when the whole bunch of
modules is already completely loaded and populated.
Consequently, "from a import x" also fails if a is incomplete.
So, the last question is: Under which circumstances does this
happen? It happens when you import a module which imports (directly
or indictly) the current module and which comes before the current
module in the import order while the program runs.
If you don't rely on imported things at top-level code (but only in
functions and methods which in turn must not be called from the
top-level), everything is fine.
Can anybody confirm that this is correct?
Duncan Booth
> So, the last question is: Under which circumstances does this
> happen? It happens when you import a module which imports (directly
> or indictly) the current module and which comes before the current
> module in the import order while the program runs.
>
> If you don't rely on imported things at top-level code (but only in
> functions and methods which in turn must not be called from the
> top-level), everything is fine.
>
> Can anybody confirm that this is correct?
Imports are pretty straightforward really. Just remember the following:
'import' and 'from xxx import yyy' are executable statements. They execute
when the running program reaches that line.
If a module is not in sys.modules, then an import creates the new module
entry in sys.modules and then executes the code in the module. It does not
return control to the calling module until the execution has completed.
If a module does exist in sys.modules then an import simply returns that
module whether or not it has completed executing. That is the reason why
cyclic imports may return modules which appear to be partly empty.
Finally, the executing script runs in a module named __main__, importing
the script under its own name will create a new module unrelated to
__main__.
Take that lot together and you shouldn't get any surprises when importing
modules.
kanchan....@gmail.com
> Hallöchen!
>
> I have a rather fat module that represents a document parser --
> inline elements, block elements, and the like. Now I want to split
> it into many modules to make everything more manageable.
>
> But at the moment I don't see how to avoid cyclic imports: A
> document element A, which is represented my module parser.A, may
> contain the element B, as defined in parser.B. And vice versa. So
> both modules must import each other, as far as I can see.
>
> I know that cyclic imports work in Python under certain
> circumstances. Can anyone refer me to a page which explains *when*
> this works? Because at least once, the imported module was not
> "finished" and thus largely unusual.
>
> Thank you!
>
> Tschö,
> Torsten.
>
The problem is when instead you do "from foo import abc" and "from bar import xyz". Because now each module requires the other module to already be compiled (so that the name we are importing exists) before it can be compiled.
from
http://stackoverflow.com/questions/744373/circular-or-cyclic-imports-in-python
Dave Angel
top-level code in the second module to be imported tries to use symbols
defined in the first module. And it's definitely bad practice, with
weird bugs if either one of these files is the main script being loaded.
--
DaveA
Oscar Benjamin
> On Tuesday, April 8, 2008 10:06:46 PM UTC+2, Torsten Bronger wrote:
> If you do "import foo" inside bar and "import bar" inside foo, it will work fine. By the time anything actually runs, both modules will be fully loaded and will have references to each other.
>
> The problem is when instead you do "from foo import abc" and "from bar import xyz". Because now each module requires the other module to already be compiled (so that the name we are importing exists) before it can be compiled.
>
> from
> http://stackoverflow.com/questions/744373/circular-or-cyclic-imports-in-python
Or is it just a joke that you've created a cyclic import advice link
by referring to a SO question where the top answer is actually a quote
linking back to the previous post in this same thread?
Oscar
kanchan....@gmail.com
and sorry i didnt see the date of the question
and guessing if people find this post first then it would be helpful for them to get a good answer
Dave Angel
The problem is much more subtle than those answers imply, unless you
make certain assumptons about the user's code. And if I'm going to do
that, the thing I'm going to figure is that the rational user avoids any
cyclic imports, period.
if a.py imports b.py, then b.py is not permitted, directly or
indirectly, to import a.py. If it does, and something doesn't work,
then 50 lashes with a wet noodle.
Whenever you find a cycle, try to move the common stuff into a 3rd
module and have the first two import that one instead of each other.
Avoid having any non-trivial code at the top level, and in no case
reference anything you've imported from there. Move everything into
functions, and don't have any class-static code or data initialization.
If you have to break any of these (such as referring to sys.argv),
make sure that the module you're using is not one that's importing you.
Never, ever import the original script from another module.
--
DaveA
Cameron Simpson
| On 4 July 2013 13:48, <kanchan....@gmail.com> wrote:
| > On Tuesday, April 8, 2008 10:06:46 PM UTC+2, Torsten Bronger wrote:
| > http://stackoverflow.com/questions/744373/circular-or-cyclic-imports-in-python
|
| Is there some reason you're responding to a post from 5 years ago?
Certainly if someone has new input on an old Python thread, which
is legitimately part of the old thread, I am _glad_ if they reply
to the old post. It pulls the whole prior thread content up the top
for my perusal should it be necessary. If they make a new post I
have to go digging through archives if I need to do that.
Cheers,
--
Cameron Simpson <c...@zip.com.au>
Oscar Benjamin
> On 04Jul2013 16:03, Oscar Benjamin <oscar.j....@gmail.com> wrote:
> |
> | Is there some reason you're responding to a post from 5 years ago?
>
> Is there some reason not to, if no newer solutions are available?
forum/newsgroup/mailing list doesn't give me a way to respond to very
old posts but others seem to do it every now and again. I see now that
if you're looking at an old thread in Google Groups (rather than e.g.
the python.org archives) it makes the thread seem more like a forum
than a newsgroup or a mailing list so that it's easy and seems more
natural to respond to old posts.
Oscar