code in a module is executed twice (cyclic import problems) ?

[Stef Mientki]

hello,

I always thought code in a module was only executed once,
but doesn't seem to be true.

I'm using Python 2.5.

And this is the example:

== A.py ==
My_List = []

== B.py ==
from A import *
My_List.append ( 3 )
print 'B', My_List
import C

== C.py ==
from A import *
from B import *
print 'C', My_List

Now when you start with B.py as the main program,
this is the resulting output:

B [3]
B [3, 3]
C [3, 3]

Why is the B.py executed twice ?

thanks,
Stef

[Benjamin Peterson]

Stef Mientki <stef.mientki <at> gmail.com> writes:
> Why is the B.py executed twice ?

Because it's executed once as a script and once as a module.

[Hendrik van Rooyen]

On Sunday, 11 October 2009 02:24:34 Stephen Hansen wrote:

> It's really better all around for "modules" to be considered like
> libraries, that live over There, and aren't normally executed. Then you
> have scripts over Here which may just be tiny and import a module and call
> that module's "main" method. Okay, I'm not arguing you should never execute
> a module, sometimes its useful and needful-- especially for testing or more
> complex project organization. But in general... this is just gonna cause no
> end of suffering if you don't at least try to maintain the "script" vs
> "module" mental separation. Modules are the principle focus of code-reuse
> and where most things happen, scripts are what kickstart and get things
> going and drive things. IMHO.
>
> If not that, then at least make sure that nothing is ever /executed/ at the
> top-level of your files automatically; when imported call 'module.meth()'
> to initialize and/or get My_List or whatever, and when executed use the
> __name__ == "__main__" block to do whatever you want to do when the file is
> started as a main script.
>
> Otherwise, things'll get weird.
>
> ... gah snip huge write-up I threw in about how we organize our code around
> the office. Not important! Your use-case is probably different enough that
> you'd surely organize differently. But still, I really recommend treating
> "modules" like static repositories of code that "scripts" call into /
> invoke / execute. Even if sometimes you execute the modules directly (and
> thus use a main() function to run in whatever default way you choose).

I have been using the __name__== "main" part of what you call modules to do
the test code for the module. So if you execute it at the top level, it does
its testing, and if you import it, it is like a library. This works for me,
but it is probably not of general use - the stuff we do tends to be smallish
and long running.

- Hendrik

[ryles]

FYI, there was actually a related discussion about this just recently:

http://groups.google.com/group/comp.lang.python/browse_thread/thread/e24be42ecbee7cad

[Stef Mientki]

thanks very much Stephen,

This is the first time I become aware of the difference between script
and module !!
Starting with the wrong book "Learning Python" second edition, from Lutz
and Ascher, based on Python 2.3"
in combination with using Python only from a high level IDE
(PyScripter), (never seeing the m-switch),
I never was aware of this important difference.
A quick Googling on "python module vs script" doesn't reveal many (good)
links,
the best one I found is
http://effbot.org/zone/import-confusion.htm

thanks again,
Stef Mientki

Stephen Hansen wrote:

> On Sat, Oct 10, 2009 at 4:36 PM, Stef Mientki <stef.m...@gmail.com
> <mailto:stef.m...@gmail.com>> wrote:
>
> hello,
>
> I always thought code in a module was only executed once,
> but doesn't seem to be true.
>
>

> This is one of the reasons why that whole big mess of a ton separate
> scripts that all call each-other and are sometimes imported and
> sometimes executed is just a bad way to achieve code re-use and
> organization... :) IMHO :)

>
>
> I'm using Python 2.5.
>
> And this is the example:
>
> == A.py ==
> My_List = []
>
> == B.py ==
> from A import *
> My_List.append ( 3 )
> print 'B', My_List
> import C
>
> == C.py ==
> from A import *
> from B import *
> print 'C', My_List
>
> Now when you start with B.py as the main program,
> this is the resulting output:
>
> B [3]
> B [3, 3]
> C [3, 3]
>
> Why is the B.py executed twice ?
>
>

> Because when you start B, it's not the module "B". Its a script that
> is being run. Python doesn't byte-compile such scripts, nor do those
> scripts count really as the modules you're expecting them to be.
>
> When you "import B" after executing B as a main module, it won't find
> that module "B" has already been loaded. When you execute B directly,
> its actually the module "__main__". When you "import B", it's the
> module "B".

>
> It's really better all around for "modules" to be considered like
> libraries, that live over There, and aren't normally executed. Then
> you have scripts over Here which may just be tiny and import a module
> and call that module's "main" method. Okay, I'm not arguing you should
> never execute a module, sometimes its useful and needful-- especially
> for testing or more complex project organization. But in general...
> this is just gonna cause no end of suffering if you don't at least try
> to maintain the "script" vs "module" mental separation. Modules are
> the principle focus of code-reuse and where most things happen,
> scripts are what kickstart and get things going and drive things. IMHO.
>
> If not that, then at least make sure that nothing is ever /executed/
> at the top-level of your files automatically; when imported call
> 'module.meth()' to initialize and/or get My_List or whatever, and when
> executed use the __name__ == "__main__" block to do whatever you want
> to do when the file is started as a main script.
>
> Otherwise, things'll get weird.
>
> ... gah snip huge write-up I threw in about how we organize our code
> around the office. Not important! Your use-case is probably different
> enough that you'd surely organize differently. But still, I really
> recommend treating "modules" like static repositories of code that
> "scripts" call into / invoke / execute. Even if sometimes you execute
> the modules directly (and thus use a main() function to run in
> whatever default way you choose).
>

> HTH,
>
> --S
>

[Dave Angel]

(please don't top-post. Put your reply *after* the message you're quoting.)

Stef Mientki wrote:
> <div class="moz-text-flowed" style="font-family: -moz-fixed">thanks

> very much Stephen,
>
> This is the first time I become aware of the difference between script
> and module !!
> Starting with the wrong book "Learning Python" second edition, from
> Lutz and Ascher, based on Python 2.3"
> in combination with using Python only from a high level IDE
> (PyScripter), (never seeing the m-switch),
> I never was aware of this important difference.
> A quick Googling on "python module vs script" doesn't reveal many
> (good) links,
> the best one I found is
> http://effbot.org/zone/import-confusion.htm
>
> thanks again,
> Stef Mientki

> <snip>
The point you should get from that link is

"Don't do circular imports. Ever."

It's generally worse if the circle includes the original script, treated
as a module, but even between modules it can get you into trouble in
many subtle ways. Some of them cause runtime errors, so you'll "fix"
them. Some of them will just fail quietly, and you'll be searching for
subtle bugs. I don't agree with the author's "advice" that sometimes
moving the import to the end helps. Best advice is to break the circle,
by "rearranging" the modules, moving commonly needed symbols into
someplace else that both import.

It's probably safe if none of the modules in the circle has any
top-level code that references imported symbols. But top-level code
includes default values for function definitions, and class initializers
for class definitions.

Does anybody know if there's a way to activate a warning for this?

DaveA

[Stephen Fairchild]

ryles wrote:

>> I always thought code in a module was only executed once,
>> but doesn't seem to be true.
>>
>> I'm using Python 2.5.
>>
>> And this is the example:
>>
>> == A.py ==
>> My_List = []
>>
>> == B.py ==
>> from A import *
>> My_List.append ( 3 )
>> print 'B', My_List
>> import C
>>
>> == C.py ==
>> from A import *
>> from B import *
>> print 'C', My_List
>>
>> Now when you start with B.py as the main program,
>> this is the resulting output:
>>
>> B [3]
>> B [3, 3]
>> C [3, 3]
>>
>> Why is the B.py executed twice ?

B.py is the entry point of the program and it is known internally as
__main__. There is no record of a B.py.

If you really must import objects from the main module you can do it like
this.

from __main__ import *
--
Stephen Fairchild

[greg]

Dave Angel wrote:

> The point you should get from that link is
>
> "Don't do circular imports. Ever."

No, I would say the lesson to be learned from this is
"don't use the same file as both a main script and an
imported module".

I would create another file called e.g. 'main.py' that
simply contains

import B

and use that as the main script.

Another benefit is that the compiled bytecode of B will
be saved, so it will start up faster the next time you
run it. That won't happen if you use B directly as a
main script.

(Circular imports are a different issue. There are perfectly
legitimate uses for them, and they're fine as long as you
understand the potential pitfalls and take steps to avoid
them. Although it's probably best to avoid them until you
do understand all the issues.)

--
Greg

[Dave Angel]

greg wrote:
> <div class="moz-text-flowed" style="font-family: -moz-fixed">Dave

You prove my point by trying to contradict it. Importing a script from
the same or another module is a circular import, and therefore should be
avoided. If you understand all the issues, you can get away with it,
but few programmers understand them all like they think they do.

The *only* difference between this circular import and any other is that
the problems it can trigger are a little more obvious, and a lot more
likely.

DaveA