How do you develop in Python?
Lou Pecora
here's a development conundrum I run into a lot (there are more complex
examples, but I'll give an easy one). I am developing module A.py
which imports B.py in the IDE. I am running test code which is also in
A.py as I incrementally develop A.py. Then I need to change B.py and,
of course, the import does not import the new changes since import only
works once. Hence, to "refresh" B.py I have to quit the IDE and
restart, open A.py and run. This is clumsy. However, the above
scenario is not uncommon and more complex interdependencies of modules
appear to make it unavoidable. Anyone know a way to get around this
"import" problem? I know there is a "reload", but then I have to
import and change my modules' code to "reload" or something each time I
work with them. Any help appreciated.
I use Python on a Macintosh. Nice IDE, otherwise.
Michael Hudson
If module A.py goes
import B
... uses B ...
and then in the top-level you type
>>> import A
>>> import B
then change B.py
then in the top-level typing
>>> reload(B)
modifies B in place, so code inside A should get the new version of
the module.
This doesn't work if you've used
from B import ...
in A; then you need to reload A too.
I don't see where you get the
> Hence, to "refresh" B.py I have to quit the IDE and restart, open
> A.py and run.
but I'd have to admit that I don't use the MacPython IDE, so I might
be missing some specific detail of that implementation. But I don't
think so.
Cheers,
M.
--
. <- the point your article -> .
|------------------------- a long way ------------------------|
-- Cristophe Rhodes, ucam.chat
Just van Rossum
It's simple: if you modify B.py, _run_ B.py to refresh it (*). A.py will see
the changes. It's hardly ever neccesary to quit the MacPython IDE.
*) In other words: if B.py has already been imported, running it it will
execute it in B's old namespace, effectively refreshing it for everybody
to see.
Just
Louis M. Pecora
<m...@python.net> wrote:
> Lou Pecora <pec...@anvil.nrl.navy.mil> writes:
>
> > I program in Python occassionally and would like to do more of it. But
> > here's a development conundrum I run into a lot (there are more complex
> > examples, but I'll give an easy one). I am developing module A.py
> > which imports B.py in the IDE. I am running test code which is also in
> > A.py as I incrementally develop A.py. Then I need to change B.py and,
> > of course, the import does not import the new changes since import only
> > works once. Hence, to "refresh" B.py I have to quit the IDE and
> > restart, open A.py and run. This is clumsy. However, the above
> > scenario is not uncommon and more complex interdependencies of modules
> > appear to make it unavoidable. Anyone know a way to get around this
> > "import" problem? I know there is a "reload", but then I have to
> > import and change my modules' code to "reload" or something each time I
> > work with them. Any help appreciated.
>
> If module A.py goes
>
> import B
> ... uses B ...
>
> and then in the top-level you type
>
> >>> import A
> >>> import B
>
> then change B.py
>
> then in the top-level typing
>
> >>> reload(B)
>
Let me try to rephrase your solution. I go into the "commandline"
window and do a reload of the imported modules. I guess that would
work. I will try it. I was looking for a more automatic way of
reloading things as they changed rather than having to manually do it
when I altered code in another module. Thanks.
Louis M. Pecora
<ju...@letterror.com> wrote:
Hi, Just,
I see what you're saying. That sounds like a nice easy step for
someone like me. :-) Thanks very much for a simple, but very useful
pointer.
Chris Barker
> but I'd have to admit that I don't use the MacPython IDE, so I might
> be missing some specific detail of that implementation. But I don't
> think so.
This is exactly the same problem as you can get using PythonWin, and why
I desperately wish there was a way for the IDE to NOT share the same
interpreter as the code you are writing. Then you could just re-start
the second interpreter.
Anyway, my solution is to use:
import B
reload(B)
# and then, of required:
from B import *
everywhere
when I'm pretty confident that B is finished (maybe not untill I'm
giving the code to someone else) I get rid of all the reloads. It's a
little slower, but it makes sure I'm always using an up to date version
of all my modules that are under develoment.
Just van Rossum wrote:
> It's simple: if you modify B.py, _run_ B.py to refresh it (*). A.py will see
> the changes. It's hardly ever neccesary to quit the MacPython IDE.
>
> *) In other words: if B.py has already been imported, running it it will
> execute it in B's old namespace, effectively refreshing it for everybody
> to see.
This is a classic, why didn't I think of that? solution. I think I will
still use my method for stuff I'm changing a lot, it is less likely that
I will forget to do the refresh.
-Chris
--
Christopher Barker,
Ph.D.
ChrisH...@home.net --- --- ---
http://members.home.net/barkerlohmann ---@@ -----@@ -----@@
------@@@ ------@@@ ------@@@
Oil Spill Modeling ------ @ ------ @ ------ @
Water Resources Engineering ------- --------- --------
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------
Greg Ewing
IDLE, I created a module containing a big list
of 'reload' statements for all my modules in an
appropriate order. Whenever I changed anything,
I reloaded that module.
Crude, perhaps, but it worked!
--
Greg Ewing, Computer Science Dept, University of Canterbury,
Christchurch, New Zealand
To get my email address, please visit my web page:
http://www.cosc.canterbury.ac.nz/~greg
Gerrit Muller
"Lou Pecora" <pec...@anvil.nrl.navy.mil> schreef in bericht
news:050620011516094693%pec...@anvil.nrl.navy.mil...
I certainly recognize your problem. Occasionaly I have used reload() to
"solve" this behavior. However for more "naive" programmers (CP4E-like) this
behavior can give many unexpected results. I do recognize the power of the
current dynamic loader behavior, but I think that a more "cold" runscript
command in IDLE, which imports/reloads anything again (and maybe clears the
rest of the environment as well) would make IDLE even more accessible for
this type of programmers.
Regards Gerrit
Louis M. Pecora
<chrish...@home.net> wrote:
> Michael Hudson wrote:
>
> This is exactly the same problem as you can get using PythonWin, and why
> I desperately wish there was a way for the IDE to NOT share the same
> interpreter as the code you are writing. Then you could just re-start
> the second interpreter.
>
> Anyway, my solution is to use:
>
> import B
> reload(B)
> # and then, of required:
> from B import *
>
> everywhere
>
> when I'm pretty confident that B is finished (maybe not untill I'm
> giving the code to someone else) I get rid of all the reloads. It's a
> little slower, but it makes sure I'm always using an up to date version
> of all my modules that are under develoment.
Chris, That I will try since it is a no-brainer -- something that
workds best for me.
> Just van Rossum wrote:
>
> > It's simple: if you modify B.py, _run_ B.py to refresh it (*). A.py will see
> > the changes. It's hardly ever neccesary to quit the MacPython IDE.
> >
> > *) In other words: if B.py has already been imported, running it it will
> > execute it in B's old namespace, effectively refreshing it for everybody
> > to see.
>
> This is a classic, why didn't I think of that? solution. I think I will
> still use my method for stuff I'm changing a lot, it is less likely that
> I will forget to do the refresh.
Just's solution works, but it has the problem that if you change B and
C and D depend on B and are used in A, then C and D have to be run, too
before you run A. I stumbled on that one. But it beats my stupid
solution of quiting the IDE and restarting. Talk about doing it the
hard way (see why I need a no-brainer? :-) ). I will try your
reload/from technique. Thanks.
Louis M. Pecora
the "To," "Cc," and "Newsgroups" headers for details. ]]
In article <9fjfer$gt0$1...@nereid.worldonline.nl>, Gerrit Muller
<gmu...@worldonline.nl> wrote:
Good point. I agree.
Just van Rossum
> > Just van Rossum wrote:
> >
> > > It's simple: if you modify B.py, _run_ B.py to refresh it (*). A.py will see
> > > the changes. It's hardly ever neccesary to quit the MacPython IDE.
> > >
> > > *) In other words: if B.py has already been imported, running it it will
> > > execute it in B's old namespace, effectively refreshing it for everybody
> > > to see.
> >
> > This is a classic, why didn't I think of that? solution. I think I will
> > still use my method for stuff I'm changing a lot, it is less likely that
> > I will forget to do the refresh.
>
> Just's solution works, but it has the problem that if you change B and
> C and D depend on B and are used in A, then C and D have to be run, too
> before you run A. I stumbled on that one.
It depends: this is only true if C and D either derive from classes in B or do
"from B import *". The latter you should avoid anyway, but the former can be
a real pain indeed. *However* there is another little known feature in the
MacPython IDE that can be extremely handy: you can "reload" individual methods.
You do this by selecting the method(s) and choose "Run selection" (or hit enter).
This inserts the updated code into the *existing* class object. Beware, though:
methods below the section you changed now have different line numbers, but the
code objects don't know that, so tracebacks might point you to the wrong line.
It's a hack, but I find it extremely useful. Think about it: it even updates
existing *instances* as it updates a class in place! It's hacking on live
objects...
Just
Chris Barker
> but I think that a more "cold" runscript
> command in IDLE, which imports/reloads anything again (and maybe clears the
> rest of the environment as well) would make IDLE even more accessible for
> this type of programmers.
Also the MacPython IDE, PythonWin, etc. I've been asking for this for a
while, but I think it's harder than it sounds for two reasons:
1) I don't think the Python Interpreter has any kind of "clear all"
facilities.
2) All three of these IDEs (and others, I'm sure) use the same
interpreter to run the IDE as the program you are working on. If you
were to "clear all" you'de wipe out the IDE as well. I'd lke to see an
IDE that used a separate copy of the interpreter, but it would make
writting a debugger (and other introspective tools) a lot harder.
Gerrit Muller
"Chris Barker" <chrish...@home.net> schreef in bericht
news:3B2129AC...@home.net...
> Gerrit Muller wrote:
> > but I think that a more "cold" runscript
> > command in IDLE, which imports/reloads anything again (and maybe clears
the
> > rest of the environment as well) would make IDLE even more accessible
for
> > this type of programmers.
>
> Also the MacPython IDE, PythonWin, etc. I've been asking for this for a
> while, but I think it's harder than it sounds for two reasons:
>
> 1) I don't think the Python Interpreter has any kind of "clear all"
> facilities.
>
manually, stop and restart the interpreter :-(.
> 2) All three of these IDEs (and others, I'm sure) use the same
> interpreter to run the IDE as the program you are working on. If you
> were to "clear all" you'de wipe out the IDE as well. I'd lke to see an
> IDE that used a separate copy of the interpreter, but it would make
> writting a debugger (and other introspective tools) a lot harder.
>
I understand the problem from implementers point of view. However from CP4E
point of view I don't "want" to understand, I simply would like to have this
"reset/clear" button. Hunting for the CP4E vision requires thinking from
"naive" programmer point of view. Some brilliant idea appears to be
necessary to satisfy this need, without making the IDE's too complex.
Thinking out loudly: For many naive users the automated stop/restart might
be fully acceptable, while it might also be simple to implement. Although,
how much unexpected state needs to be persistent, like open files and
windows; when would this same naive user start hating to see windows
disappear and reappear? May-be it is already too complex ).
> -Chris
Regards Gerrit
Chris Barker
> I understand the problem from implementers point of view.
Being a free softwaqre project, the implimenters are the users. As a
rule, free software is very good when it is software for programmers,
and not so good when it is for "naive" users. The general response form
developers that I have gotten when asking for this feature is: "It would
be a lot of work, adn I don't really need it, so I'm not going to do it.
Feel free to work it out and contribute it yourself." I don't have the
time or skills to do that, so until someone does that wants the feature,
it won't happen.
> Hunting for the CP4E vision requires thinking from
> "naive" programmer point of view. Some brilliant idea appears to be
> necessary to satisfy this need, without making the IDE's too complex.
sure, but with no funding for CP4E, who's going to impliment them?
> Thinking out loudly: For many naive users the automated stop/restart might
> be fully acceptable, while it might also be simple to implement.
It's not easy, because the IDE uses the saem interpreter.
> Although,
> how much unexpected state needs to be persistent, like open files and
> windows; when would this same naive user start hating to see windows
> disappear and reappear? May-be it is already too complex ).
Yes, they would. Starting and re-starting the entire IDE really would be
unacceptable.
Gerrit, you and I are on the same page here, but for some reason, the
ability to run a program "from scratch" doesn't seem to be important
enough to the IDE developers for them to put the work into making it
possible. I think that's because it is a lot of work.
D-Man
...
| Gerrit, you and I are on the same page here, but for some reason, the
| ability to run a program "from scratch" doesn't seem to be important
| enough to the IDE developers for them to put the work into making it
| possible. I think that's because it is a lot of work.
Some people just use gvim and an xterm (or just emacs) and thus run a
program "from scratch" all the time with no trouble.
It might be easier for you to open a shell (MacOS X has one) and run
your program from the shell, though do development in the IDE. Just a
thought on another alternative...
-D
Mats Wichmann
>2) All three of these IDEs (and others, I'm sure) use the same
>interpreter to run the IDE as the program you are working on. If you
>were to "clear all" you'de wipe out the IDE as well. I'd lke to see
an
>IDE that used a separate copy of the interpreter, but it would make
>writting a debugger (and other introspective tools) a lot harder.
I believe this is somewhere around the corner. Of course, since I
can't /see/ around the corner, I don't know how far away it actually
is.
-- mats
Nick Perkins
This is a serious shortcoming of Python. I know, the language itself is not
resposible for giving me a wonderful free IDE, but in the mind of a Python
user, the IDE is part and parcel of the language, and integral to the
*experience of Python. Python, the laguage is wonderful, but weak IDEs make
it look bad. I have used PythonWin, IDLE, Boa Constructor, and SciTe.
With all apologies to those who developed these FREE IDE's... (it is not
fair to complain about free software, and that is not my intention here -- I
am sure that I could not do any better).. I simply, (and selfishly), wish
that I could use Python to the full potential of the language itself,
without being hindered by an IDE that freezes up, leaves behind runaway
processes, or crashes.
IDLE freezes up frequently.
PythonWin freezes a bit less, and seems unable to kill a runaway program.
Boa Constructor is more stable than these two, and allows you to kill a
runaway program without killing the editor. However, it won't take you to
the line that caused an error, which is very irritating. (broken feature)
SciTe seems interesting, but I haven't figured out how to configure it, yet,
and util I do, it's useless. (me being lazy, but still...)
I want an IDE to provide at least the following:
1. Each run takes place in a fresh, clean environment.
2. The IDE stays responsive while a program is run.
3. The IDE can kill a running program.
Another thing I would like, but that may have, i dunno, ..implications..?
1. The IDE should handle the sys.path so that Python does not fail to find a
file that I currently have open, or is in the same directory.
Is it not possible to start a 'fresh' interpreter for each run? Or is that
not actually desirable for some reason? Would that make it impossible to
kill it or get tracebacks, etc? Do we need a Python IDE that does not run
itself in Python?
( i guess SciTE doesn't....maybe I'll give that another go..)
BTW, which free IDEs are currently being maintained and improved?
How much work is being done, and how many people are doing it?
( keep up the good work!, if this means you!)
p.s. i know, i know, emacs, vim, etc etc....but I have become a bit of a
windows softie, and have an aversion to such things. I am going to use
emacs in my next life, and learn lisp as my first language.....
Just van Rossum
> I want an IDE to provide at least the following:
> 1. Each run takes place in a fresh, clean environment.
> 2. The IDE stays responsive while a program is run.
> 3. The IDE can kill a running program.
For me, 1. should be entirely optional: I usually *prefer* to keep working
in an existing environments, and only clean up (start over) when I've
messed things up beyond repair. As an example of this way of working: I've
developed the MacPython IDE largely in *itself*. It's a medium-sized app,
but restarting it for every minor change would've been a pain. Instead,
I work on it while it's running, updating modules and individual classes,
and seeing effect *immediately*. This is amazingly effective with Python,
due to its dynamic nature.
Yet there are cases where this is not possible or too hard: eg. developing
GUI apps that use a different GUI/mainloop than the IDE. A separate
interpeter is simply the best option. But as has been written before in
this thread: to conveniently do this one would like to have a good
communications channel between the IDE and the app in question: for
debugging, tracebacks, object browsers, etc. Such a thing is not all
that easy to create in a cross-platform manner, but I would agree that
this is where Python IDE's should be heading.
Just
Glyph Lefkowitz
On Sun, 10 Jun 2001, Nick Perkins wrote:
> I want an IDE to provide at least the following:
> 1. Each run takes place in a fresh, clean environment.
> 2. The IDE stays responsive while a program is run.
> 3. The IDE can kill a running program.
One idea which I've been kicking around (and finally even started a little
implementation of last night) is a Python interpreter in Python. I
believe that the existing compiler is relatively robust (at least, I
haven't been able to generate any source code that can crash or hang it
yet), but the bytecode interpreter can't isolate tasks, which makes it
impossible to kill a process or even slow it down to see what it's doing.
Stackless seems like it might make this possible, but it's still using
essentially the same partially-recursive interpreter. I'm also interested
in running completely untrusted code from python bytecode. However, the
tricky parts are in making sure that this meta-interpreter isn't
recursive. (In particular, I am getting stuck on operator overloading,
since it's possible to implement __getattr__...)
Some advantages that this approach would have:
* .pyc bytecode interpreter for Jython.
* untrusted code execution
* resource control
* mobile code
* defined API for communication between isolated tasks
* ability to run "blocking" code without threads in a singlethreaded app
* educational value; easier to read than ceval.c
* task-scheduling in python without continuations
* IDEs could safely execute/debug development code w/o a separate process
* python code would run 2 orders of magnitude slower, making it even more
high-level
Has anyone attempted such a thing before, or have any advice to give me?
Also, is it easy/possible to crash the "marshal" module with bad data?
______ __ __ _____ _ _
| ____ | \_/ |_____] |_____|
|_____| |_____ | | | |
@ t w i s t e d m a t r i x . c o m
http://twistedmatrix.com/users/glyph
Bufu
Paul Prescod
>
>...
>
> One idea which I've been kicking around (and finally even started a little
> implementation of last night) is a Python interpreter in Python.
>
>
> * IDEs could safely execute/debug development code w/o a separate process
Komodo runs code in a separate process and I'm pretty sure that your
Python-in-Python wouldn't convince us to do otherwise because it would
be difficult for you to *exactly* emulate every single behavior of the
real Python interpreter especially when you consider that we an
currently allow people to debug code on any Python interpreter including
everything from 1.5.2 to experimental 2.2s.
--
Take a recipe. Leave a recipe.
Python Cookbook! http://www.ActiveState.com/pythoncookbook
Louis M. Pecora
<gmu...@worldonline.nl> wrote:
> I understand the problem from implementers point of view. However from CP4E
> point of view I don't "want" to understand, I simply would like to have this
> "reset/clear" button. Hunting for the CP4E vision requires thinking from
> "naive" programmer point of view. Some brilliant idea appears to be
> necessary to satisfy this need, without making the IDE's too complex.
>
> Thinking out loudly: For many naive users the automated stop/restart might
> be fully acceptable, while it might also be simple to implement. Although,
> how much unexpected state needs to be persistent, like open files and
> windows; when would this same naive user start hating to see windows
> disappear and reappear? May-be it is already too complex ).
>
> Regards Gerrit
I won't pretend to understand all this internal stuff, but with my
familiarity with C/C++ IDEs, they handle the dependencies throught the
header files. Why can't we have a switch that will reload
automatically in the IDE according to dependencies which could be
traced through the import statements? Is this too naive?
Jesper
(http://www.lowerstandard.com/python/index.html). I have not tried it
so I can not tell you if it works as described but it looks
straightforward (on second thought I guess it won't work with 'from
module import *'. Oh well -- can´t get it all):
reloadall.py:
import types
def status(module):
print 'reloading', module.__name__
def transitive_reload(module, visited):
if not visited.has_key(module): # trap cycles, dups
status(module) # reload this module
reload(module) # and visit children
visited[module] = None
for attrobj in module.__dict__.values(): # for all attrs
if type(attrobj) == types.ModuleType: # recur if module
transitive_reload(attrobj, visited)
def reload_all(*args):
visited = {}
for arg in args:
if type(arg) == types.ModuleType:
transitive_reload(arg, visited)
if __name__ == '__main__':
import reloadall # test code: reload myself
reload_all(reloadall) # should reload this, types
Neil Hodgson
> With all apologies to those who developed these FREE IDE's...
> (it is not fair to complain about free software, and that is
> not my intention here -- I am sure that I could not do any
> better)..
But if there weren't any complaints, then we wouldn't know what to fix
<wink>.
> SciTe seems interesting, but I haven't figured out how to
> configure it, yet, and util I do, it's useless. (me being lazy,
> but still...)
SciTE is the dumbest on your list, not attempting to be a debugger at all
and just executing a Python file in an external process and piping the
output into a window.
> I want an IDE to provide at least the following:
> 1. Each run takes place in a fresh, clean environment.
> 2. The IDE stays responsive while a program is run.
> 3. The IDE can kill a running program.
These are fairly easy although chances of a process kill differs between
OS versions.
> Is it not possible to start a 'fresh' interpreter for each run? Or is
that
> not actually desirable for some reason? Would that make it impossible to
> kill it or get tracebacks, etc? Do we need a Python IDE that does not run
> itself in Python?
What is hard if Python is run out of process is to performing debugging
operations such as single stepping and examining variables.
> ( i guess SciTE doesn't....maybe I'll give that another go..)
Go on, but I bet you'll want a real debugger after a short period and
that is something I'm unlikely to find the time to add to SciTE.
> BTW, which free IDEs are currently being maintained and improved?
> How much work is being done, and how many people are doing it?
> ( keep up the good work!, if this means you!)
SciTE gets lots of additions but there is not much quality control so at
any time several of the features are sure to work strangely. Boa is alive
but progressing slowly. PythonWin doesn't see much improvement as Mark is
very busy with a lot of work including Komodo.
PythonWin could be fairly easily made to have an option to run programs
in another process but not allowing debugging. With open source though, its
a matter of someone being sufficiently motivated (or annoyed) to invest the
effort.
Neil