image vs source
Peter Seibel
Lisp's image-based development. I take it that most folks leave their
Lisp image running for quite some time (all day? days on end?) And I
see how that supports interactive programming which is good. The thing
that continues to make me uneasy is not knowing that I'm working from
a known state. Say, for instance, that I write this:
(defun fooo () (do-something))
And elsewhere, say, in a test I use it:
(defun some-test ()
(fooo)
(make-sure-it-worked))
Later, I'm working foox and notice that I accidentally typed an extra
'o' on the end of the name. So I rename it and evaluate the new defun.
But the old fooo is still in the image and some-test still passes so I
don't notice that I forgot to change the caller. Until, of course, I
quit my image, start a new one and load my .lisp (or .fasl) files that
have a definition for 'foo' but not 'fooo'. Now some-test explodes
because it calls an undefined function.
This is obviously a trivial example of a more general problem: how do
you know that the contents of your image bear a predictable relation
to the contents of your source file. I'm sure there are practices that
make this not a problem--I'm just not sure what they are. Any advice?
-Peter
--
Peter Seibel
pe...@javamonkey.com
Kenny Tilton
Peter Seibel wrote:
> I'm still trying to wrap my head around how to make effective use of
> Lisp's image-based development.
I screamed bloody murder when I ran into this for the first time, in
Prolog, when I deleted the source for a rule and Prolog still remembered
the rule. :) Those were the days! The implementation happened to offer a
compilation option in which you could declare a certain source file to
be the only place rules of a certain name should reside. It then cleaned
up as necessary when it recompiled that source file.
> I take it that most folks leave their
> Lisp image running for quite some time (all day? days on end?)
Yes, days.
> The thing
> that continues to make me uneasy is not knowing that I'm working from
> a known state.
You'll get used to it. Hints below.
> Say, for instance, that I write this:
>
> (defun fooo () (do-something))
>
> And elsewhere, say, in a test I use it:
>
> (defun some-test ()
> (fooo)
> (make-sure-it-worked))
>
> Later, I'm working foox and notice that I accidentally typed an extra
> 'o' on the end of the name. So I rename it and evaluate the new defun.
> But the old fooo is still in the image and some-test still passes so I
> don't notice that I forgot to change the caller. Until, of course, I
> quit my image, start a new one and load my .lisp (or .fasl) files that
> have a definition for 'foo' but not 'fooo'. Now some-test explodes
> because it calls an undefined function.
After a while you'll recognize instantly what happened and just go clean
it up. It's like COBOL programmers that instinctively check for
misplaced full stops that got cut and pasted into the middle of an if
block, or any programmer that instinctively checks "is that a zero or an
'oh'".
What I get a lot is changing the specialization of a method (or deleting
a method from the source) and then re-running only to see the same bug.
I get this dumb look on my face for about two seconds and then I realize
what happened.
The worst is changing a macro. The good news is that, with today's
systems, a full build from scratch is barely time for a run to the head.
Defsystem also helps here if you lump macros into one source file and
make other source dependent on that (not that I have tried that, but
that's my plan soon.)
Finally, one thing I do is indeed exit at least once a day to clear the
detritus of my thrashing. And if I end up refactoring the wazoo out of a
system, once I have everything re-compiling clean, I then yo-yo the
image and continue the cleanup process (because more stuff always drops
out then).
--
kenny tilton
clinisys, inc
---------------------------------------------------------------
""Well, I've wrestled with reality for thirty-five years, Doctor,
and I'm happy to state I finally won out over it.""
Elwood P. Dowd
JP Massar
wrote:
>
>This is obviously a trivial example of a more general problem: how do
>you know that the contents of your image bear a predictable relation
>to the contents of your source file.
You can't really know, without doing a lot more work than giving up,
quitting the image and reloading.
(And note that just because your SOURCE file now says X, doesn't mean
that you've compile/loaded that source file into your image...)
But as a practical matter it is not that much of a problem.
(On Lisp Machines, in days of yore (for me), images would exist for
weeks or even months. Yes, rebooting and reloading would occasionally
reveal these types of problems, but they were usually simple to fix)
> I'm sure there are practices that
>make this not a problem--I'm just not sure what they are. Any advice?
>
I like to make sure my stuff compile/load's without any warnings.
If at some point I suspect my image is not in sync, I quit and do
a complete recompile/reload. Then, if I get any warnings I know I've
introduced some kind of problem, probably similar to what you are
describing.
But again, generally, it just isn't that much of an issue.
You'll learn quickly not to do things that cause these kinds of
problems.
(Hell, my stupid Windows OS crashes or flakes out reliably enough that
restarting Lisp images is a given...)
Erik Naggum
| But the old fooo is still in the image
Why do you believe this?
| This is obviously a trivial example of a more general problem: how
| do you know that the contents of your image bear a predictable
| relation to the contents of your source file.
This more general problem is known as "the problem of sanity" in
the real world. The solution is not to be found in the particular
discipline, but rather in repairing your sense of loss of control
(and predictability) of your environment. Such sense of loss of
control is a major component of fear, and it can remain and fuel
more fear a long time after the initial feeling of loss of control.
Some people make up their mind that they not trust an environment
because they get burnt by a mistake that they strongly feel should
not have caused them any harm. This belief is at the core of their
future susceptibility for loss of control, since they have ceased
to believe that they have full control to begin with.
To fix this problem, which is actually very common and occurs in
everything from ablutophobia to zoophobia, one has to examine what
caused the initial feeling of loss of control, demonstrate or have
demonstrated that one may yet regain control of the situation. The
problem with these irrational fears is that they are unassayable if
their core premise is not addressed. In the case of programmers
who have an abnormal fear of losing control if they make a mistake
in their computing environment, nothing whatsoever can be done to
address this particular fear if the person who wants to help them
does not understand that the problem is loss of control; it serves
only to aggravate the fear if the person who intends to help shows
great dexterity and foreknowledge of the machine's reactions, as
this is unattainable magic to the person who feels out of control.
More often than not, the feeling of being out of control when using
a computer is caused by lack of concentration, attention to detail,
but some people get this misguided idea that the computer "ought"
to do what they want, and that is a serious handicap as they become
angry at the computer instead of making an effort to understand
what it actually does.
So what do other people do when they have made mistakes? They do
one thing that those who repeat their mistakes because they think
are /not/ mistakes do not: They /listen/ carefully to the computer
and pay attention to how it responds under different stimuli. The
only way to learn what the computer (that is, the programmers of
the application you are using) think is right, is to accept that a
computer, to be commanded, must be obeyed. (Thus, no difference
from anything else.) The right way to command a computer is to
obey it every single step of the way to changing its behavior. For
some reason, many people believe that they can somehow command a
computer like a human being, that it kindly fills in the blanks in
your commands. This could be because most programmers have yet to
spawn children of their own, do not have time for a dog, and do not
even have cats. Most adults are exceedingly willing to entertain
their own assumptions when you give them insufficient information,
but children and dogs and cats and computers need to be given the
full set of information in their current context and you need to
keep track of the context. The reason many users seem to prefer a
graphical user interface to a textual is that they may believe that
more information about the context is available to them, but they
limit the useful context to that which is visually apparent -- what
you see is all you get. It may be hard to keep track of everything
the system "knows" at any given time, but the alternative to having
to synchronize the computer with your mental image of it is to be
given a completely /overwhelming/ visual status report unless you
are satisfied to be constrained to believe only a subset "exists".
Most Common Lisp environments provide some means of easily undoing
definitions, if not a fully general "undo" mechanism. The language
sports functions like `makunbound´, `fmakunbound´, `remove-method´,
`slot-makunbound´ and even `setf´ of `find-class´ to nuke the name
to class association, as well as `delete-package´. Other functions
also undo various other operations, such as `unintern´. You should
investigate and familiarize yourself with these functions so you
know how to clean up after your mistakes. One of the reasons to
develop your code in their own packages is that you can delete them
and re-load them from the compiled sources.
What most Common Lisp environments do not provide, however, is the
previous value of something they are more than happy to overwrite.
I miss a transaction facility where I can make a number of changes
to the system that are only visible to my thread of a multi-threaded
execution environment and then discard them or commit them all at
once. E.g., loading a file could be such a transcation. Signaling
an error during loading could cause the whole slew of operations
that preceded it to be discarded instead of leaving the system in a
partially modified state. This is not impossible to build on top of
the existing system, but it takes significant effort, so it is the
kind of thing that Common Lisp systems programmers should do.
--
Erik Naggum, Oslo, Norway
Act from reason, and failure makes you rethink and study harder.
Act from faith, and failure makes you blame someone and push harder.
Peter Seibel
> * Peter Seibel
> | But the old fooo is still in the image
>
> Why do you believe this?
To answer your question literally: because nothing I know about how my
Lisp implementation and/or ilisp work, nor anything I've read in the
CLHS would suggest that evaluating the definition of a function 'foo'
would cause an existiting definition of a function 'fooo' (note the
different spelling) to go away even though the two functions are
related insofar as the new function 'foo' was created by very slightly
modifying the text in a file that used to be the definition of 'fooo'.
Am I missing something, either in my understanding of Lisp or in the
intent of your question?
> Most Common Lisp environments provide some means of easily undoing
> definitions, if not a fully general "undo" mechanism. The language
> sports functions like `makunbound´, `fmakunbound´,
> `remove-method´, `slot-makunbound´ and even `setf´ of `find-class´
> to nuke the name to class association, as well as
> `delete-package´. Other functions also undo various other
> operations, such as `unintern´. You should investigate and
> familiarize yourself with these functions so you know how to clean
> up after your mistakes. One of the reasons to develop your code in
> their own packages is that you can delete them and re-load them
> from the compiled sources.
I looked at DELETE-PACKAGE but both my reading of the DELETE-PACKAGE
entry in the CLHS and a brief experiment convinced me that while
deleting the package would keep anyone from writing new code that
referenced the old function, existing functions whose definitions have
already been evaluated (mmmm, maybe only if they've been compiled? I
didn't check that) will continue to refer to that function since the
symbol resolution has already happened. But I definitely didn't
explore that avenue exhaustively--I thought about writing something
that would loop across the symbols in a package and MAKUNBOUND and
FMAKUNBOUND as appropriate and *then* delete the package but I wasn't
convinced that was even the right approach: no need to write a bunch
of software if my real problem is a fear of losing control. ;-)
> What most Common Lisp environments do not provide, however, is the
> previous value of something they are more than happy to overwrite.
> I miss a transaction facility where I can make a number of changes
> to the system that are only visible to my thread of a multi-threaded
> execution environment and then discard them or commit them all at
> once.
When you say you "miss" that, do you mean that in the sense of, "I
once had it and now I miss it" or merely, "I notice it's absence"? If
the former, what system had such a facility?
Erik Naggum
| To answer your question literally: because nothing I know about how
| my Lisp implementation and/or ilisp work, nor anything I've read in
| the CLHS would suggest that evaluating the definition of a function
| 'foo' would cause an existiting definition of a function 'fooo'
| (note the different spelling) to go away even though the two
| functions are related insofar as the new function 'foo' was created
| by very slightly modifying the text in a file that used to be the
| definition of 'fooo'. Am I missing something, either in my
| understanding of Lisp or in the intent of your question?
You missed the point of the question: Why do you believe that
editing source to effectively remove a function would magically
transfer to the environment when you re-load the file? There is
some core belief here that probably stem from other languages, but
which does not hold for Common Lisp. E.g., if you are perhaps used
to dynamic loading of class files which replace themselves with the
new version, that could explain why you think the old definitions
"vanish".
| I looked at DELETE-PACKAGE but [...]
When nothing references the symbols, they vanish. The purpose of
deleting the whole package is to yank the ground out from under the
symbols. Only with some difficulty can you reference symbols in a
deleted package. It would therefore be hard for anything to /call/
the functions that referenced the old values. If you have old code
running, you will want to stop it, obviously.
There are a number of tacit assumptions under what you write that I
try hard not to assume I know, so if you want to discuss the issues
that really bother you, you have to make them clear.
Tim Bradshaw
> When nothing references the symbols, they vanish. The purpose of
> deleting the whole package is to yank the ground out from under the
> symbols. Only with some difficulty can you reference symbols in a
> deleted package. It would therefore be hard for anything to /call/
> the functions that referenced the old values. If you have old code
> running, you will want to stop it, obviously.
Erik knows this, but just in case it's not clear, if you do something
like this in the same `image':
At time t_1:
(defun foo (...)
(my-package:bar ...))
At time t_2, t_2 > t_1:
(delete-package "MY-PACKAGE")
At time t_3, t_3 > t_2:
(foo ...)
Then you still call the function formerly known as MY-PACKAGE:BAR. It
would be hard for newly-defined functions to refer to symbols in the
deleted package, but ones that already refer to those symbols (or
their bindings) continue to do so.
--tim
Peter Seibel
> * Peter Seibel
> | To answer your question literally: because nothing I know about how
> | my Lisp implementation and/or ilisp work, nor anything I've read in
> | the CLHS would suggest that evaluating the definition of a function
> | 'foo' would cause an existiting definition of a function 'fooo'
> | (note the different spelling) to go away even though the two
> | functions are related insofar as the new function 'foo' was created
> | by very slightly modifying the text in a file that used to be the
> | definition of 'fooo'. Am I missing something, either in my
> | understanding of Lisp or in the intent of your question?
>
> You missed the point of the question: Why do you believe that
> editing source to effectively remove a function would magically
> transfer to the environment when you re-load the file?
I *don't* believe that. I understand that in Lisp functions (and other
things) that have been defined *don't* just go away for no reason. My
original question was about how people dealt with the consequences of
that fact: while it's all perfectly deterministic to the computer, it
seems--to me, given my current state of experience in Lisp and in
programming generally--that this is an area where a slight slip of
concentration on my part could lead to hard-to-figure-out behavior
down the road (such as when the code that is actually running in the
image was not generated from any source code I can find in my emacs
buffers or in the filesystem.)
> There is some core belief here that probably stem from other
> languages, but which does not hold for Common Lisp. E.g., if you
> are perhaps used to dynamic loading of class files which replace
> themselves with the new version, that could explain why you think
> the old definitions "vanish".
Is it possible that I need to change my email address? ;-) Just
because I happen to know how Java works (as one example of a language
that behaves as you describe) doesn't mean I think Common Lisp works
that way.
> | I looked at DELETE-PACKAGE but [...]
>
> When nothing references the symbols, they vanish. The purpose of
> deleting the whole package is to yank the ground out from under the
> symbols. Only with some difficulty can you reference symbols in a
> deleted package. It would therefore be hard for anything to /call/
> the functions that referenced the old values. If you have old code
> running, you will want to stop it, obviously.
What about functions in other packages that reference functions named
by symbols in the now-deleted package that have already been evaluated
and/or compiled? My understanding of what DELETE-PACKAGE does seems to
jibe with Tim Bradshaw's example elsewhere in this thread, i.e. those
functions will continue to work and will continue to call the same
function they always called.
> There are a number of tacit assumptions under what you write that
> I try hard not to assume I know, so if you want to discuss the
> issues that really bother you, you have to make them clear.
I'm not sure what those assumptions are--either I'm deeply
misunderstanding you or I've managed to convey in my earlier posts
something like the exact opposite of what I intended or (possibly) you
have misunderstood what I wrote. At any rate it seems that somewhere
between my brain, my fingers, your eyes, and your brain we're losing
some important bits. Thanks for your help though.
Thomas F. Burdick
I keep things consistent using a few techniques. If I have some
source form:
(defun fooo ...)
And I edit it to say:
(defun foo ...)
I kill the old definition in the image right then and there, before I
forget. I have key bindings in Emacs set up for C-c u ..., using f
for tfb:lisp-fmakunbound, v for tfb:lisp-makunbound, c for
tfb:lisp-delete-class, etc. When appropriate, I'll make
project-specific unbinding commands.
So I move point over "fooo", hit C-c u f, change it to "foo", and hit
M-C-x to get the new definition. This makes it really easy to keep
things consistent.
Occasionally, though, I go on a refactoring spree, and come out the
other end not so sure that I can bring the image up from source. In
that case, I kill the lisp image, and reload the system. Because I
keep my files compiled, except for the ones I'm actively working on,
only the files affected by the refactoring spree need to be loaded
from source, the rest are FASLs, so it's pretty fast. If you learn
good habits in keeping your sources and image in sync, it's pretty
rare to get problems here, and they're usually trivial bootstrapping
issues that take a few seconds to fix.
I usually kill my image every week or two and reload everything, just
in case. It's been some time since I've found a problem this way that
I didn't already know about.
I know your fear about keeping your system buildable from source. I
was horrified the first time I realized I didn't have all the source
around to rebuild an image I'd killed. It's easy to make sure that
the definitions in your source are loaded in the image, the only
slightly difficult thing is to delete things from your image when you
delete them from your source. Give yourself key bindings to do this,
and suddenly the whole process is easy.
--
/|_ .-----------------------.
,' .\ / | No to Imperialist war |
,--' _,' | Wage class war! |
/ / `-----------------------'
( -. |
| ) |
(`-. '--.)
`. )----'
Coby Beck
"Peter Seibel" <pe...@javamonkey.com> wrote
> Erik Naggum <er...@naggum.no> writes:
>
> > You missed the point of the question: Why do you believe that
> > editing source to effectively remove a function would magically
> > transfer to the environment when you re-load the file?
>
> I *don't* believe that. I understand that in Lisp functions (and other
> things) that have been defined *don't* just go away for no reason. My
> original question was about how people dealt with the consequences of
> that fact: while it's all perfectly deterministic to the computer, it
> seems--to me, given my current state of experience in Lisp and in
> programming generally--that this is an area where a slight slip of
> concentration on my part could lead to hard-to-figure-out behavior
> down the road
I'm quite sure you already have the answer to your question, but I'll chime
in here anyway: you just have to be careful :) Every language has
"gotcha's" this can be one of them. Similar ones are:
- fixing your buggy macros but not reloading every file that uses them
- defining FOO, setting a slot somewhere with #'FOO then redefining FOO and
having buggy code still attached to your slot (as came up recently in
another thread).
- defining a method specializing on a class, realizing you have specified to
high up in your heirarchy, redefining and ending up with surprises when
dispatch time comes.
As for dealing with it, I can't say it has ever been a huge problem...be
careful, occasionally restart your image...do things like (fmakunbound
'fooo) and (defmethod bar ((obj superclass)) (error "WRONG!")) and then
redefine bar for the correct class... in short lots of little tricks, no one
cure-all. FWIW, I think delete-package is over kill.
--
Coby Beck
(remove #\Space "coby 101 @ bigpond . com")
Paolo Amoroso
Burdick) wrote:
> I usually kill my image every week or two and reload everything, just
Have you considered removing all mirrors from your office :)
Paolo
--
EncyCMUCLopedia * Extensive collection of CMU Common Lisp documentation
http://www.paoloamoroso.it/ency/README
Peter Seibel
> I keep things consistent using a few techniques. If I have some
> source form:
>
> (defun fooo ...)
>
> And I edit it to say:
>
> (defun foo ...)
>
> I kill the old definition in the image right then and there, before I
> forget. I have key bindings in Emacs set up for C-c u ..., using f
> for tfb:lisp-fmakunbound, v for tfb:lisp-makunbound, c for
> tfb:lisp-delete-class, etc. When appropriate, I'll make
> project-specific unbinding commands.
Do you happen to use ilisp? I'm in the process of writing my
equivalent of tfb:list-fmakunbound and it seems that the only way to
get an arbitrary form (such as "(fmakunbound 'fooo)" is to put in in
the buffer, mark it as the region and then use eval-region-lisp or
compile-region-lisp to eval/compile it. That's fine but seems a bit
clunky.
Peter Seibel
Any ilisp experts out there? I'm trying to write some elisp to let me
rename a function, i.e. fmakunbound the old name and then replace the
old name with a new name in the text and and eval-defun-lisp the new
function definition. Below is what I've come up with. But notice the
incredibly nastly sleep-for call. Without it this function either
hangs or gets an error in the inferior lisp (ACL 6.2 in this case)
about:
Error: Package "CONTP" not found.
I put the sleep in a on a guess that somehow the eval-region-lisp to
do the fmakunbound and the eval-defun-lisp were happening too close
together intime for the inferior lisp's taste. Since that seems--based
on my experiments--to indeed be the problem, I'm wondering if there's
a way to get the various eval-*-lisp calls to block until the inferior
lisp is really done processing them. The doc strings are rather, er,
concise, so I tried to follow through the ilisp source but didn't see
anything too obvious. Anyway, here's my code:
(defun pbs:change-function-name (new-name)
(interactive "sNew name: ")
(save-excursion
(beginning-of-defun-lisp)
(let ((old-name (lisp-def-name t)))
(pbs:lisp-eval-in-context (concat "(fmakunbound '" old-name ")"))
(re-search-forward old-name)
(replace-match new-name nil t))
(sleep-for .125) ;;; <============= doesn't work without this!
(eval-defun-lisp)))
(defun pbs:lisp-eval-in-context (s)
(interactive "sForm: ")
(let ((start (point)) end)
(insert s)
(setq end (point))
(eval-region-lisp start end)
(delete-region start end)))
--
Peter Seibel
pe...@javamonkey.com
Martin Thornquist
> Do you happen to use ilisp? I'm in the process of writing my
> equivalent of tfb:list-fmakunbound and it seems that the only way to
> get an arbitrary form (such as "(fmakunbound 'fooo)" is to put in in
> the buffer, mark it as the region and then use eval-region-lisp or
> compile-region-lisp to eval/compile it. That's fine but seems a bit
> clunky.
I think you want ilisp-send.
Martin
--
"An ideal world is left as an exercise to the reader."
-Paul Graham, On Lisp
Thomas F. Burdick
At this point, you should look at the definition of eval-region-lisp,
and it should be easy to see how to write your own function like it.
(Not all the ilisp code is comprehensible in isolation, but the
eval-*-lisp functions are a case where it is).
Peter Seibel
> Peter Seibel <pe...@javamonkey.com> writes:
>
> > t...@conquest.OCF.Berkeley.EDU (Thomas F. Burdick) writes:
> >
> > > I keep things consistent using a few techniques. If I have some
> > > source form:
> > >
> > > (defun fooo ...)
> > >
> > > And I edit it to say:
> > >
> > > (defun foo ...)
> > >
> > > I kill the old definition in the image right then and there, before I
> > > forget. I have key bindings in Emacs set up for C-c u ..., using f
> > > for tfb:lisp-fmakunbound, v for tfb:lisp-makunbound, c for
> > > tfb:lisp-delete-class, etc. When appropriate, I'll make
> > > project-specific unbinding commands.
> >
> > Do you happen to use ilisp? I'm in the process of writing my
> > equivalent of tfb:list-fmakunbound and it seems that the only way
> > to get an arbitrary form (such as "(fmakunbound 'fooo)" is to put
> > in in the buffer, mark it as the region and then use
> > eval-region-lisp or compile-region-lisp to eval/compile it. That's
> > fine but seems a bit clunky.
>
> At this point, you should look at the definition of
> eval-region-lisp, and it should be easy to see how to write your own
> function like it. (Not all the ilisp code is comprehensible in
> isolation, but the eval-*-lisp functions are a case where it is).
I had actually already taken a little tour through the source and had
though about doing just that. I was sort of hoping for an official
entry point. Maybe I'll get on the ilisp mailing list and see if they
might be interested in a patch to provide such a thing. Anyway,
thanks. Now if I can just figure out that timing problem I mentioned
in a later post ...