Re: [sage-devel] Re: Suspicious file in a binary distribution?

[Tom Boothby]

Richard, are you still trying to convince us to switch to using Lisp?
And bragging that compiling a proper subset of Sage takes less time
than compiling Sage? And do you expect anybody to see you as anything
other than a ridiculous codger for continuing to beat this pair of
dead horses? Please keep this tripe off of sage-devel, since it isn't
even vaguely productive.


On Wed, Dec 2, 2009 at 8:01 AM, rjf <fat...@gmail.com> wrote:
> While it may not be entirely helpful to point it out, I thought that
> one of the advantages of using
> python was some kind of rapid development.  NTL is not in python so
> maybe it doesn't "count"?
>
> In the Maxima environment, if someone detects an error in a lisp
> function, FF then the function
> can be replaced in the run-time environment, e.g. at a command line
> (%i100)  by:
>  :lisp (defun FF(x y z) <newdefinition>)
>
> it could be run interpretively or compiled. Compilation can also be
> invoked from the command line.
>
> If the replacement works, the replacement (or a command to load the
> compiled version from a file)
> can be place in an initialization file so that the next time Maxima is
> loaded, the fix is automatically loaded.
>
> This doesn't fix the problem for people at remote locations.  Early in
> the Macsyma development when
> essentially all users were running on one of 2 or 3 computers at MIT
> via the ARPANET, such fixes
> took effect for all users. Maybe such a setup could be used for the
> web-based access to Sage.
>
> (It's kind of interesting to realize that this idea -- of users
> accessing a time-shared or networked-access
> computer to use a math system -- was standard between say 1966 and
> 1980, and continued on for some time..)
> (In the years after 1980 some 50 VAX-Macsyma test sites were set up,
> and later the program was sold etc.)
> RJF
>
>
> I think that compiling all of Maxima typically takes between 10
> minutes and an hour, depending on your
> machine and your choice of Lisp system.  That doesn't include re-
> making the lisp system itself (probably
> written in C) or the C compiler.
>
> On Dec 1, 10:28 pm, Dr David Kirkby <drkir...@gmail.com> wrote:
>> On Dec 2, 5:36 am, William Stein <wst...@gmail.com> wrote:
>>
>>
>>
>> > On Tue, Dec 1, 2009 at 7:18 PM, Dr. David Kirkby
>>
>> > <david.kir...@onetel.net> wrote:
>> > > Trying to create a binary distribution on Solaris, I find some rather suspicious
>> > > looking files are being added:
>>
>> > > sage-4.2.1-Solaris-10-sun4u-SunOS/devel/sage-main/build/temp.solaris-2.10-sun4u-2.6/sage/libs/symmetrica/
>> > > sage-4.2.1-Solaris-10-sun4u-SunOS/devel/sage-main/build/temp.solaris-2.10-sun4u-2.6/sage/libs/symmetrica/symmetrica.o
>> > > sage-4.2.1-Solaris-10-sun4u-SunOS/devel/sage-main/build/temp.solaris-2.10-sun4u-2.6/sage/libs/mpmath/
>> > > sage-4.2.1-Solaris-10-sun4u-SunOS/devel/sage-main/build/temp.solaris-2.10-sun4u-2.6/sage/libs/mpmath/utils.o
>> > > sage-4.2.1-Solaris-10-sun4u-SunOS/devel/sage-main/build/temp.solaris-2.10-sun4u-2.6/sage/libs/cremona/
>> > > sage-4.2.1-Solaris-10-sun4u-SunOS/devel/sage-main/build/temp.solaris-2.10-sun4u-2.6/sage/libs/cremona/homspace.o
>> > > sage-4.2.1-Solaris-10-sun4u-SunOS/devel/sage-main/build/temp.solaris-2.10-sun4u-2.6/sage/libs/cremona/mat.o
>> > > sage-4.2.1-Solaris-10-sun4u-SunOS/devel/sage-main/build/temp.solaris-2.10-sun4u-2.6/sage/libs/cremona/newforms.o
>> > > sage-4.2.1-Solaris-10-sun4u-SunOS/devel/sage-main/build/temp.solaris-2.10-sun4u-2.6/sage/libs/ntl/
>> > > sage-4.2.1-Solaris-10-sun4u-SunOS/devel/sage-main/build/temp.solaris-2.10-sun4u-2.6/sage/libs/ntl/ntl_GF2.o
>> > > sage-4.2.1-Solaris-10-sun4u-SunOS/devel/sage-main/build/temp.solaris-2.10-sun4u-2.6/sage/libs/ntl/ntl_GF2E.o
>> > > sage-4.2.1-Solaris-10-sun4u-SunOS/devel/sage-main/build/temp.solaris-2.10-sun4u-2.6/sage/libs/ntl/ntl_GF2EContext.o
>> > > sage-4.2.1-Solaris-10-sun4u-SunOS/devel/sage-main/build/temp.solaris-2.10-sun4u-2.6/sage/libs/ntl/ntl_GF2EX.o
>> > > sage-4.2.1-Solaris-10-sun4u-SunOS/devel/sage-main/build/temp.solaris-2.10-sun4u-2.6/sage/libs/ntl/ntl_GF2X.o
>> > > sage-4.2.1-Solaris-10-sun4u-SunOS/devel/sage-main/build/temp.solaris-2.10-sun4u-2.6/sage/libs/ntl/ntl_lzz_p.o
>> > > sage-4.2.1-Solaris-10-sun4u-SunOS/devel/sage-main/build/temp.solaris-2.10-sun4u-2.6/sage/libs/ntl/ntl_lzz_pContext.o
>> > > sage-4.2.1-Solaris-10-sun4u-SunOS/devel/sage-main/build/temp.solaris-2.10-sun4u-2.6/sage/libs/ntl/ntl_lzz_pX.o
>> > > sage-4.2.1-Solaris-10-sun4u-SunOS/devel/sage-main/build/temp.solaris-2.10-sun4u-2.6/sage/libs/ntl/ntl_mat_GF2.o
>> > > sage-4.2.1-Solaris-10-sun4u-SunOS/devel/sage-main/build/temp.solaris-2.10-sun4u-2.6/sage/libs/ntl/ntl_mat_GF2E.o
>> > > sage-4.2.1-Solaris-10-sun4u-SunOS/devel/sage-main/build/temp.solaris-2.10-sun4u-2.6/sage/libs/ntl/ntl_mat_ZZ.o
>> > > sage-4.2.1-Solaris-10-sun4u-SunOS/devel/sage-main/build/temp.solaris-2.10-sun4u-2.6/sage/libs/ntl/ntl_ZZ.o
>> > > sage-4.2.1-Solaris-10-sun4u-SunOS/devel/sage-main/build/temp.solaris-2.10-sun4u-2.6/sage/libs/ntl/ntl_ZZX.o
>>
>> > > There can not be any need for object files can there?
>>
>> > > I've no idea where the directory temp.solaris-2.10-sun4u-2.6 comes from.
>>
>> > The reason for these files is so that if a user modifies the Sage
>> > library and then types "sage -br" it takes a few seconds the first
>> > time instead of an *hour*.    This makes an absolutely dramatic
>> > difference in whether or not somebody will develop on Sage.  If their
>> > first
>> > experience is:
>>
>> >   1. Modify Sage's library code
>> >   2. Type "sage -br" and wait 1-2 hours to see the result...
>>
>> > they probably aren't left with a warm fuzzy feeling.   Given that an
>> > extracted Sage install is about 1.5GB (or more), this extra 130MB is
>> > well worth it.
>>
>> >  -- William
>>
>> Fair enough, I understand that now.
>>
>> I'm puzzled why the Solaris binary is so big compared to the others. I
>> checked over it briefly and found the links to libraries are links,
>> not duplicate copies. Perhaps I have a core dump or something like
>> that - I will look more closely. If all is well, I'll upload version
>> 4.2.1 built on the first release of Solaris 10.
>>
>> I just looked on the Wolfram Research site, to see the size of the
>> binaries for a trial version.
>>
>> http://www.wolfram.com/products/mathematica/experience/request.cgi
>>
>> Solaris is the biggest, but not by a large amount. The sizes are:
>>
>> Solaris 695.42 MB
>> Linux 647.57 MB
>> Mac  616.04 MB
>> Windows 474.76 MB
>>
>> Dave
>
> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to sage-devel-...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org

[rjf]

On Dec 2, 9:55 am, Tom Boothby <tomas.boot...@gmail.com> wrote:
> Richard, are you still trying to convince us to switch to using Lisp?

No, just pointing out that one of the supposed advantages of the Sage
development environment -- interactive development via Python
has a serious issue, and is apparently inferior to 40 year old
technology
in Lisp. No amount of bragging about how popular Python has become
for non-programmers(?) changes this situation. It might be
possible to fix Python, but to do that you would first have to realize
that it was, at least in this respect and for this usage, in need of a
fix.

Now you could argue that Python would be slower if it were so changed.

You might argue that Sage does not need this feature because ...??

You might argue that the feature is actually in Python, or will be
soon.



Now what was the feature? That you could interactively change a
system function.

As was pointed out by Nick Alexander, even when using Python directly
(as opposed to some library like NTL), you most likely cannot just
drop in a replacement definition for some system function. He says:

"And even when you do update a member function, there
is in general no way to update existing instances of the class,
meaning that your mis-behaving object is not fixed."



What Nick points out (and he is right about Lisp/CLOS including this),
is that a
more useful specification for object systems provides a mechanism
so that changes to methods or class definitions actually take effect
immediately, not at the next time you recompile everything.

This is a defect in Python with respect to the claim of John Palmieri,
in
the same thread just ahead of Nick, that functions can be replaced in
the runtime environment.

Of course, in the spirit of Sage, such criticisms are ignored, as
the subsequent thread in sage-devel shows.

> And bragging that compiling a proper subset of Sage takes less time
> than compiling Sage?

Huh? Just an observation for those who might not have this information
handy. I would find making a change in Maxima code, compiling just
that change,
and loading up a new system, just to test it, to be a burden, even if
it took 30 seconds.
Again, and again, stamping out bugs. If I had to recompile the whole
system and
load it, that would be painful. Perhaps it would be less painful if I
didn't know any
better and I thought the whole world debugged code by editing a file
and typing
"make clean; make all &" and going out for lunch.

> And do you expect anybody to see you as anything
> other than a ridiculous codger for continuing to beat this pair of
> dead horses?

Are you bragging that Sage is a dead horse? Or are you a dead horse?
It is true that on the internet you cannot tell if someone is a horse.

 Please keep this tripe off of sage-devel, since it isn't
> even vaguely productive.

Actually, I found the note by Nick to be highly informative, and I
hope John P found
it informative, since he seemed not to know that his suggestion on
patching Sage
would not work.

And perhaps others were similarly disabused of their notion of how
one might
use a little Python for patching Sage.

Since no one commented further on Nick's message to amend it in
any way, I assume his statement regarding Sage and Python was
correct. If it
is not correct, you should respond on sage-devel, not here.

And for those who have an interest in fixing a bug in Maxima, a part
of Sage, they
may now realize that it is quite effective to make fixes in Maxima,
which is written
in Lisp, from the command line in Maxima.

e.g. :lisp (defun $whoflames() "Tom Boothby")

defines a new lisp function, accessible from maxima, with the name
whoflames().


but more significantly, and in contrast to Sage/Python,
one can change an arbitrary system routine internal to Maxima, e.g.
solve1a, part of the
solution method for single equations, in the same way, online.

Also, such fixes can be inserted in the initialization file, but I've
already said that.

Regards
RJF

[Robert Bradshaw]

On Dec 4, 2009, at 10:27 AM, rjf wrote:

> On Dec 2, 9:55 am, Tom Boothby <tomas.boot...@gmail.com> wrote:
>> Richard, are you still trying to convince us to switch to using Lisp?
>
> No, just pointing out that one of the supposed advantages of the Sage
> development environment -- interactive development via Python
> has a serious issue, and is apparently inferior to 40 year old
> technology
> in Lisp. No amount of bragging about how popular Python has become
> for non-programmers(?) changes this situation. It might be
> possible to fix Python, but to do that you would first have to realize
> that it was, at least in this respect and for this usage, in need of a
> fix.

In some ways Lisp is better than Python, in others Python is better
than Lisp.

Sage chose Python and is thriving--I doubt choosing Lisp would have
made things better (in fact I quite suspect the opposite).

> Of course, in the spirit of Sage, such criticisms are ignored, as
> the subsequent thread in sage-devel shows.

This is because responding to us usually doesn't do any good.

>> Please keep this tripe off of sage-devel, since it isn't
>> even vaguely productive.
>
> Actually, I found the note by Nick to be highly informative,

That's because he was both wrong and supported your point of view.

> and I hope John P found
> it informative, since he seemed not to know that his suggestion on
> patching Sage would not work.
>
> And perhaps others were similarly disabused of their notion of how
> one might use a little Python for patching Sage.
>
> Since no one commented further on Nick's message to amend it in
> any way, I assume his statement regarding Sage and Python was
> correct. If it is not correct, you should respond on sage-devel, not
> here.

Done.

> And for those who have an interest in fixing a bug in Maxima, a part
> of Sage, they may now realize that it is quite effective to make
> fixes in Maxima,
> which is written in Lisp, from the command line in Maxima.

Yes, this could be useful.

- Robert

[Tom Boothby]

> Are you bragging that Sage is a dead horse? Or are you a dead horse?
> It is true that on the internet you cannot tell if someone is a horse.

lol. Go ahead and interpret anything I saw however you like. It
amuses me. But for the record, the "two dead horses" I referred to
were "lisp > python" and "maxima \subset sage"

>   Please keep this tripe off of sage-devel, since it isn't
>> even vaguely productive.
>
> Actually, I found the note by Nick to be highly informative, and I
> hope John P found
> it informative, since he seemed not to know that his suggestion on
> patching Sage
> would not work.

Agreed! I eat my hat!

[rjf]

On Dec 4, 10:58 am, Robert Bradshaw <rober...@math.washington.edu>
wrote:
...

>
> In some ways Lisp is better than Python, in others Python is better  
> than Lisp.

no doubt.

>
> Sage chose Python and is thriving--I doubt choosing Lisp would have  
> made things better (in fact I quite suspect the opposite).

No clear way of making this determination.

>
> > Of course, in the spirit of Sage, such criticisms are ignored, as
> > the subsequent thread in sage-devel shows.
>
> This is because responding to us usually doesn't do any good.

Huh? Responding to Nick?

>
> >>  Please keep this tripe off of sage-devel, since it isn't
> >> even vaguely productive.
>
> > Actually, I found the note by Nick to be highly informative,
>
> That's because he was both wrong and supported your point of view.
>

I read the thread in sage-devel again and I don't see how your
response
exactly demonstrates that Nick is wrong. Indeed it agrees that it
doesn't work for Cython.

I am unclear about whether the redefinition of is_finite fully
contradicts Nick's
assertion, or is an illustration of something similar. Cython aside.

If it is not correct, you should respond on sage-devel, not  
> > here.
>
> Done.

Thanks.

[rjf]

On Dec 4, 11:08 am, Tom Boothby <tomas.boot...@gmail.com> wrote:
.  But for the record, the "two dead horses" I referred to
> were "lisp > python" and "maxima \subset sage"
>

Huh?
I think that one way of improving a programming language P is to
notice when another language L
does something better, and try to figure out if some insight with
respect to P can be found.

I am not sure what you mean by "\", but obviously Maxima is, or
certainly could be, a subset
of Sage, if the linkages involved are appropriately general.

> >   Please keep this tripe off of sage-devel, since it isn't
> >> even vaguely productive.
>
> > Actually, I found the note by Nick to be highly informative, and I
> > hope John P found
> > it informative, since he seemed not to know that his suggestion on
> > patching Sage
> > would not work.
>
> Agreed!  I eat my hat!

Well, if John P is not informed by Nick's message, and Nick is wrong,
then
Nick would be informed by Robert's message. So someone is learning
something.

[Robert Bradshaw]

On Dec 4, 2009, at 11:37 AM, rjf wrote:

>>> Of course, in the spirit of Sage, such criticisms are ignored, as
>>> the subsequent thread in sage-devel shows.
>>
>> This is because responding to us usually doesn't do any good.
>
> Huh? Responding to Nick?

No, bring up "Lisp is better than Python" on a thread about including
binary files. You like to argue, but usually it just wastes everyone's
time and doesn't accomplish much if anything.

In general we respond to criticism, but yours more often than not just
feels like flame-bait.

>>
>>>> Please keep this tripe off of sage-devel, since it isn't
>>>> even vaguely productive.
>>
>>> Actually, I found the note by Nick to be highly informative,
>>
>> That's because he was both wrong and supported your point of view.
>>
>
> I read the thread in sage-devel again and I don't see how your
> response
> exactly demonstrates that Nick is wrong. Indeed it agrees that it
> doesn't work for Cython.
>
> I am unclear about whether the redefinition of is_finite fully
> contradicts Nick's
> assertion, or is an illustration of something similar. Cython aside.

To be clear, your interpretation of Nick's response was wrong, he
didn't say it's impossible in Python, just in Cython. I demonstrated
that one can re-define a class method and have all instances of that
class start using the new definition automatically at runtime,
contrary to your assertion that this was something that Python didn't
support.

- Robert

[Harald Schilly]

On Dec 4, 7:27 pm, rjf <fate...@gmail.com> wrote:
> No, just pointing out that one of the supposed advantages of the Sage
> development environment -- interactive development via Python
> has a serious issue, and is apparently inferior to 40 year old
> technology
> in Lisp.

ok, i'm probably unable to follow you, it's late and after a party.

i think, now we are talking about "development". So, you think, that
changing something in python is "painful"

> Perhaps it would be less painful if I
> didn't know any
> better and I thought the whole world debugged code by editing a file
> and typing
> "make clean; make all &" and going out for lunch.

I think you think that it takes a long time to compile if something is
edited in a file in the sage library.

Well...

big mixup :(

i try to dissect the pieces

1. we were talking about redefining a function at runtime in the
python environment.
1.1. a function is not a class-method.
some were talking about redefining a class method, that has no effect
on already instantiated objects - that's true!
but methods aren't functions (in my use of the language)
i have given you an example how to redefine a function definition at
runtime in the python environment. have you examined it?
i also added that in my humble opinion it makes no sense to moify a
function at runtime and add that modification to a script that is run
at startup. rather, it's better to edit the file directly! i also
pointed out that we ship a history of all modification, so that it is
easy to collect all new modifications in a patch. have you read that?

2. we are talking about the "efficiency of development"
2.1 do you know how long it takes to modify a python file in the sage
library, rebuild it and run sage again? yes or no?

In the following i'm up to editing a line in sage/algebras/
steenrod_algebra.py

harri@kyle:/opt/sage/current$ date
Sat Dec 5 00:51:16 CET 2009
harri@kyle:/opt/sage/current$ vim devel/sage-main/sage/algebras/
steenrod_algebra.py
harri@kyle:/opt/sage/current$ ./sage -br

----------------------------------------------------------
sage: Building and installing modified Sage library files.


Installing c_lib
scons: `install' is up to date.
Updating Cython code....
Time to execute 0 commands: 3.00407409668e-05 seconds
Finished compiling Cython code (time = 3.25007295609 seconds)
running install
running build
running build_py
copying sage/algebras/steenrod_algebra.py -> build/lib.linux-i686-2.6/
sage/algebras
running build_ext
Total time spent compiling C/C++ extensions: 0.109875202179 seconds.
running install_lib
copying build/lib.linux-i686-2.6/sage/algebras/steenrod_algebra.py -> /
opt/sage/current/local/lib/python2.6/site-packages/sage/algebras
byte-compiling /opt/sage/current/local/lib/python2.6/site-packages/
sage/algebras/steenrod_algebra.py to steenrod_algebra.pyc
running install_egg_info
Removing /opt/sage/current/local/lib/python2.6/site-packages/
sage-0.0.0-py2.6.egg-info
Writing /opt/sage/current/local/lib/python2.6/site-packages/sage-0.0.0-
py2.6.egg-info
----------------------------------------------------------------------
| Sage Version 4.2.1, Release Date: 2009-11-14 |
| Type notebook() for the GUI, and license() for information. |
----------------------------------------------------------------------
sage:
Exiting SAGE (CPU time 0m0.06s, Wall time 0m2.77s).
harri@kyle:/opt/sage/current$ date
Sat Dec 5 00:51:37 CET 2009

----

important lines:
Sat Dec 5 00:51:16 CET 2009
Sat Dec 5 00:51:37 CET 2009

so, it takes about 1 minute on a 5 year old laptop to modify a file in
the sage library. files have to be examined, cython files probably
recompiled, some things happen, etc. but overall, not enough time to
take a lunch, isn't it?

2.2 imagine you develop something "new" in a sage or python file. e.g.
inside a running sage instance, you can do this:

sage: attach file.py

inside that file is the function definition i was talking about
earlier ... basically f(x) = 2*x
now i change it to f(x) = 3*x and save the file. after saving the
file, the modification is instantaneous available inside sage! no need
to reload sage, reload the file or anything else!

f(2) gives 4, edit file, save, f(2) gives 6!

that helps to rapidly develop scripts, doesn't it?

editing anything in a library that sage uses is only indirectly
related to sage. it's up to those other projects how long it takes to
recompile after modifications - but that's something for their
development discussion lists and not ours.

h

[rjf]

On Dec 4, 4:26 pm, Harald Schilly <harald.schi...@gmail.com> wrote:
> On Dec 4, 7:27 pm, rjf <fate...@gmail.com> wrote:
>

.....

> I think you think that it takes a long time to compile if something is
> edited in a file in the sage library.
>
> Well...
>
> big mixup :(

...

>
> i try to dissect the pieces
>
> 1. we were talking about redefining a function at runtime in the
> python environment.
> 1.1. a function is not a class-method.

If you want to debug a function, then that's nice you can redefine a
function at runtime.

> some were talking about redefining a class method, that has no effect
> on already instantiated objects - that's true!

Um what was Robert saying about redefining is_finite ? If that is a
function, not a class method, then we are talking about different
things. When I (and Nick, I think) are talking about re-definitions,
we are talking about parts of the system which you might find to be
erroneous and wish to patch without much ado. Nick (and now you,
Harald) seem to be saying that redefining a class method has no effect
on already instantiated objects. So you can't patch those parts of the
system which require redefining a class method. Right? Especially you
can't if you used cython.

Robert, on the other hand, is either contradicting this if: is_finite
is a "class method" or he has decided to answer a different question
(Can you redefine a "function"), thereby suggesting that the answer to
his chosen question about functions should influence what Nick,
Harald, and I believe about our chosen question about "class
methods", even though they have different answers.

> but methods aren't functions (in my use of the language)
> i have given you an example how to redefine a function definition at
> runtime in the python environment. have you examined it?

Well, in some programming languages there is no distinction between
methods and functions. Again, if you are saying you can redefine
functions but you can't redefine methods (in Python OR Cython), then
that's cool with me. It is what I believed from Nick's note. But
what was Robert saying about is_finite ??

I'm sorry if this seems obtuse, but if you can't agree among
yourselves, what difference does it make if I've examined your
redefinition of
(something).

> i also added that in my humble opinion it makes no sense to moify a
> function at runtime and add that modification to a script that is run
> at startup.

Depends on how sure you are about your fix. Do you want to put it in
place in the file, or do you want to compose something as a fix file
that (in lisp) might look like this:

;;; following stuff fixes bug reported on x/x/x by zzz
;;; ... description of bug W ....
;; redefinition of function foo in file foofile.lisp
;; (defun foo .....)
;; redefinition of function bar in file barfile.lisp
;; (defun bar ....)
;;; end of fixes for bug W ...

........
after some testing, and assuming the fix doesn't break anything else,
the changes to file foofile and barfile can be committed to the source
code.

[actually, the fix file may also be compiled by the lisp compiler to
see if there
are some warnings generated, etc. ]

> rather, it's better to edit the file directly!

I suppose that if you can back-out your changes gracefully, that would
be possible. It doesn't seem to me to be better, but whatever your
opinion ...

> i also
> pointed out that we ship a history of all modification, so that it is
> easy to collect all new modifications in a patch. have you read that?

I do not always want to collect all new modifications. I generally
want to use the previous "released" system, and try to fix only the
particular bug I am tracking down.

Other peoples' bug fixes may be distractions. Especially if, as is
too often the case, that there is a bug in their fix.

>
> 2. we are talking about the "efficiency of development"
> 2.1 do you know how long it takes to modify a python file in the sage
> library, rebuild it and run sage again? yes or no?

No, but your anecdotal evidence says...
.......

> important lines:
> Sat Dec  5 00:51:16 CET 2009
> Sat Dec  5 00:51:37 CET 2009
>
> so, it takes about 1 minute

huh?

I guess you are allowing 39 seconds to edit the file and
21 seconds for the computer to incorporate the change.

> but overall, not enough time to
> take a lunch, isn't it?

You see to miss a major point, which is the following:

You don't want to test your bug fix in a freshly compiled and loaded
system. You want to test your bug fix in the system that just
exhibited the bug, at the
point where the bug manifested itself. It may have taken you 10
minutes of fiddling, and computing stuff to get to the bug spot. It
may take you another 10 minutes to get to that spot again. So you
have lost your train of thought and it may require substantial
intellectual effort to regain it.

No, 21 seconds is not even time to take a pee, unless maybe you happen
to be using your laptop in the lavatory.

Even if your bug/fix is exhibited by a 0-time command to Sage, 21
seconds (really, any time>0) is long enough to degrade your attention
to the task at hand, even if it is slight.
Studies have suggested that faster responses are "better" and that it
even matters down to fractions of a second. (I don't have references
handy, but you could find an HCI specialist at UW.)

>
> 2.2 imagine you develop something "new" in a sage or python file. e.g.
> inside a running sage instance, you can do this:
>
> sage: attach file.py
>
> inside that file is the function definition i was talking about
> earlier ... basically f(x) = 2*x
> now i change it to f(x) = 3*x and save the file. after saving the
> file, the modification is instantaneous available inside sage! no need
> to reload sage, reload the file or anything else!

In Maxima I suppose you could have a command edit_then_reload
("patchfile")
by which your changes are available inside Maxima as soon as you quit
the editor.

If Sage is constantly monitoring the file file.py for changes, even if
the file is altered
by (say) another user, that's an interesting feature. Is that what it
does? I'm not
sure it is terribly useful since it seems kind of non-deterministic.
For example,
if f is some kind of recursive function, and the definition of f is
changed in the file,
does the definition of f get changed while it is running?

>
> f(2) gives 4, edit file, save, f(2) gives 6!

again, does "edit file" get initiated from Sage? and could someone
else edit it?

>
> that helps to rapidly develop scripts, doesn't it?

Well, I have found your suggested method of developing a script to be
unsatisfactory, in Maxima and in Mathematica.

I develop a script by fiddling with some commands to try some things
out, but before I go too far, I write out a script in a file. The
file
begins with a command that resets the system as nearly as possible
to a standard state, so I can run the script, repeatedly and get the
same result. When I edit the script and run it again, I know that
someone else running the script will get the same result (assuming
deterministic results etc...).
Starting up a computation and midway editing some attached file
sounds like a really bad idea. How would you know what you had
computed along the way.

Maybe all your computations are one-shot single commands?
Try to debug a memory allocation issue your way..


RJF

[Harald Schilly]

On Dec 5, 2:28 am, rjf <fate...@gmail.com> wrote:
> Um what was Robert saying about redefining is_finite ?  If that is a
> function, not a class method, then we are talking about different
> things.  When I (and Nick, I think) are talking about re-definitions,
> we are talking about parts of the system which you might find to be
> erroneous and wish to patch without much ado.

Uhm ok, I don't know who is talking about what, but there are
different types of methods. I don't want to read into all details, but
the basic mechanism is, that a method is either available with the
module (or "by default" there), or part of an object (then it depends
on the instantiation)
"classmethods" -> http://docs.python.org/library/functions.html#classmethod

> Well, in some programming languages there is no distinction between
> methods and functions.

My use of the language isn't perfect. I think the problem is that
there are more things than words and the words are used too general.

> after some testing, and assuming the fix doesn't break anything else,
> the changes to file foofile and barfile can be committed to the source
> code.
>

yes

> > rather, it's better to edit the file directly!
>
> I suppose that if you can back-out your changes gracefully, that would
> be possible. It doesn't seem to me to be better, but whatever your
> opinion ...

we use the mercurial version control system. in particular, it has a
mechanism called "mercurial queues". it allows one to keep a stack of
patches on top of the base version without actually committing
anything. you can remove such a stack and you get the unmodified
source code, you can switch between stacks. you can add a stack of
patches, edit something, update the last patch and more.
http://wiki.sagemath.org/MercurialQueues

i think that's a good mechanism, because you actually do the work
exactly there were the patch is and you do not have to work with an
intermediate initialization file that is not the real deal.

> I do not always want to collect all new modifications.  I generally
> want to use the previous "released" system, and try to fix only the
> particular bug I am tracking down.
>
> Other peoples' bug fixes may be distractions.  Especially if, as is
> too often the case, that there is a bug in their fix.

you only get the last released system, other things are local to the
other users on their computers or they are uploaded to
trac.sagemath.org ... all bugs that are there are part of a release
sage system!

> You see to miss a major point, which is the following:
>
> You don't want to test your bug fix in a freshly compiled and loaded
> system. You want to test your bug fix in the system that just
> exhibited the bug, at the
> point where the bug manifested itself.  It may have taken you 10
> minutes of fiddling, and computing stuff to get to the bug spot. It
> may take you  another 10 minutes to get to that spot again.  So you
> have lost your train of thought and it may require substantial
> intellectual effort to regain it.

you can go back in the history, or even better write a script that
executes all the code which brought up the code. it's side effect free
to restart from 0 each time.

additionally, our policy at sage is that you have to generate a test
case that shows that the bug is fixed. therefore you have to supply
the list of commands that nail down the bug in a buggy version and
this test is executed in every version later on to make sure, that the
bug doesn't show up again.

> If Sage is constantly monitoring the file file.py for changes, even if
> the file is altered
> by (say) another user, that's an interesting feature. Is that what it
> does?

I think it's a feature of the posix file system to notify observers of
changes. It's like "tail -f logfile.log".

Well, isn't it a bit crazy to edit a function definition while it is
running? That's something I'll try, but I think it is no problem. When
the function is parsed and created in memory when executed, it uses
closures to keep references to itself and so on.

> > f(2) gives 4, edit file, save, f(2) gives 6!
>
> again, does "edit file"   get initiated from Sage?

no, i have a window system and two terminals open. in the first one is
sage running, in the second one vi or emacs. after attach, i don't
have to close sage or the editor, just do "save" and sage reads the
file. no processes are restarted!

> and could someone
> else edit it?

if someone else logs in on the same machine and has edit rights to the
file, yes

> I develop a script by fiddling with some commands to try some things
> out, but before I go too far, I write out a script in a file.  The
> file
> begins with a command that resets the system as nearly as possible
> to a standard state, so I can run the script, repeatedly and get the
> same result.

you cannot be 100% sure that your handwritten initialization really
resets everything.
it's better to restart the entire process to be 100% sure.

> Starting up a computation and midway editing some attached file
> sounds like a really bad idea.  How would you know what you had
> computed along the way.

what along the way? it depends what's in the file! usually, no global
variables are defined and therefore it has no side effects and what
you already calculated is essentially lost. it's nice to develop in
one window the script, and execute it in the other one until it works.
the shorter this cycle is, the more you can do without distractions.

>
> Maybe all your computations are one-shot single commands?

no, but they can be if the computation is stored in one file - i.e. an
"attached" one - that is called by one command. everything can be
reduced to a one-shot single command!

> Try to debug a memory allocation issue your way..

we do not explicitly allocate memory in python and python has a
garbage collector

h

[rjf]

On Dec 5, 3:49 am, Harald Schilly <harald.schi...@gmail.com> wrote:
....

>
> Uhm ok, I don't know who is talking about what, but there are
> different types of methods. I don't want to read into all details, but
> the basic mechanism is, that a method is either available with the
> module (or "by default" there), or part of an object (then it depends
> on the instantiation)
> "classmethods" ->http://docs.python.org/library/functions.html#classmethod

So you it appears that maybe some things -- methods of some kind --
cannot
be patched interactively. OK, that was my impression. Tom seems to
disagree with you though.

> we use the mercurial version control system. in particular, it has a
> mechanism called "mercurial queues".

......

> i think that's a good mechanism, because you actually do the work
> exactly there were the patch is and you do not have to work with an
> intermediate initialization file that is not the real deal.

This may be a benefit, however you have to re-load the whole file in
which the
patch occurs, and this may not be a good thing. For example, there
may be
dependencies on the order in which files are loaded, and a long chain
of
activities may be triggered. But this is not so likely, and
computers are
so fast these days...

....

> > You see to miss a major point, which is the following:
>
> > You don't want to test your bug fix in a freshly compiled and loaded
> > system. You want to test your bug fix in the system that just
> > exhibited the bug, at the
> > point where the bug manifested itself.  It may have taken you 10
> > minutes of fiddling, and computing stuff to get to the bug spot. It
> > may take you  another 10 minutes to get to that spot again.  So you
> > have lost your train of thought and it may require substantial
> > intellectual effort to regain it.
>
> you can go back in the history, or even better write a script that
> executes all the code which brought up the code. it's side effect free
> to restart from 0 each time.

You still seem to be missing the point. Can you automatically and
instantly take a newly loaded system and return it to the point at
which
you noticed an error? In a newly loaded system you don't have any
history,
so I assume that you would have to somehow dump out the history from
one system and construct and script and reload that in the new system.

You can spend a long time preparing to duplicate the bug, writing a
script
etc. Or you can interactively patch a program, right away.

>
> additionally, our policy at sage is that you have to generate a test
> case that shows that the bug is fixed. therefore you have to supply
> the list of commands that nail down the bug in a buggy version and
> this test is executed in every version later on to make sure, that the
> bug doesn't show up again.

Well that is a fine idea to do AFTER you've fixed the bug. What I'm
pointing out is that the process of figuring out how to fix the bug is
helped by being able to try things out interactively, not reloading
the system each time. (And, to point out the obvious to Tom, that
this is apparently harder to do with Python/Sage than Lisp).

>
>
>
> Well, isn't it a bit crazy to edit a function definition while it is
> running?

Sure. Which is why I'm curious about what the semantics would be.

>That's something I'll try, but I think it is no problem. When
> the function is parsed and created in memory when executed, it uses
> closures to keep references to itself and so on.

It is possible to edit a function in Lisp while it is running. Not a
great idea,
though.

>
> no, i have a window system and two terminals open. in the first one is
> sage running, in the second one vi or emacs. after attach, i don't
> have to close sage or the editor, just do "save" and sage reads the
> file. no processes are restarted!

I suppose that could be a minor convenience, though I think there
might
be times when I would save a file and NOT want it loaded back in to
Sage.

>
> > I develop a script by fiddling with some commands to try some things
> > out, but before I go too far, I write out a script in a file.  The
> > file
> > begins with a command that resets the system as nearly as possible
> > to a standard state, so I can run the script, repeatedly and get the
> > same result.
>
> you cannot be 100% sure that your handwritten initialization really
> resets everything.

Actually, there is a command like kill(all) or clear(all) that
resets everything.
It's part of the system. If Sage does not have it, I guess it could
be simulated by
some script mechanism in which a Sage is killed/rebooted/continues to
read a script?

> it's better to restart the entire process to be 100% sure.

But then you have to do
kill Sage
start Sage
start script.

Instead of start script
where the first line in the script is clear(all).

>
> > Starting up a computation and midway editing some attached file
> > sounds like a really bad idea.  How would you know what you had
> > computed along the way.
>
> what along the way?

You type something. You define the function f. You run the function
f.
You attach a file. You define the function f in the file.
You run the function f.
You look at the history.
There are several calls to the function f.
What definition is used?
Who knows? Midway you edited it in an attached file.

> it depends what's in the file!

It also depends on when something is done in the file.

> usually, no global
> variables are defined and therefore it has no side effects

I don't know about Sage generally, but Maxima has LOTS of global
variables and flags controlling behavior, and the user can define
additional global (to Maxima) variables.

>and what
> you already calculated is essentially lost.

Is a fresh Maxima is started up each time Maxima is used from the same
Sage session?? If that is the case, I would have to say
that Maxima is not by any means a subset of Sage. A Maxima without
persistent memory of calculations between commands
is severely handicapped.

> it's nice to develop in
> one window the script, and execute it in the other one until it works.
> the shorter this cycle is, the more you can do without distractions.

I agree. I do just this. In fact, one way I work is to have an emacs
with two buffers, one with a script and the other
with a buffer linked to a computer algebra system -- Mathematica,
Maxima, or some other system.

>
>
>
> > Maybe all your computations are one-shot single commands?
>
> no, but they can be if the computation is stored in one file - i.e. an
> "attached" one - that is called by one command. everything can be
> reduced to a one-shot single command!

yes, this is possible, but exploratory computations are another quite
useful way of interacting with the computer,
unscripted.

>
> > Try to debug a memory allocation issue your way..
>
> we do not explicitly allocate memory in python and python has a
> garbage collector

I have seen reported problems that are mentioned in sage-devel that
look like this:

I did X, Y, Z, and then after a long time Maxima gave some strange
message that seemed to be related
to memory allocation. (Perhaps a bug in the way Sage called Maxima?)

Anyway, you could put X,Y,Z in a script, but each debugging attempt
would take a long time to get started.
The person trying to figure out that situation would be hampered
substantially.

Lisp, as you know, does not provide for explicit allocation of memory
either, and also has a garbage collector.
that does not mean memory allocation issues disappear. There are no
user-caused bugs of dangling pointers etc,
but if you run out of memory (or some memory component like stack, or
array space or string space or ....)
then that may be because of a bug that needs fixing.

RJF

[Marshall Hampton]

I've stopped actually reading your posts, so I don't know what you
just said. I'll begin to do so after you have made a truly
constructive contribution to sage - a useful patch in trac. You often
claim to have helpful intentions, but that's just hot air and
meaningless until you actually _do_ something. You also often make
statements that make it clear that you don't understand about 95% of
sage and what it is and what it is for. Show me the code, or go away.

-Marshall Hampton

[William Stein]

On Sat, Dec 5, 2009 at 9:27 PM, Marshall Hampton <hamp...@gmail.com> wrote:
> I've stopped actually reading your posts, so I don't know what you
> just said.  I'll begin to do so after you have made a truly
> constructive contribution to sage - a useful patch in trac.  You often
> claim to have helpful intentions, but that's just hot air and
> meaningless until you actually _do_ something.  You also often make
> statements that make it clear that you don't understand about 95% of
> sage and what it is and what it is for.  Show me the code, or go away.

+1

I would like to encourage everybody else to follow Marshall's example.

-- William

> --
>
> You received this message because you are subscribed to the Google Groups "sage-flame" group.
> To post to this group, send email to sage-...@googlegroups.com.
> To unsubscribe from this group, send email to sage-flame+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/sage-flame?hl=en.
>
>
>



--
William Stein
Associate Professor of Mathematics
University of Washington
http://wstein.org

[Harald Schilly]

I'm pretty sure that's my last email in this thread here, too.

On Dec 5, 5:28 pm, rjf <fate...@gmail.com> wrote:

>  In a newly loaded system you don't have any
> history

false

>  Or you can interactively patch a program, right away.

and the next time you run the program the fix is still there?

> ... the process of figuring out how to fix the bug is

> helped by being able to try things out interactively, not reloading
> the system each time.

I told you how it can be done interactively.

Do you concur with me that it is important to have a test to actually
know if the bug is fixed? (Now and any time in the future!)

> I think there
> might
> be times when I would save a file and NOT want it loaded back in to
> Sage.

Don't use attach.

> Actually, there is a command like   kill(all)   or   clear(all) that
> resets everything.

> It's part of the system.  If Sage does not have it ...

help of reset() says:

"Delete all user defined variables, reset all globals variables back
to their default state, and reset all interfaces to other computer
algebra systems. [...]"

> Maxima has LOTS of global
> variables and flags controlling behavior, and the user can define
> additional global (to Maxima) variables.

That's a bad design, especially flags that modify the behavior. e.g.
for each reported problem you have to additionally provide the set of
all global flags. Testing is also much more complicated, since you
have to essentially test all possible combinations of all global
flags ...

> Is a fresh Maxima is started up each time Maxima is used from the same
> Sage session??

No

> I did X, Y, Z, and then after a long time Maxima gave some strange
> message that seemed to be related
> to memory allocation.

Please investigate such problems and send us a patch. You know maxima
better and you are a better lisp programmer, therefore you probably
know how to improve and fix it!

H

[rjf]

On Dec 6, 4:22 am, Harald Schilly <harald.schi...@gmail.com> wrote:
> I'm pretty sure that's my last email in this thread here, too.
>
> On Dec 5, 5:28 pm, rjf <fate...@gmail.com> wrote:
>
> >  In a newly loaded system you don't have any
> > history
>
> false

Look. the point is, the time you must wait for the computer to get to
the point in the computation that previously
exposed a bug, to see if the bug fix repairs the error, is critical to
human efficiency. You wait longer -- you lose
some of the context, the attention, etc. If you can keep this down to
a minimum, say the time to type the
fix, you win. If you need to allow extra time to reload a system,
load some script (perhaps based on some history file?)
you are not as efficient.

Are you disputing this?

>
> >  Or you can interactively patch a program, right away.
>
> and the next time you run the program the fix is still there?

Ordinarily, you test the patch right away, and if it is a correct
patch, you would take the emacs buffer in which it exists,
and append it to a fix file, which could be loaded "next time" either
automatically (if in an initialization file) or on demand.

>
> > ... the process of figuring out how to fix the bug is
> > helped by being able to try things out interactively, not reloading
> > the system each time.
>
> I told you how it can be done interactively.

Someone said, in this thread,

"1.1. a function is not a class-method.

some were talking about redefining a class method, that has no effect
on already instantiated objects - that's true! "

So my reading of this is that already instantiated objects have the
wrong class method attached to them.
So whoever wrote this, must disagree with you that fixes can be done
interactively if the fix requires
that already instantiated objects be repaired.

Oh dear, YOU said that.

>
> Do you concur with me that it is important to have a test to actually
> know if the bug is fixed? (Now and any time in the future!)

Sure, I am not opposed to regression testing.

>
> > I think there
> > might
> > be times when I would save a file and NOT want it loaded back in to
> > Sage.
>
> Don't use attach.

OK, the choice is between using attach, with automatic reloading,
or just editing, with occasional explicit loading.
I have not found the latter to be a problem.

>
> > Actually, there is a command like   kill(all)   or   clear(all) that
> > resets everything.
> > It's part of the system.  If Sage does not have it ...
>
> help of reset() says:
>
> "Delete all user defined variables, reset all globals variables back
> to their default state, and reset all interfaces to other computer
> algebra systems. [...]"

OK, I don't know what it means to reset the interface to Maxima --
does it kill the Maxima system, reload a new one, and
attach to it? Or does it issue a kill(all) in Maxima?

Why don't you just kill the whole Sage process and all its hangers-
on? Wouldn't that be a better guarantee that everything was actually
reset ? Or is that actually what reset() does?

>
> > Maxima has LOTS of global
> > variables and flags controlling behavior, and the user can define
> > additional global (to Maxima) variables.
>
> That's a bad design, especially flags that modify the behavior.

OK, let's say you were around 40 years ago, and could have influenced
the design of Macsyma.
You sit around with the design team and say "global flags are a bad
design". They say to you,
Harald, for sure you are right. What do you suggest? And you
say, .....

> e.g.
> for each reported problem you have to additionally provide the set of
> all global flags.

It would seem necessary in general, but most global flags are
irrelevant to any particular bug... they are set to their default
values.
It is also possible to save an environment (which includes such
matters), by a single command, save("filename",all).

Testing is also much more complicated, since you
> have to essentially test all possible combinations of all global
> flags ...

Yes, that would seem to be necessary. It isn't done. Some settings
of global flags (e.g. turning simplifier off)
break many things.

Oh, if Sage does such a great job of testing, it seems to me that Sage
would have to test all the commands that
access Maxima under all possible flag settings. So I guess Sage
doesn't test so thoroughly.

Some flags have an infinite number of possible settings, e.g. modulus
can be set to any prime number.

>
> > Is a fresh Maxima is started up each time Maxima is used from the same
> > Sage session??
>
> No
>
> > I did X, Y, Z, and then after a long time Maxima gave some strange
> > message that seemed to be related
> > to memory allocation.
>
> Please investigate such problems and send us a patch.

You don't pay me enough to fix bugs reported in the Sage+Maxima
environment..

You know maxima
> better and you are a better lisp programmer, therefore you probably
> know how to improve and fix it!

You flatter me~!

R

[rjf]

On Dec 5, 6:27 pm, Marshall Hampton <hampto...@gmail.com> wrote:
> I've stopped actually reading your posts, so I don't know what you
> just said.  

No problem. My comments and yours are part of the great internet
archive.

If you think that contributions to a computer system must consist of
bug fixes, so be it.

Do you ever read Dilbert?

[rjf]

On Dec 5, 6:41 pm, William Stein <wst...@gmail.com> wrote:

>
> I would like to encourage everybody else to follow Marshall's example.
>

Are you auditioning for the part of the pointy-haired boss? :)

[Tom Boothby]

> Look.  the point is, the time you must wait for the computer to get to
> the point in the computation that previously
> exposed a bug, to see if the bug fix repairs the error, is critical to
> human efficiency.  You wait longer -- you lose
> some of the context, the attention, etc.

tl;dr

[rjf]

are you sure you didn't mean

tms;du
?