Re: [sage-devel] Two articles of interest to Sage in latest Notices
William Stein
> On Wed, 2012-01-18 at 23:45 +0100, Burcin Erocal wrote:
>> On Wed, 18 Jan 2012 12:26:33 -0800 (PST)
>> kcrisman <kcri...@gmail.com> wrote:
>>
>> > Publishing Computational Mathematics, by Tim Daly (of Axiom, a
>> > frequent contributor on sage-devel)
>> > http://www.ams.org/notices/201202/rtx120200320p.pdf
>>
>> Literate programming is not just adding comments to code, but here are
>> a few numbers nevertheless...
>>
>> FriCAS (I couldn't find Axiom on ohloh.net):
>> https://www.ohloh.net/p/fricas
>> # of lines of code: 1218007
>> # of comment lines: 205615
>
> http://en.wikipedia.org/wiki/Axiom_computer_algebra_system
> http://axiom-developer.org
>
> Axiom has a project focus to become a literate program.
> FriCAS has a project focus to compete with Mathematica.
> Due to this philosophical difference the Axiom project
> forked.
>
> New computational mathematics is always interesting but
> it is more important to make sure that the code can be
> modified and maintained without devoting your life to
> the task. Projects die when the original authors leave.
> Sourceforge has 100,000 piles of "dead code". In order
> to keep Axiom "alive" it must be possible to understand,
> modify, and maintain it. I firmly believe that literate
> programming is the only technology that can achieve this.
>
> Axiom is being rewritten to be fully literate. The code
> is being moved from a "tree of little files" to volumes
> with a particular focus. There are currently 21 volumes.
>
> These volumes contain actual executable source code which
> is extracted at build time. So the code you see in the
> books is the actual source code.
>
> There are still many small files to rewrite and distribute
> into their proper locations and there is still a great
> deal of explanation to be added to the volumes. However,
> this project has "a 30 year horizon" so this huge task
> is "right on schedule".
>
>>
>> Singular:
>> https://www.ohloh.net/p/Singular
>> # of lines of code: 456764
>> # of comment lines: 73429
>>
>> Sage:
>> https://www.ohloh.net/p/sage
>> # of lines of code: 474866
>> # of comment lines: 551813
>
> Literate programming has nothing to do with comments.
> They are completely orthogonal. Think of literate programs
> as works of literature. You are telling a story (of why
> the code exists and what you are trying to solve). You
> need to motivate the characters (so procedures need to be
> introduced at the proper time in the explanation, not when
> the compiler needs it). You are trying to write a story for
> another human being, not code comments for the programmer.
> Write down the theory behind the code.
>
> The "Hawaii test" is a good benchmark. Can you bring a new
> person on the project, give them the literate books, send
> them away to an all-expense paid, 2 week vacation to Hawaii,
> and when they return they can maintain and modify the
> program as well as the original authors?
>
> I have found 2 literate programs that I consider examples:
> "Lisp in Small Pieces" by Queinnec (ISBN 0-521-56247-3)
> "Implementing Elliptic Curve Cryptography" by Rosing
> (ISBN 1-884777-69-4)
>
> I claim that literate programming improves code in 3 ways:
> A) the original programmer discovers errors while writing
> up the full explanation for peer review
> B) peer reviewers gain a deep understanding of the problem
> and can critique the original motivation for the code
> including how well the implementation covers it.
> C) future programmers can correctly modify and maintain
> the code in the spirit it was written.
>
> All of these are just opinions at this point but they can
> be tested. I am trying to find a university willing to
> run a test of hypothesis B. Students would be given either
> a literate program (e.g. Rosing's book) or the original
> source code (e.g. Rosing's downloadable source) and asked
> questions to test their comprehension, such as given
> #DEFINE NUMBITS 158
> what is the significance of "158"? Do you know?
>
> Elliptic curve code uses modular polynomial arithmetic
> which Rosing explains but would be really hard to
> "reverse engineer" from the source code.
>
>>
>> Gap wasn't available for comment at the time of publication:
>> https://www.ohloh.net/p/gap-system
>>
>>
>> This is also a good occasion to paste "sage -coverageall" output from
>> 5.0.prealpha0:
>>
>> Overall weighted coverage score: 86.3%
>> Total number of functions: 28917
>> We need 1056 more function to get to 90% coverage.
>> We need 2502 more function to get to 95% coverage.
>> We need 3658 more function to get to 99% coverage.
>
> Test coverage is an excellent idea. Commenting code is an
> excellent idea. Choosing descriptive variable names is an
> excellent idea. Code standards are an excellent idea. All
> of these have nothing to do with a literate program.
>
> Literate programming can be done in any language.
> I wrote an example using HTML:
> http://axiom-developer.org/axiom-website/litprog.html
>
> Are you creating computational mathematics useful for
> future generations or will they start yet-another-
> from-scratch-computer-algebra-system to replace Sage?
>
> Raise your game.
> Make your code "live".
> Make it literate.
I'm really glad that Tim published that article in the Notices to
raise awareness about computational mathematics.
<flamebait>
Unfortunately, I'm not convinced by the promise of "literate
programming". I think that code is most valuable if it is:
* written in a readable programming language and *style*,
especially one that is clean enough that it doesn't require excess
additional explanation,
* peer reviewed
* tested, by having yet other people work on the code successfully
It is much better to have no comments than misleading or wrong
comments. Literate programs have more potential to mislead.
Regarding Tim's Hawaii test, last time I spent two weeks in Hawaii I
got a tan and didn't get anything done.
Moreover, many algorithms I work on would take a professional years to
understand (if they aren't already in the area) no matter how well
they are made literate.
</flamebait>
rjf
Tim says..
> > A) the original programmer discovers errors while writing
> > up the full explanation for peer review
> > B) peer reviewers gain a deep understanding of the problem
> > and can critique the original motivation for the code
> > including how well the implementation covers it.
> > C) future programmers can correctly modify and maintain
> > the code in the spirit it was written.
>
William says..
> I'm really glad that Tim published that article in the Notices to
> raise awareness about computational mathematics.
>
> <flamebait>
> Unfortunately, I'm not convinced by the promise of "literate
> programming". I think that code is most valuable if it is:
>
> * written in a readable programming language and *style*,
> especially one that is clean enough that it doesn't require excess
> additional explanation,
>
> * peer reviewed
>
> * tested, by having yet other people work on the code successfully
>
> It is much better to have no comments than misleading or wrong
> comments. Literate programs have more potential to mislead.
>
> Regarding Tim's Hawaii test, last time I spent two weeks in Hawaii I
> got a tan and didn't get anything done.
> Moreover, many algorithms I work on would take a professional years to
> understand (if they aren't already in the area) no matter how well
> they are made literate.
>
> </flamebait>
literate
program is similar in nature to a proof. I suppose one could make them
coincide,
but I suspect that one would end up with a proof that was overly
verbose and a
program that was inefficient, in that case.
It has been my goal to make programs that express mathematics look
like the
mathematics, to the extent possible. This is more plausible if the
mathematics
is of the more constructive approach (I do not mean "constructive
mathematics (TM)")
Existence-proof style math is often unhelpful algorithmically.
The use of programming languages like Python subvert this paradigm
because anyone
programming in Python must think of how to do something in Python,
i.e. with dots,
indentation, etc. You want to write the mathematics unambiguously,
and bend the
programming language to accept this notation. It is not always
possible, but the
modifications to the math should be easily explained mathematically.
Not by
telling people (say) that there is a difference between a python
integer and a
mathematical integer, and they better use the right coercions.
Maxima allows one to declare new syntax, and of course new semantics
(by defining
subroutines etc.) This helps but does not solve all issues.
Ken Wilson (Nobelist in physics, 1982) thought he understood the
relationship of
physics to computational mathematics. He also got put in charge of
some supercomputer
stuff. His view of computing struck me as rather odd. E.g. claiming
that
FORTRAN EQUIVALENCE statements were somehow important to physics
models, and deserve
prominent discussion in chapter 2 of some book of his..
The idea that future generations are going to read your code,
misunderstand it, and "correct"
it (and the literate document that includes it) gives me the heebie-
jeebies .
I've seem it happen with Macsyma, where people rip out pieces of code
that they incorrectly
believe is unused or wrong. I've also seen pieces of code which have
outlived any
possibility of being executed (unless someone in the future
resurrects a DEC PDP-10,
a GE 645, a VAX running Franz [non-Common] Lisp).
I disagree with William, too.
the book Numerical Recipes has an excellent reputation in some circles
for being
literate, readable, reliable, authoritative, executable...and in your
choice of language that
you presumably already know, e.g. C or Fortran. What could be better?
(There is
even a NR in Lisp, somewhere..)
And yet it is (at least in the places that I've looked recently) STILL
somewhat flaky, in terms of robustness and speed. It is not state of
the art.
Why not? I think that to some extent some (much of?) the highest
quality code is simply
not going to be readable casually, or with a few dedicated Hawaiian
weeks. It is hard
to explain, and so NR leaves it out.
Casual testing, as apparently is done by pythonistas, is clearly
hazardous.
What I've seen is instances where people promote (scientific) python
code that
is portable, readable, and "good enough" in preference to code that is
state-of-the-art. My guess is that the promoters are simply ignorant
of what
has been left out. (To be specific, I happened to look at Bessel
function
evaluation code.)
William Stein
Nothing you say below seems to be an argument against what I wrote, so
I'm not sure what you're disagreeing with. I guess you "disagree with
William" (on general principle?) rather than with what I wrote.
Here's what I wrote:
"I think that code is most valuable if it is:
1. written in a readable programming language and *style*,
especially one that is clean enough that it doesn't require excess
additional explanation,
2. peer reviewed
3. tested, by having yet other people work on the code successfully"
I've added numbers so you can more easily referee to which point you
disagree with.
-- William
rjf
On Jan 19, 1:56 pm, William Stein <wst...@gmail.com> wrote:
can be written to solve a specific problem. Being easy to explain is
not a requirement, though of course it would be a bonus. A Piper Cub
airplane is presumably easier to explain than a Boeing 747. That does
not make it better.
Peer review? So if it is written by Moe, and reviewed by Larry and
Curly,
that's a plus? I would prefer that it be written as well as reviewed
by experts. But hey, I guess I'm elitist.
Tested? I don't know what having other people "work on the code"
means. Does that include altering it? If it just means randomly
trying it out, that doesn't mean much. For example the Bessel
function code
in scipy probably works usually; in fact I haven't tried it but I
looked
at the code in deciding whether to use it for a project. It is
not worth using. It is not high quality. It presumably passes all your
points (though maybe it is not documented adequately. I didn't look
around
for extra stuff.)
My guess is that if you use it to try evaluating in double-float J_3
at
16.223466160318768 that you will get about 1 decimal digit correct.
Why? Well, that number is not random. It is close to a zero of J_3.
There are an infinite number of zeros of J_3 on the real line, and it
will screw up on all of them. And if I'm recalling that code, it
becomes
increasingly inaccurate and inefficient for ALL arguments x as x
increases.
I don't mind having a really good expert piece of code that I don't
have
to read, understand, or test. I would much rather stand on shoulders
of giants..
than on their feet.
(Re giants business .. Newton said this to Robert Hooke, a man of
apparently short stature.)
I hope I've clarified my points here.
William Stein
>> Here's what I wrote:
>>
>> "I think that code is most valuable if it is:
>>
>> 1. written in a readable programming language and *style*,
>> especially one that is clean enough that it doesn't require excess
>> additional explanation,
>>
>> 2. peer reviewed
>>
>> 3. tested, by having yet other people work on the code successfully"
>>
>> I've added numbers so you can more easily referee to which point you
>> disagree with.
>>
>> -- William
>
> Code is most valuable if it is the most efficient and robust code that
> can be written to solve a specific problem. Being easy to explain is
> not a requirement, though of course it would be a bonus. A Piper Cub
> airplane is presumably easier to explain than a Boeing 747. That does
> not make it better.
>
So you claim that code that is "most efficient and robust" is far more
valuable than code that can be read, has been peer reviewed, and has
passed the test that other people can work with the code? I would
definitely prefer slightly less efficient or robust code in exchange
for actually being able to read it!
> Peer review? So if it is written by Moe, and reviewed by Larry and
> Curly,
> that's a plus?
It depends entirely who Larry and Curly are.
> I would prefer that it be written as well as reviewed
> by experts. But hey, I guess I'm elitist.
It is of course better if the reviewers and authors are skilled.
That's obvious.
> Tested? I don't know what having other people "work on the code"
> means. Does that include altering it?
Yes.
> If it just means randomly
> trying it out, that doesn't mean much.
It obviously does not mean that.
> I don't mind having a really good expert piece of code that I don't
> have
> to read, understand, or test. I would much rather stand on shoulders
> of giants..
> than on their feet.
What if that "good expert piece of code" doesn't solve exactly the
problem you want to solve, and the only way to solve your problem is
to either modify the code or start from scratch? In that case, I
would be in a much better position than you.
> I hope I've clarified my points here.
Thanks. In summary, you believe the best code is code written by
giants that you don't understand. You're definitely no Tim Daly.
-- William
rjf
On Jan 19, 3:22 pm, William Stein <wst...@gmail.com> wrote:
are high school
students, then peer review is not worth much. Freshman reading each
others' code
does not generally result in quality.
Now if you said "expert review" then you would agree with me.
>
> > I would prefer that it be written as well as reviewed
> > by experts. But hey, I guess I'm elitist.
>
> It is of course better if the reviewers and authors are skilled.
> That's obvious.
>
> > Tested? I don't know what having other people "work on the code"
> > means. Does that include altering it?
>
> Yes.
it.
Sounds like a bad idea to me unless the person testing the code is at
least
as much of an expert as the reviewers and perhaps the author(s).
>
> > If it just means randomly
> > trying it out, that doesn't mean much.
>
> It obviously does not mean that.
thoroughly testing it. They generate large numbers of random tests.
>
> > I don't mind having a really good expert piece of code that I don't
> > have
> > to read, understand, or test. I would much rather stand on shoulders
> > of giants..
> > than on their feet.
>
> What if that "good expert piece of code" doesn't solve exactly the
> problem you want to solve, and the only way to solve your problem is
> to either modify the code or start from scratch? In that case, I
> would be in a much better position than you.
would write the program. I might refer to the mathematical literature
for a simple one-off kind of inefficient program. Or I might adapt
my problem so that it fits the existing program. I suppose there are
cases where one benefits from having a piece of crap open source
program,
but as a computer scientist I would prefer having good programs.
> > I hope I've clarified my points here.
>
> Thanks. In summary, you believe the best code is code written by
> giants that you don't understand. You're definitely no Tim Daly.
want
well-intentioned persons who don't understand it
(even if it is documented) pissing on our code.
RJF
Tom Boothby
> On Jan 19, 3:22 pm, William Stein <wst...@gmail.com> wrote:
>> That's obvious.
>
> So why don't you say so, instead of "peer review"?
Man, kids just don't read shit these days. Oh sorry... I meant
crotchety grumps, not kids.
rjf
Oh, by the way... what kinds of mistakes do people make when they
write scientific code but don't know much about computing / numerical
computation ?
They make mistakes in adding together a bunch of numbers, like
computing a
(partial) sum.
That some algorithms crucially depend on the order of operations.
Maybe this will be solved if Tim writes out everything everyone knows,
literately.
rjf
I just had occasion to need computation of eigenvalues of a
symmetric tridiagonal real matrix. There is a routine in
Numerical Recipes in Fortran that is about 1.5 pages long.
The C version is somewhat shorter.
The LAPACK version of the same (QL) algorithm -- it's hard to
say how long it is because it uses bunches of routines, some
only for error reporting -- but the netlib.org download of
dstevd and all dependencies is 105 pages long (much of it comments,
to be sure, but still, a lot of pages).
So what gives here? Well, let's say you want your program to take
note of the parameters of precision, overflow, etc of your machine
to get the most accurate results possible.
Let's say you want NaNs, scaling, and all the nasty little bits
to be handled in some appropriate fashion. LAPACK.
On the other hand, if you want a program that just goes through
the motions, the NR version may be adequate, In fact if you are
doing "arbitrary precision" floats, where under/overflow is
essentially impossible, NR may be a good start. But it is not
an expertly written robust program; Arguably it is a blight
on the landscape of numerical computing.
Neither of them is, formally, literate programming.
Would the LAPACK program be better if written in python?
William seems to be arguing that.
Would the NR program be better if it were reviewed/tested by
"peers" who would presumably be physicists? Not by much.
RJF
Ondřej Čertík
> Apologies for responding, in some sense, to my own previous post.
> I just had occasion to need computation of eigenvalues of a
> symmetric tridiagonal real matrix. There is a routine in
> Numerical Recipes in Fortran that is about 1.5 pages long.
> The C version is somewhat shorter.
> The LAPACK version of the same (QL) algorithm -- it's hard to
> say how long it is because it uses bunches of routines, some
> only for error reporting -- but the netlib.org download of
> dstevd and all dependencies is 105 pages long (much of it comments,
> to be sure, but still, a lot of pages).
>
> So what gives here? Well, let's say you want your program to take
> note of the parameters of precision, overflow, etc of your machine
> to get the most accurate results possible.
>
> Let's say you want NaNs, scaling, and all the nasty little bits
> to be handled in some appropriate fashion. LAPACK.
Lapack is a great piece of software. It's written in f77 though,
for historical reasons. These days, I would choose f90
and later for it.
>
> On the other hand, if you want a program that just goes through
> the motions, the NR version may be adequate, In fact if you are
> doing "arbitrary precision" floats, where under/overflow is
> essentially impossible, NR may be a good start. But it is not
> an expertly written robust program; Arguably it is a blight
> on the landscape of numerical computing.
>
> Neither of them is, formally, literate programming.
>
> Would the LAPACK program be better if written in python?
> William seems to be arguing that.
I don't think so. William said 1., 2., and 3., see his post.
>
> Would the NR program be better if it were reviewed/tested by
> "peers" who would presumably be physicists? Not by much.
I think it would be better than not reviewed and tested by peers.
But how does that relate to 1., 2. or 3., or literate programming?
Ondrej
rjf
On Jan 20, 10:54 pm, Ondřej Čertík <ondrej.cer...@gmail.com> wrote:
> > Apologies for responding, in some sense, to my own previous post.
> > I just had occasion to need computation of eigenvalues of a
> > symmetric tridiagonal real matrix. There is a routine in
> > Numerical Recipes in Fortran that is about 1.5 pages long.
> > The C version is somewhat shorter.
> > The LAPACK version of the same (QL) algorithm -- it's hard to
> > say how long it is because it uses bunches of routines, some
> > only for error reporting -- but the netlib.org download of
> > dstevd and all dependencies is 105 pages long (much of it comments,
> > to be sure, but still, a lot of pages).
>
> > So what gives here? Well, let's say you want your program to take
> > note of the parameters of precision, overflow, etc of your machine
> > to get the most accurate results possible.
>
> > Let's say you want NaNs, scaling, and all the nasty little bits
> > to be handled in some appropriate fashion. LAPACK.
>
> Lapack is a great piece of software. It's written in f77 though,
> for historical reasons. These days, I would choose f90
> and later for it.
>
http://www.netlib.org/lapack/
>
> > On the other hand, if you want a program that just goes through
> > the motions, the NR version may be adequate, In fact if you are
> > doing "arbitrary precision" floats, where under/overflow is
> > essentially impossible, NR may be a good start. But it is not
> > an expertly written robust program; Arguably it is a blight
> > on the landscape of numerical computing.
>
> > Neither of them is, formally, literate programming.
>
> > Would the LAPACK program be better if written in python?
> > William seems to be arguing that.
>
> I don't think so. William said 1., 2., and 3., see his post.
"written in a readable programming language " which I interpret as
code for "written in Python." For his statement to have any meaning
there
must be some unreadable programming languages in more or less wide use
which William wishes to disqualify. If there is a programming
language
being disqualified, what might that be? FORTRAN?
>
>
>
> > Would the NR program be better if it were reviewed/tested by
> > "peers" who would presumably be physicists? Not by much.
>
> I think it would be better than not reviewed and tested by peers.
unqualified people masquerading as quality control experts of some
sort is hazardous, especially if it was written by an unqualified
person in the first place. Sadly, much of the "contributed program"
libraries fall dangerously close to this model.
It might be more direct and honest to label programs "use at your own
risk"
rather than "tested and approved".
> But how does that relate to 1., 2. or 3., or literate programming?
valuable if it comes out of some mechanistic writing/formatting/
interspersing
of code and comment.
Important characteristics include: correct, robust, efficient,
accurate, well-documented. Maybe portable. One can argue about free,
open-source, etc.
One could also argue about such issues (just read the news!) "not
infringing on
patents".
RJF
>
> Ondrej
Ondřej Čertík
>
>
> On Jan 20, 10:54 pm, Ondřej Čertík <ondrej.cer...@gmail.com> wrote:
>> On Fri, Jan 20, 2012 at 3:51 PM, rjf <fate...@gmail.com> wrote:
>> > Apologies for responding, in some sense, to my own previous post.
>> > I just had occasion to need computation of eigenvalues of a
>> > symmetric tridiagonal real matrix. There is a routine in
>> > Numerical Recipes in Fortran that is about 1.5 pages long.
>> > The C version is somewhat shorter.
>> > The LAPACK version of the same (QL) algorithm -- it's hard to
>> > say how long it is because it uses bunches of routines, some
>> > only for error reporting -- but the netlib.org download of
>> > dstevd and all dependencies is 105 pages long (much of it comments,
>> > to be sure, but still, a lot of pages).
>>
>> > So what gives here? Well, let's say you want your program to take
>> > note of the parameters of precision, overflow, etc of your machine
>> > to get the most accurate results possible.
>>
>> > Let's say you want NaNs, scaling, and all the nasty little bits
>> > to be handled in some appropriate fashion. LAPACK.
>>
>> Lapack is a great piece of software. It's written in f77 though,
>> for historical reasons. These days, I would choose f90
>> and later for it.
>>
> Current version is available in F90. See
> http://www.netlib.org/lapack/
I thought the latest is lapack 3.4.0. If you download the tarball, it
is written in f77.
I know that they write "LAPACK is written in Fortran 90", but I don't
see any features of f90 in the code (maybe I missed some?),
I however see lots of f77 features, that are now deprecated (old style
array passing, no modules, ...).
>>
>>
>> > On the other hand, if you want a program that just goes through
>> > the motions, the NR version may be adequate, In fact if you are
>> > doing "arbitrary precision" floats, where under/overflow is
>> > essentially impossible, NR may be a good start. But it is not
>> > an expertly written robust program; Arguably it is a blight
>> > on the landscape of numerical computing.
>>
>> > Neither of them is, formally, literate programming.
>>
>> > Would the LAPACK program be better if written in python?
>> > William seems to be arguing that.
>>
>> I don't think so. William said 1., 2., and 3., see his post.
>
> I think that is exactly what he is saying.
> "written in a readable programming language " which I interpret as
> code for "written in Python." For his statement to have any meaning
> there
> must be some unreadable programming languages in more or less wide use
> which William wishes to disqualify. If there is a programming
> language
> being disqualified, what might that be? FORTRAN?
Ah I see. I interpreted it differently -- I thought any widely used
language is fine,
if it allows readable and manageable code (in my case, I use Python
and modern Fortran a lot).
>
>
>
>>
>>
>>
>> > Would the NR program be better if it were reviewed/tested by
>> > "peers" who would presumably be physicists? Not by much.
>>
>> I think it would be better than not reviewed and tested by peers.
>
> I disagree. A program with a certificate of review and testing by
How can the program get worse by reviewing it (even by unqualified
people as you say)?
Ondrej
rjf
On Jan 21, 8:16 am, Ondřej Čertík <ondrej.cer...@gmail.com> wrote:
>
> How can the program get worse by reviewing it (even by unqualified
> people as you say)?
>
qualifications,
perhaps with conflicts of interest) review (say) restaurants,
dentists, ...
I have seen situations in which totally bogus articles have been
submitted
to journals where they are then reviewed by friends of the authors,
and
then published.
See
http://retractionwatch.wordpress.com/2011/03/15/faked-data-unsubstantiated-claims-and-spirituality-add-up-to-a-math-journal-retraction/
search on google for the author's name for more discussion..
> Ondrej
Bill Hart
(including code comments) and literate programming are all distinct
and each have their place.
In the case of flint we do some kind of code review, lots of testing
and documentation only.
The C language is not particularly suited to formal techniques, other
than static typing.
I think that reference implementations should be refereed. Some
journals do provide a kind of mechanism for doing this but we are
obviously in the early days.
There is a lot of disagreement on these sorts of issues because of a
great deal of pseudoscience, urban myths and ulterior motives.
Bill.
> --
> You received this message because you are subscribed to the Google Groups "sage-flame" group.
> To post to this group, send email to sage-...@googlegroups.com.
> To unsubscribe from this group, send email to sage-flame+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/sage-flame?hl=en.
>
Tom Boothby
>
>
> On Jan 21, 8:16 am, Ondřej Čertík <ondrej.cer...@gmail.com> wrote:
>
>>
>> How can the program get worse by reviewing it (even by unqualified
>> people as you say)?
>>
>
> Are you familiar with Yelp? Where people (perhaps without
> qualifications,
> perhaps with conflicts of interest) review (say) restaurants,
> dentists, ...
>
Perhaps good code can get negatively reviewed. The developer will
push the issue, and in the worst case, bring it to sage-devel, where
we hash it out. This happens, feelings get hurt, but good code wins.
Quality of the submitted code doesn't go down as a result of the
review process here.
>
> I have seen situations in which totally bogus articles have been
> submitted
> to journals where they are then reviewed by friends of the authors,
> and
> then published.
Shit code gets sloppily reviewed sometimes. It was shit code to begin
with, and it's a shame for it to go in... but the code wasn't harmed
by the review process.
>
> See
> http://retractionwatch.wordpress.com/2011/03/15/faked-data-unsubstantiated-claims-and-spirituality-add-up-to-a-math-journal-retraction/
>
> search on google for the author's name for more discussion..
Fun example! Fortunately, we've never had that level of crank submit
code to Sage. But in all likelihood, it'd be found quickly.
rjf
On Jan 21, 10:19 am, Tom Boothby <tomas.boot...@gmail.com> wrote:
> Shit code gets sloppily reviewed sometimes. It was shit code to begin
> with, and it's a shame for it to go in... but the code wasn't harmed
> by the review process.
reviewers?
> Fortunately, we've never had that level of crank submit
> code to Sage.
> But in all likelihood, it'd be found quickly.
I think it is pretty much unarguable
that Sage has some bad code in it.
RJF
Tom Boothby
>
>
> On Jan 21, 10:19 am, Tom Boothby <tomas.boot...@gmail.com> wrote:
>
>
>> Shit code gets sloppily reviewed sometimes. It was shit code to begin
>> with, and it's a shame for it to go in... but the code wasn't harmed
>> by the review process.
>
> What if good code is rejected in favor of bad code by crummy
> reviewers?
It happens. But as a whole, it's better than no review process.
>
>> Fortunately, we've never had that level of crank submit
>> code to Sage.
>
> How can you be sure?
We gossip like teenage girls about such things. I'm sure I'd have heard of it.
>
>> But in all likelihood, it'd be found quickly.
>
> There are levels of craziness and of bad code.
> I think it is pretty much unarguable
> that Sage has some bad code in it.
Yes, Sage has some bad code in it. It includes Maxima.
>
> RJF
rjf
On Jan 21, 6:03 pm, Tom Boothby <tomas.boot...@gmail.com> wrote:
>
> Yes, Sage has some bad code in it. It includes Maxima.
non-experts, then peer reviewed and tested (by physicists).
Tom Boothby
>
>
> On Jan 21, 6:03 pm, Tom Boothby <tomas.boot...@gmail.com> wrote:
>
>>
>> Yes, Sage has some bad code in it. It includes Maxima.
>
> There are many parts of Maxima that were contributed by
> non-experts, then peer reviewed and tested (by physicists).
>
Yeah... open source software would be great, if only there weren't so
many amateurs contributing.
rjf
On Jan 21, 10:37 pm, Tom Boothby <tomas.boot...@gmail.com> wrote:
>
> Yeah... open source software would be great, if only there weren't so
> many amateurs contributing.
projects on sourceforge.
2. Professional programmers are probably, on the whole, better
programmers than amateurs.
Considering that professionals tend to be the ones who get paid, it is
likely that they
have some business perspective and are less likely to give away what
they produce.
Not always, of course. Some professionals provide open source code
sometimes,
for various reasons.
Academics operate in a different setting. Whether they are
professional or not is
hard to say.
Tom Boothby
>
>
> On Jan 21, 10:37 pm, Tom Boothby <tomas.boot...@gmail.com> wrote:
>
>>
>> Yeah... open source software would be great, if only there weren't so
>> many amateurs contributing.
>
> 1. Open source software, by and large, is sucky. Look at the dead
> projects on sourceforge.
Human endeavors, by and large, are sucky. Most businesses fail in
their first decade. Over half of marriages end in divorce. Over half
of the landers we've tried to put on Mars have failed. We're
destroying the planet through greed and poor resource management.
Woe, why do we even get out of bed in the morning?!?!
> 2. Professional programmers are probably, on the whole, better
> programmers than amateurs.
And professional programmers rely on a review process. More eyes on
code is good. This is something open source folks have learned from
the pros. And here, Mr. "You jerks should read more about what other
people do" is telling us that code review is shit.
> Considering that professionals tend to be the ones who get paid, it is
> likely that they
> have some business perspective and are less likely to give away what
> they produce.
> Not always, of course. Some professionals provide open source code
> sometimes,
> for various reasons.
>
> Academics operate in a different setting. Whether they are
> professional or not is
> hard to say.
>
>
Dima Pasechnik
On Sunday, 22 January 2012 00:40:27 UTC+8, rjf wrote:
> Ondrej
Tom Boothby
The other alternative is to not scrutinize anything anybody else does,
and take it on faith. I think that's what Fateman's suggesting: he's
world's solitary expert in everything he's into, and everybody else's
code is shit not to be trusted.
I, for one, am ok with him thinking that. I've written books on the
topic, and received several Nobel prizes for them. So you should
agree with me.
>>
>>
>>
>> > Ondrej
>
> --
> You received this message because you are subscribed to the Google Groups
> "sage-flame" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/sage-flame/-/cgTGR8meBeIJ.
rjf
On Jan 21, 11:42 pm, Tom Boothby <tomas.boot...@gmail.com> wrote:
>
> And professional programmers rely on a review process.
to
try to make the project better. Nevertheless, a professional
programmer
may be better at writing programs than an amateur, on average. Are
you
really arguing that?
> More eyes on
> code is good.
mind) make
it nearly impossible for amateurs (or anyone) to report errors on
their own
initiative. The reality is, a huge percentage of the "eyes" are
ignorant. The
"errors" they report are so called PICNIC errors. Problem In Chair
Not In Computer.
Devoting the time of developers to examining these has very low
payoff.
> This is something open source folks have learned from
> the pros. And here, Mr. "You jerks should read more about what other
> people do" is telling us that code review is shit.
necessarily result in quality programs.
It is odd that you choose to argue on these points.
"There is an old saying that teaching a pig to dance is a fruitless
endeavor/
It is a waste of your time and it irritates the pig.
rjf
On Jan 21, 11:47 pm, Dima Pasechnik <dimp...@gmail.com> wrote:
....
> an obviously wrong lemma somewhere,
where
the conclusions might be true or not, but the proof is defective.
> and use it to go on to become a Clay
> Scholar etc etc.
experts.
> (And it will take a lot of persistence from someone to
> yet, there is no better model known, in science or in software.
relevance
of a web page to a query. EBay uses a model to determine the
reliability of
a seller. I sometimes use a model that says if a piece of code is in
netlib.org
that it is worth looking at.
> The
> alternative of peer-reviewed code is to get software from a commercial
> provider, who will do whatever it takes to shut you up if you found an
> error in it...
on
the market for a purpose, so you will buy it rather than the
competition.
Ideally, at least. This assumes a fair market, disclosure, and the
existence of competition, assumptions that may not be correct
sometimes.
RJF
rjf
On Jan 22, 12:52 am, Tom Boothby <tomas.boot...@gmail.com> wrote:
>
> The other alternative is to not scrutinize anything anybody else does,
> and take it on faith.
> I think that's what Fateman's suggesting: he's
> world's solitary expert in everything he's into, and everybody else's
> code is shit not to be trusted.
>
> I, for one, am ok with him thinking that. I've written books on the
> topic, and received several Nobel prizes for them. So you should
> agree with me.
>
Bill Hart
<SNIP>
>> This is something open source folks have learned from
>> the pros. And here, Mr. "You jerks should read more about what other
>> people do" is telling us that code review is shit.
>
> Incompetent review of incompetently written programs does not
> necessarily result in quality programs.
>
> It is odd that you choose to argue on these points.
>
> "There is an old saying that teaching a pig to dance is a fruitless
> endeavor/
> It is a waste of your time and it irritates the pig.
OK, so you have a point and are winning the argument and scoring lots
of flamage.
But seriously, what are you proposing? Let's say a mathematician has a
very complex algorithm they have implemented. How are you proposing
that this gets to the point where it meets your standards for release?
What process should it undergo?
Dima Pasechnik
On Sunday, 22 January 2012 23:46:03 UTC+8, rjf wrote:
On Jan 21, 11:47 pm, Dima Pasechnik <dim...@gmail.com> wrote:
....
> Basically, if one cannot finish off a difficult proof, one can still insert
> an obviously wrong lemma somewhere,
I am not really concerned about misleading articles about matroids,
where
the conclusions might be true or not, but the proof is defective.
> and use it to go on to become a Clay
> Scholar etc etc.
The Clay Institute, last I looked, relied heavily on carefully chosen
experts.
> (And it will take a lot of persistence from someone to
> force this one to acknowledge the wrong: seehttp://arxiv.org/abs/0709.1291).
>
> yet, there is no better model known, in science or in software.
No, I think there are other models. Google uses one to determine the
relevance
of a web page to a query. EBay uses a model to determine the
reliability of
a seller. I sometimes use a model that says if a piece of code is in
netlib.org
that it is worth looking at.
> The
> alternative of peer-reviewed code is to get software from a commercial
> provider, who will do whatever it takes to shut you up if you found an
> error in it...
No, a vendor will attempt to convince you that its program is the best
on
the market for a purpose, so you will buy it rather than the
competition.
Ideally, at least. This assumes a fair market, disclosure, and the
existence of competition, assumptions that may not be correct
sometimes.
RJF
rjf
On Jan 22, 8:01 am, Dima Pasechnik <dimp...@gmail.com> wrote:
...
> well, do you mean to say that Google will review debug your code better
> than peers?
> This seems to be going in the totally opposite direction...
solve
a particular problem, Google will identify the ones that are
(according to
some criterion) the "best".
For example, I was recently searched for ..
tridiagonal symmetric eigenvalues
>
> > > The
> > > alternative of peer-reviewed code is to get software from a commercial
> > > provider, who will do whatever it takes to shut you up if you found an
> > > error in it...
>
> > No, a vendor will attempt to convince you that its program is the best
> > on
> > the market for a purpose, so you will buy it rather than the
> > competition.
> > Ideally, at least. This assumes a fair market, disclosure, and the
> > existence of competition, assumptions that may not be correct
> > sometimes.
>
> I'd say "almost always" rather than "sometimes". Unfortunately.
but
lots of smaller segments have competition. e.g. players for music,
editors,
OCR scanning, photo management, maybe web browsers.
rjf
On Jan 22, 7:57 am, Bill Hart <goodwillh...@googlemail.com> wrote:
...
> But seriously, what are you proposing? Let's say a mathematician has a
> very complex algorithm they have implemented. How are you proposing
> that this gets to the point where it meets your standards for release?
> What process should it undergo?
OK, here's what I would suggest. Mathematician A has written a
program.
He/she finds another Mathematician B of significant reputation. Asks
for
help certifying the program. If refused, finds Mathematician B' etc.
Mathematician A must provide not only the program, but documentation,
test data, alternative versions, the opportunity for discussion etc.
Much of this would be unavailable to a reviewer of a submitted
article.
If B agrees that the program is (whatever.. certifiable), then it
could
be publicized, e.g. posted on a web page owned by A. linked to from
a web page owned by B.
B may not be a person, but an organization (e.g. NAG, SLATEC, IBM,
Apple's App Store??)
This does not prevent subsequent testing and review, but a potential
user (or purchaser) can look at the reputation of A and B and have
some useful information. This is a well known phenomenon in related
areas. E.g. "I have not seen a proof, but Professor X said it is
surely
true that .....".
Simple example. You may not care to believe that I am an authority on
Lisp,
but you probably would accept the information in
http://dl.acm.org/citation.cfm?id=1074543&CFID=79710250&CFTOKEN=15212924
which is an entry in an encyclopedia of computer science published by
John Wiley,
an article coauthored by Richard Fateman and John McCarthy.
If I had a great new algorithm, I would try to run it by one of my
students and then one of my colleagues first.
A truly large and complex program e.g. FLINT may not fit this method,
but
maybe getting (say) Richard Brent to look at it might convince people
it
is the right stuff.
Tom Boothby
>
>
> On Jan 22, 7:57 am, Bill Hart <goodwillh...@googlemail.com> wrote:
> ...
>
>> But seriously, what are you proposing? Let's say a mathematician has a
>> very complex algorithm they have implemented. How are you proposing
>> that this gets to the point where it meets your standards for release?
>> What process should it undergo?
>
> Are we allowed to be serious?
>
> OK, here's what I would suggest. Mathematician A has written a
> program.
> He/she finds another Mathematician B of significant reputation. Asks
> for
> help certifying the program. If refused, finds Mathematician B' etc.
>
> Mathematician A must provide not only the program, but documentation,
> test data, alternative versions, the opportunity for discussion etc.
> Much of this would be unavailable to a reviewer of a submitted
> article.
>
> If B agrees that the program is (whatever.. certifiable), then it
> could
> be publicized, e.g. posted on a web page owned by A. linked to from
> a web page owned by B.
>
> B may not be a person, but an organization (e.g. NAG, SLATEC, IBM,
> Apple's App Store??)
Do you realize that you've just described Sage's review process?
>
> This does not prevent subsequent testing and review, but a potential
> user (or purchaser) can look at the reputation of A and B and have
> some useful information. This is a well known phenomenon in related
> areas. E.g. "I have not seen a proof, but Professor X said it is
> surely
> true that .....".
>
> Simple example. You may not care to believe that I am an authority on
> Lisp,
> but you probably would accept the information in
> http://dl.acm.org/citation.cfm?id=1074543&CFID=79710250&CFTOKEN=15212924
> which is an entry in an encyclopedia of computer science published by
> John Wiley,
> an article coauthored by Richard Fateman and John McCarthy.
>
> If I had a great new algorithm, I would try to run it by one of my
> students and then one of my colleagues first.
>
> A truly large and complex program e.g. FLINT may not fit this method,
> but
> maybe getting (say) Richard Brent to look at it might convince people
> it
> is the right stuff.
>
>
>
>
rjf
On Jan 22, 2:49 pm, Tom Boothby <tomas.boot...@gmail.com> wrote:
...
>
> Do you realize that you've just described Sage's review process?
>
>
Or does Sage just include huge gobs of unreviewed stuff too?
Tom Boothby
Nope, it's neither complete nor perfect.
William Stein
> On 22 January 2012 15:37, rjf <fat...@gmail.com> wrote:
> <SNIP>
>>> This is something open source folks have learned from
>>> the pros. And here, Mr. "You jerks should read more about what other
>>> people do" is telling us that code review is shit.
>>
>> Incompetent review of incompetently written programs does not
>> necessarily result in quality programs.
>>
>> It is odd that you choose to argue on these points.
>>
>> "There is an old saying that teaching a pig to dance is a fruitless
>> endeavor/
>> It is a waste of your time and it irritates the pig.
>
> OK, so you have a point and are winning the argument and scoring lots
> of flamage.
Just out of curiosity, how do you come to such a conclusion reading
this thread? When I look at it I get the sense that RJF is winning
only in "number of posts", but everything he writes is total
bullshit.
-- William
>
> But seriously, what are you proposing? Let's say a mathematician has a
> very complex algorithm they have implemented. How are you proposing
> that this gets to the point where it meets your standards for release?
> What process should it undergo?
>
> --
> You received this message because you are subscribed to the Google Groups "sage-flame" group.
> To post to this group, send email to sage-...@googlegroups.com.
> To unsubscribe from this group, send email to sage-flame+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/sage-flame?hl=en.
>
--
William Stein
Professor of Mathematics
University of Washington
http://wstein.org
rjf
On Jan 22, 11:34 pm, William Stein <wst...@gmail.com> wrote:
>
> Just out of curiosity, how do you come to such a conclusion reading
> this thread? When I look at it I get the sense that RJF is winning
> only in "number of posts", but everything he writes is total
> bullshit.
>
> -- William
are apparently not understanding it.
RJF
Tom Boothby
rjf
"I'm rubber you're glue"...
quel riposte!
Searching for humor? Here's a riddle for your friends..
Why should you serve eggs Benedict on a hubcap?
..ans:
Because there's no plates like chrome for the Hollandaise.
On Jan 23, 9:37 pm, Tom Boothby <tomas.boot...@gmail.com> wrote:
Tom Boothby
> ah, sarcasm, irony... they fall by the wayside in the face of
> "I'm rubber you're glue"...
> quel riposte!
>
> Searching for humor? Here's a riddle for your friends..
>
> Why should you serve eggs Benedict on a hubcap?
>
>
> ..ans:
> Because there's no plates like chrome for the Hollandaise.
Chrome? The gangster culture of California pervades even its ivory
towers? Please tell me that your hoopty gots 20" spinners and
switches. I'm really enjoying the image.
Also... yes, I'll almost certainly share that gem with my friends.
The only thing better than a pun is an oft-repeated pun.
>
>
>
> On Jan 23, 9:37 pm, Tom Boothby <tomas.boot...@gmail.com> wrote:
>> On Mon, Jan 23, 2012 at 9:32 PM, rjf <fate...@gmail.com> wrote:
>>
>> > On Jan 22, 11:34 pm, William Stein <wst...@gmail.com> wrote:
>>
>> >> Just out of curiosity, how do you come to such a conclusion reading
>> >> this thread? When I look at it I get the sense that RJF is winning
>> >> only in "number of posts", but everything he writes is total
>> >> bullshit.
>>
>> >> -- William
>>
>> > Nice to know you are reading this thread, even if you
>> > are apparently not understanding it.
>>
>> Nice to know you are reading this thread, even if you are apparently
>> not understanding it.
>
Tom Boothby
> "I'm rubber you're glue"...
> quel riposte!
http://www.toothpastefordinner.com/012412/not-now-mom.gif
>
> Searching for humor? Here's a riddle for your friends..
>
> Why should you serve eggs Benedict on a hubcap?
>
>
> ..ans:
> Because there's no plates like chrome for the Hollandaise.
>
>
>
> On Jan 23, 9:37 pm, Tom Boothby <tomas.boot...@gmail.com> wrote:
>> On Mon, Jan 23, 2012 at 9:32 PM, rjf <fate...@gmail.com> wrote:
>>
>> > On Jan 22, 11:34 pm, William Stein <wst...@gmail.com> wrote:
>>
>> >> Just out of curiosity, how do you come to such a conclusion reading
>> >> this thread? When I look at it I get the sense that RJF is winning
>> >> only in "number of posts", but everything he writes is total
>> >> bullshit.
>>
>> >> -- William
>>
>> > Nice to know you are reading this thread, even if you
>> > are apparently not understanding it.
>>
>> Nice to know you are reading this thread, even if you are apparently
>> not understanding it.
>