sage constraints

[Ondrej Certik]

On Tue, Aug 11, 2009 at 3:50 PM, William Stein<wst...@gmail.com> wrote:
>
> On Tue, Aug 11, 2009 at 9:19 AM, rjf<fat...@gmail.com> wrote
>> Continuing off topic, at least given the subject line...
>> Since so many pieces of code are currently part of Sage,  it seems
>> prudent to  tell those developers what the constraints are, or else
>> they may suddenly find themselves excluded from Sage.
>> Or worse, they may be included in Sage and jeopardize uh, the security
>
> You seem to be making the claim that on a whim I will just decide to
> suddenly delete large portions of code or packages from Sage, because
> of some made up constraints.  For the last two years,  packages get
> added to Sage and removed from Sage via a voting procedure which is
> public and documented on the mailing list.
>
> It's actually interesting to summarize, the specific constraints I am aware of:
>
>   1. A major government agency -- we can't use Sage unless you
> provide a version that contains no binary components and that builds
> from source using the latest released version of GCC.
>
>   2. Microsoft -- we can't run Sage if it contains GPLv3 code, so please make
> a version for us that uses only GPLv2+ (or other) code.
>
>   3. Enthought -- we can't use parts of Sage unless they are BSD licensed.
>
> I can't think of any other specific technical constraints that I've
> heard about from various companies or organizations.
>
> Though it is difficult to balance what companies and organizations
> want within the constraints imposed by what developers are
> enthusiastic about creating, I think so far the Sage community is
> doing an awesome job!

Seems like noone is using this list besides me. Anyways, some thoughts
for contemplation:

I think it'd bad, if companies like Microsoft pay enormous taxes,
which are then used by the government to fund projects, that are
released using a license (GPL3), so that Microsoft can't use it.
(That's not the Sage case).

The same about enthought and BSD. I think government funded software
should not use any other license (if one has the choice to choose the
license) besides public domain, or permissive licenses.

With Mathematica, it's more involved, since lots of universities buy
it (using government money), so is it bad, if Mathematica uses some
software which is not "protected" by GPL? Like gmp. I tend to think
it's not bad, but I can see the argument, that since Mathematica also
lives by government money (indirectly), it should release its'
software as BSD, counter argument is that the universities don't have
to buy it.

Ondrej

[William Stein]

Huh? So you believe that software that receives government funding
should not be allowed to use the GPL license? Is that what you're
flaming?

William

[Ondrej Certik]

I think so yes.

Ondrej

[David Joyner]

On Tue, Aug 11, 2009 at 9:41 PM, Ondrej Certik<ond...@certik.cz> wrote:
>
> On Tue, Aug 11, 2009 at 7:35 PM, William Stein<wst...@gmail.com> wrote:
>>

..

>>
>> Huh?  So you believe that software that receives government funding
>> should not be allowed to use the GPL license?   Is that what you're
>> flaming?
>
> I think so yes.

It is hard to argue against that but IMHO the GPL serves the public interest
in terms of taxpayer's value. The BSD has greater commercial value but
I'm not convinced that every taxpayer-supported program must also
support also commercial interests. If commercial interests are set aside then
it seems the the GPL is a sufficient license to guarantee taxpayer's value.
If the commercial value is not to be set aside then what is the argument
that the taxpayers must *also* support commercial companies?


>
> Ondrej
>
> >
>

[William Stein]

On Tue, Aug 11, 2009 at 6:41 PM, Ondrej Certik<ond...@certik.cz> wrote:
>> Huh?  So you believe that software that receives government funding
>> should not be allowed to use the GPL license?   Is that what you're
>> flaming?
>
> I think so yes.
>
> Ondrej

Do you also think that the Sage project should thus spend about 10
years (or much more?) of hard work by a bunch of people to rewrite all
functionality we need that is currently in GAP, Singular, PARI, NTL,
GAP, GMP, and Maxima from scratch? Moreover, do you think we should
thus sever our relationships with the communities that have created
all that software? Because that is really the only option if Sage is
to be releasable under a BSD or PSF license.

-- William

[Ondrej Certik]

On Tue, Aug 11, 2009 at 8:05 PM, William Stein<wst...@gmail.com> wrote:
>
> On Tue, Aug 11, 2009 at 6:41 PM, Ondrej Certik<ond...@certik.cz> wrote:
>>> Huh?  So you believe that software that receives government funding
>>> should not be allowed to use the GPL license?   Is that what you're
>>> flaming?
>>
>> I think so yes.
>>
>> Ondrej
>
> Do you also think that the Sage project should thus spend about 10
> years (or much more?) of hard work by a bunch of people to rewrite all
> functionality we need that is currently in GAP, Singular, PARI, NTL,
> GAP, GMP, and Maxima from scratch?   Moreover, do you think we should

No, I don't believe that and I never said that, e.g. see in my email
where I said --- if it has the choice of the license, because I was
expecting that you'll reply exactly that:)

> thus sever our relationships with the communities that have created
> all that software?  Because that is really the only option if Sage is
> to be releasable under a BSD or PSF license.

I don't think it's a good option.

On Tue, Aug 11, 2009 at 8:04 PM, David Joyner<wdjo...@gmail.com> wrote:
>
> On Tue, Aug 11, 2009 at 9:41 PM, Ondrej Certik<ond...@certik.cz> wrote:
[...]

>> I think so yes.
>
>
> It is hard to argue against that but IMHO the GPL serves the public interest
> in terms of taxpayer's value. The BSD has greater commercial value but

Yes, I think in the first place, everything that is supported by
government money (=tax payers money, because government has no money)
should be opensource and GPL is an opensource license. So GPL servers
the public interest imho, to some extent.

> I'm not convinced that every taxpayer-supported program must also
> support also commercial interests. If commercial interests are set aside then
> it seems the the GPL is a sufficient license to guarantee taxpayer's value.
> If the commercial value is not to be set aside then what is the argument
> that the taxpayers must *also* support commercial companies?

I think this is the root of the question. I don't know, I guess
everyone has to decide for himself/herself. I tend to think that
government/taxpayers should not restrict taxpayer in using the "public
product" in any way they wish, it's their money.

Ondrej

[William Stein]

>> I'm not convinced that every taxpayer-supported program must also
>> support also commercial interests. If commercial interests are set aside then
>> it seems the the GPL is a sufficient license to guarantee taxpayer's value.
>> If the commercial value is not to be set aside then what is the argument
>> that the taxpayers must *also* support commercial companies?
>
> I think this is the root of the question. I don't know, I guess
> everyone has to decide for himself/herself. I tend to think that
> government/taxpayers should not restrict taxpayer in using the "public
> product" in any way they wish, it's their money.
>
> Ondrej

Here's another question for you. In the case of Sage, perhaps 90% of
the effort is volunteer work, often in people's free time, as a hobby.
It would be generous to say that about 10% is directly being funded
by government grant money. How does this sort of "partial funding"
impact your thinking?

-- William

[Ondrej Certik]

Exactly. I don't know the answer.

I don't even think there is an answer, at the end of the day, one just
has to use his own judgement and intuition, to best use the government
money to server public interest. I think Sage is doing well with this
regard.

Ondrej

[William Stein]

It's certainly doing better than Magma in this regard, imho.

William

[Ondrej Certik]

Exactly, GPL or BSD, is a little thing, on top of the iceberg, because
both is opensource. I think that getting government money, creating
something, and then selling it without giving away the sourcecode, I
think that's a wrong model (=definitely I do not support such model).

But I have thought about your question --- so my answer is that I
believe that government funded work on software should be BSD (or
public domain) and in most cases one can satisfy that pretty easily,
e.g. I get some funding on implementing magnetohydrodynamics, so I do
that, release all my new code as BSD, and yes, in order to actually
run it, I have to use some GPLed software (umfpack, FEM solver), and
since that is the best that I can get for the government money (e.g.
the only alternatives are commercial), then I just use GPL. E.g.
exactly like Sage, there are no alternatives to gmp, maxima and other
libraries.

The idea is, that let's say some company would like to get something
for their taxes, so yes, they get exactly my work, e.g. they can take
it, and use it (it only depends on opensource) and since it is BSD
licensed, they can do whatever they want with it. I couldn't make it
depend on BSD stuff only, because then I couldn't get my job done,
which is the most important what I got the grant for.

Anyway, thanks for discussing this with me, it helped me to form a
better opinion on this.

Ondrej

[William Stein]

Hey, there is a slashdot story right now called "Leaving the GPL
behind" about just this sort of discussion:

http://news.slashdot.org/story/09/08/12/018255/Leaving-the-GPL-Behind?from=rss

[Harald Schilly]

On Aug 12, 4:27 am, William Stein <wst...@gmail.com> wrote:
> > I think this is the root of the question. I don't know, I guess
> > everyone has to decide for himself/herself. I tend to think that
> > government/taxpayers should not restrict taxpayer in using the "public
> > product" in any way they wish, it's their money.
>
> > Ondrej
>
> Here's another question for you.  In the case of Sage, perhaps 90% of
> the effort is volunteer work, often in people's free time, as a hobby.
>  It would be generous to say that about 10% is directly being funded

> by government grant money. ...


There is possibly a third option solving this: Every contributed code
that is government sponsored has to do this under something like a
"contributor agreement" with the government.
e.g.: http://www.sun.com/software/opensource/contributor_agreement.jsp#ca_1
"Consolidated copyright of code also allows for the possibility of
relicensing the whole code base should that become desirable."
IANAL, but if the government consolidates all its sponsored code it
has the authority to relicense these code parts. I think that's
legally possible but just not done. The complicated question is, what
happens with additional patches on top of that code which are not
sponsored.

H

[Ondrej Certik]

I think it's the same as if you pull some BSD code and integrate it,
and create some (GPL) patches on top of it. E.g the code stays bsd, so
the government (and me) are happy, and what other people contribute in
their own government sponsored activity, they are also free to use any
license they want (e.g. GPL).

Ondrej

Ondrej

[Ondrej Certik]

I mean non government sponsored activity

Ondrej

[kro...@uni-math.gwdg.de]

Do you also think that the Sage project should thus spend about 10
years (or much more?) of hard work by a bunch of people to rewrite all
functionality we need that is currently in GAP, Singular, PARI, NTL,
GAP, GMP, and Maxima from scratch?   Moreover, do you think we should
thus sever our relationships with the communities that have created
all that software?  Because that is really the only option if Sage is
to be releasable under a BSD or PSF license.

 -- William

Not for the sake of a certain license model but for the sake of correctness,
 modularity and maintainability at least some  parts of GAP and Singular
could benefit from rewriting from scratch or extensible testing.

My first impressions with Sage were positive, especially due to the cultural aspects (e.g. taking quality assurance more serious, using reviews and doctesting )
Then I discovered that Singular (and GAP) are incorporated in Sage and I was shocked.

 I won't claim I would do a
better job than the Singular or GAP developers, but if I extrapolate my experience,
Singular is full of issues and bugs and   GAP  contains at least several pointer bugs (e.g. out-of bound access);
 the 'object' and namespacing designs are not optimal.


  -- Jack

[William Stein]

Unfortunately, very large software projects are difficult, resources
are limited, and there are many bugs in them. For example, Sage has
1586 known bugs right now:

http://trac.sagemath.org/query?status=new&status=needs_work&status=needs_review&status=needs_info&status=positive_review&type=defect&max=0&order=id


Definitely if there were massively more money/people/resource, then
all kinds of amazing things could happen.

William


>
> -- Jack
>
> --
> You received this message because you are subscribed to the Google Groups
> "sage-flame" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-flame+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sage-flame.
> For more options, visit https://groups.google.com/groups/opt_out.



--
William Stein
Professor of Mathematics
University of Washington
http://wstein.org

[kro...@uni-math.gwdg.de]

Unfortunately, very large software projects are difficult, resources
are limited, and there are many bugs in them.  For example, Sage has
1586 known bugs right now:

William Stein 

If the project size and complexity would be the only reason...

The Singular library routines do mostly not check for valid parameters;
many routines fail already for the simplest input, e.g. see (my) bug reports

http://www.singular.uni-kl.de:8002/trac/ticket/527, http://www.singular.uni-kl.de:8002/trac/ticket/526

I have the impression, that they do not test/review their routines seriously (a typical banana product)

They have serious problems with variable shadowing and ring variable conflicts, e.g.
http://www.singular.uni-kl.de:8002/trac/ticket/508

Two years ago I held a small talk in Kaiserslautern about test code coverage,
and I guess things did not improve much since then (I have to check first, though)

So clearly, the project size and complexity is not their biggest problem,
but the blatant ignorance of the basic lessons from the past in software engineering.


Jack

[kro...@uni-math.gwdg.de]

Unfortunately, very large software projects are difficult, resources
are limited, and there are many bugs in them.  For example, Sage has
1586 known bugs right now:

   http://trac.sagemath.org/query?status=new&status=needs_work&status=needs_review&status=needs_info&status=positive_review&type=defect&max=0&order=id


Definitely if there were massively more money/people/resource, then
all kinds of amazing things could happen.

William

<offtopic>It seems my post was withdrawn or get lost due to some browser issues or recent updates of google group (I'm using Firefox)
 </offtopic>

If the size and complexity of the Singular project were the only problems...
It seems that several Singular developers ignore the known lessons from software engineering in the past.


Just to give some examples:

- Most interfaces to not check if the input parameters are valid

-It seems (at least serveral) Singular contributors do not test or review appropriately  their code:

 some methods fail for very basic input

http://www.singular.uni-kl.de:8002/trac/ticket/527
http://www.singular.uni-kl.de:8002/trac/ticket/526

-Singular has serious issues with variable shadowing, ring variable name conflicts, global(!) options,
see e.g.

http://www.singular.uni-kl.de:8002/trac/ticket/508

-Singular maintails reference output (in a git repo) only in packed binary format(!) so that is almost impossible to
retrace changes.

http://www.singular.uni-kl.de:8002/trac/ticket/531


-years ago I held in Kaiserslautern about test code coverage, and I guess
things get not much improved (I have to check first, through).


Should I continue?


What I'm fearing is that relying on external libraries with quite poor code quality( e.g. Singular )
will have major negative impact to the Sage project.


Jack

[William Stein]

On Fri, Nov 15, 2013 at 8:39 AM, "Jakob Kröker"
<kro...@uni-math.gwdg.de> wrote:
> <offtopic>

> It seems my post got lost due to some browser issues, recent updates of
> google group (I'm using firefox) or it was blocked by a moderator
> I'm now trying to answer directly via email and apologize if someone will
> get duplicate messages.
> </offtopic>

>
>
>> Unfortunately, very large software projects are difficult, resources
>> are limited, and there are many bugs in them. For example, Sage has
>> 1586 known bugs right now:
>>
>> http://trac.sagemath.org/query?status=new&status=needs_work&status=needs_review&status=needs_info&status=positive_review&type=defect&max=0&order=id
>>
>>
>> Definitely if there were massively more money/people/resource, then
>> all kinds of amazing things could happen.
>>
>> William
>
>

> If the size and complexity of the Singular project were the only problems...

> It seems that several Singular developers ignore basic known lessons from
> software engineering.

>
>
> Just to give some examples:
>

> - Most interfaces do not check if the input parameters are valid
>
> -It seems (at least several) Singular contributors do not test or review

> appropriately their code:
>
> some methods fail for very basic input
>
> http://www.singular.uni-kl.de:8002/trac/ticket/527
> http://www.singular.uni-kl.de:8002/trac/ticket/526
>
> -Singular has serious issues with variable shadowing, ring variable name
> conflicts, global(!) options,
> see e.g.
>
> http://www.singular.uni-kl.de:8002/trac/ticket/508
>

> -Singular maintains reference output (in a git repo) only in packed binary
> format(!) so that is almost impossible to retrace changes:
>
> http://www.singular.uni-kl.de:8002/trac/ticket/531
>
> -years ago I held in Kaiserslautern a talk about test code coverage, and I

> guess things get not much improved (I have to check first, through).
>

> I will stop here for now.

>
>
> What I'm fearing is that relying on external libraries with quite poor
> code quality( e.g. Singular )
> will have major negative impact to the Sage project.

This is an argument that might have made sense to make 8 years ago.
However, since we've been relying on Singular for 8 years, you must
instead point out specific ways in which relying on Singular has
*already* had a major negative impact on the Sage project.

It is easy to find a lot of Sage bugs in http://trac.sagemath.org over
the years that are completely the fault of Singular. However, I
don't have the impression that they have had a "major negative impact"
on the project.

There have been numerous instances of joint workshops and other
collaborations between Sage developers and the Singular group over the
years, which I view as a positive impact.

If we had not used Singular in Sage, leaving us with no non-toy
Groebner basis functionality at all, or even fast multivariate
polynomial arithmetic, then *that* would definitely have had a major
negative impact on Sage.

That said, if somebody (you?) wants to sit down and work very hard for
several years to do a good job implementing the same sort of
functionality from scratch that we use Singular for, and in a better
quality way, then this would be a very valuable contribution to the
community. Something quite similar has happened -- over the years --
with the FLINT library (http://www.flintlib.org/), which is partly
meant to be a more modern alternative to PARI. As it has turned out
though, FLINT is more a complement to PARI, rather than a replacement,
since PARI has a very wide range of capabilities and is still under
active development (e.g., much recent work on L-series).

-- William

["Jakob Kröker"]

<offtopic>
It seems my post got lost due to some browser issues, recent updates of
google group (I'm using firefox) or it was blocked by a moderator
I'm now trying to answer directly via email and apologize if someone will
get duplicate messages.
</offtopic>

> Unfortunately, very large software projects are difficult, resources
> are limited, and there are many bugs in them. For example, Sage has
> 1586 known bugs right now:
>
> http://trac.sagemath.org/query?status=new&status=needs_work&status=needs_review&status=needs_info&status=positive_review&type=defect&max=0&order=id
>
>
> Definitely if there were massively more money/people/resource, then
> all kinds of amazing things could happen.
>
> William

If the size and complexity of the Singular project were the only problems...
It seems that several Singular developers ignore basic known lessons from
software engineering.


Just to give some examples:

- Most interfaces do not check if the input parameters are valid

-It seems (at least several) Singular contributors do not test or review
appropriately their code:

some methods fail for very basic input

http://www.singular.uni-kl.de:8002/trac/ticket/527
http://www.singular.uni-kl.de:8002/trac/ticket/526

-Singular has serious issues with variable shadowing, ring variable name
conflicts, global(!) options,
see e.g.

http://www.singular.uni-kl.de:8002/trac/ticket/508

-Singular maintains reference output (in a git repo) only in packed binary
format(!) so that is almost impossible to retrace changes:

http://www.singular.uni-kl.de:8002/trac/ticket/531

-years ago I held in Kaiserslautern a talk about test code coverage, and I
guess things get not much improved (I have to check first, through).

I will stop here for now.


What I'm fearing is that relying on external libraries with quite poor
code quality( e.g. Singular )
will have major negative impact to the Sage project.

Jack

[rjf]

On Friday, November 15, 2013 8:51:17 AM UTC-8, William Stein wrote:

On Fri, Nov 15, 2013 at 8:39 AM, "Jakob Kröker"
<kro...@uni-math.gwdg.de> wrote:
> <offtopic>
> It seems my post got lost due to some browser issues, recent updates of
> google group (I'm using firefox) or it was blocked by a moderator
> I'm now trying to answer directly via email and apologize if someone will
> get duplicate messages.
> </offtopic>
>
>
>>    Unfortunately, very large software projects are difficult, resources
>>    are limited, and there are many bugs in them.  For example, Sage has
>>    1586 known bugs right now:
>>
>>       http://trac.sagemath.org/query?status=new&status=needs_work&status=needs_review&status=needs_info&status=positive_review&type=defect&max=0&order=id
>>
>>
>>    Definitely if there were massively more money/people/resource, then
>>    all kinds of amazing things could happen.
>>
>>   William
>
>
> If the size and complexity of the Singular project were the only problems...
> It seems that several Singular developers ignore basic known lessons from
> software engineering.

Much of software engineering is a crock.  I've looked at the texts, and even
taught a course on the subject.  Some (much?) of it is an attempt to help
intellectually deficient managers and programmers and users from doing too
much damage to each other while pursuing projects of zero technical
algorithmic, mathematical, numerical, scientific ... contents.   (Or engineering
those parts of possibly larger projects with zero content...)

Thus there are parts of Sage that are of essentially no content; perhaps the
code linking stuff?  Or routine web interfaces or display.  For some people
writing parsers is a technical challenge, but mostly because of their ignorance of
the available technology.

For example, search through the software engineering literature for the word
"roundoff".

I found it in one textbook once, in a sentence that said, in effect,
Since floating point numbers have roundoff, you should always use integers.

Now getting to the complaints about Singular...

>
>
> Just to give some examples:
>
> - Most interfaces do not check if the input parameters are valid

Does this mean that SOME interfaces DO check?   If so, it could be that
the other interfaces do not need to check, and inserting checks would
(a) bloat the source code,  (b) slow down the code (c) possibly introduce
bugs along with the code bloat (d) discourage programmers.

To the extent that the bugs reported are USER ERRORS, it may not
be a problem with Singular's code, but the model that Singular has
of the user.  For example, Singular may assume that its users know
something about the domain of applicability of the programmer, and not
guard against strange inputs.

I do not recall the last time I personally used Singular, and in fact I may
not have ever used it.

 

>
> -It seems (at least several) Singular contributors do not test or review
> appropriately  their code:
>
>  some methods fail for very basic input
>
> http://www.singular.uni-kl.de:8002/trac/ticket/527
> http://www.singular.uni-kl.de:8002/trac/ticket/526

I have not looked at these and don't know if the input is erroneous, or
these are just bugs or illustrations of design issues.

>
> -Singular has serious issues with variable shadowing, ring variable name
> conflicts, global(!) options,
> see e.g.
>
> http://www.singular.uni-kl.de:8002/trac/ticket/508

There may indeed be design issues.

>
> -Singular maintains reference output (in a git repo) only in packed binary
> format(!) so that is almost impossible to retrace changes:
>
> http://www.singular.uni-kl.de:8002/trac/ticket/531
>
> -years ago I held in Kaiserslautern a talk about test code coverage, and I
> guess things get not much improved (I have to check first, through).
>
> I will stop here for now.
>
>
> What I'm fearing is that relying on external libraries with quite poor
> code quality( e.g. Singular )
> will have major negative impact to the Sage project.

This is an argument that might have made sense to make 8 years ago.
However, since we've been relying on Singular for 8 years, you must
instead point out specific ways in which relying on Singular has
*already* had a major negative impact on the Sage project.

I don't know anything about the statistics for people accessing Singular
from Sage.  However, there is a good argument to be made that Sage
has an obligation to try to call Singular (etc) only with well-formed requests.
After all Sage is making far more expansive promises about doing so
many things for people who might not know anything about Singular per se.

After all, what would one do with a bug report that says,

"bouquet garni - thyme -bay leaves  "

should simplify to  sage."

or
 "beef + young 

should simplify to veal"

Is the response, Oh, Sage isn't about cooking?  Or do you go about fixing the bugs.





 

It is easy to find a lot of Sage bugs in http://trac.sagemath.org over
the years that are completely the fault of Singular.   However, I
don't have the impression that they have had a "major negative impact"
on the project.

There have been numerous instances of joint workshops and other
collaborations between Sage developers and the Singular group over the
years, which I view as a positive impact.

If we had not used Singular in Sage, leaving us with no non-toy
Groebner basis functionality at all, or even fast multivariate
polynomial arithmetic, then *that* would definitely have had a major
negative impact on Sage.

Are you familiar with CoCoA ?

That said, if somebody (you?) wants to sit down and work very hard for
several years to do a good job implementing the same sort of
functionality from scratch that we use Singular for, and in a better
quality way, then this would be a very valuable contribution to the
community.  

It seems that your message above contradicts this "valuable contribution"
notion.  1. Singular seems to work well enough;  2. We like the people;

So it seems that working on this project would be almost pointless
from the perspective of Sage. Beyond Sage,  I am not sure about the availability
of competitive programs that have "the same sort of functionality"
but they too should be considered if you are even remotely interested
in spending years writing a program.


RJF

[Tom Boothby]

On Fri, Nov 15, 2013 at 9:49 AM, rjf <fat...@gmail.com> wrote:
>
> Much of software engineering is a crock. I've ... even taught a course on the subject.

Proof enough for me!

[Bill Hart]

I taught a course on C once. By the end of it I think my students knew more about C than they had ever wanted to know.

Bill.

[William Stein]

On Fri, Nov 15, 2013 at 9:49 AM, rjf <fat...@gmail.com> wrote:

> I do not recall the last time I personally used Singular, and in fact I may
> not have ever used it.

That should be your signature, but with Singular replaced by "anything".

>> If we had not used Singular in Sage, leaving us with no non-toy
>> Groebner basis functionality at all, or even fast multivariate
>> polynomial arithmetic, then *that* would definitely have had a major
>> negative impact on Sage.
>
>
> Are you familiar with CoCoA ?

Of course, and I know the developers. However, CoCoA was closed
source for several years during which time we were using Singular in
Sage. I did write a pexpect interface to Cocoa though. The cocoa
devs started a complete new open source rewrite from scratch. I
don't know how useful that is today.

>>
>>
>> That said, if somebody (you?) wants to sit down and work very hard for
>> several years to do a good job implementing the same sort of
>> functionality from scratch that we use Singular for, and in a better
>> quality way, then this would be a very valuable contribution to the
>> community.
>
>
> It seems that your message above contradicts this "valuable contribution"
> notion. 1. Singular seems to work well enough; 2. We like the people;

I don't see a contradiction. This is identical the situation with
FLINT (versus PARI) -- it turns out that because PARI already solves
quite a lot of mundane problems, the FLINT devs have focuses mainly on
solving problems that you can't solve anywhere else, e.g., designing
and implementing new algorithms that are faster than anything ever
done before (anywhere).

> So it seems that working on this project would be almost pointless
> from the perspective of Sage. Beyond Sage, I am not sure about the

That's wrong. For example, Magma is (dramatically) faster than any
open source software at certain classes of Groebner basis. Somebody
who writes a new quality library might produce something that is
competitive with Magma. This happened, e.g., with Faugere, though
instead of open sourcing his work, he sells it (e.g., to Maple), which
is fine, but isn't a contribution to open source. However, I have no
reason to think Jakob might not write something of great value that is
better than Singular at certain problems, and is open source. I
strongly encourage him (and anybody else!) to try, and if it turns out
they succeed, we benefit.

William

[rjf]

On Friday, November 15, 2013 9:59:07 AM UTC-8, William Stein wrote:

On Fri, Nov 15, 2013 at 9:49 AM, rjf <fat...@gmail.com> wrote:

> I do not recall the last time I personally used Singular, and in fact I may
> not have ever used it.

That should be your signature, but with Singular replaced by "anything".

But then it would be false, so why would I do that .

>> If we had not used Singular in Sage, leaving us with no non-toy
>> Groebner basis functionality at all, or even fast multivariate
>> polynomial arithmetic, then *that* would definitely have had a major
>> negative impact on Sage.
>
>
> Are you familiar with CoCoA ?

Of course, and I know the developers.  However, CoCoA was closed
source for several years during which time we were using Singular in
Sage.  I did write a pexpect interface to Cocoa though.   The cocoa
devs started a complete new open source rewrite from scratch.    I
don't know how useful that is today.

So you wouldn't mind if CoCoA were truly top of the line, and you also
encouraged someone to spend several years to implement something
duplicating its functionality, but was perhaps inferior?   I guess
mathematicians have a wide range of opportunities to do mostly pointless
things, but even mostly-pointless things can be ranked.

>>
>>
>> That said, if somebody (you?) wants to sit down and work very hard for
>> several years to do a good job implementing the same sort of
>> functionality from scratch that we use Singular for, and in a better
>> quality way, then this would be a very valuable contribution to the
>> community.
>
>
> It seems that your message above contradicts this "valuable contribution"
> notion.  1. Singular seems to work well enough;  2. We like the people;

I don't see a contradiction.  This is identical the situation with
FLINT (versus PARI) -- it turns out that because PARI already solves
quite a lot of mundane problems, the FLINT devs have focuses mainly on
solving problems that you can't solve anywhere else, e.g., designing
and implementing new algorithms that are faster than anything ever
done before (anywhere).

> So it seems that working on this project would be almost pointless
> from the perspective of Sage. Beyond Sage,  I am not sure about the

That's wrong.   For example, Magma is (dramatically) faster than any
open source software at certain classes of Groebner basis.  Somebody
who writes a new quality library might produce something that is
competitive with Magma.   This happened, e.g., with Faugere, though
instead of open sourcing his work, he sells it (e.g., to Maple), which
is fine, but isn't a contribution to open source.

So if anyone had any interest in a Grobner basis computation, an interest
worth more than a few dollars, he should either
(a) buy or borrow the use of maple
   or
(b) spend a few years of his life trying to figure out and improve on Faugere's
stuff.

Hard choice.

 

 However, I have no
reason to think Jakob might not write something of great value that is
better than Singular at certain problems, and is open source.  I
strongly encourage him (and anybody else!) to try, and if it turns out
they succeed, we benefit.

Primarily in a political, not a mathematical sense, it seems.

William

[William Stein]

On Fri, Nov 15, 2013 at 10:25 AM, rjf <fat...@gmail.com> wrote:
>
>
> On Friday, November 15, 2013 9:59:07 AM UTC-8, William Stein wrote:
>>
>> On Fri, Nov 15, 2013 at 9:49 AM, rjf <fat...@gmail.com> wrote:
>>
>> > I do not recall the last time I personally used Singular, and in fact I
>> > may
>> > not have ever used it.
>>
>> That should be your signature, but with Singular replaced by "anything".
>
> But then it would be false, so why would I do that .
>
>>
>> >> If we had not used Singular in Sage, leaving us with no non-toy
>> >> Groebner basis functionality at all, or even fast multivariate
>> >> polynomial arithmetic, then *that* would definitely have had a major
>> >> negative impact on Sage.
>> >
>> >
>> > Are you familiar with CoCoA ?
>>
>> Of course, and I know the developers. However, CoCoA was closed
>> source for several years during which time we were using Singular in
>> Sage. I did write a pexpect interface to Cocoa though. The cocoa
>> devs started a complete new open source rewrite from scratch. I
>> don't know how useful that is today.
>
>
> So you wouldn't mind if CoCoA were truly top of the line, and you also
> encouraged someone to spend several years to implement something
> duplicating its functionality, but was perhaps inferior?

Hell yes. I would not "mind" someone doing this. The person would
have done something more valuable than playing World of Warcraft for
several years. I encourage innovation, experimentation, etc.
Perhaps this hypothetical person would produce something that is not a
proper subset of CoCoA... Software is complicated and multifaceted;
programs are rarely proper subsets of other programs.

>> That's wrong. For example, Magma is (dramatically) faster than any
>> open source software at certain classes of Groebner basis. Somebody
>> who writes a new quality library might produce something that is
>> competitive with Magma. This happened, e.g., with Faugere, though
>> instead of open sourcing his work, he sells it (e.g., to Maple), which
>> is fine, but isn't a contribution to open source.
>
>
> So if anyone had any interest in a Grobner basis computation, an interest
> worth more than a few dollars, he should either
> (a) buy or borrow the use of maple
> or
> (b) spend a few years of his life trying to figure out and improve on
> Faugere's
> stuff.
>
> Hard choice.

Seriously?

>> However, I have no
>> reason to think Jakob might not write something of great value that is
>> better than Singular at certain problems, and is open source. I
>> strongly encourage him (and anybody else!) to try, and if it turns out
>> they succeed, we benefit.
>
>
> Primarily in a political, not a mathematical sense, it seems.

Nope, mathematical and technical. Political is irrelevant.

Well, you've succeeded at wasting 10 minutes of my time...

["Jakob Kröker"]

>At the time, I chose Python, I was
>also planning to not depend much on existing third party math
>libraries, but instead to write everything "from scratch", based on
>looking at their code, etc. That approach didn't last long, since I
>don't have infinite time to waste.
> [...]
> -- William

Probably it was inevitably in the early days of the Sage project to build
up on other related projects, even if they had some significant issues.
So far, so good; the pragmatism forced you to avoid the "not invented
here"-syndrome. But I hope that the Sage project or other groups will
have the will, theoretical and technical ability and also adequate
funding to rewrite some deficient parts of the used libraries.

<offtopic>
As a user I would also like to get a hint about the reliability (up to
some subjective degree) of a certain function, if this is possible at all.
(if code parts are experimental, this may be fine, but nothing really
useful can be build up on top of experimental/untested code unless you
exactly know what are you doing)
</offtopic>

To return to the topic ("the most dangerous Sage language flaws"),
I guess that many bugs will be related to the Python<>Sage integer types
interaction and Python's implicit data type conversion rules,
especially to the "everything is a bool" philosophy.

Comments?
Or more suggestions (see also Bill Hart's post)?

What about antidotes? I would expect, there exists a checklist (is there
one?) which describes how to deal with this (and other) distinctive
features of the Python programming language properly. In addition to code
reviews and code testing, maybe it would be also fine to have a link to an
online test where contributors may check their Python skills. At a first
look I didn't find a free online test, but maybe this needs an extensive
googling session..



Jack

[kro...@uni-math.gwdg.de]

what the hell..

I'm sorry, I posted in the wrong thread (to "sage constraints" instead of "most dangerous Sage language flaws")
by replying accidentally to a different answer (changing the email subject seems to have no effect)

Can this be fixed somehow?

Jack

[William Stein]

On Fri, Nov 15, 2013 at 2:37 PM, "Jakob Kröker"
<kro...@uni-math.gwdg.de> wrote:
>>At the time, I chose Python, I was
>>also planning to not depend much on existing third party math
>>libraries, but instead to write everything "from scratch", based on
>>looking at their code, etc. That approach didn't last long, since I
>>don't have infinite time to waste.
>> [...]
>> -- William
>
> Probably it was inevitably in the early days of the Sage project to build
> up on other related projects, even if they had some significant issues.
> So far, so good; the pragmatism forced you to avoid the "not invented
> here"-syndrome. But I hope that the Sage project or other groups will
> have the will, theoretical and technical ability and also adequate
> funding to rewrite some deficient parts of the used libraries.

So do I. This is one reason I've been looking for alternative
commercial sources of funding (some of the motivation for
https://cloud.sagemath.com).

> <offtopic>
> As a user I would also like to get a hint about the reliability (up to
> some subjective degree) of a certain function, if this is possible at all.
> (if code parts are experimental, this may be fine, but nothing really
> useful can be build up on top of experimental/untested code unless you
> exactly know what are you doing)
> </offtopic>

With Sage at least it is theoretically possible via reading relevant
source code to make a judgement for yourself in any particular case
You can see how good the tests, are, etc. That's a first step at
least, which isn't possible with (say) Mathematica or Magma. It's
clear you've been doing exactly this with the Singular source code,
and you're not happy with what you see. Fair enough.

I once asked a CoCoA developer why CoCoa was closed source; they said
the *main* reason was that the source code was "embarrassing".

> To return to the topic ("the most dangerous Sage language flaws"),
> I guess that many bugs will be related to the Python<>Sage integer types
> interaction and Python's implicit data type conversion rules,
> especially to the "everything is a bool" philosophy.
>
> Comments?
> Or more suggestions (see also Bill Hart's post)?
>
> What about antidotes? I would expect, there exists a checklist (is there
> one?) which describes how to deal with this (and other) distinctive
> features of the Python programming language properly. In addition to code
> reviews and code testing, maybe it would be also fine to have a link to an
> online test where contributors may check their Python skills. At a first
> look I didn't find a free online test, but maybe this needs an extensive
> googling session..

When I teach Python and Cython programming to undergrads (which I do
for at least a month every year), I spent a lot of the time covering
the standard gotchas, which I've seen confuse a lot of people and
waste much time over the years. See
https://github.com/williamstein/2013-480

Some examples:

- len(v) to get the length of v, instead of v.len[tab key]

- using lists and dicts as default arguments to functions (they get
mutated and change)

- Python's unusual scoping rules, and recursion limitations.
Especially in math it is tempting to use recursion, but for math where
the number of calls might exceed 1000, Python is not so good.

- that lists are really an ordered list of *references*, and that
everything is a reference, causes a lot of confusion (since it is
different than matlab). v=[1,[2,3]]; w=v[1]; w[1]=10; now v is
different...

- When defining your own classes, do not define __hash__ on
anything that is mutable, or you'll run into subtle bugs.

- 2/3 is 0 in Python

- If you do this, then ValueError will get overwritten with the
actual exception:

try:
blah
except TypeError, ValueError: # !! should be "(TypeError, ValueError):"
blah blah


Many of these issues have been addressed in python3.x.

-- William

>
>
>
> Jack
>
>
>
>
>
> --
> You received this message because you are subscribed to the Google Groups "sage-flame" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-flame+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sage-flame.
> For more options, visit https://groups.google.com/groups/opt_out.

[rjf]

I sometimes look around at the attendees at a particularly poor lecture
and image that if we all were given shovels, we could spend our time
more usefully digging a ditch.  How long / deep would that ditch be,
given reasonable assumptions about soil, etc.
 

    I encourage innovation, experimentation, etc.

I have heard it said  (paraphrasing) that two weeks programming can save an hour
in the library.

 

Perhaps this hypothetical person would produce something that is not a
proper subset of CoCoA...    Software is complicated and multifaceted;
programs are rarely proper subsets of other programs.

Most programs don't work very well, if at all.
 

>> That's wrong.   For example, Magma is (dramatically) faster than any
>> open source software at certain classes of Groebner basis.  Somebody
>> who writes a new quality library might produce something that is
>> competitive with Magma.   This happened, e.g., with Faugere, though
>> instead of open sourcing his work, he sells it (e.g., to Maple), which
>> is fine, but isn't a contribution to open source.
>
>
> So if anyone had any interest in a Grobner basis computation, an interest
> worth more than a few dollars, he should either
> (a) buy or borrow the use of maple
>    or
> (b) spend a few years of his life trying to figure out and improve on
> Faugere's
> stuff.
>
> Hard choice.

Seriously?

No, that was sarcasm.  If I had to compute a GB and  it took too long
on whatever I had in front of me, and someone said that
Maple had the best program, I'd use Maple.

>>  However, I have no
>> reason to think Jakob might not write something of great value that is
>> better than Singular at certain problems, and is open source.  I
>> strongly encourage him (and anybody else!) to try, and if it turns out
>> they succeed, we benefit.

Sure, it costs you nothing, and costs Jakob two years of his life.  Sounds
like a benefit. Maybe. but probably not, and the downside to Jakob is
substantial.
 

>
>
> Primarily in a political, not a mathematical sense, it seems.

Nope, mathematical and technical.  Political is irrelevant.

You said it yourself just now.  "open source".  That's political,
not technical.
 

Well, you've succeeded at wasting 10 minutes of my time...

I would hope you get some enjoyment from um, banter.

RJf

 

[rjf]

On Friday, November 15, 2013 3:20:22 PM UTC-8, William Stein wrote:

....

 

With Sage at least it is theoretically possible via reading relevant
source code to make a judgement for yourself in any particular case
You can see how good the tests, are, etc.    That's a first step at
least, which isn't possible with (say) Mathematica or Magma.    It's
clear you've been doing exactly this with the Singular source code,
and you're not happy with what you see.   Fair enough.

 

I think the first step would certainly not be to read the source code.
It would more likely be the testing of known examples.  In some cases
the solution, once known, can be tested.  Proving someone else's code
to be correct by reading it -- well, theoretically, that would be an
advantage of open source over closed source, but that is a mostly
illusory advantage.
It's too hard for humans, and too hard to automate.

I once asked a CoCoA developer why CoCoa was closed source; they said
the *main* reason was that the source code was "embarrassing".

That's fair.  Just out of curiousity did he/she say what were the other reasons?

 

> To return to the topic ("the most dangerous Sage language flaws"),
> I guess that many bugs will be related to  the Python<>Sage integer types
> interaction and Python's implicit data type conversion rules,
> especially to the "everything is a bool" philosophy.
>
> Comments?
> Or more  suggestions (see also Bill Hart's post)?
>
> What about antidotes?  I would expect, there exists a checklist (is there
> one?) which describes how to deal with this (and other) distinctive
> features of the Python programming language properly. In addition to code
> reviews and code testing, maybe it would be also fine to have a link to an
> online test where contributors may check their Python  skills. At a first
> look I didn't find a free online test, but maybe this needs an extensive
> googling session..

When I teach Python and Cython programming to undergrads (which I do
for at least a month every year), I spent a lot of the time covering
the standard gotchas, which I've seen confuse a lot of people and
waste much time over the years.   See
https://github.com/williamstein/2013-480

Too bad.  I dislike teaching something I have to make excuses for.

RJF
 

[William Stein]

On Fri, Nov 15, 2013 at 5:07 PM, rjf <fat...@gmail.com> wrote:
>
>
> On Friday, November 15, 2013 3:20:22 PM UTC-8, William Stein wrote:
>>
>> ....
>
>
>>
>> With Sage at least it is theoretically possible via reading relevant
>> source code to make a judgement for yourself in any particular case
>> You can see how good the tests, are, etc. That's a first step at
>> least, which isn't possible with (say) Mathematica or Magma. It's
>> clear you've been doing exactly this with the Singular source code,
>> and you're not happy with what you see. Fair enough.
>
>
> I think the first step would certainly not be to read the source code.
> It would more likely be the testing of known examples. In some cases
> the solution, once known, can be tested. Proving someone else's code
> to be correct by reading it -- well, theoretically, that would be an

Boggle. The original poster wrote "get a hint about the reliability
(up to some subjective degree)" and you deleted their quote and
pretended that we're talking about "proof".

[rjf]

In general a bug in a program can be (in a way that can be
defined rigorously) extremely difficult  (in a Turing-computable
sense), it's not clear what is meant by a hint of subjective reliability.

Unless the source code includes comments like
"I know this only works for simple cases"  or
"This part has never been tested" or
"Boothby wrote this part"

Then  what is one to use to get a hint?   Maybe the source code has
smudgy fingerprints?


Getting to the main question:  How does Sage test for correct behavior?
A regression test suite perhaps.  Anything else?

I get the feeling that no Sage-ist ever looked at the source
code for Maxima. I don't know about other parts.

RJF