Should we still support old-style packages?

[Jeroen Demeyer]

Hello,

as you might know, we currently have two kinds of packages in Sage: the
old-style .spkg files which were the norm in Sage 5.x and the new-style
$SAGE_ROOT/build/pkgs packages which are currently the norm.

The question is: to what extent should we continue supporting old-style
packages?

(A) sage -i OLDSTYLEPKGNAME should just install the package, there is no
difference with new-style packages from the user's point of view.
(B) sage -i OLDSTYLEPKGNAME should still work but with a clear
deprecation warning.
(C) sage -i OLDSTYLEPKGNAME should still work but only after a "are you
sure?" confirmation, like we have for experimental packages.
(D) sage -i OLDSTYLEPKGNAME should give an error.

In Sage 6.8 we have (A) and in the latest betas, we have (D). I think
that (D) is an over-reaction to the problem that some old-style packages
are confusing or broken.

My personal vote goes to (B) since it still allows non-interactive
scripts to work like before but it shows a clear message that packages
should migrate to new-style packages. This is also implemented in #19158.

Jeroen..

[Nathann Cohen]

Another way to "fix" the problem:

(E) Convert the remaining old-style packages to new-style packages.

Nathann

[Volker Braun]

IMHO showing a warning that you are about to do something stupid but then proceeding to do so is a terrible user interface. Also, I don't like command line utilities that ask stupid questions (a.k.a. the windows UI style).

Really, I still don't know a use case where you actually want to search the list of old-style packages. All the old-style packages that I recently tried don't work. Back when we switched to the new-style packages I already went through the list old-style packages and converted everything that made sense to me and worked on my machine.  E.g. chomp doesn't build.

I do, however, see lots of very good examples for why its terribly confusing to install old-style packages for unsuspecting users. For starters:

* The "python" package is ancient, you really want the new "python2" or "python3" packages. Nobody is going to figure that one out

* TOPCOM vs. topcom, which one is current?

If you really want to use old-style packages then thats fine, just specify the full url to the spkg. The only old-style spkg on the Sage mirrors that is actively developed is Simon's, and he is most likely better served by providing the spkg at some place that is under his direct control and telling his users the full url.

[Jeroen Demeyer]

On 2015-09-09 10:15, Nathann Cohen wrote:
> Another way to "fix" the problem:
> (E) Convert the remaining old-style packages to new-style packages.

Of course we should do that, but that's orthogonal to the question I
asked. I don't think that (E) is going to happen short-term.

As long as there is still at least one old-style package, we need to
define what should happen when a user types

sage -i $OLDSTYLEPACKAGE

[Nathann Cohen]

> Of course we should do that, but that's orthogonal to the question I asked.
> I don't think that (E) is going to happen short-term.
>
> As long as there is still at least one old-style package, we need to define
> what should happen when a user types

I suspect that it is not so orthogonal, and that it is the main source
of disagreement between Volker and you. He seems to say that "all
packages which work are now new-style", so that there is no "useful"
old-style package left.

He thus considers them not as old-style but as "broken", which solves
the problem (for him). If there is any of them which is not broken,
well, turning it into a new-style package does the job.

Nathann

[John Cremona]

Re (B): a deprecation warning is normally a message to the user that they should start to do things in a different way, as the old way will stop working.  This does not seem appropriate here, since it is not the user who is expected to change behaviour (unless you regard all users as developers!).  So rather than a deprecation warning as such why not a message saying

"OLDSTYLEPACKAGE is no longer supported by SageMath;  if it was important for your work, please contact <sage-support?>  with a request to upgrade the package."

Then, if many such requests come in for a package we will at least know what is missed and could be given priority.

John

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To post to this group, send email to sage-...@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[Jeroen Demeyer]

On 2015-09-09 10:50, John Cremona wrote:
> Re (B): a deprecation warning is normally a message to the user that
> they should start to do things in a different way, as the old way will
> stop working. This does not seem appropriate here, since it is not the
> user who is expected to change behaviour (unless you regard all users as
> developers!). So rather than a deprecation warning as such why not a
> message saying
>
> "OLDSTYLEPACKAGE is no longer supported by SageMath; if it was
> important for your work, please contact <sage-support?> with a request
> to upgrade the package."

I think the wording is just a detail and not relevant for this vote, so
you can think of option (B) as: display some kind of message and just
install the package.


Anyway, the proposed wording in #19158 is

You are about to download and install an old-style package. While this
might still work fine, old-style packages are unmaintained and deprecated.

This package will be removed in future versions of SageMath. If you care
about this package, you should make a proper new-style package instead.
For more information about making Sage packages, see
http://doc.sagemath.org/html/en/developer/packaging.html

[John Cremona]

On 9 September 2015 at 10:22, Jeroen Demeyer <jdem...@cage.ugent.be> wrote:

On 2015-09-09 10:50, John Cremona wrote:

Re (B): a deprecation warning is normally a message to the user that
they should start to do things in a different way, as the old way will
stop working.  This does not seem appropriate here, since it is not the
user who is expected to change behaviour (unless you regard all users as
developers!).  So rather than a deprecation warning as such why not a
message saying

"OLDSTYLEPACKAGE is no longer supported by SageMath;  if it was
important for your work, please contact <sage-support?>  with a request
to upgrade the package."

I think the wording is just a detail and not relevant for this vote, so you can think of option (B) as: display some kind of message and just install the package.

Not quite -- I was proposing to display the message and *not* install the package -- since I thought that essentially none of the old .spkg will now install anyway?

 

Anyway, the proposed wording in #19158 is

You are about to download and install an old-style package. While this might still work fine, old-style packages are unmaintained and deprecated.

This package will be removed in future versions of SageMath. If you care about this package, you should make a proper new-style package instead. For more information about making Sage packages, see
http://doc.sagemath.org/html/en/developer/packaging.html

I like the second sentence better, and some version of the first...

John

[Jeroen Demeyer]

On 2015-09-09 11:26, John Cremona wrote:
> Not quite -- I was proposing to display the message and *not* install
> the package

OK, so this is
(D'): sage -i OLDSTYLEPKGNAME should give an error. However, it should
determine whether an old-style package named OLDSTYLEPKGNAME exists and
adjust the error message in case it does exist.

> since I thought that essentially none of the old .spkg
> will now install anyway?

Who are you to decide that? I really dislike this attitude of "we,
developers, know better than the user what the user wants".

If the "essentially none" in your sentence would be literally none, then
you are right. But even if there is just a single old-style package
which works and is useful, I think we (the Sage developers) should not
actively prevent the user from installing that package.

That's also why we have experimental packages: those are packages which
may or not work properly or work only partially. Do you think we should
remove all of them just because they might not work?

No offense meant,
Jeroen.

[Volker Braun]

On Wednesday, September 9, 2015 at 11:56:28 AM UTC+2, Jeroen Demeyer wrote:

But even if there is just a single old-style package
which works and is useful

I disagree with that rationale. We want to maximize value for the scientific community. If that means inconveniencing a single user in order to make Sage accessible to 100 other users then we should do that. 

[Jeroen Demeyer]

On 2015-09-09 12:01, Volker Braun wrote:
> in
> order to make Sage accessible to 100 other users then we should do that.

How does option (D) make "Sage accessible to 100 other users" compared
with options (B) or (C)?

[Volker Braun]

On Wednesday, September 9, 2015 at 12:13:39 PM UTC+2, Jeroen Demeyer wrote:

How does option (D) make "Sage accessible to 100 other users" compared
with options (B) or (C)?

Because not even 1 out of 100 will figure out that "python" actually tries to install the old and unsupported package.

[Nathann Cohen]

Another option: would it be possible to make "fake new-style
packages", whose upstream/ tarball would be the .spkg?

This way they would all become new-style packages at a very low cost.

Nathann

> --
> You received this message because you are subscribed to a topic in the
> Google Groups "sage-devel" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/sage-devel/_5SidQWFYDY/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to

[Jeroen Demeyer]

On 2015-09-09 13:33, Nathann Cohen wrote:
> Another option: would it be possible to make "fake new-style
> packages", whose upstream/ tarball would be the .spkg?
>
> This way they would all become new-style packages at a very low cost.

And which problem would that solve?

[Jeroen Demeyer]

On 2015-09-09 13:14, Volker Braun wrote:
> Because not even 1 out of 100 will figure out that "python" actually
> tries to install the old and unsupported package.

There are other solutions to that problem, any of the following will do:
* a proper implementation of (B) to make them figure it out
* the proposed #19105
* removing the old python spkg from the server

[Nathann Cohen]

>> This way they would all become new-style packages at a very low cost.
>
> And which problem would that solve?

It would then be possible to install them with "sage -i <package>",
which is I belive the reason you created this thread?

Nathann

[Jeroen Demeyer]

But by reverting #19004 (which is essentially what #19158 does), it is
also possible to install them using "sage -i <package>".

We don't have to fix "sage -i", we just need to undo the breaking.

[Nathann Cohen]

> But by reverting #19004 (which is essentially what #19158 does), it is also
> possible to install them using "sage -i <package>".

There are multiple ways out indeed. All I try to do here is help you
two work together if I can, by suggesting "middle ways". What happens
with unmaintained packages does not concern me much otherwise.

Nathann

[Volker Braun]

On Wednesday, September 9, 2015 at 2:14:23 PM UTC+2, Jeroen Demeyer wrote:

There are other solutions to that problem, any of the following will do:
* a proper implementation of (B) to make them figure it out
* the proposed #19105
* removing the old python spkg from the server

None of those solves the TOPCOM vs topcom issue, or the giant grab bag of issues with other old-style packages. Other than deleting them all from the server, of course. The latter *would* also solve the problem. 

[Jeroen Demeyer]

On 2015-09-09 15:06, Volker Braun wrote:
> Other than deleting them all
> from the server, of course. The latter *would* also solve the problem.

Just like reformatting your hard drive will instantly remove all
software issues that you might have with your computer.

[Volker Braun]

On Wednesday, September 9, 2015 at 2:47:20 PM UTC+2, Jeroen Demeyer wrote:

But by reverting #19004 (which is essentially what #19158 does), it is
also possible to install them using "sage -i <package>". 

It is possible to try, and almost always fail miserably, to install them. 

[Travis Scrimshaw]

   A possible (hack) fix is to put something in the skpg script to explicitly ban it from installing python without specifying 2 or 3. It keeps the old python spkg for backwards compatibility for 5.x and prevents 6.x users from using the old outdated version. Either that or we provide a backport patch which changes the packaging system on older versions of Sage to the latest, but that doesn't sound very feasible.

   To the discussion at hand, I think the current system of not allowing old spkgs to be installed is a waste. What are the current old style (non-experimental) spkgs, which of them have you tried, and which of them don't work on what system. I'm willing to put in some time and effort to convert some of them to new style (specifically chomp and Simon's cohomology) to try and get this resolved. However, if we are going to go with the current route, we should have a major version number bump for the next stable release.

Best,
Travis

[kcrisman]

But by reverting #19004 (which is essentially what #19158 does), it is
also possible to install them using "sage -i <package>". 

It is possible to try, and almost always fail miserably, to install them. 

It would be helpful for this discussion to know what packages we are actually talking about.  For instance, for some of them (e.g. beautifulsoup or brian) we could just make new-style packages that say "this is no longer supported this way, please do sage -sh and pip install packagename."

I agree that if python is the only one that is an actual disaster (TOPCOM vs. topcom seems very unlikely to cause a Sage installation to self-destruct to me) then that should be solved separately from this issue.

There is also the question of how long/much to support earlier versions of Sage that might want some of these packages.  I'm not sure what the answer to that is - and Volker, don't just say they should upgrade their system if it won't build from scratch, because (as you very well know) some people don't have admin authority or access to a newer system (e.g. if it's hardware provided by their employer).  XP is still out there in the wild so many places... and maybe the virtual machine image even works there :-)

- kcrisman 

[kcrisman]

   To the discussion at hand, I think the current system of not allowing old spkgs to be installed is a waste. What are the current old style (non-experimental) spkgs, which of them have you tried, and which of them don't work on what system. I'm willing to put in some time and effort to convert some of them to new style (specifically chomp and Simon's cohomology) to try and get this resolved. However, if we are going to go with the current route, we should have a major version number bump for the next stable release.

Hmm, that's a good point. 

[Jeroen Demeyer]

On 2015-09-09 16:42, Travis Scrimshaw wrote:
> A possible (hack) fix is to put something in the skpg script to
> explicitly ban it from installing python without specifying 2 or 3.

Assuming #19119, it's very easy to this. Just two short lines need to be
added to build/make/deps (one for sage -i python and one for sage -f
python).

[John H Palmieri]

I think that we should try to support old-style packages. Given that many of them are broken, and given that there are only 30 optional old-style packages (at least on http://files.sagemath.org/spkg/optional/), we can just look at them and white-list the ones that work and/or are not ancient. I just tried on OS X, and the following installed without apparent problems:

(listed with most recently modified first)

ore_algebra (modified less than a year ago)
chomp
biopython
cluster_seed
coxeter
lie
phc
pybtex
nzmath
qhull (last modified five years ago)

I think that we should white-list at least some of these, and also try to convert some of them (all of them?) to new-style packages. Same with Simon's p_group_cohomology package, which didn't build correctly for me. I don't have objections to leaving things as they are (i.e., completely opaque, maybe also stop them from being listed in "sage --optional") for the broken old-style spkgs (many of the others, although I didn't test all of the ones that were last modified more than 5 years ago).

By white-list, I mean that in sage-spkg, deal with these cases explicitly. Maybe that means deleting the broken ones from the server and rolling back #19004, I don't know.

  John

[Simon King]

Hi Jeroen,

On 2015-09-09, Jeroen Demeyer <jdem...@cage.ugent.be> wrote:
> The question is: to what extent should we continue supporting old-style
> packages?
>
> (A) sage -i OLDSTYLEPKGNAME should just install the package, there is no
> difference with new-style packages from the user's point of view.
> (B) sage -i OLDSTYLEPKGNAME should still work but with a clear
> deprecation warning.
> (C) sage -i OLDSTYLEPKGNAME should still work but only after a "are you
> sure?" confirmation, like we have for experimental packages.
> (D) sage -i OLDSTYLEPKGNAME should give an error.
>
> In Sage 6.8 we have (A) and in the latest betas, we have (D). I think
> that (D) is an over-reaction to the problem that some old-style packages
> are confusing or broken.
>
> My personal vote goes to (B) since it still allows non-interactive
> scripts to work like before but it shows a clear message that packages
> should migrate to new-style packages. This is also implemented in #19158.

+1.

(A) is fine, too (but I am biased since I am author of an old-style
spkg), (C) is fine as well (but, as you said, is a problem for scripts).

And (D) means breaking things on purpose without a long deprecation
period. That's not acceptable.

So, I prefer (B).

Best regards,
Simon

[Simon King]

On 2015-09-09, Nathann Cohen <nathan...@gmail.com> wrote:
> Another way to "fix" the problem:
> (E) Convert the remaining old-style packages to new-style packages.

Sure, but that means to teach noobs like me to use github in order
to artificially create an "upstream" source (basically the spkg *is*
uptream).

The new style is good for third party packages, but I believe bad for
original contributions.

Best regards,
Simon

[Volker Braun]

On Wednesday, September 9, 2015 at 4:43:47 PM UTC+2, kcrisman wrote:

TOPCOM vs. topcom seems very unlikely to cause a Sage installation to self-destruct to me

It doesn't cause data loss but it is confusing enough that even experienced developers can't tell which one to use (see sage-devel) 

[Simon King]

On 2015-09-09, Volker Braun <vbrau...@gmail.com> wrote:
> IMHO showing a warning that you are about to do something stupid but then
> proceeding to do so is a terrible user interface. Also, I don't like
> command line utilities that ask stupid questions (a.k.a. the windows UI
> style).

Are you really using the word "stupid"? You, who helped to impose a code of
conduct? Just saying...

For the record: A deprecation warning does *not* mean that the user is
about to do something stupid, but that (s)he is doing something that used
to be supported but soon will not be.

> If you really want to use old-style packages then thats fine, just specify
> the full url to the spkg. The only old-style spkg on the Sage mirrors that
> is actively developed is Simon's, and he is most likely better served by
> providing the spkg at some place that is under his direct control and
> telling his users the full url.

Indeed. I plan to try and learn how the new system works. But before I
relocate my spkg, I will first relocate myself (starting a new job in
October).

Best regards,
Simon

[Simon King]

On 2015-09-09, Nathann Cohen <nathan...@gmail.com> wrote:

> I suspect that it is not so orthogonal, and that it is the main source
> of disagreement between Volker and you. He seems to say that "all
> packages which work are now new-style", so that there is no "useful"
> old-style package left.
>
> He thus considers them not as old-style but as "broken", which solves
> the problem (for him).

... and entitles him to label the usage of an old-style package (or not
just the usage but even the users and the developer?) as stupid.

[Simon King]

Hi John,

On 2015-09-09, John Cremona <john.c...@gmail.com> wrote:
> Not quite -- I was proposing to display the message and *not* install the
> package -- since I thought that essentially none of the old .spkg will now
> install anyway?

That old .spkg do not install is not just a fact, but in the first place it
is a *decision* that created a fact.

Best regards,
Simon

[Simon King]

On 2015-09-09, Jeroen Demeyer <jdem...@cage.ugent.be> wrote:

Concerning my old-style spkg: I wouldn't mind if it was removed from the
server---it currently doesn't install anyway, because of backward
incompatible changes that have happened in SageMath half a year ago.

However, I believe that "sage -i <full url of an old style spkg>" should
still be supported.

Best regards,
Simon

[Simon King]

Hi Nathann,

On 2015-09-09, Nathann Cohen <nathan...@gmail.com> wrote:

Unmaintained packages will bitrot and break, sooner or *later*.

However, if I understand correctly, what is currently happening is
more than bitrotting: SageMath *actively* prevents packages from working
that used to work.

Best regards,
Simon

[Volker Braun]

On Wednesday, September 9, 2015 at 5:30:54 PM UTC+2, Simon King wrote:

For the record: A deprecation warning does *not* mean that the user is
about to do something stupid, but that (s)he is doing something that used
to be supported but soon will not be.

Exactly, which is why a deprecation warning is not appropriate here. The intent is NOT to tell the user "use this new command to install an outdated package". We just want the user to not install the outdated package.

[Volker Braun]

On Wednesday, September 9, 2015 at 5:45:14 PM UTC+2, Simon King wrote:

However, I believe that "sage -i <full url of an old style spkg>" should
still be supported. 

Agree, and this is not what this thread is about. 

[Simon King]

Hi Travis,

On 2015-09-09, Travis Scrimshaw <tsc...@ucdavis.edu> wrote:
> To the discussion at hand, I think the current system of not allowing
> old spkgs to be installed is a waste. What are the current old style
> (non-experimental) spkgs, which of them have you tried, and which of them
> don't work on what system. I'm willing to put in some time and effort to
> convert some of them to new style (specifically chomp and Simon's
> cohomology) to try and get this resolved.

Thank you, your help is very welcome! Due to incompatible changes in
SageMath, p_group_cohomology needs an upgrade anyway. It is at #18514.

Best regards,
Simon

[Simon King]

On 2015-09-09, Volker Braun <vbrau...@gmail.com> wrote:

> I disagree with that rationale. We want to maximize value for the
> scientific community. If that means inconveniencing a single user in order
> to make Sage accessible to 100 other users then we should do that.

Are you saying that the fact that something *works* (namely the
installation of some old-style packages) is *decreasing* the value of
SageMath?

[Simon King]

On 2015-09-09, Volker Braun <vbrau...@gmail.com> wrote:

> On Wednesday, September 9, 2015 at 5:30:54 PM UTC+2, Simon King wrote:
>>
>> For the record: A deprecation warning does *not* mean that the user is
>> about to do something stupid, but that (s)he is doing something that used
>> to be supported but soon will not be.
>>
>
> Exactly, which is why a deprecation warning is not appropriate here. The
> intent is NOT to tell the user "use this new command to install an outdated
> package".

The version of my cohomology spkg on the sage server (2.1.4) is broken because
of backward incompatible changes in SageMath that were labelled as
"internal changes". So, no deprecation needed.

However, if I understand correctly, there are old style spkgs that *would*
install correctly (version 2.1.5 of my spkg does work). In these cases, I
believe the right thing to do is: Give a warning that, although the spkg
works now, SageMath will stop to support it in about one year.

> We just want the user to not install the outdated package.

Exactly. It is a *conscious* decision to make something fail that used to
work. And it is also a conscious decision to go around and say that
using old-style packages is stupid (well, it used to be not stupid for
years...) and that preventing an optional package from working has a
benefit for the average user.

Best regards,
Simon

[Simon King]

Hi Volker,

On 2015-09-09, Volker Braun <vbrau...@gmail.com> wrote:

Is it not? OK, then I indeed misunderstood.

So, what is the thread about? Removing all official optional old-style
packages from the server?

There are several less drastic ways to proceed:
1) Have a deprecation period. I.e., the old-style optional spkgs remain
on the server, a warning is printed, SageMath tries to install it
(which may fail if the package did not cope with recent internal
changes in SageMath), and after one year the spkg is removed from
the server (unless the maintainer transforms it to new-style).
2) Demote all optional old-style packages to experimental packages. If I
understand correctly, installing an experimental package requests
conformation of the user. Which is fine, unless we have a script.
3) Raise an error that points the user to an url (or to the address of
the maintainer) from which the old-style package can be obtained.

IMHO, each of these solutions is better than bluntly saying that the
spkg doesn't install or doesn't exist.

Cheers,
Simon

[Dima Pasechnik]

On Wednesday, 9 September 2015 03:01:53 UTC-7, Volker Braun wrote:

On Wednesday, September 9, 2015 at 11:56:28 AM UTC+2, Jeroen Demeyer wrote:

But even if there is just a single old-style package
which works and is useful

I disagree with that rationale. We want to maximize value for the scientific community. If that means inconveniencing a single user in order to make Sage accessible to 100 other users then we should do that. 

if there is just one old-style package left, then the effort to convert it into new style is small, compared to the effort of
keeping the whole support for them...

[Nathann Cohen]

> if there is just one old-style package left, then the effort to convert it
> into new style is small, compared to the effort of
> keeping the whole support for them...

+1

Let us remove those that are not supposed to work (like the old
version of Simon's) and see what remains. Perhaps there will not
remain enough to debate over.

Nathann

[William Stein]

On Wed, Sep 9, 2015 at 8:22 AM, Simon King <simon...@uni-jena.de> wrote:
> On 2015-09-09, Nathann Cohen <nathan...@gmail.com> wrote:
>> Another way to "fix" the problem:
>> (E) Convert the remaining old-style packages to new-style packages.
>

I personally really like this suggestion.

> Sure, but that means to teach noobs like me to use github in order
> to artificially create an "upstream" source (basically the spkg *is*
> uptream).
>
> The new style is good for third party packages, but I believe bad for
> original contributions.
>
> Best regards,
> Simon
>

> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.



--
William (http://wstein.org)

[Jeroen Demeyer]

On 2015-09-09 18:36, Dima Pasechnik wrote:
> compared to the effort of
> keeping the whole support for them...

There is no "effort" needed, we just need to undo the intentional
breaking of the support.

[Jeroen Demeyer]

On 2015-09-09 18:24, Simon King wrote:
> Hi Volker,
>
> On 2015-09-09, Volker Braun <vbrau...@gmail.com> wrote:
>> On Wednesday, September 9, 2015 at 5:45:14 PM UTC+2, Simon King wrote:
>>>
>>> However, I believe that "sage -i <full url of an old style spkg>" should
>>> still be supported.
>>
>>
>> Agree, and this is not what this thread is about.
>
> Is it not? OK, then I indeed misunderstood.

This thread is about "sage -i <bare package name>"

So, to use your package as an example, about "sage -i p_group_cohomology".

The command "sage -i <full URL>" still works.

[Jeroen Demeyer]

On 2015-09-09 17:06, John H Palmieri wrote:
> I think that we should white-list at least some of these, and also try
> to convert some of them (all of them?) to new-style packages. Same with
> Simon's p_group_cohomology package, which didn't build correctly for me.
> I don't have objections to leaving things as they are (i.e., completely
> opaque, maybe also stop them from being listed in "sage --optional") for
> the broken old-style spkgs (many of the others, although I didn't test
> all of the ones that were last modified more than 5 years ago).

OK, I am adding this as a variation on (D):

(D''): sage -i OLDSTYLEPKGNAME should give an error, except if
OLDSTYLEPKGNAME is in a pre-determined white-list of packages.

But the hard question remains: which packages should be white-listed?

[Jeroen Demeyer]

On 2015-09-09 19:03, William Stein wrote:
> On Wed, Sep 9, 2015 at 8:22 AM, Simon King <simon...@uni-jena.de> wrote:
>> On 2015-09-09, Nathann Cohen <nathan...@gmail.com> wrote:
>>> Another way to "fix" the problem:
>>> (E) Convert the remaining old-style packages to new-style packages.
>>
>
> I personally really like this suggestion.

That's of course a noble goal, but it doesn't answer the question about
what to do on the short term (unless you volunteer to port all packages
before the next Sage release).

[Nathann Cohen]

> That's of course a noble goal, but it doesn't answer the question about what
> to do on the short term (unless you volunteer to port all packages before
> the next Sage release).

Is there a list of those old spkg somewhere? We could see what there
is to do by putting it on a wiki page and crowdsourcing the filing:
which ones do we port, and which ones do we throw away?

Nathann

[Jeroen Demeyer]

On 2015-09-09 19:14, Nathann Cohen wrote:
> Is there a list of those old spkg somewhere?

http://files.sagemath.org/spkg/index.html

> We could see what there
> is to do by putting it on a wiki page

Like this one:
http://wiki.sagemath.org/Classify%20old-style%20packages

[John H Palmieri]

I posted a list of some which worked for me. The full list of old-style optional packages is here: http://files.sagemath.org/spkg/optional/.

One thing that would help is, if we restore the on-line search of old-style packages, to not search "archived" packages. This would prevent the old python package from being installed, I believe.

--
John

[John H Palmieri]

From the list I posted, I would suggest

  ore_algebra  (pretty small, should be easy to convert, I can work on that)
  chomp

  Also p_group_cohomology, once it's been fixed.

Does anyone want to advocate for any of these?

  biopython
  cluster_seed
  coxeter
  lie
  phc

  nzmath
  qhull

I've never used them and I have no idea if they are still useful.

--
John

[Dima Pasechnik]

On Wednesday, 9 September 2015 10:25:53 UTC-7, John H Palmieri wrote:

On Wednesday, September 9, 2015 at 10:09:21 AM UTC-7, Jeroen Demeyer wrote:

On 2015-09-09 17:06, John H Palmieri wrote:
> I think that we should white-list at least some of these, and also try
> to convert some of them (all of them?) to new-style packages. Same with
> Simon's p_group_cohomology package, which didn't build correctly for me.
> I don't have objections to leaving things as they are (i.e., completely
> opaque, maybe also stop them from being listed in "sage --optional") for
> the broken old-style spkgs (many of the others, although I didn't test
> all of the ones that were last modified more than 5 years ago).

OK, I am adding this as a variation on (D):

(D''): sage -i OLDSTYLEPKGNAME should give an error, except if
OLDSTYLEPKGNAME is in a pre-determined white-list of packages.

But the hard question remains: which packages should be white-listed?

From the list I posted, I would suggest

  ore_algebra  (pretty small, should be easy to convert, I can work on that)
  chomp

  Also p_group_cohomology, once it's been fixed.

Does anyone want to advocate for any of these?

instead of reading and posting here, I just spent time converting  jones_numfield to new style. (#19174)
If every poster here does spend half a day or a day on this, the issue will be moot... :-)

[Travis Scrimshaw]

Thank you Jeroen and John for posting those links.

coxeter3 and lie are useful to me and are things I'd want to keep and I will convert over. I will also be handling cluster_seed.

Best,
Travis

[John H Palmieri]

See http://trac.sagemath.org/ticket/19175 for ore_algebra.

[Simon King]

On 2015-09-09, John H Palmieri <jhpalm...@gmail.com> wrote:
> See http://trac.sagemath.org/ticket/19175 for ore_algebra.

See http://trac.sagemath.org/ticket/18514 for p_group_cohomology.

[Dima Pasechnik]

the links to the spkg no longer work there, please update

[Volker Braun]

On Wednesday, September 9, 2015 at 6:00:22 PM UTC+2, Simon King wrote:

Are you saying that the fact that something *works* 

If FEATURE gives you 1 case that works and 99 that fail then it is not a good feature. Just working rarely is not good enough. 

[Simon King]

On 2015-09-09, Dima Pasechnik <dim...@gmail.com> wrote:
>> See http://trac.sagemath.org/ticket/18514 for p_group_cohomology.
>>
>
> the links to the spkg no longer work there, please update

That has been discussed here already. Since sage.math.washington doesn't
allow access, I need to find a different place. The spkg is too large
for an attachment to trac, but it seems that the university of Jena is
willing to host it, although I'm leaving Jena by end of this month.

Cheers,
Simon

[Jeroen Demeyer]

For adding a new feature, what you say makes sense.

But this argument completely fails when backwards compatibility is
involved. Actively removing a feature (even if broken 99% of the time)
without deprecation is a very exceptional move which should require
exceptional justification.

[Simon King]

On 2015-09-09, Simon King <simon...@uni-jena.de> wrote:
> That has been discussed here already. Since sage.math.washington doesn't
> allow access, I need to find a different place. The spkg is too large
> for an attachment to trac, but it seems that the university of Jena is
> willing to host it, although I'm leaving Jena by end of this month.

It turned out that the quota given to the presentation of our
computational results was sufficient to additionally host the spkg.

So, it is at
http://users.minet.uni-jena.de/cohomology/p_group_cohomology-2.1.5.spkg

Cheers,
Simon

[Thierry]

Hi,

sorry for not participating actively to this long thread, i am currently
pretty far from computers. Let me just mention that this discussion
already appears some time ago, and that somehow, the mailing-list is
probably not the appropriate tool for solving such an issue, while the
wiki is definitely more appropriate for classifying things, see how big is
the work to be accomplished, and eventually getting things done. During a
previous similar thread, i opened the following page, which might a good
tool if we keep it up-to-date:

http://wiki.sagemath.org/Classify%20old-style%20packages

The benefits of such list is that i started with the complete list of
(both old-style or new-style) packages, so it is unlikely that we will
miss something.

As you may see, the list of uncassified packages is not very big, nor is
the list of packages that need to be repackaged. At least, they are much
less impressive than looking to the raw list of old-style spkgs and
discussing on an abstract basis.

Ciao,
Thierry

[kcrisman]

instead of reading and posting here, I just spent time converting  jones_numfield to new style. (#19174)
If every poster here does spend half a day or a day on this, the issue will be moot... :-)

:)

Is biopython pip-installable? 

[Jeroen Demeyer]

On 2015-09-09 22:17, Thierry wrote:
> sorry for not participating actively to this long thread, i am currently
> pretty far from computers. Let me just mention that this discussion
> already appears some time ago, and that somehow, the mailing-list is
> probably not the appropriate tool for solving such an issue, while the
> wiki is definitely more appropriate for classifying things, see how big is
> the work to be accomplished, and eventually getting things done. During a
> previous similar thread, i opened the following page, which might a good
> tool if we keep it up-to-date

I never meant this thread to be about
classifying/upgrading/porting/fixing old-style packages, but somehow
many people started thinking in that direction.

This thread was meant to answer the very simple question: what should
happen if a user does

sage -i OLDSTYLEPACKAGE

As long as there is still at least one old-style package left, this
question makes sense and should be answered on the short term before the
next Sage release.

I think that deciding what to do with concrete old-style packages is a
longer-term issue.

[Volker Braun]

On Wednesday, September 9, 2015 at 9:39:49 PM UTC+2, Jeroen Demeyer wrote:

But this argument completely fails when backwards compatibility is
involved. Actively removing a feature (even if broken 99% of the time)
without deprecation is a very exceptional move which should require
exceptional justification.

You don't need to deprecate bugs; The sooner you cut out an infected appendix the better.

[Volker Braun]

On Wednesday, September 9, 2015 at 10:20:26 PM UTC+2, kcrisman wrote:

Is biopython pip-installable? 

Yes, and it gives you a newer version than our old spkg as well.

[Thierry]

On Wed, Sep 09, 2015 at 01:20:26PM -0700, kcrisman wrote:

> Is biopython pip-installable?

Yes. If you want to test without changing your Sage install, just install
virtualenv and do:

virtualenv venv
. venv/bin/activate
pip install biopython
# ipython
# whathever test
deactivate
rm -rf venv

If it works, then there should not be any problem within Sage. If it does
not work because of some dependencies (e.g. Cython, gmp,...), then you
should try within Sage (`sage -pip install blah`) to see if everything is
there already.

Ciao,
Thierry

[Simon King]

On 2015-09-09, Volker Braun <vbrau...@gmail.com> wrote:

If a package used to work with the previous Sage release, then making it
fail on purpose without deprecation period *IS* a bug.

[Travis Scrimshaw]

LiE is ready at http://trac.sagemath.org/ticket/19176.

cluster_seed is next on http://trac.sagemath.org/ticket/19177.

coxeter3 is after that on http://trac.sagemath.org/ticket/19178.

Best,
Travis

[John H Palmieri]

On Wednesday, September 9, 2015 at 1:24:09 PM UTC-7, Jeroen Demeyer wrote:

On 2015-09-09 22:17, Thierry wrote:
> sorry for not participating actively to this long thread, i am currently
> pretty far from computers. Let me just mention that this discussion
> already appears some time ago, and that somehow, the mailing-list is
> probably not the appropriate tool for solving such an issue, while the
> wiki is definitely more appropriate for classifying things, see how big is
> the work to be accomplished, and eventually getting things done. During a
> previous similar thread, i opened the following page, which might a good
> tool if we keep it up-to-date

I never meant this thread to be about
classifying/upgrading/porting/fixing old-style packages, but somehow
many people started thinking in that direction.

Although this new effort is an improvement for Sage, and if enough old-style packages get upgraded, we've solved the problem.
 

This thread was meant to answer the very simple question: what should
happen if a user does

sage -i OLDSTYLEPACKAGE

Among other things, don't search archived packages. I'm not sure why that was ever a good idea.

In principle, removing the ability to easily install old-style packages is bad. If we upgrade enough of those packages, we essentially fix this problem. That is, there is a balance between philosophy and practicality.
 

As long as there is still at least one old-style package left, this
question makes sense and should be answered on the short term before the
next Sage release.

As long as there is one non-broken old-style package, yes. Or as long as there is one non-broken old-style package that anyone cares about, but I'm not sure how to measure that.

I've lost track of what the options are, but I would be happy if "sage -i PACKAGE" said something like

"PACKAGE is no longer a maintained Sage package. If you want to install it, type
    sage -i full/url/to/package"

[Jeroen Demeyer]

Let's try to summarize what we have so far.

I am going to ignore responses of the form "just port all/some packages
to new-style". That's certainly a good thing to do, but it doesn't
answer the question. It's also a more long-term solution, while we need
something by the next stable Sage release.

So far, nobody was in favour of (A) and (C), so that gives us 2
remaining options:

(B) sage -i OLDSTYLEPKGNAME should still work but with a clear warning
message.
(D) sage -i OLDSTYLEPKGNAME should give an error (possibly a more
helpful error message than what we currently have).

Votes so far are:
(B): Jeroen Demeyer, Simon King
(D): Volker Braun, John Cremona, John Palmieri

Anybody else wants to give his/her opinion?


Jeroen.

[John Cremona]

On 10 September 2015 at 11:36, Jeroen Demeyer <jdem...@cage.ugent.be> wrote:

Let's try to summarize what we have so far.

 

Good idea.

 

I am going to ignore responses of the form "just port all/some packages to new-style". That's certainly a good thing to do, but it doesn't answer the question. It's also a more long-term solution, while we need something by the next stable Sage release.

So far, nobody was in favour of (A) and (C), so that gives us 2 remaining options:

(B) sage -i OLDSTYLEPKGNAME should still work but with a clear warning message.
(D) sage -i OLDSTYLEPKGNAME should give an error (possibly a more helpful error message than what we currently have).

Votes so far are:
(B): Jeroen Demeyer, Simon King
(D): Volker Braun, John Cremona, John Palmieri

Am I right that Simon's package is one (perhaps the only one) for which (B) would be useful?  If that is true then I don't wish to make things hard for Simon.  But then I don't know what the obstacles are to converting his spkg.

John

 

Anybody else wants to give his/her opinion?


Jeroen.

[Simon King]

On 2015-09-10, John Cremona <john.c...@gmail.com> wrote:
> Am I right that Simon's package is one (perhaps the only one) for which (B)
> would be useful? If that is true then I don't wish to make things hard for
> Simon. But then I don't know what the obstacles are to converting his spkg.

Partially my ignorance.

Partially the fact that the code is divided into three parts. The first
has a proper upstream (MeatAxe), but we use a very old version (that said,
even the latest version is several years old, and does not appear to be
faster, but has almost all function names changed). The other two parts are
not published elsewhere, and somehow it seems odd to me to artificially
create an upstream repository, as the spkg *IS* the upstream repository
(but perhaps I am the only one who finds it odd...).

Anyway, it currently looks like the spkg will be converted to a
new-style package.

Something else: Of course each package has a maintainer resp. an
upstream contact. Can this information be automatically extracted? Then,
if an error occurs during installation, for whatever reason, the error
message could clearly state: "Please contact f...@bar.com about the
problem." If I recall correctly, the current error messages only
mention sage-devel.

Best regards,
Simon

[Nathann Cohen]

Hello,

> Something else: Of course each package has a maintainer resp. an
> upstream contact. Can this information be automatically extracted?

Err. Well, "not always". Yesterday Dima converted a .spkg to
new-style, but that package did not include a SPKG.txt file (so no
official maintainer either). And the 'graphs' package contains my
name, though without email. This, to say that we probably can't assume
that the current data exists.

> Then,
> if an error occurs during installation, for whatever reason, the error
> message could clearly state: "Please contact f...@bar.com about the
> problem." If I recall correctly, the current error messages only
> mention sage-devel.

I do not have your faith that *all* of the guys who appear as
maintainers of a package would, when receiving such a bug report,
immediately fix it or make sure that it will not be forgotten. Asking
people to write to sage-devel sounds safer to me.

Nathann

[John Cremona]

On 10 September 2015 at 14:02, Simon King <simon...@uni-jena.de> wrote:

On 2015-09-10, John Cremona <john.c...@gmail.com> wrote:
> Am I right that Simon's package is one (perhaps the only one) for which (B)
> would be useful?  If that is true then I don't wish to make things hard for
> Simon.  But then I don't know what the obstacles are to converting his spkg.

Partially my ignorance.

Partially the fact that the code is divided into three parts. The first
has a proper upstream (MeatAxe), but we use a very old version (that said,
even the latest version is several years old, and does not appear to be
faster, but has almost all function names changed). The other two parts are
not published elsewhere, and somehow it seems odd to me to artificially
create an upstream repository, as the spkg *IS* the upstream repository
(but perhaps I am the only one who finds it odd...).

Anyway, it currently looks like the spkg will be converted to a
new-style package.

Has anyone started?  AFter reading the above I just downloaded MeatAxe 2.4.24 (2011) and very easily built it (make check OK);  all one has to do is copy Makefile to Makefile.conf and edit 2 lines, as I am sure you know.

One way to proceed would be to make a new-style package out of the MeatAxe source first.  The your 2nd and 3rd parts would be separate, but with MeatAxe as a dendency.  The first part would take about 20 minutes to do....

 

John

Something else: Of course each package has a maintainer resp. an
upstream contact. Can this information be automatically extracted? Then,
if an error occurs during installation, for whatever reason, the error
message could clearly state: "Please contact f...@bar.com about the
problem." If I recall correctly, the current error messages only
mention sage-devel.

Best regards,
Simon

[Jeroen Demeyer]

On 2015-09-10 15:14, Nathann Cohen wrote:
> I do not have your faith that *all* of the guys who appear as
> maintainers of a package would, when receiving such a bug report,
> immediately fix it or make sure that it will not be forgotten. Asking
> people to write to sage-devel sounds safer to me.

+1

Besides, the "maintainers" section is usually very outdated.

[John H Palmieri]

To clarify, I would really want option (D) if the error message were much more helpful, including the URL for a direct link to the package. If I had to rank my choices:
(D) with helpful error message
(B)
(D) as it currently is

--
John

[Simon King]

Hi John,

On 2015-09-10, John Cremona <john.c...@gmail.com> wrote:
> Has anyone started? AFter reading the above I just downloaded MeatAxe
> 2.4.24 (2011) and very easily built it (make check OK); all one has to do
> is copy Makefile to Makefile.conf and edit 2 lines, as I am sure you know.

... and then edit literally 1000s of lines in David Green's code and in my
code. And rewrite the patches that are to be applied to MeatAxe.

The problem doesn't lie in building MeatAxe. MeatAxe is a collection of
executables, but what we need in the applications is a library of C-functions
that my Cython code uses.

So, part of the patches turns MeatAxe into a library. And the patches also add
Strassen multiplication.

But the interface of MeatAxe has thoroughly changed: Really, I don't
think that there is any function or any type whose name has not changed.
That's why it would involve a lot of work to use the most recent MeatAxe.

> One way to proceed would be to make a new-style package out of the MeatAxe
> source first. The your 2nd and 3rd parts would be separate, but with
> MeatAxe as a dendency. The first part would take about 20 minutes to do....

No, since the patches need to be rewritten. And the 2nd and 3rd parts would
also take rather long.

That's why I suggest the following approach:
1. Turn the spkg into a new-style package with *least* effort. I.e.,
put the existing sources into some repository. In that way, the spkg
can be used *NOW*.
2. Create a separate libmeataxe spkg (new-style of course), with patches
rebased on the most recent upstream sources, and also providing the
Cython wrapper that is currently burried in the cohomology spkg.
3. Adapt David Green's C programs and my Cython programs to the new
version of MeatAxe. In the best case, it merely amounts in hundreds
of search-and-replace actions.

If I recall correctly, a couple of years ago I did some steps towards
2., so, in the best case, I can come up with libmeataxe-1.0 in a week or
so.

Best regards,
Simon

[Travis Scrimshaw]

I want option (B) and I do not want option (D) (especially since I recently tried to install the LiE spkg and was left wondering what happened, which I attributed to my internet connection).

Best,
Travis

[Nathann Cohen]

Hello everybody,

I just created a new thread with a script.

What this script does is create a working new-style package from an old .spkg.

It is a possible way out: with this script, we can turn all old .spkg
into new-style ones, and have it settled in one big commit. Then, we
will have all the time we need to remove the spkg we do not want to
keep, those we need to repackage properly, etc.

https://groups.google.com/d/topic/sage-devel/MrgGDIckT5c/discussion

Nathann

> --
> You received this message because you are subscribed to a topic in the
> Google Groups "sage-devel" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/sage-devel/_5SidQWFYDY/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to

[John H Palmieri]

I could also be easily persuaded to (B) if we don't allow "sage -i python" to install the old python-2.5 package. (There is also the TOPCOM vs. topcom issue. Maybe delete TOPCOM from the file server? And as I've said before, don't search archived packages for possible installation. The website lists these as "old/broken/deprecated", so they should be hard to install.)

--
John

[Jeroen Demeyer]

On 2015-09-10 20:30, John H Palmieri wrote:
> I could also be easily persuaded to (B) if we don't allow "sage -i
> python" to install the old python-2.5 package.

Are you willing to review #19119? It's a dependency for #19105 which
would do exactly that.

[Sébastien Labbé]

OK, let me vote too.

Votes so far are:
(B): Jeroen Demeyer, Simon King, Sébastien Labbé

(D): Volker Braun, John Cremona, John Palmieri

Anybody else wants to give his/her opinion?

There are packages out there that are not listed on official sagemath.org/packages_lists :

http://www.slabbe.org/Sage/slabbe-0.1.1.spkg
http://www.liafa.univ-paris-diderot.fr/~paperman/cpaut.spkg

To my knowledge, they are not broken on sage-6.8.

Maybe others?

Personnally, I prefer the old style because I do not need to change Sage (update checksums) for somebody (like myself!) to install a modified version of my package while I develop it. I do read sage-devel. And I am willing to change to the new style if you force me to (or maybe try pip if I can install an updated version of an optional package without changing sage). But I would also prefer a transition time with a deprecation warning allowing people that you will never see on that mailing list to adapt.

Charles Paperman wrote his package [1] on semigroup and automata stuff. Charles never read sage-devel. Charles never contributed to sage. But he is able to write code and managed to create that spkg last spring which allows him to easily share his code to his colleagues.

[1] http://www.liafa.univ-paris-diderot.fr/~paperman/index.php?page=sage

I still feel that the new style is not made for small scale hidden optional packages that people may want to do. Why should sage know about the checksum every optional packages that users may want to create?

Sébastien

[Jeroen Demeyer]

On 2015-09-11 12:37, Sébastien Labbé wrote:
> There are packages out there that are not listed on official
> sagemath.org/packages_lists

True, but that's not what this thread is about. Nothing will change to
externally-hosted old-style packages.

[Sébastien Labbé]

This thread is about "should we still support old-style packages" and my answer *is* about that thread.

Sébastien

[Nathann Cohen]

> This thread is about "should we still support old-style packages" and my
> answer *is* about that thread.

"Should we still support old-style packages with 'sage -i <pkg_name>'
? (no changes were made to 'sage -i <url>') "

Nathann

[Jeroen Demeyer]

On 2015-09-11 14:07, Sébastien Labbé wrote:
>
>
> On Friday, September 11, 2015 at 1:10:47 PM UTC+2, Jeroen Demeyer wrote:
>
> On 2015-09-11 12:37, Sébastien Labbé wrote:
> > There are packages out there that are not listed on official

> > sagemath.org/packages_lists <http://sagemath.org/packages_lists>

> True, but that's not what this thread is about. Nothing will change to
> externally-hosted old-style packages.
>
>
> This thread is about "should we still support old-style packages" and my
> answer *is* about that thread.

To be more precise, this thread is about

sage -i <package name>

not about

sage -i <full URL to package>

nor

sage -i <local file.spkg>

[Volker Braun]

Jeroen gave the thread a misleading name (always pose loaded questions). The real subject is: Should the old-style packages be searched for. Only applies to old-style packages that are hosted on files.sagemath.org.

[Jeroen Demeyer]

On 2015-09-11 14:16, Volker Braun wrote:
> always pose loaded questions.
I hope that's not an accusation.

[Volker Braun]

No its a compliment --- I would do the same thing ;-)

[Sébastien Labbé]

Ok, I see. Thanks for clarifying. It was not clear from the initial question.

Then, I am fine with (B) or (D) for the next official Sage release and for (D) thereafter.

Sébastien

[kcrisman]

Did this one come up?  Social media user asks:

Is it possible to get the full Jones number field database on Sage Math? typing JonesDatabase() only gives a small slice of it.

Looks like it is converted to new-style in the latest beta, but may not be installable in 6.8?  Along these lines see the weird way the first sentence in http://doc.sagemath.org/html/en/reference/databases/sage/databases/jones.html looks. 

[John H Palmieri]

I think it's not that weird, actually, although it's not the advice I would give: while running Sage, "!cmd" will run the shell command "cmd". So "!sage -i PKG" could in principle install PKG, while running Sage.

By the way, I think the old database is broken: I think it tries to install into some directory which now no longer exists. So I think the one in the newest beta is the only option.

--
John

[Thierry]

Hi,

According to the following thread, it seems that there was a consensus to
remove the "maintainers" section (which was done in quite a few packages
already):

https://groups.google.com/forum/#!searchin/sage-devel/maintainer/sage-devel/-MZAvj8qxVk/EdbRFw0hGucJ
http://thread.gmane.org/gmane.comp.mathematics.sage.devel/75324/

Ciao,
Thierry

[Simon King]

Hi John,

On 2015-09-12, John H Palmieri <jhpalm...@gmail.com> wrote:
> By the way, I think the old database is broken: I think it tries to install
> into some directory which now no longer exists. So I think the one in the
> newest beta is the only option.

In the same way (changed directories, changed import locations) version 2.1.3
and 2.1.4 of my group cohomology spkg got broken. I was told that changing
these directories was an *internal* change and thus did not require a
deprecation period. By other internal changes (Cython upgrade, Singular
upgrade), versions 2.1.1 and 2.1.2 broke as well.

In other words: All versions from 1.0 till 2.1.1 were motivated by
improvements. All versions from 2.1.2 till now were at least partially
motivated by coping with backward incompatible changes.

For the record: I think ANY backward incompatible internal change that *can*
be covered by a deprecation warning (thus, changed import locations,
changed function names or argspecs, etc) *has* to be covered by a
deprecation period, in particular when it breaks some optional package.
Admittedly, internal changes in third party packages (cython, singular) do
not fall into that category.

I thought some patchbots did test optional packages on a more or less regular
basis?

Anyway. Isn't it very convenient that one can say: "The old-style optional
packages are now all broken (by internal changes that did not require
deprecation), thus, let's expunge them."

Cheers,
Simon

[Jeroen Demeyer]

On 2015-09-12 09:48, Simon King wrote:
> All versions from 2.1.2 till now were at least partially
> motivated by coping with backward incompatible changes.

I am probably responsible for several of these backward incompatible
changes. However, they were changes to the basic Sage infrastructure,
like moving of include directories, the removal of c_lib. Really, things
which cannot be deprecated. And these changes were good changes, not
just changes for the sake of changing stuff. For example, the inclusion
of sage_include_directories() in sage.env should make the include
directories future-proof.

And, as you mentioned, if an upstream package breaks things (for example
Cython no longer allowing foo(mutable=True, mutable=True)), then there
is not much that Sage can do.

Jeroen.

[Volker Braun]

On Saturday, September 12, 2015 at 9:48:38 AM UTC+2, Simon King wrote:

For the record: I think ANY backward incompatible internal change that *can*
be covered by a deprecation warning (thus, changed import locations,
changed function names or argspecs, etc) *has* to be covered by a
deprecation period

Thats madness, this means that we cannot change anything in Sage at all because surely somebody imported that private function somewhere. If its not globally exported (i.e. you can't import it from sage.all) then there is no guarantee of stability. 

I dare you find any project that makes stability guarantees for non-public APIs. I wonder why nobody else does it...

 

I thought some patchbots did test optional packages on a more or less regular
basis?

No, the build/patchbots only test standard packages. Of course we would be happy to have you volunteer as the person in change of testing all optional packages.

[kcrisman]

I thought some patchbots did test optional packages on a more or less regular
basis?

No, the build/patchbots only test standard packages. Of course we would be happy to have you volunteer as the person in change of testing all optional packages.

R has the resources to do this, and is very picky about their packages (even though they have thousands).  The new thing where all installed optional packages are tested is a good start, it's clearly already helped. 

[Nathann Cohen]

> The new thing where all installed optional
> packages are tested is a good start, it's clearly already helped.

Here is a (stalled) attempt at pushing it further. Needs volunteers.
http://trac.sagemath.org/ticket/18904

Nathann

[William Stein]

On Saturday, September 12, 2015, kcrisman <kcri...@gmail.com> wrote:

I thought some patchbots did test optional packages on a more or less regular
basis?

No, the build/patchbots only test standard packages. Of course we would be happy to have you volunteer as the person in change of testing all optional packages.

R has the resources to do this, and is very picky about 

R has vastly more resources than us to put mildly. 

I apologize - It was a mistake how we released sage and broke a large number of spkg's just because we didn't want to include a backwards compatible symlink (which I do with my sage install recipe still).   If we want to make changes that break a lot of stuff then we should also do the work to update all the packages.  

William  

 

their packages (even though they have thousands).  The new thing where all installed optional packages are tested is a good start, it's clearly already helped. 

--

You received this message because you are subscribed to the Google Groups "sage-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

To post to this group, send email to sage-...@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

--
Sent from my massive iPhone 6 plus.

[kcrisman]

R has the resources to do this, and is very picky about 

R has vastly more resources than us to put mildly. 

To put it mildly :)