How much do we support optional packages.

[Maarten Derickx]

Hi Dear fellow sage-developers,

The question in the subject is related to the discussion in the Patchbot failures metaticket thread. 

I remember that I read (somewhere a long time ago so my memory might be off) that optional packages where expected to always build and not cause any doctest failures and pas their own testsuite, because optional packages were considered to have the same amount of support as sage itself, the main reason that they are not installed by default is just because not everybody needs them and installing by default makes the sage install even bigger then it already is. However I seem to not be able to find this statement anymore in either one of the places [1][2] where I would expect it. The fact this level of support does not hold for experimental packages is made very clear at [3].

So the main questions: do we consider an optional package not building, not passing it's own testsuite or causing sage to have doctest failures a bug?

In the other thread it was mentioned that patchbot failures should be considered blocker status defects (at least if the failures is not due to a buggy patchbot/patchbot on an unsupported platform). Does this also hold for optional packages?

[1] http://files.sagemath.org/spkg/optional/

[2] https://wiki.sagemath.org/spkg

[3] http://files.sagemath.org/spkg/experimental/

[Maarten Derickx]

And related to this, how much do we support pip packages. With this I don't mean any random pip package that happens to be on pypi, but just the ones that are returned by:

sage -optional 

that happen to be pip packages. At the moment:

['beautifulsoup',

 'biopython',

 'brian',

 'guppy',

 'mercurial',

 'mpi4py',

 'nibabel',

 'pybtex',

 'pyflakes',

 'pyopenssl',

 'sqlalchemy',

 'trac']

[Vincent Delecroix]

Sided note: as far as I know, these pip packages are not involved in any
doctest.

[Jeroen Demeyer]

On 2017-09-13 15:28, Maarten Derickx wrote:
> So the main questions: do we consider an optional package not building,
> not passing it's own testsuite or causing sage to have doctest failures
> a bug?

Yes.

> In the other thread it was mentioned that patchbot failures should be
> considered blocker status defects (at least if the failures is not due
> to a buggy patchbot/patchbot on an unsupported platform). Does this also
> hold for optional packages?

I mentioned that, so I obviously think so.

[rjf]

Just because a package builds, loads, and passes some tests

doesn't mean that it also includes some security attack. 

Does anyone care about / have any useful thoughts about /.  that?  

Sage includes Maxima and Lisp, which generally provides

access to system routines, for example.

RJF

[Jeroen Demeyer]

On 2017-09-13 21:56, rjf wrote:
> Just because a package builds, loads, and passes some tests
> doesn't mean that it also includes some security attack.
> Does anyone care about / have any useful thoughts about /. that?

What would security even mean for a mathematics program? Sage is not
meant for security, so your question makes little sense.

[Thierry]

Hi,

On Wed, Sep 13, 2017 at 04:51:09PM +0200, Vincent Delecroix wrote:
> Sided note: as far as I know, these pip packages are not involved in any
> doctest.

Indeed. Most are leftovers from the old system. I am in favor to remove
them from the build/pkgs/ directory for better clarity.

Ciao,
Thierry

> On 13/09/2017 16:42, Maarten Derickx wrote:
> >And related to this, how much do we support pip packages. With this I don't
> >mean any random pip package that happens to be on pypi, but just the ones
> >that are returned by:
> >
> >sage -optional
> >
> >that happen to be pip packages. At the moment:
> >
> >
> >['beautifulsoup',
> >
> > 'biopython',
> >
> > 'brian',
> >
> > 'guppy',
> >
> > 'mercurial',
> >
> > 'mpi4py',
> >
> > 'nibabel',
> >
> > 'pybtex',
> >
> > 'pyflakes',
> >
> > 'pyopenssl',
> >
> > 'sqlalchemy',
> >
> > 'trac']
> >
>

> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.

[kcrisman]

Well, in principle someone could use a bug in an outside program to hack into Sage, and then use that to gain access (e.g. via Sage shell abilities or os.* in Python) to gain access to a system, right?  I agree that it's relatively unlikely compared to the likelihood of a Sage user clicking on a phishing link. 

[Jeroen Demeyer]

On 2017-09-14 15:21, kcrisman wrote:
> Well, in principle someone could use a bug in an outside program to hack
> into Sage

If that "outside program" is executing arbitrary strings in Sage, the
bug is with that outside program, not with Sage.

There is a reason why things like the Sage Cell server or CoCalc run in
a container.

[rjf]

If I were an administrator of a computer network at

an industrial, governmental, or educational operation,

I would not want people downloading risky software

to their workstations.  Theft of passwords, confidential

information, access to health, financial, accounts etc

is a growing concern.  It's not that anything has

necessarily changed technically, just there's more attention

to this security issue.

It has been policy for years now, that you cannot even

take your laptop computer into some (US) government

offices.

I don't know of any workable solutions (and only one

not-so-workable one involving a trusted virtual machine).  

RJF

[Vincent Delecroix]

On 13/09/2017 15:28, Maarten Derickx wrote:
> So the main questions: do we consider an optional package not building, not
> passing it's own testsuite or causing sage to have doctest failures a bug?

I do. And as well for me it is the frontier between optional and
experimental packages.

> In the other thread it was mentioned that patchbot failures should be
> considered blocker status defects (at least if the failures is not due to a
> buggy patchbot/patchbot on an unsupported platform). Does this also hold
> for optional packages?

It does for me.

Vincent

[Maarten Derickx]

Hi Jeroen and Vincent,

Thanks for your concrete answers to my questions.

If people want to keep discussing security issues of sage feel free to do so but please do it in a different thread.  I would like this thread to focus on my initial question and not get cluttered by related discussions.

[Jori Mäntysalo]

On Wed, 13 Sep 2017, Jeroen Demeyer wrote:

> What would security even mean for a mathematics program?

I think it could be possible to use Sage as a part of teaching program,
i.e. have it as a part of automatic checking of some exercises. If so, a
bug in Sage could -- at least in theory -- lead a compromise to the
server.

--
Jori Mäntysalo

[Jeroen Demeyer]

On 2017-09-15 09:00, Jori Mäntysalo wrote:
> If so, a
> bug in Sage could -- at least in theory -- lead a compromise to the server.

It's a *feature* of Sage (actually Python) that it can run arbitrary
code, not a bug.

[Vincent Delecroix]

In case you did not notice, you are just preventing an important
discussion concerning the management of optional packages to happen...

[Maarten Derickx]

In my ongoing quest of trying to get fewer patchbot failures I started at least with installing all optional packages. I now have every optional package returned by::

   sage: optional_packages()

except the following:

    gmp - this package installs fine but one has to choose between mpir and gmp

    igraph -this needs libxml2 could by solved by apt get libxml2-dev

    python_igraph - you wouldn't guess, but this depends on igraph

    libtheora - a known problem #23732

    mpfrcx - added so recently (#11806) that the tar is not on the mirror network yet

Sage builds fine after this but the documentation suddenly does not build anymore because it misses Graphviz. The total memory used by this install is 15 GB (7 GB more then the 8.3 GB it had before installing the optional packages). 

The doctest don't pass anymore, to be precise there are 48 failures, there were 3 that I saw before and I know the trac tickets of. The other 45 can be found at https://trac.sagemath.org/ticket/23871 if any of the files affected look familiar and you know an open ticket for it please let me know. I am now working on sorting out the 45 failures at #23871 .

[Maarten Derickx]

The packages needing packages external to sage to be installed raise the question wether optional packages are allowed to have dependencies other then sage. To me it feels a bit weird if you first need to install something outside of sage before you can install the package, however this seems like a minor issue related to the patchbot failures.

[Maarten Derickx]

Ok, most errors have been adressed. They turned out to be mostly due to some optional packages having dependencies outside of sage, but not complaining about these at installation time, but only during runtime.

[Vincent Delecroix]

Apparently Volker does not agree with what was a kind of agreement here

https://trac.sagemath.org/ticket/24903#comment:3
https://trac.sagemath.org/ticket/23533#comment:13

It would be nice that we take a concrete decision about how much we
support optional packages (and write it in the developer manual). So far
Maarten, Jeroen and I are in favor of as much support with optional as
standard. And Volker seems to be against. It would be nice to have more
opinions.

Vincent

[Simon King]

Hi Vincent,

On 2018-03-25, Vincent Delecroix <20100.d...@gmail.com> wrote:
> Apparently Volker does not agree with what was a kind of agreement here
>
> https://trac.sagemath.org/ticket/24903#comment:3
> https://trac.sagemath.org/ticket/23533#comment:13
>
> It would be nice that we take a concrete decision about how much we
> support optional packages (and write it in the developer manual). So far
> Maarten, Jeroen and I are in favor of as much support with optional as
> standard.

I think that the only difference between an optional and a standard package
should be (and I hope: currently *is*!) that an optional package is not
(yet) installed by default, for reasons that are independent of the
overall quality and portability of the package.

In other words: Installation of an optional package is supposed to work
and keep SageMath fully functional on all supported platforms. If it doesn't
then it is an experimental package.

Best regards,
Simon

[Dima Pasechnik]

IMHO in these two cases these tickets should not be blockers for just purely practical reasons:

one can install autotools on archlinux systemwide

one can use a better gcc version to build that other package.

[Volker Braun]

On Sunday, March 25, 2018 at 2:51:46 PM UTC+2, Dima Pasechnik wrote:

one can install autotools on archlinux systemwide

Also, autotools aren't even required to build Sage. 

Whats the point of delaying a release for weeks/months to fix an optional package? Presumably you agree that broken standard packages should have the highest priority, so broken optional packages necessarily have a lower priority. 

[Dr. David Kirkby]

On 25 March 2018 at 10:03, Vincent Delecroix <20100.d...@gmail.com> wrote:

Apparently Volker does not agree with what was a kind of agreement here

https://trac.sagemath.org/ticket/24903#comment:3
https://trac.sagemath.org/ticket/23533#comment:13

It would be nice that we take a concrete decision about how much we
support optional packages (and write it in the developer manual). So far
Maarten, Jeroen and I are in favor of as much support with optional as
standard. And Volker seems to be against. It would be nice to have more
opinions.

Vincent

As someone who did a lot of development on Sage years ago, my understanding was that optional packages were meant to work. In much the same way as you buy a car, and can pay extra for optional features, such as leather seats, heated seats, alloy wheels, metallic paint etc.

If something does not work reliably on all supported platforms, then it should be considered 'experimental' in my opinion. So in my opinion at least, if something does not work properly on all platforms, there are currently only two sensible options.

1) Fix the problem.

2) Downgrade the package to experimental.

Otherwise you have no distinction between optional and experimental.

Dave

[William Stein]

On Sun, Mar 25, 2018 at 6:53 AM, Dr. David Kirkby <drki...@gmail.com> wrote:
> On 25 March 2018 at 10:03, Vincent Delecroix <20100.d...@gmail.com>
> wrote:
>>
>> Apparently Volker does not agree with what was a kind of agreement here
>>
>> https://trac.sagemath.org/ticket/24903#comment:3
>> https://trac.sagemath.org/ticket/23533#comment:13
>>
>> It would be nice that we take a concrete decision about how much we
>> support optional packages (and write it in the developer manual). So far
>> Maarten, Jeroen and I are in favor of as much support with optional as
>> standard.
>> And Volker seems to be against. It would be nice to have more
>> opinions.

My initial intention with ooptional packages was definitely that they
do *not* have as much support as standard.
The point of optional packages was that they are packages that are of
significantly less importance to most users,
so don't need to be installed in every copy of Sage.

I just searched for a while (e.g., via google [1] and looking
sagemath.org) for the list of optional packages, in
order to give some examples of how there is clearly no intention of
them working on all platforms, but I couldn't
even find the optional package list... However, if I remember
correctly, I have added long ago some packages
that were OS specific as optional packages. Those can't work on all
supported platforms.

Experimental is just a place to put packages with absolutely no
expectation of them working at all.

All that said, these days package systems (and the internet) have
gotten massively better, and it is more reasonable
to deliver a small core to users, and have them install what they
need. I don't have the impresison that *Sage's* package
system has necessarily got significantly good though, e.g., as
compared to npm, pip, anaconda's, debian, etc., which all
have binary options.

In summary: this is definitely a question that deserves to be
revisted. The answer definitely used to be what Volker seems to
think, namely "optional packages are less supported". However, how
supported something is, is partly just a function of how much time we
have to put into supporting things -- if we have the time now to
support all the optional packages better, and our packaging is good
enough that it actually works, then maybe we can aspire to make
optional packages much better!

[1] https://www.google.com/search?q=sage+optional+packages&rlz=1CAZZAF_enUS778US778&oq=sage+optional+packages&aqs=chrome..69i57j69i61l2j0j69i64.2232j0j4&sourceid=chrome&ie=UTF-8

>>
>> Vincent
>>
> As someone who did a lot of development on Sage years ago, my understanding
> was that optional packages were meant to work. In much the same way as you
> buy a car, and can pay extra for optional features, such as leather seats,
> heated seats, alloy wheels, metallic paint etc.
>
> If something does not work reliably on all supported platforms, then it
> should be considered 'experimental' in my opinion. So in my opinion at
> least, if something does not work properly on all platforms, there are
> currently only two sensible options.
>
> 1) Fix the problem.
>
> 2) Downgrade the package to experimental.
>
> Otherwise you have no distinction between optional and experimental.
>
> Dave
>

> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.

--
William (http://wstein.org)

[Dr. David Kirkby]

On 25 March 2018 at 18:06, William Stein <wst...@gmail.com> wrote:

>> opinions.

My initial intention with ooptional packages was definitely that they
do *not* have as much support as standard.

The developers guide.

http://doc.sagemath.org/html/en/developer/packaging.html

says:

"

• optional packages are subject to the same requirements, they should also work on all supported platforms. If there are optional doctests in the Sage library, those tests must pass. Note that optional packages are not tested as much as standard packages, so in practice they might break more often than standard packages.
• for experimental packages, the bar is much lower: even if there are some problems, the package can still be accepted.

"

I would interpret that as meaning if a test reveals it does not work, then it should not be optional, as optional packages should work on all supported platforms. In this instance, it seems there are some problems, so it should either be fixed or changed to experimental.

Dave

Dave

[Dima Pasechnik]

TLDR; "supported platform" and "blocker ticket" are merely engineering terms. There are not and cannot be as precise

as mathematical theorems :-)

 

The list of supported platforms (https://wiki.sagemath.org/SupportedPlatforms)

is severely outdated.

As well, no attempt to promise that a particular version of gcc (or other compiler) can be used to

build Sage on a particular supported platform is made.

In such a situation, the statement "Sage is fully supported on platform X" has at best only fuzzy meaning.

And it's OK, as it seems that the commitment to fully support Sage on every flavour of Linux out there is

not realistic, no matter how popular this flavour is.

 From this (and other) discussions here, it seems that "X is supported" came to mean "there is at least one patchbot

running platform X".

This is too rigid and not really the case, in my view.

I would actually like to see the definition of "supported platform" to be made realistic, and mean "popular distributions of Linux on x86/x86_64, ..." without any particular emphasis to versions, toolsets used, etc.

(e.g. the already recognised fact that compilers can be broken ought to be extended to other tools, e.g. make --- cf. the

recent ticket related to guile-supporting make on Archlinux)

------------------------

Given this, there should be no tickets made blockers merely on the basis that Sage broke on your favourite patchbot or  laptop...

Dima

[Ralf Stephan]

On Monday, March 26, 2018 at 1:09:32 AM UTC+2, Dima Pasechnik wrote:

TLDR; "supported platform" and "blocker ticket" are merely engineering terms. There are not and cannot be as precise

as mathematical theorems :-)

Still, if a standard package fails doctests on all systems it's a blocker. Also, if an optional package fails to build on all platforms it's a blocker as well.

And that's it in my opinion. Everything else is only handled as critical, from my experience.

As well, no attempt to promise that a particular version of gcc (or other compiler) can be used to

build Sage on a particular supported platform is made.

That is not true because we make the promise that it builds with the shipped gcc (i.e. 7.2.0 atm).

 

In such a situation, the statement "Sage is fully supported on platform X" has at best only fuzzy meaning.

And it's OK, as it seems that the commitment to fully support Sage on every flavour of Linux out there is

not realistic, no matter how popular this flavour is.

 From this (and other) discussions here, it seems that "X is supported" came to mean "there is at least one patchbot

running platform X".

That goes in circles because e.g. the fact that Sage still crashes in two doctests on OpenSuSE prevents people on that system from running a patchbot. And that ticket is not handled as a blocker.

Given this, there should be no tickets made blockers merely on the basis that Sage broke on your favourite patchbot or  laptop...

Can we agree on "blocker" == "on all system"?

[Dima Pasechnik]

Sure, a failure on all systems, or even on all Linux/OSX systems (see, more fuzziness here :-)) ought to be a reason for making something a blocker.

[Martin R]

There is another problem, which should be treated separately: sometimes doctests of optional packages fail because of newly introduced bugs or modifications in sage core. An example is https://trac.sagemath.org/ticket/24827 (now fixed and closed).

I think one point of having the label "optional" is that such regressions are dealt with.  In particular, in this case it would be wrong to demote the package to "experimental".

Martin

[Dr. David Kirkby]

On 26 March 2018 at 00:09, Dima Pasechnik <dim...@gmail.com> wrote:

Given this, there should be no tickets made blockers merely on the basis that Sage broke on your favourite patchbot or  laptop...

Dima

I think the method Wolfram Research follow with Mathematica has a lot of merit. With respect to Linux, they say:

http://www.wolfram.com/mathematica/system-requirements.html

"Ubuntu 12.04–17.10
RHEL 6–7
CentOS 6–7
Debian 7–9
openSUSE 12.1–13.2/Leap 42.3
Fedora 14–27

Mathematica 11.3 has been fully tested on the Linux distributions listed above. On new Linux distributions, additional compatibility libraries may need to be installed. It is likely that Mathematica will run successfully on other distributions based on the Linux kernel 2.6 or later.

So in other words, I would suggest

1) Pick a set of distributions you want to support - not necessarily the same as Wolfram Research.

2) Make sure there are patchbots for all of them, which test all optional packages.

3) Make tickets blockers if they fail to work 100% on one of the fully supported platforms.

4) If it is impossible to get some aspect of Sage working properly on a fully supported platform, either drop that part of Sage, or remove that distribution from the list of fully supported platforms.

Some of the ideas presented above by some people, would in my opinion at least, have a negative aspect to the quality of the code.

Dave

[Dima Pasechnik]

On Monday, March 26, 2018 at 11:34:04 AM UTC+1, Dr David Kirkby wrote:

On 26 March 2018 at 00:09, Dima Pasechnik <dim...@gmail.com> wrote:

Given this, there should be no tickets made blockers merely on the basis that Sage broke on your favourite patchbot or  laptop...

Dima

I think the method Wolfram Research follow with Mathematica has a lot of merit. With respect to Linux, they say:

http://www.wolfram.com/mathematica/system-requirements.html

"Ubuntu 12.04–17.10
RHEL 6–7
CentOS 6–7
Debian 7–9
openSUSE 12.1–13.2/Leap 42.3
Fedora 14–27

Mathematica 11.3 has been fully tested on the Linux distributions listed above. On new Linux distributions, additional compatibility libraries may need to be installed. It is likely that Mathematica will run successfully on other distributions based on the Linux kernel 2.6 or later.

I think their task at Wolfram is much easier than here, as compatible to Sage's requirements would be certifying a

particular Linux distribution to be fully fit for Sage development rather than simple Sage usage.

[Simon King]

++1!

It has happened *repeatedly* spkg that changes in the SageMath core
(ostensibly "internal changes", but I still believe that at least some of
them were backwards incompatible API changes) broke my group cohomology
spkg, so that it didn't even build (fixed by an spkg version that has
recently been ready for review) resp. so that it fails tests (before the
review could be finished).

The spkg used to be optional, but was downgraded to "experimental" when
it was decided to break all old-style spkgs.

Best regards,
Simon

[Simon King]

Hi!

In my previous post I didn't understand that the actual point of this
thread apparently is not "How much support are we supposed to provide for
optional packages?", but "Are optional packages important enough to
constitute a blocker?".

As a maintainer of a former optional package, I repeatedly found it
annoying that the package got broken by changes in SageMath. It feels
like EACH TIME I fix upstream to make it work with the latest Sage version
and open a ticket for upgrading the package, BEFORE THE REVIEW IS FINISHED
there will be yet another change in Sage that breaks the just-fixed package
yet again. For that reason, I lost the impetus at some point, and now the
package doesn't properly work since several years.

Therefore I do believe that Sage development should treat optional packages
with more respect.

However, I do agree that broken optional packages have a lower priority
than broken standard packages. The latter is a blocker, thus the former
should be regarded as "critical", but not necessarily "blocker".

Best regards,
Simon

[Erik Bray]

On Sun, Mar 25, 2018 at 4:49 PM, Simon King <simon...@uni-jena.de> wrote:
> Hi!
>
> On 2018-03-25, Volker Braun <vbrau...@gmail.com> wrote:
>> On Sunday, March 25, 2018 at 2:51:46 PM UTC+2, Dima Pasechnik wrote:
>>>
>>> one can install autotools on archlinux systemwide
>>>
>>
>> Also, autotools aren't even required to build Sage.
>>
>> Whats the point of delaying a release for weeks/months to fix an optional
>> package? Presumably you agree that broken standard packages should have the
>> highest priority, so broken optional packages necessarily have a lower
>> priority.
>
> In my previous post I didn't understand that the actual point of this
> thread apparently is not "How much support are we supposed to provide for
> optional packages?", but "Are optional packages important enough to
> constitute a blocker?".
>
> As a maintainer of a former optional package, I repeatedly found it
> annoying that the package got broken by changes in SageMath. It feels
> like EACH TIME I fix upstream to make it work with the latest Sage version
> and open a ticket for upgrading the package, BEFORE THE REVIEW IS FINISHED
> there will be yet another change in Sage that breaks the just-fixed package
> yet again. For that reason, I lost the impetus at some point, and now the
> package doesn't properly work since several years.
>
> Therefore I do believe that Sage development should treat optional packages
> with more respect.

I agree--I think there should be at least one buildbot run
per-platform--maybe not for every issue but at least run once a week,
that tests building all optional packages and running tests that use
them (i.e. tagged with # optional - <packagename>).

A build with broken optional packages could then be considered a broken build.

It's possible we could still "downgrade" some formerly "optional"
packages that no one is interested in maintaining, but that shouldn't
be the default assumption.

[kcrisman]

I agree--I think there should be at least one buildbot run
per-platform--maybe not for every issue but at least run once a week,
that tests building all optional packages and running tests that use
them (i.e. tagged with # optional - <packagename>).

A build with broken optional packages could then be considered a broken build.

 

Or even starting at rc time.  But if an optional package is broken on most platforms we provide builds for, then either there is something wrong with the package (-> experimental) or we changed API or there is a new bug in Sage (-> blocker).  However, that does presuppose the "maintainer" model we talked about a number of years ago, where someone really does say they take responsibility for the optional package for (say) 3 years.  I don't know if that has happened in practice.  By the way, Simon has been remarkably patient w.r.t. his package, which has a lot of moving parts for complex computations.