Modularization project: III. The hooks

[Matthias Koeppe]

(index of previous posts: https://github.com/sagemath/sage/issues/29705)

In the open source world, as we all know, a powerful mechanism is that people often start out as users of a package, then gradually start contributing to it, and then some find a path to becoming major contributors or maintainers.

Using a package U, in Python, often means that you create your own package D to be published on PyPI with a dependency on U. 

I argued in post I that (without the modularization) no portion of Sage is viable to be used as the dependency U of any Python package D, and that therefore the developers of D face the difficult decision: "Are the facilities that a portion of Sage offers really important enough for my project D --- that it makes sense to accept being isolated from the rest of the Python world?"

So let's first take a look at the portions of Sage with an eye on their potential to find many new users if we make them usable separately.

I will preface my remarks with the following:

(1) Of course, all portions of Sage are the product of wonderful contributions by dedicated individuals over the years.

(2) Some portions of Sage serve a very important purpose within Sage but cannot be plausibly marketed to users who are not already using Sage. That's OK and nobody's at fault.

(3) I may have edited your docstrings and doctests and may know a lot about the dependencies of your code, but I don't necessarily claim to know the mathematics that this code is about or uses, or what its standing of the code in the field is, etc.

So consider this as a mere start of a discussion, with the goal of eventually determining a strategy.

sage.geometry.polyhedron: 

- Obvious strong potential for use outside of Sage. There's nothing like it in the Python world, and it's competitive with the standalone package Polymake (which uses Perl as the interaction language).

- Core functionality just needs linear algebra over QQ, ZZ, and a few well-known specialized small libraries. And everyone knows that; it's very hard to explain that in order to use it, they need to pull in gigabytes of unrelated mathematics.

sage.graphs: 

- Suppose we make all or most of its functionality available to users by "pip install sagemath-graphs". 

- Would we be able to explain to potential users (who are not already using Sage) why (or when) they should use it rather than using https://networkx.org/ or https://python.igraph.org/en/stable/) 

- This is not a rhetorical question; I don't know the answer, and if/when we prepare sagemath-graphs, this answer would need to go on the front page of the package.

sage.numerical.mip: 

- SageMath's in-house multiple-backend mixed integer programming interface is important for sage.graphs and sage.combinat, but it currently has no distinguishing features that would make it interesting to users who are not already using Sage. (There are lots of alternatives in Python, see https://github.com/sagemath/sage/issues/26511, many with lots of users and some with lots of developers.)

sage.finance, sage.logic, sage.plot, sage.probability, sage.stats:

- Probably not.

sage.symbolic:

- It's complicated.

sage.schemes, sage.lfunctions, sage.modular:

- Hoping for experts to comment.

[William Stein]

sage.plot provides functions for plotting that are very similar to the Mathematica's plotting functions.  This is analogous to how matplotlib's pyplot provides a UI on top of the core matplotlib library that is similar to matlab's plotting UI.  In many cases the implementations in sage.plot (might be) nontrivial and (last I checked) did not exist anywhere else in the Python ecosystem, e.g., at sage days 9 a lot of work by grad students went into adaptive refinements algorithms for plotting a function f(x).  The approach to plotting using by Mathematica is extremely different than Matlab's, and they both have their pros and cons for different use cases.    There would be significant value and interest in sage.plot outside of sage.   

other packages you mention: it's probably mostly done better elsewhere already, except maybe sage.probability.

sage.symbolic:

- It's complicated.

The competitor is of course sympy.  Probably the main advantage over sympy is that sage.symbolic can be much, much faster in some situations.  Also, the fast_float (and related functionality) that compiles symbolic functions to a fast floating point evaluating "machine" is very useful for numerical algorithms and plotting, and I don't know how or if sympy compares in that regard.  Symbolics also can leverage sage's interval arithmetic to do nontrivial things involving equality testing...

 

sage.schemes, sage.lfunctions, sage.modular:

- Hoping for experts to comment.

Probably if you're using  sage.schemes, sage.lfunctions, sage.modular, then having all of sage as a dependency is the least of your concerns.

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/a120bc50-3482-4ed6-bdf0-9c68f23c847cn%40googlegroups.com.

--

William (http://wstein.org)

[Kwankyu Lee]

Would there be one-to-one correspondence between distribution packages and features (for modules of sage library)?

For example, for `sage.plot`, would there be one distribution package `sagemath-plot`? 

One-to-one correspondence would make things simpler to understand.

[Matthias Koeppe]

My design (https://github.com/sagemath/sage/pull/35095) does have a distribution "sagemath-plot". But other top-level packages such as "sage.modules" are simultaneously too small and too big: It makes no sense to separate "sage.modules" and "sage.matrix" from each other; and on the other hand, distributing all free module implementations in one distribution would pull in lots of libraries.

I'm preparing a post that explains the proposed design in more detail.

[Matthias Koeppe]

On Wednesday, June 14, 2023 at 8:32:17 AM UTC-7 William Stein wrote:

On Tue, Jun 13, 2023 at 12:05 PM Matthias Koeppe <matthia...@gmail.com> wrote:

let's first take a look at the portions of Sage with an eye on their potential to find many new users if we make them usable separately.

 

sage.plot provides functions for plotting that are very similar to the Mathematica's plotting functions.  This is analogous to how matplotlib's pyplot provides a UI on top of the core matplotlib library that is similar to matlab's plotting UI.  In many cases the implementations in sage.plot (might be) nontrivial and (last I checked) did not exist anywhere else in the Python ecosystem, e.g., at sage days 9 a lot of work by grad students went into adaptive refinements algorithms for plotting a function f(x).  The approach to plotting using by Mathematica is extremely different than Matlab's, and they both have their pros and cons for different use cases.    There would be significant value and interest in sage.plot outside of sage.   

Thanks, William. I have added a bit from what you wrote to the sagemath-plot README. It doesn't quite read like ad copy yet, but it's a start.