For proper releases, I am hoping for separate tarballs (eventually) which means that there won’t be any issues with symbolic links.
To package these I pull the full tree :( and then I don’t go to SAGE_ROOT/pkgs/pkg_name and build from there.
If I want to patch it doesn’t work from there because patch doesn’t apply across symbolic links. So, instead I go to SAGE_ROOT/src, copy the appropriate setup.py (and any other files needed) and patch and build from there.
it is possible to use Git to only pull a selected directory
(or directories) from a Git repo with no history. See, e.g., the
discussion of "sparse checkout" here.
https://stackoverflow.com/questions/600079/how-do-i-clone-a-subdirectory-only-of-a-git-repository
As we are making progress on preparing the Sage library for modularization (https://trac.sagemath.org/ticket/29705), here's a discussion of three things that need to be decided whenever a portion of the Sage library is modularized:(1) Namespace: Will it be imported as (+) "from sage.hermeneutics.quantum import QuantumGravity" or (-) "from transformative_hermeneutics.quantum import QuantumGravity"?(2) Distribution name (= project name): Will it be installed using "pip install transformative-hermeneutics" or "pip install sagemath-hermeneutics"?(3) Source repository: Is the source maintained as part of the Sage repository using Trac tickets (MONOREPO); or in a separate repository on GitHub, GitLab, ... (MULTIREPO)?This post is dedicated to the MONOREPO vs. MULTIREPO question, and how it relates to (1) and (2).Some details about MONOREPO:The Sage repository already contains several separate distribution source trees. Since Sage 9.4 they are located in the subdirectory SAGE_ROOT/pkgs (see https://wiki.sagemath.org/ReleaseTours/sage-9.4#New_location_for_distribution_package_sources:_SAGE_ROOT.2Fpkgs).For example, SAGE_ROOT/pkgs/sage-sws2rst is a self-contained source tree of this distribution package: It has standard Python packaging files such as setup.py.SAGE_ROOT/pkgs/sagemath-standard is also a source tree of this distribution package; but it is created in part using symbolic links into SAGE_ROOT/src. This is the trick that has allowed the modularization effort to keep the SAGE_ROOT/src tree monolithic: Modularization has been happening behind the scenes and will not change where Sage developers find the source files.In favor of MONOREPO:+ The Sage developer community is used to using Trac+ The process is well defined: No code "ownership" (all Sage developers can change any code, subject to peer review); changes are synchronized+ There is implicit integration testing with all of SageDrawbacks of MONOREPO:- Extra hurdle for attracting new developers from outside the Sage developer community (everyone who is not already a Sage developer uses GitHub or similar)- Testing infrastructure needs to be set up and maintained separately; in particular, integration testing with Sage.
Evidence:- The pynac situation -- now finally resolved by merging it into the Sage library (https://wiki.sagemath.org/ReleaseTours/sage-9.5#Modularization_and_packaging_changes)
- Even for separate git repositories hosted in the SageMath GitHub organization (https://github.com/sagemath/), code ownership and review workflow are unclear.- For example, https://github.com/sagemath/sage-numerical-backends-coin provides no information on how to contribute (commonly expected in a file CONTRIBUTING.md)- Neither does https://github.com/sagemath/cysignals, which moreover looks abandoned: Issues and PRs are not tended to. Is it subject to peer review like other parts of Sage? Who is in charge of merging PRs?- Similar issues with https://github.com/sagemath/cypari2
Recommendation: Keep MONOREPO for all distributions that fill the sage.PAC.KAGE.MODULE namespace (= distribution packages named sagemath-... -- according to my recommendation in part I).
Recommendation: For distributions that do not use the sage.PAC.KAGE.MODULE namespace (= distribution packages named something other than sagemath-... -- according to my recommendation in part I), weigh the benefits vs. drawbacks of MONOREPO vs. MULTIREPO. It is also simple enough for a distribution to start out being developed inside of the Sage MONOREPO and to be split out to a separate repository later.Recommendation: Discuss the development workflow for projects in the https://github.com/sagemath organization, and document it in files CONTRIBUTING.md in the individual repositories.
On Mon, Oct 11, 2021 at 1:17 AM Matthias Koeppe <matthia...@gmail.com> wrote:(3) Source repository: Is the source maintained as part of the Sage repository using Trac tickets (MONOREPO); or in a separate repository on GitHub, GitLab, ... (MULTIREPO)?This post is dedicated to the MONOREPO vs. MULTIREPO question, and how it relates to (1) and (2).Some details about MONOREPO:The Sage repository already contains several separate distribution source trees. Since Sage 9.4 they are located in the subdirectory SAGE_ROOT/pkgs (see https://wiki.sagemath.org/ReleaseTours/sage-9.4#New_location_for_distribution_package_sources:_SAGE_ROOT.2Fpkgs).For example, SAGE_ROOT/pkgs/sage-sws2rst is a self-contained source tree of this distribution package: It has standard Python packaging files such as setup.py.SAGE_ROOT/pkgs/sagemath-standard is also a source tree of this distribution package; but it is created in part using symbolic links into SAGE_ROOT/src. This is the trick that has allowed the modularization effort to keep the SAGE_ROOT/src tree monolithic: Modularization has been happening behind the scenes and will not change where Sage developers find the source files.In favor of MONOREPO:+ The Sage developer community is used to using Trac+ The process is well defined: No code "ownership" (all Sage developers can change any code, subject to peer review); changes are synchronized+ There is implicit integration testing with all of SageDrawbacks of MONOREPO:- Extra hurdle for attracting new developers from outside the Sage developer community (everyone who is not already a Sage developer uses GitHub or similar)- Testing infrastructure needs to be set up and maintained separately; in particular, integration testing with Sage.you put aside the very fact that dependence on Trac is a problem in itself
On Mon, Oct 11, 2021 at 1:17 AM Matthias Koeppe <matthia...@gmail.com> wrote:Recommendation: Keep MONOREPO for all distributions that fill the sage.PAC.KAGE.MODULE namespace (= distribution packages named sagemath-... -- according to my recommendation in part I).I believe that this hinges on reliance of Trac a lot, but we should not be bound by it.With Trac gone, most benefits of monorepo will go, as well.
--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/4cc79323-fe8f-46e4-9865-1d9482b08b1bn%40googlegroups.com.
On Monday, 11 October 2021 at 09:36:54 UTC-7 Matthias Koeppe wrote:On Monday, October 11, 2021 at 1:31:14 AM UTC-7 Dima Pasechnik wrote:I believe that this hinges on reliance of Trac a lot, but we should not be bound by it.With Trac gone, most benefits of monorepo will go, as well.Not at all. Still the same even with a switch to using GitHub.I *really* appreciate that I can install sage from source and have it ready for development by issuing a single git clone (well, a second one to get git-trac. That's a bit annoying
) and then a combination of make and configure that I can never remember, but the system helps me along. I find it a little annoying that it recommends a "yum install..." of a whole bunch of stuff, where that really should be a "dnf install ...", but that's again an easy edit to make.I also really appreciate that fixing something and/or stepping between different branches is all done through a single git repo, so that "git branch ..." gets my entire relevant tree in a sensible state (and I don't have to figure out where to submit fixes in different parts of the tree). I absolutely dread having to navigate multiple git repositories for making a fix and/or having to figure out WHICH repository a fix should go into.Currently, having code spun off in "spkgs" is already a barrier to fixing things. Recently there was a ticket where some maxima/ecl patch needed work. The only reason I was able to contribute there was that the whole situation could be recreated by monkey-patching maxima on the spot. Figuring out how to pull/patch/build an spkg would have been a prohibitive barrier. I understand the rationale behind the spkgs, but I would think splitting up sagemath in more repositories would pose a higher barrier of entry to contributions. So PLEASE make it possible to pretend sagemath is still in one big repo, where the same build tools all apply.
Another worry that I have is that modularizing sage will lead to destabilization. The original reason for putting all components of sagemath into one big environment was to reduce the number of variables in getting things to work. I expect that if modularization takes off and sage ends up split in more components, then 10 years down the road there will be a group of developers that throw their hands up in desperation because it's just so hard to get all different components together with versions that actually work well together, and trying to keep different components working across version ranges is just so tricky (plus it's too hard to get exactly the right versions from all places, because the newest version of some components will always just have made some incompatible change that the others still have to catch up to). So PLEASE make it easy to have a reference of known-to-work components together, like sage-the-distribution does. I think we'll need it in the future again.
--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/866ac025-586e-4836-b09c-a59e834f3a07n%40googlegroups.com.
Another worry that I have is that modularizing sage will lead to destabilization. The original reason for putting all components of sagemath into one big environment was to reduce the number of variables in getting things to work. I expect that if modularization takes off and sage ends up split in more components, then 10 years down the road there will be a group of developers that throw their hands up in desperation because it's just so hard to get all different components together with versions that actually work well together, and trying to keep different components working across version ranges is just so tricky (plus it's too hard to get exactly the right versions from all places, because the newest version of some components will always just have made some incompatible change that the others still have to catch up to). So PLEASE make it easy to have a reference of known-to-work components together, like sage-the-distribution does. I think we'll need it in the future again.
other Python projects somehow manage
On Monday, October 11, 2021 at 12:08:29 PM UTC-7 Dima Pasechnik wrote:other Python projects somehow manageDima, the dependencies of the Sage library are real -- they cannot be eliminated by wishful thinking or by eliminating the mechanism that makes it easy for developers to install them.
The way to address the dependencies is ... modularization: Define distributions of parts of the Sage library, each of which has a small number of dependencies only, and fix the problems of the Sage library in which "everything depends on everything else".
--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/b9291220-b92f-419a-bbae-3792a3adce11n%40googlegroups.com.
unvendoring Jupyter is removing about ~50 Sage spkgs and replacing their installation by one command,> python3.10 -m pip install jupyter --userThat's about 15% of all the spkgs. No need to care about individual versions of these Jupyter components
On Monday, October 11, 2021 at 12:54:24 PM UTC-7 Dima Pasechnik wrote:unvendoring Jupyter is removing about ~50 Sage spkgs and replacing their installation by one command,> python3.10 -m pip install jupyter --userThat's about 15% of all the spkgs. No need to care about individual versions of these Jupyter componentsJupyter has really nothing to do with anything in this thread.
But yes, it also needs work, see https://trac.sagemath.org/ticket/30306 "Meta-ticket: Use system Jupyter notebook / JupyterLab".
--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/f20a1f9d-5bf4-44e7-a36a-0f57ee2a24f9n%40googlegroups.com.
On Mon, Oct 11, 2021 at 9:01 PM Matthias Koeppe <matthia...@gmail.com> wrote:Jupyter has really nothing to do with anything in this thread.at least I got an impression from your proposal that among other modules you want to create a sage-jupyter package, as a module?
Recommendation: Keep MONOREPO for all distributions that fill the sage.PAC.KAGE.MODULE namespace (= distribution packages named sagemath-... -- according to my recommendation in part I).Recommendation: For distributions that do not use the sage.PAC.KAGE.MODULE namespace (= distribution packages named something other than sagemath-... -- according to my recommendation in part I), weigh the benefits vs. drawbacks of MONOREPO vs. MULTIREPO. It is also simple enough for a distribution to start out being developed inside of the Sage MONOREPO and to be split out to a separate repository later.
- Even for separate git repositories hosted in the SageMath GitHub organization (https://github.com/sagemath/), code ownership and review workflow are unclear.- For example, https://github.com/sagemath/sage-numerical-backends-coin provides no information on how to contribute (commonly expected in a file CONTRIBUTING.md)- Neither does https://github.com/sagemath/cysignals, which moreover looks abandoned: Issues and PRs are not tended to. Is it subject to peer review like other parts of Sage? Who is in charge of merging PRs?- Similar issues with https://github.com/sagemath/cypari2