Source tarballs with different sizes

[OHappyDay]

Dear all,

I noticed that the tarballs distribute here

       https://www.sagemath.org/download-source.html 

and here

      https://github.com/sagemath/sage/releases

are of significant different filesizes. 

Is there a specific reason for that? Error in the distribution process?

Klaus

[Dima Pasechnik]

No, the latter tarballs are unconfigured, the former are (you don't
need to run autotools, that is).

HTH
Dima

>
> Klaus
>
> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/sage-devel/ee2ecdd6-335a-40ee-9f78-ba38c8d2b61an%40googlegroups.com.

[John H Palmieri]

I think also that the former contains tarballs for all of the Sage packages (in the "upstream" directory) whereas the latter downloads them as needed during the build process. So the former should be able to build without an internet connection, once you've downloaded the single large tarball.

  John

[Dima Pasechnik]

On Wed, May 13, 2026 at 12:56 PM John H Palmieri <jhpalm...@gmail.com> wrote:
>
> I think also that the former contains tarballs for all of the Sage packages (in the "upstream" directory) whereas the latter downloads them as needed during the build process. So the former should be able to build without an internet connection, once you've downloaded the single large tarball.

Since one of the latest standard package addition, rpds_py, this is no
longer true that you can build everything without an internet
connection.
(it's needed to pull the correct binary wheel for rpds_py - which you
can't build without a Rust toolchain).

Actually, I would like to see more such packages, e.g. scipy/numpy, be
converted to such binary wheels packages.

Dima

>
> John
>
>
> On Wednesday, May 13, 2026 at 10:47:57 AM UTC-7 dim...@gmail.com wrote:
>>
>> On Wed, May 13, 2026 at 11:56 AM 'OHappyDay' via sage-devel
>> <sage-...@googlegroups.com> wrote:
>> >
>> > Dear all,
>> >
>> > I noticed that the tarballs distribute here
>> >
>> > https://www.sagemath.org/download-source.html
>> >
>> > and here
>> >
>> > https://github.com/sagemath/sage/releases
>> >
>> > are of significant different filesizes.
>> > Is there a specific reason for that? Error in the distribution process?
>>
>> No, the latter tarballs are unconfigured, the former are (you don't
>> need to run autotools, that is).
>>
>> HTH
>> Dima
>>
>> >
>> > Klaus
>> >
>> > --
>> > You received this message because you are subscribed to the Google Groups "sage-devel" group.
>> > To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
>> > To view this discussion visit https://groups.google.com/d/msgid/sage-devel/ee2ecdd6-335a-40ee-9f78-ba38c8d2b61an%40googlegroups.com.
>
> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

> To view this discussion visit https://groups.google.com/d/msgid/sage-devel/00127d67-c402-4ae3-82e1-b1735f3fff9fn%40googlegroups.com.

[OHappyDay]

Thanks for the clarification.

[John Cremona]

I would find it helpful to have a tarball available from which Sage could be built and installed with no internet connection (some firewall issues on a university server, please do not ask for details).  If that is not possible, and not going to be supported, should there not be a vote?  Anyway, is it possible for me to build and install Sage on machine 1 which does have a direct internet connection and then create my own tarball from that, which I am able to copy to machine 2 without the direct connection, so that I can build again on machine 2?

John

[Tobia...@gmx.de]

There are (relatively simple) ways to do an offline install of the conda env / copy a conda env from a similar machine. Then just follow https://doc.sagemath.org/html/en/installation/source.html#using-conda. 

Perhaps with this method you don't even need the offline install, if your university is not blocking conda-forge.

[Michael Orlitzky]

On 2026-05-14 01:25:02, John Cremona wrote:
> I would find it helpful to have a tarball available from which Sage could
> be built and installed with no internet connection (some firewall issues on
> a university server, please do not ask for details). If that is not
> possible, and not going to be supported, should there not be a vote?
> Anyway, is it possible for me to build and install Sage on machine 1 which
> does have a direct internet connection and then create my own tarball from
> that, which I am able to copy to machine 2 without the direct connection,
> so that I can build again on machine 2?

The decision was taken out of out hands when several important python
packages started including rust code. To build them, you need a rust
toolchain and a long list of dependencies that aren't feasible to add
to the sage distro.

The binary wheels avoid that problem, but introduce a new one in that
they depend on your architecture and libc. It's still possible to put
together a tarball that works off-line, but that tarball will only be
for one platform -- whichever platform the included wheels are
for. (If your machines are similar, you won't have a problem copying
the install from one to the other.)

It may also be possible to include every binary wheel for every such
package, but the space requirements will increase, and distributing
binaries can create licensing issues. (As in, maybe not, but someone
has to do the thinking about it.)

[John Cremona]

On Thu, 14 May 2026 at 12:55, Michael Orlitzky <mic...@orlitzky.com> wrote:
>
> On 2026-05-14 01:25:02, John Cremona wrote:
> > I would find it helpful to have a tarball available from which Sage could
> > be built and installed with no internet connection (some firewall issues on
> > a university server, please do not ask for details). If that is not
> > possible, and not going to be supported, should there not be a vote?
> > Anyway, is it possible for me to build and install Sage on machine 1 which
> > does have a direct internet connection and then create my own tarball from
> > that, which I am able to copy to machine 2 without the direct connection,
> > so that I can build again on machine 2?
>
> The decision was taken out of out hands when several important python
> packages started including rust code. To build them, you need a rust
> toolchain and a long list of dependencies that aren't feasible to add
> to the sage distro.

Thanks for the explanation.

>
> The binary wheels avoid that problem, but introduce a new one in that
> they depend on your architecture and libc. It's still possible to put
> together a tarball that works off-line, but that tarball will only be
> for one platform -- whichever platform the included wheels are
> for. (If your machines are similar, you won't have a problem copying
> the install from one to the other.)

I could do that for one of my "hidden" machines as that is the same as
the one which is not hidden, but there are two more hidden ones, which
I can only log into via the non-hidden one. I guess that if I knew
more about how things work then I could set up the hidden machines to
route requests via the non-hidden one but this is not important enough
to me for that (unless it is trivial). All these computers are at
least 10 years old anyway. For the same reason, I cannot (easily)
update ubuntu packages on them (they have 22.04 only).

>
> It may also be possible to include every binary wheel for every such
> package, but the space requirements will increase, and distributing
> binaries can create licensing issues. (As in, maybe not, but someone
> has to do the thinking about it.)

I still have my first ever USB memory stick, capacity 256MB, onto
which I used to put entire Sage tarballs to carry home to install on
my home machine, as my home internet was very slow. Those were the
days (2006 or so).

John

>
> --
> You received this message because you are subscribed to a topic in the Google Groups "sage-devel" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/sage-devel/EYxb7cubxfI/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to sage-devel+...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/sage-devel/agW4OfZMhd4OlNx0%40mertle.

[John H Palmieri]

On Thursday, May 14, 2026 at 4:55:46 AM UTC-7 Michael Orlitzky wrote:

On 2026-05-14 01:25:02, John Cremona wrote:
> I would find it helpful to have a tarball available from which Sage could
> be built and installed with no internet connection (some firewall issues on
> a university server, please do not ask for details). If that is not
> possible, and not going to be supported, should there not be a vote?
> Anyway, is it possible for me to build and install Sage on machine 1 which
> does have a direct internet connection and then create my own tarball from
> that, which I am able to copy to machine 2 without the direct connection,
> so that I can build again on machine 2?

The decision was taken out of out hands when several important python
packages started including rust code. To build them, you need a rust
toolchain and a long list of dependencies that aren't feasible to add
to the sage distro.

The binary wheels avoid that problem, but introduce a new one in that
they depend on your architecture and libc. It's still possible to put
together a tarball that works off-line, but that tarball will only be
for one platform -- whichever platform the included wheels are
for. (If your machines are similar, you won't have a problem copying
the install from one to the other.)

Sage could help with this: it could provide a command to download the necessary files once it has determined the architecture. I don't know if this would solve John C's firewall problems (although the auto-generated file for that command would presumably include all of the relevant URLs, which I assume would help), but more broadly it might help someone who wants to get all of their downloading done at once for whatever reason.

I don't remember a discussion on sage-devel letting everyone know that building with no internet connection would no longer be possible, and a quick search doesn't yield any hits. There was https://groups.google.com/g/sage-devel/c/JiSryImYGe0/m/QX-eqWDbFAAJ, but it left things unresolved, it looks like (with an unanswered question about, if certain Rust-based packages became standard, how one could install Sage without an internet connection after the initial download).

Sage developers are supposed to form a community and make major decisions as a community. Changing the structure of Sage so that it can't be built without an internet connection is such a decision. Changing Sage so that it no longer builds the documentation by default is such a decision. I'm sure there are others like these that should have been discussed but weren't. I ask once again that developers bring such issues to sage-devel for a discussion before imposing major changes.

[Michael Orlitzky]

On 2026-05-14 08:19:51, John H Palmieri wrote:
>
> I don't remember a discussion on sage-devel letting everyone know that
> building with no internet connection would no longer be possible, and a
> quick search doesn't yield any hits. There
> was https://groups.google.com/g/sage-devel/c/JiSryImYGe0/m/QX-eqWDbFAAJ,
> but it left things unresolved, it looks like (with an unanswered question
> about, if certain Rust-based packages became standard, how one could
> install Sage without an internet connection after the initial download).
>
> Sage developers are supposed to form a community and make major decisions
> as a community. Changing the structure of Sage so that it can't be built
> without an internet connection is such a decision. Changing Sage so that it
> no longer builds the documentation by default is such a decision. I'm sure
> there are others like these that should have been discussed but weren't. I
> ask once again that developers bring such issues to sage-devel for a
> discussion before imposing major changes.

Python packages that were *already standard* began requiring rust
without any involvement on our part.

[John H Palmieri]

And how does that explain that there was no announcement to this list?

 

[Dima Pasechnik]

On Thu, May 14, 2026 at 3:25 AM John Cremona <john.c...@gmail.com> wrote:
>
> I would find it helpful to have a tarball available from which Sage could be built and installed with no internet connection (some firewall issues on a university server, please do not ask for details). If that is not possible, and not going to be supported, should there not be a vote?

There is no point in vote on unavoidable things.
It was discussed at some length why complete tarballs are suboptimal
in such a setup, why this is unavoidable to have packages with Rust
extensions if we want to continue to provide up to date notebook,
which depends on jupyterlab (Jupyterlab depends on a number of
packages which depend on rpds_py) and everything *with all
dependencies*.

In fact we should stop shipping notebook and its dependencies, and
ditto for sphinx, matplotlib, scipy/numpy, etc. - and this is the plan
going forward.
Maintaining by hand hundreds of dependencies we never modify makes no
sense, we don't have resources for this.

> Anyway, is it possible for me to build and install Sage on machine 1 which does have a direct internet connection and then create my own tarball from that, which I am able to copy to machine 2 without the direct connection, so that I can build again on machine 2?

All you need to do is to populate with such packages, in some way, the
upstream/ directory of the machine in question.
You can look up the URLs of the binary wheels in
build/pkgs/rpds_py/checksums.ini


Dima

> To view this discussion visit https://groups.google.com/d/msgid/sage-devel/202dd440-fe28-4e5a-a91f-32b31b5025b2n%40googlegroups.com.

[Dima Pasechnik]

Well, we also don't have announcements of breaking changes in cpython
here, or weather reports :-)

That's another example why we need a checklist for the releases to
exist and be checked off.

By the way, one reason it was not announced is that I don't like to
post anyhow remotely controversial topics on sage-develop, as
thanks to the CoC committee my posts here are moderated, and I then
I'm helplessly watching a pileup of negative reactions
without an ability to efficiently reply.

---

As the red line of no self-contained tarballs is crossed anyway, it's
time to drastically cut from sage-distro the bulk
of spkgs which we don't control, and which are not directly related to sagelib
(i.e. everything related to notebook, sphinx, and matplotlib - as
these are pip-installable).

Dima

>
>
>
> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

> To view this discussion visit https://groups.google.com/d/msgid/sage-devel/ea510fa9-eab2-436c-9848-05287d2c1a5bn%40googlegroups.com.

[Michael Orlitzky]

On 2026-05-14 10:19:18, John H Palmieri wrote:
>
> And how does that explain that there was no announcement to this list?

Hindsight is 20/20, but at that point, the damage was done. Aside from
"leave it broken" and "delete the notebook," binary wheels were the
only solution on the table.

[John H Palmieri]

One "solution" is to just let people know that the large tarball downloads no longer allow people to build without an internet connection. This should have been done.

Another solution, which requires some work — I have no idea how easy or hard, so I have no idea if it's worth it — would be to get a list of URLs for files to download and/or create a script that will do the downloads if you want. "make download" tries to download tarballs for everything including optional packages (and then fails for me because it can't find one of them). It also doesn't seem to download architecture-specific wheel files, but maybe this make target could be repaired (and advertised?): maybe it should just download what is needed given the particular settings given to ./configure rather than downloading everything, and it would be great if it could download the right wheel files. (As an example, there is no reason that the default should be to download the Stein-Watkins database, which is 2.6GB, and at least some of our mirrors are pretty slow to download from.) "make download-for-sdist" is better, but it downloads wheels for many architectures, as well as tarballs for packages that won't be built on the particular system — it is designed to ignore the information coming from ./configure.