Guidance on packaging snapshots of external packages

[Tony Luo]

Hi all,

What is the idiomatic way to package a specific revision of an external package or rather update an existing external package to a specific revision? For context, I am working on a PR to bump Singular to commit 0a4bc0e in order to fix #39801. I have rebuilt sage against the specified Singular revision and verified the issue is fixed, but I am not sure how to go about updating the spkg directory for Singular. Should I make upstream_url in checksum.ini point to the source tarball (autogenerated by GitHub) at the specified commit? I made this work locally by running autogen.sh before sdh_configure in spkg-install.in but don't know if this is the right thing to do. Any guidance or pointers would be greatly appreciated.

Sincerely,
Tony

[Michael Orlitzky]

On 2025-05-07 12:10:00, Tony Luo wrote:
> Hi all,
>
> What is the idiomatic way to package a specific revision of an external
> package or rather update an existing external package to a specific
> revision? For context, I am working on a PR to bump Singular to commit
> 0a4bc0e

Ultimately, you should probably just wait for the next Singular
release and then update to that. It's much better if sage-the-distro
and sage on linux/conda/homebrew/etc. all act the same way, and it's
unlikely that everyone involved in the packaging effort is going to
update to the same snapshot.

But to answer the larger question, updating to a specific revision
(rather than a release) will be a little difficult in this case
because Singular uses autotools, and typically the upstream developers
have to perform some steps to generate ./configure and the associated
Makefile.in files. You could do that yourself, but then you would have
to host your release somewhere, and then -- at least in theory --
someone would have to verify that you haven't backdoored the package.

If possible, it's better to just patch the existing release. You can
do that by dropping a patch into the "patches" directory of the spkg
(see pari for an example). But even this is discouraged if it changes
the behavior of sage because, assuming you add a test case for
whatever bug is fixed, everyone packaging sage then has to quickly
adopt your patch or else the tests begin to fail.

In this instance, adding a patch and then *not* adding a test could be
a good compromise. (You can obtain a patch by sticking ".patch" on the
end of the github commit URL.)

[Tony Luo]

Thank you for the detailed response. I actually tried to patch the existing release with the generated ".patch" file from the GitHub commit URL prior to just bumping the source. A naive attempt fails (as expected) since there is quite a large set of changes between the commit and the current packaged release. As I am not familiar with the Singular codebase, I haven't attempted to trace out the affected files/dependencies for the fix back to v4.4.0. Admittedly this is something out of my depth at the moment.

Regarding the use of autotools, would generating ./configure during the build with a spkg-src script be acceptable here or would you still recommend waiting until the next Singular release? My main concern here is that the Singular bug fixed in the aforementioned commit is nonfatal but nonetheless corrupt the state of objects on the sage side of the interface (see the issue for more details). I don't know how severe this issue is on the grand scale of things but it seems less than ideal to maintain the status quo until the next Singular release. If waiting for the next Singular release is better, would a PR that checks the condition that triggers the bug and aborts the computation be helpful in the meantime? Of course, the changes introduced would have to be reverted when actually bumping Singular.

[Michael Orlitzky]

On 2025-05-07 13:28:13, Tony Luo wrote:
> Thank you for the detailed response. I actually tried to patch the existing
> release with the generated ".patch" file from the GitHub commit URL prior
> to just bumping the source. A naive attempt fails (as expected) since there
> is quite a large set of changes between the commit and the current packaged
> release.

It looks like the only conflicts are in the files under "Tst", which I
assume are test files. If you edit those hunks out of the patch the
rest should apply cleanly on top of singular-4.4.1. I did a quick
build & test and nothing bad happened.

> Regarding the use of autotools, would generating ./configure during the
> build with a spkg-src script be acceptable here or would you still
> recommend waiting until the next Singular release?

It's acceptable and we do it for a few other packages. The return on
investment would be comparatively low in this case because the changes
are already upstream, unlike (say) when we rewrite an entire dead
package's build system. But if you want to do the work and if you can
get someone to review it, sure.

[Tony Luo]

On Wednesday, May 7, 2025 at 9:42:17 PM UTC-4 Michael Orlitzky wrote:
> It looks like the only conflicts are in the files under "Tst", which I
> assume are test files. If you edit those hunks out of the patch the
> rest should apply cleanly on top of singular-4.4.1. I did a quick
> build & test and nothing bad happened.

Thanks for catching this. I must have been working with the wrong tarball. I opened a PR #40076 adding the patch following your suggestion.

Cheers,
Tony