Making executables available to Sage

[Nico Van Cleemput]

Dear list

Due to various reasons I haven't been able to keep up to date with the development of Sage. However, I still use Sage in my research and mainly various of my co-authors use Sage. Our workflow often consists of me writing some external programs (usually in C) and additional several Python scripts which interface with these programs. When the program and scripts are mature enough, some of these are released as supplementary material for some papers, but usually they are too niche to be included in Sage. The way this needs to be handled has changed several times through the years, and I wanted to inform myself as to what the best way to handle this would be.

It used to be that my co-authors could simply install such a package with the command:

sage -f my-package.spkg

This changed about five years ago, and since then my instructions to them read roughly as

* copy the folder mypackage (containing ) into SAGE_ROOT/build/pkgs

* copy the file mypackage.tar.gz (containing the source code) into SAGE_ROOT/upstream.

* run the command: $ sage --package fix-checksum mypackage

* run the command: $ sage -i mypackage

The third step changed at version 7.3 (it used to be $ sage -sh sage-fix-pkg-checksums)

Nowadays, it seems that this has changed again, because I get reports that installing the package this way hangs indefinitely.

I mainly have two questions:

* Is there another way to handle this, or am I doing the right thing and do I just need to keep the instructions up to date.

* Is there a clear place where this is documented? As I said I haven't been able to keep up with Sage development recently, so each time I had to dive into the documentation and puzzle the instructions for these kinds of external packages together from different places in the documentation. There is the repository https://github.com/sagemath/sage_sample which gives a sample of how to create a package for Sage, however this is about packaging Python code and not about binaries that need to be compiled and made available to Sage.

Kind regards

Nico

[Dima Pasechnik]

pip is able  to install more than just python, it basically can do arbitrary data installs.

People routinely use pip to install scripts in ~/.local/bin/, say.

We should add this to https://github.com/sagemath/sage_sample

Unfortunately it needs hands to work on:

https://github.com/sagemath/sage_sample/issues/37

Then there is pipx, a pip-installable package that could come to your rescue - something that specifically

takes care of the PATH stuff.

https://pipxproject.github.io/pipx/installation/

Not sure how well it plays along with Sage, though.

 

Kind regards

Nico

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CADXCEk-uOqo7duHXtV5W8hP3R8OeM3O6b986b_8oQ-w0sQviWA%40mail.gmail.com.

[Nico Van Cleemput]

Do you have an example of using pip to compile and install such a binary. I've spent quite some time trying to find whether it was possible to use pip for this and couldn't find anything.

Nico

Op wo 8 jul. 2020 om 14:38 schreef Dima Pasechnik <dim...@gmail.com>:

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAAWYfq1v3XwT1DGUoj1BPKCopSmKWc1-9O7%2B9grYY5hO8si4Eg%40mail.gmail.com.

[Markus Wageringel]

One example is https://github.com/mwageringel/fgb_sage, which is a small wrapper for an external C binary and which uses sage_sample as a template. The relevant part is the setup.py file, which downloads the external files and makes them available for compiling the Cython code. It does not use pipx (which I do not know about), but is implemented in a somewhat ad-hoc fashion, so probably this is not the optimal way to achieve this but could be streamlined a bit.

[Nico Van Cleemput]

Thanks. I had been googling again yesterday, but still hadn't found any concrete examples, so I had started reading the full documentation of packaging for Python. This example will help me focus that effort a lot.

Cheers

Nico

Op do 9 jul. 2020 om 09:33 schreef Markus Wageringel <markus.w...@gmail.com>:

One example is https://github.com/mwageringel/fgb_sage, which is a small wrapper for an external C binary and which uses sage_sample as a template. The relevant part is the setup.py file, which downloads the external files and makes them available for compiling the Cython code. It does not use pipx (which I do not know about), but is implemented in a somewhat ad-hoc fashion, so probably this is not the optimal way to achieve this but could be streamlined a bit.

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/33bca7b6-33d9-4ae2-a734-e809ec5f120co%40googlegroups.com.

[TB]

I have created https://gitlab.com/mathzeta2/zetalib whose purpose is to be a pip-installable SageMath package of Zeta. It is mostly Python, but has one Cython file and one C file. I hope it works... any issues or suggestions are welcome.

It is also based on (an old fork of) sage_sample, and I am sure there are better ways to do some of the packaging acrobatics in there.

Currently to install zetalib, first install the dependencies:

$ sage -i latte_int
$ sage -i normaliz

or you can have them installed from elsewhere (you can later manually change the dependencies' commands being called) and then run

$ sage -pip install zetalib # Will compile the .c file

In the future it might be better to make zetalib into a Sage experimental package, so one could just run

$ sage -i zetalib

which will list latte_int and normaliz as dependencies, automatically install them and then fetch zetalib from PyPI. I think there are patchbots that test with many optional packages installed, which is a good way to catch regressions, instead of letting old packages rot.

Off-topic tip: The zetalib tarball on PyPI includes the generated .c file of the .pyx file (and the .pyx file), so users will only need a C compiler to install it. I am not sure this is relevant to Sage packages, because Cython is always included.

Regards,

TB

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CADXCEk9uuXEPwg69a4--U1SEMOE01VgvG%2ByAF-5X-iD6Vtz94Q%40mail.gmail.com.

[Dima Pasechnik]

On Thu, Jul 9, 2020 at 8:38 AM Nico Van Cleemput
<nico.van...@gmail.com> wrote:
>
> Thanks. I had been googling again yesterday, but still hadn't found any concrete examples, so I had started reading the full documentation of packaging for Python. This example will help me focus that effort a lot.

the relevant entries in setuptools.setup()
are entry_points=... and package_data=...

HTH
Dima

>
> Cheers
> Nico
>
> Op do 9 jul. 2020 om 09:33 schreef Markus Wageringel <markus.w...@gmail.com>:
>>
>> One example is https://github.com/mwageringel/fgb_sage, which is a small wrapper for an external C binary and which uses sage_sample as a template. The relevant part is the setup.py file, which downloads the external files and makes them available for compiling the Cython code. It does not use pipx (which I do not know about), but is implemented in a somewhat ad-hoc fashion, so probably this is not the optimal way to achieve this but could be streamlined a bit.
>>
>> --
>> You received this message because you are subscribed to the Google Groups "sage-devel" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/33bca7b6-33d9-4ae2-a734-e809ec5f120co%40googlegroups.com.
>
> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CADXCEk9uuXEPwg69a4--U1SEMOE01VgvG%2ByAF-5X-iD6Vtz94Q%40mail.gmail.com.