external libraries - is Rust allowed?

[Curran McConnell]

Hi folks!

I am scoping out a project to write an algebra/combinatorics package. It would be a shame if the project was a success, and then there was difficulty integrating with Sage if people wanted to redistribute the routines there. Are Rust additions to the Sage ecosystem welcomed?

The reason why I want to write my package in Rust is because it needs to be fast and parallelized. That constraint suggests choosing among Rust, C and C++. If I were making the decision purely for my own sake, I would use Rust. I find it easier to succeed in projects using that language, due to its guardrails preventing memory management bugs.

However, I don't see Rust explicitly mentioned on this page describing external libraries:

https://doc.sagemath.org/html/en/developer/coding_in_other.html

Has there ever been conversation in the community about attitudes towards integrating binaries from the Rust compiler? Is there a reason why it's not on this list, or was it maybe just not considered by the author of this list?

Thanks in advance for your advice.

[Michael Orlitzky]

On 2023-03-30 10:50:32, Curran McConnell wrote:
> Hi folks!
>
> I am scoping out a project to write an algebra/combinatorics package. It
> would be a shame if the project was a success, and then there was
> difficulty integrating with Sage if people wanted to redistribute the
> routines there. Are Rust additions to the Sage ecosystem welcomed?
>

It depends on what you mean by an addition to the ecosystem. You can
pretty easily write an external library or program in rust, and then
create a Sage interface to that using Cython or popen. In that
scenario the language used to implement your package is irrelevant.

If instead you want to write a standard piece of Sage in rust, that
would be more of a problem. Sage is built from source and ships its
own miniature linux distribution down to the compilers. It would have
to be an extemely compelling package to justify adding the whole rust
toolchain to the mix. I'd go farther and say that there are serious
maturity problems in the rust ecosystem that make it impossible in the
near future. Even if sage was willing to use system packages in this
case, there are, for those same reasons, no distributions attempting
to correctly package rust software right now.

[Dima Pasechnik]

On Thu, 30 Mar 2023, 20:07 Michael Orlitzky, <mic...@orlitzky.com> wrote:

On 2023-03-30 10:50:32, Curran McConnell wrote:
> Hi folks!
>
> I am scoping out a project to write an algebra/combinatorics package. It
> would be a shame if the project was a success, and then there was
> difficulty integrating with Sage if people wanted to redistribute the
> routines there. Are Rust additions to the Sage ecosystem welcomed?
>

It depends on what you mean by an addition to the ecosystem. You can
pretty easily write an external library or program in rust, and then
create a Sage interface to that using Cython or popen. In that
scenario the language used to implement your package is irrelevant.

If instead you want to write a standard piece of Sage in rust, that
would be more of a problem. Sage is built from source and ships its
own miniature linux distribution down to the compilers.

I believe that this minilinux will disappear sooner or later, and it's not really complete in any way, either (e.g. we don't ship C/C++ compilers on macOS - for it should be clang, as gcc cannot cope with some important macOS headers)

So not distributing a compiler does not bother me.

The maturity of the rust toolchain is another story,

and probably such a rust part cannot be in a standard package for the time being.

Dima 

It would have
to be an extemely compelling package to justify adding the whole rust
toolchain to the mix. I'd go farther and say that there are serious
maturity problems in the rust ecosystem that make it impossible in the
near future. Even if sage was willing to use system packages in this
case, there are, for those same reasons, no distributions attempting
to correctly package rust software right now.

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/ZCXdywyRyIr91vI0%40stitch.

[Michael Orlitzky]

On Thu, 2023-03-30 at 23:11 -0400, Curran McConnell wrote:
> Thanks so much for breaking that down!
> I will take a closer look at the distinction between the different
> kinds of
> "package". There is a good likelihood I won't ever need this to be a
> "standard" package, but either way that is a helpful way to frame the
> problem.
>
> Also, I didn't mean to start a private conversation--I haven't used a
> Google group before, I think I might have misused the interface
> slightly.
> Should we repost this thread to the main chat in case other people
> have
> this question in future?
>

No problem. I only interact with the group via email, but I'll CC this
message to the list (sage-...@googlegroups.com) so that the quoted
conversation below shows up.




> Best,
>
> Curran
>
> On Thu, Mar 30, 2023 at 4:23 PM Michael Orlitzky
> <mic...@orlitzky.com>
> wrote:
>
> > On 2023-03-30 12:43:45, Curran McConnell wrote:
> > > Hi Michael, thanks for your explanation. I have a followup
> > > question.
> > >
> > > You distinguish between creating a Sage interface to an external
> > > library
> > vs
> > > writing a standard piece of Sage in Rust.
> > > If my package exported a native Python extension module
> > > (https://docs.python.org/3/extending/extending.html), packaged
> > > via
> > wheels
> > > so the user never has to run the Rust toolchain themselves, would
> > > that
> > fall
> > > into the "interface to an external library" category, from your
> > > point of
> > > view?
> >
> > The devil is in the details but I'd bet that it does.
> >
> > The two things that would cause problems are,
> >
> >   1. Adding an *.rs file directly to sage.git.
> >   2. Creating a type=standard package that needs to compile an *.rs
> > file.
> >
> > Sage is slowly modularizing, and if you look in build/pkgs, you'll
> > see
> > a list of separate "packages" that can be installed. The
> > terminology
> > is a bit confusing nowadays, but if you look in (for example)
> > build/pkgs/flint/type, you'll see "standard". A standard package is
> > one that is essentially a part of sage itself. Those packages
> > should
> > also not be written in rust, at least for now, since everyone needs
> > to
> > be able to build them from source.
> >
> > In your case you probably just want people to be able to `pip
> > install`
> > your package and to have it show up in their Sage interface. That's
> > possible without an entry in build/pkgs at all, and you can
> > implement
> > it however you want. So long as it's not shipped with and an
> > essential
> > part of sage itself, an independent python module/extension has a
> > lot
> > of freedom.
> >