Dependencies (mpmath and rubi/MatchPy)

[Oscar Gustafsson]

Dear all,

two issues that I've been thinking about that are maybe related.

1. It seems like the maintenance of mpmath is a bit on hold. I seem to recall that it has been discussed for SymPy to take over the maintenance of it. Is this correct? Is now the time? (I assume that maybe we do not have the competence to actually develop it further, but more sorting out the few bugs that emerge etc.)

2. It has been discussed adding matchplay as a dependency. Related to that: would it make sense to move the rubi code out to its own repository? The code seems highly useful if/when it works and this can maybe spark some interest in it (or at least reduce the size of SymPy...). Then, matchpy can easily be made a dependency there. (And possibly for the remaining sympy as well, as it is sort of independent, but definitely the largest direct benefit is currently with rubi.)

Both these require that "someone" (at least Aaron I assume) creates repos and are willing to assist in setting up deployment chains etc. I may personally not be willing to take on a larger role for any of these repos, but can definitely be part of them and assist in moving rubi.

BR Oscar Gustafsson

[Oscar Gustafsson]

Directly after sending the email, I found this highly relevant issue regarding mpmath:

https://github.com/fredrik-johansson/mpmath/issues/550

[Aaron Meurer]

I'm in favor of moving RUBI out. IMO it should have been developed
that way in the first place. We should move it back in once it is in a
functioning state, but right now it mostly doesn't work and just adds
unnecessary bloat to the library.

Ideally anyone with push access should be able to do this? The GitHub
permissions system is a bit complicated, but I *think* I just enabled
it so that anyone with push access can create new repos. Do you see
the green "New" button here https://github.com/sympy? If it doesn't
work, or if you need permissions to do something else like setting up
CI, let me know. Ideally anyone should be able to do these things
without being blocked on me.

Aaron Meurer

>
> BR Oscar Gustafsson
>
> --
> You received this message because you are subscribed to the Google Groups "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAFjzj-K8iEL-ZoyNqSFgL9eGRYfw1tK86cO0KUDy3Bmd9qriCg%40mail.gmail.com.

[Oscar Benjamin]

On Mon, 31 Jan 2022 at 20:53, Aaron Meurer <asme...@gmail.com> wrote:
>
> On Mon, Jan 31, 2022 at 2:16 AM Oscar Gustafsson
> <oscar.gu...@gmail.com> wrote:
> >
> > Dear all,
> >
> > two issues that I've been thinking about that are maybe related.
> >
> > 1. It seems like the maintenance of mpmath is a bit on hold. I seem to recall that it has been discussed for SymPy to take over the maintenance of it. Is this correct? Is now the time? (I assume that maybe we do not have the competence to actually develop it further, but more sorting out the few bugs that emerge etc.)

Taking over maintenance of mpmath has been discussed and it is agreed
to do that. It just needs someone with the bandwidth to figure out
exactly how transferring a repo works. Ideally we preserve all issues
and other things in the migration.

> > 2. It has been discussed adding matchplay as a dependency. Related to that: would it make sense to move the rubi code out to its own repository? The code seems highly useful if/when it works and this can maybe spark some interest in it (or at least reduce the size of SymPy...). Then, matchpy can easily be made a dependency there. (And possibly for the remaining sympy as well, as it is sort of independent, but definitely the largest direct benefit is currently with rubi.)
> >
> > Both these require that "someone" (at least Aaron I assume) creates repos and are willing to assist in setting up deployment chains etc. I may personally not be willing to take on a larger role for any of these repos, but can definitely be part of them and assist in moving rubi.
>
> I'm in favor of moving RUBI out. IMO it should have been developed
> that way in the first place. We should move it back in once it is in a
> functioning state, but right now it mostly doesn't work and just adds
> unnecessary bloat to the library.

I'm also in favour of moving RUBI out. More importantly than bloat I
think it reflects badly on SymPy in general if the main project ships
with things that don't work yet.

I would like it to be developed but it isn't being developed currently
and I don't think having it in the sympy codebase helps it to get
developed. If it was clearly outside then maybe even someone would
fork it and work on it independently more easily.

--
Oscar

[Aaron Meurer]

My question is just how (dis)functional is RUBI in SymPy right now? Is
there any way someone could be using the RUBI part of SymPy in its
current state? Removing it would break such code if they were.

I just tried

from sympy.integrals.rubi.rubimain import *
rubi_integrate(sin(x), x)

and it took 8 minutes (no joke) to produce -cos(x). I then tried

rubi_integrate(sin(x)*cos(x), x)

and it immediately produced an error "AttributeError: 'NoneType'
object has no attribute 'is_Relational'"

I then did

rubi_integrate(x, x)

it also took 8 minutes, then it produced Integral(0, x), which is
obviously completely wrong!

So based just on that, parts of it don't work at all, and other parts
"work" but are so slow that it's almost unusable (to be fair, the
existing integrators in SymPy can also sometimes be slow, although
generally for much more complicated integrals). The wrong answer is
the most worrying to me. Even if someone is willing to wait 8 minutes,
the integrator shouldn't be producing results that are incorrect.

I'm obviously not really exercising the true capabilities of the
integrator here, but if very simple integrals don't work, that isn't a
good sign for more complicated ones. And given that RUBI is basically
a huge lookup table, how can we be sure of correctness, unless there
is extremely extensive testing. It looks like there are a lot of tests
which are copied from the main RUBI test suite. But are they passing?
Is it even physically possible to tell if every integral takes 8
minutes to run? Is the above integrate(x) example tested, or were
integrals that are "too trivial" overlooked?

If anyone is actually using sympy.integrals.rubi, or even part of it,
please let us know. For now, I can't see how removing it would break
too much user code.

Aaron Meurer

>
> --
> Oscar

>
> --
> You received this message because you are subscribed to the Google Groups "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAHVvXxSvxpTq6r4yfoy5P5nsFtMtXcq2j3n1s_2mFOoRPDw%3DRw%40mail.gmail.com.

[Francesco Bonazzi]

I think there might be some issues with MatchPy when a constraint object is shared across patterns. Probably every pattern needs to have its own CustomConstraint objects even if they share the same constraints.

How do you create a new repo? How do you split the subfolder a keep the git history?

[Oscar Gustafsson]

It was possible to create a new repository, so there is now one: https://github.com/sympy/sympy_rubi

I made an attempt to move everything with history and it didn't fully work. I used ths: https://gist.github.com/trongthanh/2779392 but I could not do

git merge repo1 --allow-unrelated-histories

as my git didn't like the --allow-unrelated-histories switch. Will have to try slightly different approaches, but I expect that in the end it will be possible.

Then there will be a bit of work to set up all the related scripts, but I think it should be manageable.

The temporary result is here (will not try anymore today): https://github.com/oscargus/sympy_rubi and although it has 212 commits, no file seems to have any history. Also, your commit today is not in the commit list, which is a bit surprising.

BR Oscar

To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/cb62b0cf-cc2b-489a-99cf-68777a18bb0cn%40googlegroups.com.

[Aaron Meurer]

I think the "allow unrelated histories" part of that gist is unneeded.
That's only relevant if you are wanting to merge the SymPy git history
into the already existing history of some other repo. But since we are
starting the other repo anew, we can just use the one history
extracted from the SymPy history as the only git history of the repo.

Aaron Meurer

> To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAFjzj-LcX4KxTvh47UXc3G3rdU36e9WWbFGb%2B2HSDivp7%3DLVsQ%40mail.gmail.com.